2024-09-10T11:20:53.466 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T11:20:53.491 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T11:20:53.584 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086 branch: wip-rishabh-testing-20240909.165402-squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-full} email: null first_in_suite: false job_id: '7899086' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlFjNnscouiMRY0pAe90+tzbIv1vkfdlkF+pFCU20uQTeQ6aHxlKLQp2zUwCk30OECOszJ4FL7JVMXUbEdC82M= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9vUsvZ0x57VBxzkLyzVPazLOq1BugG+PWld+sAMJ4aY4LQedzFueRyB0mLJ9f8HF6q8Jej6rrCVd+8r7pCHv8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T11:20:53.584 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T11:20:53.585 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T11:20:53.585 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T11:20:53.587 INFO:teuthology.task.internal:Checking packages... 2024-09-10T11:20:53.609 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T11:20:53.610 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T11:20:53.610 INFO:teuthology.packaging:ref: None 2024-09-10T11:20:53.610 INFO:teuthology.packaging:tag: None 2024-09-10T11:20:53.610 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:20:53.610 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:20:53.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:20:53.750 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T11:20:53.752 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T11:20:53.758 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T11:20:53.759 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T11:20:53.799 INFO:teuthology.task.internal:Saving configuration 2024-09-10T11:20:53.818 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T11:20:53.833 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T11:20:53.857 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 11:15:54.008971', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlFjNnscouiMRY0pAe90+tzbIv1vkfdlkF+pFCU20uQTeQ6aHxlKLQp2zUwCk30OECOszJ4FL7JVMXUbEdC82M='} 2024-09-10T11:20:53.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 11:15:54.007234', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9vUsvZ0x57VBxzkLyzVPazLOq1BugG+PWld+sAMJ4aY4LQedzFueRyB0mLJ9f8HF6q8Jej6rrCVd+8r7pCHv8='} 2024-09-10T11:20:53.876 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T11:20:53.882 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T11:20:53.882 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T11:20:53.882 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T11:20:53.967 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff92843370>, signals=[15]) 2024-09-10T11:20:53.967 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T11:20:53.973 INFO:teuthology.task.internal:Opening connections... 2024-09-10T11:20:53.973 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:20:53.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:20:54.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:20:54.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:20:54.125 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T11:20:54.132 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-10T11:20:54.151 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-10T11:20:54.151 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-10T11:20:54.208 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T11:20:54.209 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T11:20:54.210 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-10T11:20:54.230 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-10T11:20:54.249 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-10T11:20:54.249 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-10T11:20:54.306 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T11:20:54.307 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T11:20:54.307 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-10T11:20:54.331 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T11:20:54.339 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T11:20:54.365 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T11:20:54.366 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-10T11:20:54.368 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-10T11:20:54.385 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T11:20:54.403 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T11:20:54.403 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-10T11:20:54.425 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-10T11:20:54.539 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T11:20:54.591 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T11:20:54.604 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T11:20:54.604 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T11:20:54.604 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T11:20:54.605 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T11:20:54.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:20:54.621 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-10T11:20:54.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:20:54.622 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-10T11:20:54.677 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-503.el9.x86_64 2024-09-10T11:20:54.678 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-503.el9.x86_64 2024-09-10T11:20:54.678 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-09-10T11:20:54.680 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-503.el9.x86_64 2024-09-10T11:20:54.680 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.14.0-503.el9.x86_64 2024-09-10T11:20:54.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2024-09-10T11:20:58.283 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-10T11:20:58.374 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-10T11:21:01.658 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 20 MB 00:02 2024-09-10T11:21:05.975 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-10T11:21:06.449 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 20 MB 00:07 2024-09-10T11:21:07.492 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-10T11:21:08.355 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-10T11:21:12.324 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-10T11:21:13.643 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-10T11:21:13.775 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 19 kB 00:00 2024-09-10T11:21:14.729 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T11:21:15.666 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T11:21:15.666 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T11:21:15.755 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-10T11:21:15.761 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-09-10T11:21:15.762 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:21:15.763 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 79 M 2024-09-10T11:21:15.763 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 126 M 2024-09-10T11:21:15.763 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-10T11:21:16.045 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 87 kB/s | 23 kB 00:00 2024-09-10T11:21:20.093 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T11:21:22.133 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T11:21:22.133 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T11:21:22.219 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T11:21:22.225 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T11:21:22.225 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-10T11:21:22.225 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout:Install 4 Packages 2024-09-10T11:21:22.226 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:21:22.227 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 79 M 2024-09-10T11:21:22.227 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 126 M 2024-09-10T11:21:22.227 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-10T11:21:22.570 INFO:teuthology.orchestra.run.smithi099.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 259 kB/s | 23 kB 00:00 2024-09-10T11:21:24.611 INFO:teuthology.orchestra.run.smithi099.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-10T11:21:25.186 INFO:teuthology.orchestra.run.smithi099.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-10T11:21:26.394 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.3 MB/s | 35 MB 00:10 2024-09-10T11:21:27.020 INFO:teuthology.orchestra.run.smithi099.stdout:(4/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.4 MB/s | 16 MB 00:04 2024-09-10T11:21:27.020 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:21:27.020 INFO:teuthology.orchestra.run.smithi099.stdout:Total 16 MB/s | 79 MB 00:04 2024-09-10T11:21:27.418 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.3 MB/s | 16 MB 00:11 2024-09-10T11:21:27.699 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T11:21:28.054 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T11:21:28.054 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T11:21:29.002 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T11:21:29.003 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T11:21:31.446 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T11:21:32.088 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:21:32.171 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:21:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:21:35.093 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:21:39.909 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:21:40.001 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:21:44.868 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:21:47.093 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 942 kB/s | 29 MB 00:31 2024-09-10T11:21:47.093 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:21:47.093 INFO:teuthology.orchestra.run.smithi196.stdout:Total 2.5 MB/s | 79 MB 00:31 2024-09-10T11:21:47.778 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-10T11:21:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-10T11:21:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-10T11:21:49.116 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-10T11:21:49.116 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-10T11:21:51.572 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-10T11:21:52.262 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:21:52.343 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:21:55.129 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:21:55.261 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:22:00.064 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:22:00.102 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:04.917 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:22.934 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:22.974 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:22:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:22:23.343 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:22:24.175 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T11:22:24.508 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:22:24.603 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.603 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:22:24.603 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T11:22:24.603 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T11:22:24.604 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T11:22:24.604 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T11:22:24.604 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T11:22:24.604 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:22:24.662 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:24.662 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:22:24.662 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-09-10T11:22:24.694 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 82.el9 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8274670 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T11:22:24.695 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:22:24.696 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T11:22:24.697 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T11:22:24.697 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 5.14.0-505.el9.x86_64 2024-09-10T11:22:24.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T11:22:25.378 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-10T11:22:26.758 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T11:22:26.784 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-10T11:22:26.790 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-10T11:22:26.805 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.pKu6QddIeF 2024-09-10T11:22:26.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.pKu6QddIeF 2024-09-10T11:22:26.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.pKu6QddIeF 2024-09-10T11:22:27.015 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.pKu6QddIeF is 6KB 2024-09-10T11:22:27.064 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.pKu6QddIeF 2024-09-10T11:22:27.078 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T11:22:27.141 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T11:22:27.141 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T11:22:27.141 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T11:22:27.141 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T11:22:27.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T11:22:27.292 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-09-10T11:22:43.175 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:43.214 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:43.569 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:43.569 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:22:43.569 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:22:43.569 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:22:44.411 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-10T11:22:44.776 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:22:44.928 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.928 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:22:44.928 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T11:22:44.928 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T11:22:44.929 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T11:22:44.929 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T11:22:44.929 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T11:22:44.929 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:22:44.988 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:22:44.988 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:22:44.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 82.el9 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T11:22:45.021 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8274670 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:22:45.022 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T11:22:45.023 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T11:22:45.023 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T11:22:45.023 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-09-10T11:22:45.023 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:22:45.023 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T11:22:45.024 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T11:22:45.024 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-505.el9.x86_64 2024-09-10T11:22:45.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T11:22:45.661 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-09-10T11:22:46.982 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T11:22:47.014 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-09-10T11:22:47.016 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-10T11:22:47.031 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.eQKOrD8tXX 2024-09-10T11:22:47.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eQKOrD8tXX 2024-09-10T11:22:47.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.eQKOrD8tXX 2024-09-10T11:22:47.238 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.eQKOrD8tXX is 6KB 2024-09-10T11:22:47.286 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.eQKOrD8tXX 2024-09-10T11:22:47.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T11:22:47.372 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T11:22:47.372 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T11:22:47.372 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T11:22:47.372 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T11:22:47.374 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T11:22:47.534 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-09-10T11:22:57.303 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T11:22:57.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:22:57.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:23:15.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T11:23:17.539 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T11:23:17.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:23:17.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:23:24.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:23:24.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:23:27.709 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T11:23:35.869 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-10T11:23:39.713 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:23:39.714 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:23:44.873 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:23:44.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:23:47.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-10T11:23:58.141 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-10T11:23:59.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:23:59.936 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:24:13.153 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:24:13.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:24:13.565 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-10T11:24:13.932 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:24:13.933 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T11:24:13.933 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-10T11:24:13.947 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-505.el9.x86_64 2024-09-10T11:24:13.947 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T11:24:13.948 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T11:24:13.948 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T11:24:14.948 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T11:24:14.949 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T11:24:15.068 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-09-10T11:24:15.100 DEBUG:teuthology.parallel:result is None 2024-09-10T11:24:18.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-10T11:24:33.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:24:33.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:24:33.632 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-10T11:24:34.152 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:24:34.153 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T11:24:34.153 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-10T11:24:34.178 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-505.el9.x86_64 2024-09-10T11:24:34.178 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T11:24:34.179 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T11:24:34.179 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T11:24:35.180 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T11:24:35.180 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T11:24:35.242 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-09-10T11:24:35.259 DEBUG:teuthology.parallel:result is None 2024-09-10T11:24:35.259 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T11:24:35.266 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T11:24:35.266 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T11:24:35.269 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T11:24:35.286 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T11:24:35.292 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T11:24:35.325 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T11:24:35.325 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T11:24:35.329 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T11:24:35.374 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T11:24:35.381 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T11:24:35.381 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T11:24:35.392 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T11:24:35.425 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:24:35.437 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:24:35.447 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:24:35.457 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:24:35.458 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T11:24:35.465 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T11:24:35.465 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T11:24:35.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T11:24:35.523 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T11:24:35.532 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T11:24:35.532 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T11:24:35.548 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T11:24:35.578 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:24:35.641 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:24:35.697 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:24:35.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:24:35.821 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:24:35.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T11:24:35.887 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:24:35.927 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:24:35.975 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:24:36.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:24:36.097 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:24:36.097 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T11:24:36.162 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-10T11:24:36.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-10T11:24:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:24:36.250 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:24:36.753 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T11:24:36.762 INFO:teuthology.task.internal:Starting timer... 2024-09-10T11:24:36.762 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T11:24:36.804 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T11:24:36.836 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-10T11:24:36.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-10T11:24:36.901 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-10T11:24:36.902 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-10T11:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-10T11:24:36.963 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T11:24:36.963 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-10T11:24:36.985 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-10T11:24:36.985 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-10T11:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-10T11:24:37.019 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-10T11:24:37.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:24:37.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:37.053 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:24:37.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:24:37.085 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T11:24:37.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-10T11:24:37.122 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-10T11:24:37.160 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T11:24:37.169 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T11:24:37.173 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T11:24:37.288 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T11:24:37.307 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T11:24:37.308 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T11:28:19.287 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-10T11:28:19.288 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:28:19.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:28:19.368 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-10T11:28:19.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-10T11:28:19.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:28:19.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:28:19.525 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-10T11:28:19.609 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-10T11:28:19.609 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T11:28:19.621 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T11:28:19.621 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T11:28:19.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:28:19.624 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T11:28:19.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:28:19.654 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T11:28:19.669 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T11:28:19.678 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T11:28:19.692 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T11:28:19.695 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-10T11:28:19.706 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-10T11:28:19.716 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-10T11:28:19.721 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T11:28:19.727 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-10T11:28:19.735 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T11:28:19.741 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T11:28:19.755 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T11:28:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.794 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.812 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.816 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:28:19.817 INFO:teuthology.run_tasks:Running task install... 2024-09-10T11:28:19.825 DEBUG:teuthology.task.install:project ceph 2024-09-10T11:28:19.825 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T11:28:19.825 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T11:28:19.825 INFO:teuthology.task.install:Using flavor: default 2024-09-10T11:28:19.830 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T11:28:19.830 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T11:28:19.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T11:28:19.831 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:19.833 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T11:28:19.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:19.983 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T11:28:19.983 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T11:28:19.987 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T11:28:19.987 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T11:28:20.097 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T11:28:20.097 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:28:20.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T11:28:20.100 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T11:28:20.100 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:28:20.100 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T11:28:20.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T11:28:20.124 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T11:28:20.127 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T11:28:20.127 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T11:28:20.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T11:28:20.194 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T11:28:20.266 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T11:28:20.270 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T11:28:20.330 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-10T11:28:20.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-10T11:28:20.334 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-10T11:28:20.336 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-10T11:28:20.559 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-10T11:28:20.563 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-10T11:28:20.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T11:28:20.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T11:28:21.140 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-10T11:28:21.150 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 304 kB/s | 84 kB 00:00 2024-09-10T11:28:21.408 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 53 kB/s | 12 kB 00:00 2024-09-10T11:28:21.426 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-10T11:28:21.667 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-10T11:28:21.693 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T11:28:22.596 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.2 MB 00:00 2024-09-10T11:28:23.433 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-10T11:28:24.611 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T11:28:27.700 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-10T11:28:28.979 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T11:28:30.788 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T11:28:31.533 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-10T11:28:31.934 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-10T11:28:33.552 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-10T11:28:34.431 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T11:28:36.914 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T11:28:39.241 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T11:28:39.241 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T11:28:39.253 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T11:28:39.253 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T11:28:39.254 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T11:28:39.255 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T11:28:39.257 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T11:28:39.259 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T11:28:39.260 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T11:28:39.323 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-10T11:28:39.338 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T11:28:39.339 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T11:28:39.340 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T11:28:39.341 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T11:28:39.342 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T11:28:39.343 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T11:28:39.344 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T11:28:39.345 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T11:28:39.346 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T11:28:39.347 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T11:28:39.348 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T11:28:39.349 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T11:28:39.352 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T11:28:39.353 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T11:28:39.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T11:28:39.355 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:====================================================================================== 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-09-10T11:28:39.356 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 8 Packages 2024-09-10T11:28:39.357 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:28:39.358 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 226 M 2024-09-10T11:28:39.358 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-10T11:28:39.895 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T11:28:40.266 INFO:teuthology.orchestra.run.smithi196.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2024-09-10T11:28:40.582 INFO:teuthology.orchestra.run.smithi196.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.8 MB/s | 897 kB 00:00 2024-09-10T11:28:40.691 INFO:teuthology.orchestra.run.smithi196.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-09-10T11:28:40.716 INFO:teuthology.orchestra.run.smithi196.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.1 MB/s | 145 kB 00:00 2024-09-10T11:28:40.883 INFO:teuthology.orchestra.run.smithi196.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-10T11:28:41.200 INFO:teuthology.orchestra.run.smithi196.stdout:(6/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 15 MB/s | 4.7 MB 00:00 2024-09-10T11:28:41.325 INFO:teuthology.orchestra.run.smithi196.stdout:(7/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 1.5 MB 00:00 2024-09-10T11:28:41.676 INFO:teuthology.orchestra.run.smithi196.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 21 MB 00:01 2024-09-10T11:28:41.718 INFO:teuthology.orchestra.run.smithi196.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 606 kB/s | 25 kB 00:00 2024-09-10T11:28:42.143 INFO:teuthology.orchestra.run.smithi196.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 18 MB/s | 17 MB 00:00 2024-09-10T11:28:42.201 INFO:teuthology.orchestra.run.smithi196.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.8 MB/s | 228 kB 00:00 2024-09-10T11:28:42.231 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T11:28:42.231 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T11:28:42.244 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T11:28:42.245 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T11:28:42.246 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T11:28:42.247 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T11:28:42.249 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T11:28:42.252 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T11:28:42.253 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T11:28:42.285 INFO:teuthology.orchestra.run.smithi196.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 324 kB/s | 27 kB 00:00 2024-09-10T11:28:42.317 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-10T11:28:42.338 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-10T11:28:42.338 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-10T11:28:42.338 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T11:28:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T11:28:42.340 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T11:28:42.341 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T11:28:42.342 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T11:28:42.343 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T11:28:42.344 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T11:28:42.345 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T11:28:42.346 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T11:28:42.347 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T11:28:42.348 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T11:28:42.349 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T11:28:42.350 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T11:28:42.351 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 8 Packages 2024-09-10T11:28:42.352 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:28:42.353 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 226 M 2024-09-10T11:28:42.353 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-10T11:28:42.360 INFO:teuthology.orchestra.run.smithi196.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 423 kB/s | 32 kB 00:00 2024-09-10T11:28:42.452 INFO:teuthology.orchestra.run.smithi196.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T11:28:42.502 INFO:teuthology.orchestra.run.smithi196.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 3.2 MB/s | 164 kB 00:00 2024-09-10T11:28:42.553 INFO:teuthology.orchestra.run.smithi196.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T11:28:42.661 INFO:teuthology.orchestra.run.smithi196.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 4.6 MB/s | 506 kB 00:00 2024-09-10T11:28:42.879 INFO:teuthology.orchestra.run.smithi099.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2024-09-10T11:28:43.055 INFO:teuthology.orchestra.run.smithi196.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-10T11:28:43.120 INFO:teuthology.orchestra.run.smithi196.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 694 kB/s | 45 kB 00:00 2024-09-10T11:28:43.187 INFO:teuthology.orchestra.run.smithi196.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.0 MB/s | 136 kB 00:00 2024-09-10T11:28:43.262 INFO:teuthology.orchestra.run.smithi196.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T11:28:43.346 INFO:teuthology.orchestra.run.smithi196.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 3.8 MB/s | 320 kB 00:00 2024-09-10T11:28:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.8 MB/s | 897 kB 00:00 2024-09-10T11:28:43.413 INFO:teuthology.orchestra.run.smithi196.stdout:(23/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 4.4 MB/s | 301 kB 00:00 2024-09-10T11:28:43.454 INFO:teuthology.orchestra.run.smithi099.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T11:28:43.488 INFO:teuthology.orchestra.run.smithi196.stdout:(24/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.3 MB/s | 99 kB 00:00 2024-09-10T11:28:43.563 INFO:teuthology.orchestra.run.smithi196.stdout:(25/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-10T11:28:43.864 INFO:teuthology.orchestra.run.smithi196.stdout:(26/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-10T11:28:43.931 INFO:teuthology.orchestra.run.smithi196.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T11:28:44.013 INFO:teuthology.orchestra.run.smithi099.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 4.3 MB/s | 2.4 MB 00:00 2024-09-10T11:28:44.171 INFO:teuthology.orchestra.run.smithi099.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 9.6 MB/s | 1.5 MB 00:00 2024-09-10T11:28:44.297 INFO:teuthology.orchestra.run.smithi099.stdout:(6/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 3.1 MB/s | 5.4 MB 00:01 2024-09-10T11:28:44.365 INFO:teuthology.orchestra.run.smithi196.stdout:(28/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 18 MB/s | 49 MB 00:02 2024-09-10T11:28:44.389 INFO:teuthology.orchestra.run.smithi196.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 69 kB/s | 32 kB 00:00 2024-09-10T11:28:44.415 INFO:teuthology.orchestra.run.smithi196.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T11:28:44.630 INFO:teuthology.orchestra.run.smithi099.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-10T11:28:44.665 INFO:teuthology.orchestra.run.smithi196.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 13 MB/s | 3.6 MB 00:00 2024-09-10T11:28:44.749 INFO:teuthology.orchestra.run.smithi196.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.9 MB/s | 249 kB 00:00 2024-09-10T11:28:44.832 INFO:teuthology.orchestra.run.smithi196.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 592 kB/s | 49 kB 00:00 2024-09-10T11:28:44.924 INFO:teuthology.orchestra.run.smithi196.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-10T11:28:44.958 INFO:teuthology.orchestra.run.smithi196.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 131 kB/s | 16 kB 00:00 2024-09-10T11:28:45.092 INFO:teuthology.orchestra.run.smithi196.stdout:(36/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 2.8 MB/s | 11 MB 00:03 2024-09-10T11:28:45.117 INFO:teuthology.orchestra.run.smithi196.stdout:(37/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 1.4 MB/s | 285 kB 00:00 2024-09-10T11:28:45.151 INFO:teuthology.orchestra.run.smithi196.stdout:(38/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 3.9 MB/s | 766 kB 00:00 2024-09-10T11:28:45.176 INFO:teuthology.orchestra.run.smithi196.stdout:(39/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 218 kB/s | 18 kB 00:00 2024-09-10T11:28:45.251 INFO:teuthology.orchestra.run.smithi196.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 597 kB/s | 80 kB 00:00 2024-09-10T11:28:45.335 INFO:teuthology.orchestra.run.smithi196.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-10T11:28:45.360 INFO:teuthology.orchestra.run.smithi196.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-10T11:28:45.410 INFO:teuthology.orchestra.run.smithi196.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T11:28:45.455 INFO:teuthology.orchestra.run.smithi099.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 7.2 MB/s | 21 MB 00:02 2024-09-10T11:28:45.494 INFO:teuthology.orchestra.run.smithi196.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-10T11:28:45.519 INFO:teuthology.orchestra.run.smithi196.stdout:(45/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 801 kB 00:00 2024-09-10T11:28:45.569 INFO:teuthology.orchestra.run.smithi196.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-10T11:28:45.572 INFO:teuthology.orchestra.run.smithi099.stdout:(9/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9. 11 MB/s | 11 MB 00:00 2024-09-10T11:28:45.597 INFO:teuthology.orchestra.run.smithi099.stdout:(10/156): ceph-selinux-19.1.1-305.g1cf819b7.el9 177 kB/s | 25 kB 00:00 2024-09-10T11:28:45.603 INFO:teuthology.orchestra.run.smithi196.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T11:28:45.648 INFO:teuthology.orchestra.run.smithi099.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 4.5 MB/s | 228 kB 00:00 2024-09-10T11:28:45.753 INFO:teuthology.orchestra.run.smithi196.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-10T11:28:45.815 INFO:teuthology.orchestra.run.smithi099.stdout:(12/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 17 MB 00:01 2024-09-10T11:28:45.820 INFO:teuthology.orchestra.run.smithi196.stdout:(49/156): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-09-10T11:28:45.840 INFO:teuthology.orchestra.run.smithi099.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 141 kB/s | 27 kB 00:00 2024-09-10T11:28:45.845 INFO:teuthology.orchestra.run.smithi196.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-09-10T11:28:45.865 INFO:teuthology.orchestra.run.smithi099.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 636 kB/s | 32 kB 00:00 2024-09-10T11:28:45.871 INFO:teuthology.orchestra.run.smithi196.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-10T11:28:45.904 INFO:teuthology.orchestra.run.smithi196.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-10T11:28:45.915 INFO:teuthology.orchestra.run.smithi099.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-10T11:28:45.930 INFO:teuthology.orchestra.run.smithi196.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-10T11:28:45.941 INFO:teuthology.orchestra.run.smithi099.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T11:28:45.957 INFO:teuthology.orchestra.run.smithi196.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-10T11:28:45.974 INFO:teuthology.orchestra.run.smithi099.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.1 MB/s | 127 kB 00:00 2024-09-10T11:28:45.980 INFO:teuthology.orchestra.run.smithi196.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-10T11:28:45.997 INFO:teuthology.orchestra.run.smithi196.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-10T11:28:46.016 INFO:teuthology.orchestra.run.smithi099.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 6.6 MB/s | 506 kB 00:00 2024-09-10T11:28:46.022 INFO:teuthology.orchestra.run.smithi196.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-09-10T11:28:46.039 INFO:teuthology.orchestra.run.smithi196.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-10T11:28:46.066 INFO:teuthology.orchestra.run.smithi099.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 904 kB/s | 45 kB 00:00 2024-09-10T11:28:46.098 INFO:teuthology.orchestra.run.smithi196.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-10T11:28:46.131 INFO:teuthology.orchestra.run.smithi196.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 737 kB/s | 24 kB 00:00 2024-09-10T11:28:46.133 INFO:teuthology.orchestra.run.smithi099.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.0 MB/s | 136 kB 00:00 2024-09-10T11:28:46.165 INFO:teuthology.orchestra.run.smithi196.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-10T11:28:46.209 INFO:teuthology.orchestra.run.smithi099.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T11:28:46.224 INFO:teuthology.orchestra.run.smithi196.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-10T11:28:46.266 INFO:teuthology.orchestra.run.smithi196.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-09-10T11:28:46.292 INFO:teuthology.orchestra.run.smithi099.stdout:(22/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-10T11:28:46.307 INFO:teuthology.orchestra.run.smithi196.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T11:28:46.326 INFO:teuthology.orchestra.run.smithi099.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 2.7 MB/s | 320 kB 00:00 2024-09-10T11:28:46.333 INFO:teuthology.orchestra.run.smithi196.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 573 kB/s | 38 kB 00:00 2024-09-10T11:28:46.351 INFO:teuthology.orchestra.run.smithi099.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T11:28:46.376 INFO:teuthology.orchestra.run.smithi099.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T11:28:46.402 INFO:teuthology.orchestra.run.smithi099.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T11:28:46.452 INFO:teuthology.orchestra.run.smithi099.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T11:28:46.535 INFO:teuthology.orchestra.run.smithi099.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-10T11:28:46.561 INFO:teuthology.orchestra.run.smithi099.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 294 kB/s | 32 kB 00:00 2024-09-10T11:28:46.586 INFO:teuthology.orchestra.run.smithi099.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T11:28:46.600 INFO:teuthology.orchestra.run.smithi196.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.3 MB/s | 3.0 MB 00:00 2024-09-10T11:28:46.811 INFO:teuthology.orchestra.run.smithi099.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 14 MB/s | 3.6 MB 00:00 2024-09-10T11:28:46.895 INFO:teuthology.orchestra.run.smithi099.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.9 MB/s | 249 kB 00:00 2024-09-10T11:28:47.037 INFO:teuthology.orchestra.run.smithi099.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-10T11:28:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-10T11:28:47.062 INFO:teuthology.orchestra.run.smithi099.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 295 kB/s | 49 kB 00:00 2024-09-10T11:28:47.087 INFO:teuthology.orchestra.run.smithi099.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 330 kB/s | 16 kB 00:00 2024-09-10T11:28:47.113 INFO:teuthology.orchestra.run.smithi099.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T11:28:47.154 INFO:teuthology.orchestra.run.smithi099.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T11:28:47.180 INFO:teuthology.orchestra.run.smithi099.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 272 kB/s | 18 kB 00:00 2024-09-10T11:28:47.264 INFO:teuthology.orchestra.run.smithi099.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 734 kB/s | 80 kB 00:00 2024-09-10T11:28:47.309 INFO:teuthology.orchestra.run.smithi196.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-10T11:28:47.384 INFO:teuthology.orchestra.run.smithi196.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-10T11:28:47.601 INFO:teuthology.orchestra.run.smithi196.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-10T11:28:47.697 INFO:teuthology.orchestra.run.smithi099.stdout:(40/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 23 MB/s | 49 MB 00:02 2024-09-10T11:28:47.722 INFO:teuthology.orchestra.run.smithi099.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 417 kB/s | 191 kB 00:00 2024-09-10T11:28:47.756 INFO:teuthology.orchestra.run.smithi099.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-10T11:28:47.759 INFO:teuthology.orchestra.run.smithi196.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-10T11:28:47.798 INFO:teuthology.orchestra.run.smithi099.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T11:28:47.818 INFO:teuthology.orchestra.run.smithi196.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T11:28:47.857 INFO:teuthology.orchestra.run.smithi099.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-10T11:28:47.882 INFO:teuthology.orchestra.run.smithi099.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-10T11:28:47.907 INFO:teuthology.orchestra.run.smithi099.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T11:28:47.941 INFO:teuthology.orchestra.run.smithi099.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-10T11:28:47.951 INFO:teuthology.orchestra.run.smithi196.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.9 MB/s | 1.0 MB 00:00 2024-09-10T11:28:47.966 INFO:teuthology.orchestra.run.smithi099.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T11:28:47.993 INFO:teuthology.orchestra.run.smithi196.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T11:28:48.083 INFO:teuthology.orchestra.run.smithi099.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-09-10T11:28:48.094 INFO:teuthology.orchestra.run.smithi196.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T11:28:48.117 INFO:teuthology.orchestra.run.smithi099.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-10T11:28:48.133 INFO:teuthology.orchestra.run.smithi099.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-10T11:28:48.175 INFO:teuthology.orchestra.run.smithi099.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-10T11:28:48.177 INFO:teuthology.orchestra.run.smithi196.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T11:28:48.193 INFO:teuthology.orchestra.run.smithi099.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-09-10T11:28:48.217 INFO:teuthology.orchestra.run.smithi099.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-10T11:28:48.219 INFO:teuthology.orchestra.run.smithi196.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 617 kB/s | 26 kB 00:00 2024-09-10T11:28:48.243 INFO:teuthology.orchestra.run.smithi099.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-10T11:28:48.261 INFO:teuthology.orchestra.run.smithi196.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 462 kB/s | 19 kB 00:00 2024-09-10T11:28:48.268 INFO:teuthology.orchestra.run.smithi099.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T11:28:48.302 INFO:teuthology.orchestra.run.smithi099.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-10T11:28:48.327 INFO:teuthology.orchestra.run.smithi099.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-09-10T11:28:48.337 INFO:teuthology.orchestra.run.smithi196.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T11:28:48.360 INFO:teuthology.orchestra.run.smithi099.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-10T11:28:48.378 INFO:teuthology.orchestra.run.smithi196.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-09-10T11:28:48.419 INFO:teuthology.orchestra.run.smithi099.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-10T11:28:48.444 INFO:teuthology.orchestra.run.smithi099.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-10T11:28:48.469 INFO:teuthology.orchestra.run.smithi099.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-10T11:28:48.495 INFO:teuthology.orchestra.run.smithi099.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T11:28:48.520 INFO:teuthology.orchestra.run.smithi099.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-10T11:28:48.545 INFO:teuthology.orchestra.run.smithi099.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-10T11:28:48.570 INFO:teuthology.orchestra.run.smithi099.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-10T11:28:48.629 INFO:teuthology.orchestra.run.smithi099.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-10T11:28:48.696 INFO:teuthology.orchestra.run.smithi099.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-09-10T11:28:48.763 INFO:teuthology.orchestra.run.smithi099.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-10T11:28:48.788 INFO:teuthology.orchestra.run.smithi099.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T11:28:48.814 INFO:teuthology.orchestra.run.smithi099.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-10T11:28:48.847 INFO:teuthology.orchestra.run.smithi099.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-10T11:28:48.872 INFO:teuthology.orchestra.run.smithi099.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T11:28:48.914 INFO:teuthology.orchestra.run.smithi099.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T11:28:48.990 INFO:teuthology.orchestra.run.smithi099.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-10T11:28:49.015 INFO:teuthology.orchestra.run.smithi099.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-10T11:28:49.040 INFO:teuthology.orchestra.run.smithi099.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-10T11:28:49.066 INFO:teuthology.orchestra.run.smithi099.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-10T11:28:49.090 INFO:teuthology.orchestra.run.smithi099.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-10T11:28:49.116 INFO:teuthology.orchestra.run.smithi099.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T11:28:49.137 INFO:teuthology.orchestra.run.smithi196.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-10T11:28:49.141 INFO:teuthology.orchestra.run.smithi099.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-10T11:28:49.225 INFO:teuthology.orchestra.run.smithi099.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-10T11:28:49.237 INFO:teuthology.orchestra.run.smithi196.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-10T11:28:49.250 INFO:teuthology.orchestra.run.smithi099.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-10T11:28:49.275 INFO:teuthology.orchestra.run.smithi099.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-10T11:28:49.301 INFO:teuthology.orchestra.run.smithi099.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T11:28:49.326 INFO:teuthology.orchestra.run.smithi099.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-10T11:28:49.337 INFO:teuthology.orchestra.run.smithi196.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-10T11:28:49.351 INFO:teuthology.orchestra.run.smithi099.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-10T11:28:49.376 INFO:teuthology.orchestra.run.smithi099.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T11:28:49.396 INFO:teuthology.orchestra.run.smithi196.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T11:28:49.402 INFO:teuthology.orchestra.run.smithi099.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T11:28:49.419 INFO:teuthology.orchestra.run.smithi099.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T11:28:49.436 INFO:teuthology.orchestra.run.smithi099.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-10T11:28:49.438 INFO:teuthology.orchestra.run.smithi196.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T11:28:49.519 INFO:teuthology.orchestra.run.smithi099.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-10T11:28:49.536 INFO:teuthology.orchestra.run.smithi099.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-10T11:28:49.538 INFO:teuthology.orchestra.run.smithi196.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-10T11:28:49.553 INFO:teuthology.orchestra.run.smithi099.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-10T11:28:49.578 INFO:teuthology.orchestra.run.smithi099.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-09-10T11:28:49.595 INFO:teuthology.orchestra.run.smithi099.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-09-10T11:28:49.613 INFO:teuthology.orchestra.run.smithi196.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-10T11:28:49.620 INFO:teuthology.orchestra.run.smithi099.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T11:28:49.654 INFO:teuthology.orchestra.run.smithi099.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-10T11:28:49.679 INFO:teuthology.orchestra.run.smithi099.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 991 kB/s | 58 kB 00:00 2024-09-10T11:28:49.689 INFO:teuthology.orchestra.run.smithi196.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-10T11:28:49.705 INFO:teuthology.orchestra.run.smithi099.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T11:28:49.730 INFO:teuthology.orchestra.run.smithi099.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T11:28:49.731 INFO:teuthology.orchestra.run.smithi196.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T11:28:49.755 INFO:teuthology.orchestra.run.smithi099.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-10T11:28:49.772 INFO:teuthology.orchestra.run.smithi196.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-09-10T11:28:49.798 INFO:teuthology.orchestra.run.smithi196.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 800 kB/s | 54 kB 00:00 2024-09-10T11:28:49.832 INFO:teuthology.orchestra.run.smithi196.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-10T11:28:49.839 INFO:teuthology.orchestra.run.smithi099.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-10T11:28:49.865 INFO:teuthology.orchestra.run.smithi196.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-10T11:28:49.966 INFO:teuthology.orchestra.run.smithi196.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-10T11:28:49.991 INFO:teuthology.orchestra.run.smithi196.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-10T11:28:50.014 INFO:teuthology.orchestra.run.smithi099.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T11:28:50.016 INFO:teuthology.orchestra.run.smithi196.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-10T11:28:50.033 INFO:teuthology.orchestra.run.smithi196.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-10T11:28:50.065 INFO:teuthology.orchestra.run.smithi099.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-09-10T11:28:50.066 INFO:teuthology.orchestra.run.smithi196.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-09-10T11:28:50.090 INFO:teuthology.orchestra.run.smithi099.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 855 kB/s | 19 kB 00:00 2024-09-10T11:28:50.123 INFO:teuthology.orchestra.run.smithi099.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-10T11:28:50.125 INFO:teuthology.orchestra.run.smithi196.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-10T11:28:50.148 INFO:teuthology.orchestra.run.smithi099.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T11:28:50.151 INFO:teuthology.orchestra.run.smithi196.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-10T11:28:50.174 INFO:teuthology.orchestra.run.smithi099.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T11:28:50.184 INFO:teuthology.orchestra.run.smithi196.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T11:28:50.199 INFO:teuthology.orchestra.run.smithi099.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T11:28:50.201 INFO:teuthology.orchestra.run.smithi196.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-10T11:28:50.224 INFO:teuthology.orchestra.run.smithi099.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-10T11:28:50.258 INFO:teuthology.orchestra.run.smithi099.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-10T11:28:50.300 INFO:teuthology.orchestra.run.smithi099.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T11:28:50.310 INFO:teuthology.orchestra.run.smithi196.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-10T11:28:50.325 INFO:teuthology.orchestra.run.smithi099.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.9 MB/s | 43 kB 00:00 2024-09-10T11:28:50.350 INFO:teuthology.orchestra.run.smithi099.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T11:28:50.368 INFO:teuthology.orchestra.run.smithi196.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-10T11:28:50.375 INFO:teuthology.orchestra.run.smithi099.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-10T11:28:50.401 INFO:teuthology.orchestra.run.smithi099.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-10T11:28:50.402 INFO:teuthology.orchestra.run.smithi196.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-10T11:28:50.427 INFO:teuthology.orchestra.run.smithi196.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 786 kB/s | 19 kB 00:00 2024-09-10T11:28:50.434 INFO:teuthology.orchestra.run.smithi099.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-09-10T11:28:50.460 INFO:teuthology.orchestra.run.smithi099.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T11:28:50.485 INFO:teuthology.orchestra.run.smithi099.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T11:28:50.486 INFO:teuthology.orchestra.run.smithi196.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-09-10T11:28:50.510 INFO:teuthology.orchestra.run.smithi099.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T11:28:50.528 INFO:teuthology.orchestra.run.smithi196.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 628 kB/s | 26 kB 00:00 2024-09-10T11:28:50.535 INFO:teuthology.orchestra.run.smithi099.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T11:28:50.553 INFO:teuthology.orchestra.run.smithi196.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T11:28:50.560 INFO:teuthology.orchestra.run.smithi099.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-10T11:28:50.570 INFO:teuthology.orchestra.run.smithi196.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-09-10T11:28:50.628 INFO:teuthology.orchestra.run.smithi099.stdout:(124/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-09-10T11:28:50.628 INFO:teuthology.orchestra.run.smithi196.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-10T11:28:50.645 INFO:teuthology.orchestra.run.smithi099.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-09-10T11:28:50.654 INFO:teuthology.orchestra.run.smithi196.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-10T11:28:50.661 INFO:teuthology.orchestra.run.smithi099.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 691 kB/s | 23 kB 00:00 2024-09-10T11:28:50.678 INFO:teuthology.orchestra.run.smithi099.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 582 kB/s | 19 kB 00:00 2024-09-10T11:28:50.687 INFO:teuthology.orchestra.run.smithi196.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T11:28:50.695 INFO:teuthology.orchestra.run.smithi099.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-09-10T11:28:50.712 INFO:teuthology.orchestra.run.smithi099.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T11:28:50.713 INFO:teuthology.orchestra.run.smithi196.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T11:28:50.738 INFO:teuthology.orchestra.run.smithi196.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-10T11:28:50.746 INFO:teuthology.orchestra.run.smithi099.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T11:28:50.763 INFO:teuthology.orchestra.run.smithi099.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-10T11:28:50.763 INFO:teuthology.orchestra.run.smithi196.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 638 kB/s | 32 kB 00:00 2024-09-10T11:28:50.789 INFO:teuthology.orchestra.run.smithi196.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-10T11:28:50.796 INFO:teuthology.orchestra.run.smithi099.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-10T11:28:50.814 INFO:teuthology.orchestra.run.smithi196.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-10T11:28:50.822 INFO:teuthology.orchestra.run.smithi099.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-10T11:28:50.839 INFO:teuthology.orchestra.run.smithi196.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-10T11:28:50.856 INFO:teuthology.orchestra.run.smithi099.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T11:28:50.865 INFO:teuthology.orchestra.run.smithi196.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T11:28:50.881 INFO:teuthology.orchestra.run.smithi099.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-10T11:28:50.890 INFO:teuthology.orchestra.run.smithi196.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T11:28:50.906 INFO:teuthology.orchestra.run.smithi099.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T11:28:50.923 INFO:teuthology.orchestra.run.smithi196.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T11:28:50.931 INFO:teuthology.orchestra.run.smithi099.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-09-10T11:28:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-10T11:28:50.957 INFO:teuthology.orchestra.run.smithi099.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T11:28:50.974 INFO:teuthology.orchestra.run.smithi196.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 697 kB/s | 18 kB 00:00 2024-09-10T11:28:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-10T11:28:51.007 INFO:teuthology.orchestra.run.smithi099.stdout:(139/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.9 MB/s | 2.0 MB 00:00 2024-09-10T11:28:51.025 INFO:teuthology.orchestra.run.smithi196.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-10T11:28:51.032 INFO:teuthology.orchestra.run.smithi099.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-09-10T11:28:51.050 INFO:teuthology.orchestra.run.smithi196.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-10T11:28:51.057 INFO:teuthology.orchestra.run.smithi099.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T11:28:51.083 INFO:teuthology.orchestra.run.smithi099.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-10T11:28:51.108 INFO:teuthology.orchestra.run.smithi099.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T11:28:51.109 INFO:teuthology.orchestra.run.smithi196.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-10T11:28:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-10T11:28:51.133 INFO:teuthology.orchestra.run.smithi099.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T11:28:51.159 INFO:teuthology.orchestra.run.smithi099.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-10T11:28:51.159 INFO:teuthology.orchestra.run.smithi196.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-09-10T11:28:51.184 INFO:teuthology.orchestra.run.smithi099.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T11:28:51.185 INFO:teuthology.orchestra.run.smithi196.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-10T11:28:51.202 INFO:teuthology.orchestra.run.smithi196.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.3 MB/s | 58 kB 00:00 2024-09-10T11:28:51.209 INFO:teuthology.orchestra.run.smithi099.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T11:28:51.230 INFO:teuthology.orchestra.run.smithi196.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 592 kB/s | 16 kB 00:00 2024-09-10T11:28:51.252 INFO:teuthology.orchestra.run.smithi196.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 4.0 MB/s | 90 kB 00:00 2024-09-10T11:28:51.259 INFO:teuthology.orchestra.run.smithi099.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T11:28:51.278 INFO:teuthology.orchestra.run.smithi196.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-10T11:28:51.303 INFO:teuthology.orchestra.run.smithi196.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T11:28:51.328 INFO:teuthology.orchestra.run.smithi196.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-10T11:28:51.354 INFO:teuthology.orchestra.run.smithi196.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T11:28:51.379 INFO:teuthology.orchestra.run.smithi196.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T11:28:51.404 INFO:teuthology.orchestra.run.smithi196.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T11:28:51.426 INFO:teuthology.orchestra.run.smithi099.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-10T11:28:51.429 INFO:teuthology.orchestra.run.smithi196.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T11:28:51.455 INFO:teuthology.orchestra.run.smithi196.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T11:28:51.480 INFO:teuthology.orchestra.run.smithi196.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T11:28:51.505 INFO:teuthology.orchestra.run.smithi196.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 806 kB/s | 20 kB 00:00 2024-09-10T11:28:51.530 INFO:teuthology.orchestra.run.smithi196.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-10T11:28:51.564 INFO:teuthology.orchestra.run.smithi196.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-10T11:28:51.601 INFO:teuthology.orchestra.run.smithi099.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 8.1 MB/s | 3.2 MB 00:00 2024-09-10T11:28:51.623 INFO:teuthology.orchestra.run.smithi196.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T11:28:51.677 INFO:teuthology.orchestra.run.smithi099.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 546 kB/s | 41 kB 00:00 2024-09-10T11:28:51.768 INFO:teuthology.orchestra.run.smithi099.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.7 MB/s | 249 kB 00:00 2024-09-10T11:28:51.827 INFO:teuthology.orchestra.run.smithi099.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-09-10T11:28:51.865 INFO:teuthology.orchestra.run.smithi196.stdout:(148/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 13 MB/s | 3.2 MB 00:00 2024-09-10T11:28:51.936 INFO:teuthology.orchestra.run.smithi099.stdout:(154/156): NetworkManager-1.49.90-1.el9.x86_64. 3.5 MB/s | 2.3 MB 00:00 2024-09-10T11:28:52.036 INFO:teuthology.orchestra.run.smithi099.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 1.4 MB 00:00 2024-09-10T11:28:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 5.9 MB/s | 3.4 MB 00:00 2024-09-10T11:28:52.357 INFO:teuthology.orchestra.run.smithi196.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.7 MB/s | 2.3 MB 00:00 2024-09-10T11:28:52.407 INFO:teuthology.orchestra.run.smithi196.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 818 kB/s | 41 kB 00:00 2024-09-10T11:28:52.516 INFO:teuthology.orchestra.run.smithi196.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.2 MB/s | 249 kB 00:00 2024-09-10T11:28:52.536 INFO:teuthology.orchestra.run.smithi099.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 2.4 MB 00:00 2024-09-10T11:28:52.542 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:28:52.542 INFO:teuthology.orchestra.run.smithi099.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-10T11:28:52.591 INFO:teuthology.orchestra.run.smithi196.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 4.1 MB/s | 1.8 MB 00:00 2024-09-10T11:28:52.700 INFO:teuthology.orchestra.run.smithi196.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 1.4 MB 00:00 2024-09-10T11:28:52.933 INFO:teuthology.orchestra.run.smithi196.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-09-10T11:28:53.742 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-10T11:28:53.893 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-10T11:28:53.893 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-10T11:28:54.917 INFO:teuthology.orchestra.run.smithi196.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.2 MB/s | 16 MB 00:05 2024-09-10T11:28:54.922 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:28:54.922 INFO:teuthology.orchestra.run.smithi196.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-10T11:28:56.127 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-10T11:28:56.128 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-10T11:28:56.169 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-10T11:28:56.320 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-10T11:28:56.320 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-10T11:28:58.978 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-10T11:28:58.979 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-10T11:29:02.434 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-10T11:29:02.564 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T11:29:02.611 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T11:29:02.664 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T11:29:02.706 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T11:29:02.732 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T11:29:02.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T11:29:03.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T11:29:03.333 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:03.564 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:03.592 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:03.684 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:03.729 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T11:29:03.912 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T11:29:03.976 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T11:29:04.004 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:04.109 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:04.138 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T11:29:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T11:29:04.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T11:29:04.252 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T11:29:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T11:29:04.624 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T11:29:04.787 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T11:29:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:05.227 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T11:29:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T11:29:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T11:29:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T11:29:05.529 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-10T11:29:05.567 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T11:29:05.612 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T11:29:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T11:29:05.662 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T11:29:05.705 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T11:29:05.710 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T11:29:05.738 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T11:29:05.765 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T11:29:05.770 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T11:29:05.804 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T11:29:05.830 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T11:29:05.835 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T11:29:05.885 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T11:29:05.887 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T11:29:05.919 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T11:29:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T11:29:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T11:29:06.080 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T11:29:06.123 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T11:29:06.152 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T11:29:06.219 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T11:29:06.222 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T11:29:06.289 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T11:29:06.322 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:06.367 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T11:29:06.435 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:06.466 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:06.498 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T11:29:06.540 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:06.552 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T11:29:06.574 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T11:29:06.745 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T11:29:06.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T11:29:06.827 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:06.932 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:06.961 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T11:29:07.025 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T11:29:07.054 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T11:29:07.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T11:29:07.077 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T11:29:07.157 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T11:29:07.186 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T11:29:07.229 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T11:29:07.265 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T11:29:07.266 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T11:29:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T11:29:07.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T11:29:07.473 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T11:29:07.528 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T11:29:07.580 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T11:29:07.583 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T11:29:07.663 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T11:29:07.690 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:07.695 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T11:29:07.731 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T11:29:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:07.778 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T11:29:07.814 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T11:29:07.857 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T11:29:07.899 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T11:29:07.977 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T11:29:08.035 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T11:29:08.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T11:29:08.095 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T11:29:08.117 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T11:29:08.189 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T11:29:08.220 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T11:29:08.220 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T11:29:08.252 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T11:29:08.254 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T11:29:08.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T11:29:08.317 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T11:29:08.317 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T11:29:08.341 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T11:29:08.363 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T11:29:08.396 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T11:29:08.443 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T11:29:08.447 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T11:29:08.472 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T11:29:08.500 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T11:29:08.568 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T11:29:08.578 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T11:29:08.616 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T11:29:08.621 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T11:29:08.643 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T11:29:08.653 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T11:29:08.681 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T11:29:08.688 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T11:29:08.716 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T11:29:08.748 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T11:29:08.763 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T11:29:08.778 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T11:29:08.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T11:29:08.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T11:29:08.842 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T11:29:08.871 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T11:29:08.957 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T11:29:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T11:29:09.042 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T11:29:09.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T11:29:09.079 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T11:29:09.113 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T11:29:09.202 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T11:29:09.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T11:29:09.252 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T11:29:09.262 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T11:29:09.292 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T11:29:09.395 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T11:29:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T11:29:09.778 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T11:29:09.857 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T11:29:09.895 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T11:29:09.938 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T11:29:09.976 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T11:29:10.027 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T11:29:10.179 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T11:29:10.200 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T11:29:10.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T11:29:10.254 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T11:29:10.265 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T11:29:10.308 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T11:29:10.364 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T11:29:10.381 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T11:29:10.413 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T11:29:10.432 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T11:29:10.449 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T11:29:10.476 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T11:29:10.496 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T11:29:10.532 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T11:29:10.565 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T11:29:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T11:29:10.596 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T11:29:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T11:29:10.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T11:29:10.779 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T11:29:10.794 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T11:29:10.810 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T11:29:10.835 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T11:29:10.888 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T11:29:10.936 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T11:29:10.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T11:29:11.017 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T11:29:11.042 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T11:29:11.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T11:29:11.184 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T11:29:11.476 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T11:29:11.541 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T11:29:11.543 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T11:29:11.569 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T11:29:11.613 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T11:29:11.651 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T11:29:11.659 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T11:29:11.682 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T11:29:11.712 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T11:29:11.755 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T11:29:11.789 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T11:29:11.830 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T11:29:11.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T11:29:12.020 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T11:29:12.051 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T11:29:12.155 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T11:29:12.202 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T11:29:12.904 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T11:29:12.924 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T11:29:12.975 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T11:29:12.987 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T11:29:13.008 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T11:29:13.015 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T11:29:13.040 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T11:29:13.110 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T11:29:13.172 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T11:29:13.228 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T11:29:13.315 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T11:29:13.348 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T11:29:13.353 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T11:29:13.403 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T11:29:13.494 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T11:29:13.524 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T11:29:13.530 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T11:29:13.641 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T11:29:14.045 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T11:29:14.081 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T11:29:14.152 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T11:29:14.174 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T11:29:14.293 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T11:29:14.409 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T11:29:15.648 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T11:29:15.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T11:29:15.755 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T11:29:15.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T11:29:16.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T11:29:16.150 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T11:29:16.251 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T11:29:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T11:29:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T11:29:16.508 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:16.536 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:16.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T11:29:16.822 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T11:29:16.885 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T11:29:16.915 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T11:29:17.085 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:17.132 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T11:29:17.222 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T11:29:17.222 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T11:29:17.223 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T11:29:17.223 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:17.266 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T11:29:19.278 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:19.312 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:19.858 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T11:29:19.898 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T11:29:19.992 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T11:29:19.992 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T11:29:19.992 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T11:29:19.993 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:20.033 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T11:29:28.972 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-10T11:29:28.973 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:29.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T11:29:29.160 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T11:29:29.160 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:29.160 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T11:29:29.160 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:30.630 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T11:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T11:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T11:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:29:30.657 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:29:30.658 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:30.890 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T11:29:30.912 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T11:29:30.912 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:30.913 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T11:29:30.913 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:29:30.913 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:29:30.913 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:31.357 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:29:31.380 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:31.478 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T11:29:31.535 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T11:29:31.591 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T11:29:31.776 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-10T11:29:31.796 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-10T11:29:31.797 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:31.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T11:29:31.909 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T11:29:31.970 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T11:29:31.987 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T11:29:31.987 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:31.987 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T11:29:31.987 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:31.998 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T11:29:32.138 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T11:29:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T11:29:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T11:29:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T11:29:32.737 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T11:29:32.765 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T11:29:32.797 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T11:29:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T11:29:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T11:29:33.482 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:29:33.509 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:33.732 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:29:33.756 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:33.933 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T11:29:33.965 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T11:29:34.200 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:29:34.224 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:34.330 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T11:29:34.387 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T11:29:34.450 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T11:29:34.636 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T11:29:34.685 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T11:29:34.761 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T11:29:34.858 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T11:29:34.924 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T11:29:34.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T11:29:34.997 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T11:29:35.043 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T11:29:35.051 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T11:29:35.074 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T11:29:35.111 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T11:29:35.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T11:29:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:29:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T11:29:35.321 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T11:29:35.410 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T11:29:35.469 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T11:29:35.556 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T11:29:35.584 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T11:29:35.615 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T11:29:35.677 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T11:29:35.707 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T11:29:36.401 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T11:29:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T11:29:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T11:29:36.451 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:36.452 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T11:29:36.452 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:29:36.452 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:29:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T11:29:36.518 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T11:29:36.518 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:36.518 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T11:29:36.519 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:36.563 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T11:29:36.585 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T11:29:36.586 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:36.586 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T11:29:36.586 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T11:29:36.586 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T11:29:36.586 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:36.744 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T11:29:36.775 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T11:29:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:29:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:37.739 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T11:29:37.776 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T11:29:37.878 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T11:29:38.032 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T11:29:38.053 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T11:29:38.077 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T11:29:38.077 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:38.077 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T11:29:38.077 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:29:38.078 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:29:38.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:38.129 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T11:29:38.141 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T11:29:38.230 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T11:29:38.288 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T11:29:39.218 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T11:29:39.248 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T11:29:39.272 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T11:29:39.272 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:39.272 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T11:29:39.272 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:29:39.273 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:29:39.273 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:39.315 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T11:29:39.338 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T11:29:39.338 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:39.338 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T11:29:39.338 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:39.383 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T11:29:39.406 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T11:29:39.406 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:39.406 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T11:29:39.406 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T11:29:39.407 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T11:29:39.407 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:39.718 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T11:29:39.741 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T11:29:39.741 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:29:39.741 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T11:29:39.741 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:29:39.741 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:29:39.742 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:41.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T11:29:41.558 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T11:29:41.608 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T11:29:41.639 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T11:29:41.667 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T11:29:41.853 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T11:29:41.897 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T11:29:41.927 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T11:29:41.956 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T11:29:42.003 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T11:29:42.600 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:42.633 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:42.653 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T11:29:43.563 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T11:29:43.612 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T11:29:43.703 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T11:29:43.733 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T11:29:43.758 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T11:29:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T11:29:43.792 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T11:29:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T11:29:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T11:29:43.935 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T11:29:43.961 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T11:29:43.965 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:43.966 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:44.156 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:44.157 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T11:29:44.238 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T11:29:44.304 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:44.337 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T11:29:44.378 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T11:29:44.419 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T11:29:44.452 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T11:29:44.479 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T11:29:44.660 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T11:29:44.701 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T11:29:44.731 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T11:29:44.760 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T11:29:44.808 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T11:29:45.437 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:45.488 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:45.508 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T11:29:46.345 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T11:29:46.451 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T11:29:46.498 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T11:29:46.589 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T11:29:46.621 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T11:29:46.646 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T11:29:46.681 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T11:29:46.681 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T11:29:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T11:29:46.773 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T11:29:46.866 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T11:29:46.891 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T11:29:46.896 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:46.896 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:47.076 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T11:29:47.076 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T11:29:47.160 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T11:29:47.225 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T11:29:50.084 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T11:29:50.085 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:50.085 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T11:29:50.088 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T11:29:50.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T11:29:50.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T11:29:50.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T11:29:50.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T11:29:50.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T11:29:50.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T11:29:50.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T11:29:50.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T11:29:50.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T11:29:50.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T11:29:50.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T11:29:50.100 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T11:29:50.101 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T11:29:50.102 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T11:29:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T11:29:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T11:29:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T11:29:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T11:29:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T11:29:52.335 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:52.336 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-10T11:29:52.752 DEBUG:teuthology.parallel:result is None 2024-09-10T11:29:52.824 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:52.824 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T11:29:52.824 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T11:29:52.824 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T11:29:52.825 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T11:29:52.826 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T11:29:52.838 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T11:29:52.838 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T11:29:52.838 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T11:29:52.839 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T11:29:52.840 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T11:29:52.841 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T11:29:52.842 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T11:29:52.843 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T11:29:52.844 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T11:29:52.845 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T11:29:52.846 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T11:29:52.847 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T11:29:52.848 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T11:29:52.849 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:55.462 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.463 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.464 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T11:29:55.465 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T11:29:55.466 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T11:29:55.467 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T11:29:55.468 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T11:29:55.469 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T11:29:55.470 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T11:29:55.471 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T11:29:55.472 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T11:29:55.473 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T11:29:55.474 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:55.475 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-10T11:29:55.862 DEBUG:teuthology.parallel:result is None 2024-09-10T11:29:55.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:29:56.015 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T11:29:56.041 INFO:teuthology.orchestra.run.smithi099.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T11:29:56.042 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T11:29:56.042 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T11:29:56.044 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:29:56.195 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T11:29:56.217 INFO:teuthology.orchestra.run.smithi196.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T11:29:56.217 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T11:29:56.218 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T11:29:56.219 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T11:29:56.220 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:56.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T11:29:56.253 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:56.253 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T11:29:56.284 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T11:29:56.285 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:56.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T11:29:56.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T11:29:56.392 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:56.392 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T11:29:56.417 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T11:29:56.482 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T11:29:56.482 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:56.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T11:29:56.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T11:29:56.576 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:56.576 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T11:29:56.601 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T11:29:56.666 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T11:29:56.666 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:56.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T11:29:56.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T11:29:56.762 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:56.762 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T11:29:56.787 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T11:29:56.851 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T11:29:56.929 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T11:29:56.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-10T11:29:56.931 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-09-10T11:29:56.956 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T11:29:56.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T11:29:56.999 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T11:29:57.026 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T11:29:57.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T11:29:57.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T11:29:57.099 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T11:29:57.099 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T11:29:57.099 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T11:29:57.099 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:57.099 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T11:29:57.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T11:29:57.259 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T11:29:57.284 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:57.284 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T11:29:57.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T11:29:57.407 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T11:29:57.441 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T11:29:57.441 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T11:29:57.441 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086', 'branch': 'wip-rishabh-testing-20240909.165402-squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-full}', 'email': None, 'first_in_suite': False, 'job_id': '7899086', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9893, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'sleep_before_teardown': 0, 'subset': '197/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240909.165402-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLlFjNnscouiMRY0pAe90+tzbIv1vkfdlkF+pFCU20uQTeQ6aHxlKLQp2zUwCk30OECOszJ4FL7JVMXUbEdC82M=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH9vUsvZ0x57VBxzkLyzVPazLOq1BugG+PWld+sAMJ4aY4LQedzFueRyB0mLJ9f8HF6q8Jej6rrCVd+8r7pCHv8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_06:58:10', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T11:29:57.442 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T11:29:57.446 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T11:29:57.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T11:29:57.503 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T11:29:57.556 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:57.556 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T11:29:57.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T11:29:57.580 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-10T11:29:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T11:29:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 11:29:44.667923782 +0000 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 11:27:59.131510388 +0000 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 11:27:59.131510388 +0000 2024-09-10T11:29:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 11:27:59.131510388 +0000 2024-09-10T11:29:57.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T11:29:57.703 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T11:29:57.703 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T11:29:57.704 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000329523 s, 1.6 MB/s 2024-09-10T11:29:57.705 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T11:29:57.762 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-10T11:29:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 11:29:44.668923759 +0000 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 11:27:59.464502670 +0000 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 11:27:59.464502670 +0000 2024-09-10T11:29:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 11:27:59.464502670 +0000 2024-09-10T11:29:57.818 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T11:29:57.881 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T11:29:57.881 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T11:29:57.881 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.00028947 s, 1.8 MB/s 2024-09-10T11:29:57.882 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T11:29:57.938 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-10T11:29:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T11:29:57.994 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:57.994 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T11:29:57.994 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:57.994 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:57.995 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 11:29:44.668923759 +0000 2024-09-10T11:29:57.995 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 11:27:59.782495300 +0000 2024-09-10T11:29:57.995 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 11:27:59.782495300 +0000 2024-09-10T11:29:57.995 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 11:27:59.782495300 +0000 2024-09-10T11:29:57.995 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T11:29:58.057 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T11:29:58.057 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T11:29:58.057 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000209972 s, 2.4 MB/s 2024-09-10T11:29:58.058 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T11:29:58.115 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-10 11:29:44.668923759 +0000 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-10 11:28:00.117487535 +0000 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-10 11:28:00.117487535 +0000 2024-09-10T11:29:58.170 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-10 11:28:00.117487535 +0000 2024-09-10T11:29:58.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T11:29:58.234 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-10T11:29:58.234 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-10T11:29:58.234 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000234173 s, 2.2 MB/s 2024-09-10T11:29:58.236 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T11:29:58.292 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T11:29:58.292 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:58.292 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T11:29:58.310 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T11:29:58.310 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-10 11:29:47.580044124 +0000 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-10 11:27:59.131129997 +0000 2024-09-10T11:29:58.367 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-10 11:27:59.131129997 +0000 2024-09-10T11:29:58.368 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-10 11:27:59.131129997 +0000 2024-09-10T11:29:58.368 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T11:29:58.433 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-10T11:29:58.433 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-10T11:29:58.433 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000292528 s, 1.8 MB/s 2024-09-10T11:29:58.434 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T11:29:58.492 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-10 11:29:47.581044102 +0000 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-10 11:27:59.446123112 +0000 2024-09-10T11:29:58.549 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-10 11:27:59.446123112 +0000 2024-09-10T11:29:58.550 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-10 11:27:59.446123112 +0000 2024-09-10T11:29:58.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T11:29:58.614 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-10T11:29:58.614 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-10T11:29:58.614 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000230163 s, 2.2 MB/s 2024-09-10T11:29:58.615 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T11:29:58.672 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-10T11:29:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-10 11:29:47.581044102 +0000 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-10 11:27:59.759116270 +0000 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-10 11:27:59.759116270 +0000 2024-09-10T11:29:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-10 11:27:59.759116270 +0000 2024-09-10T11:29:58.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T11:29:58.792 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-10T11:29:58.792 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-10T11:29:58.792 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000227468 s, 2.3 MB/s 2024-09-10T11:29:58.794 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T11:29:58.850 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-10T11:29:58.906 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T11:29:58.906 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:29:58.906 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T11:29:58.906 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:29:58.906 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:29:58.907 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-10 11:29:47.581044102 +0000 2024-09-10T11:29:58.907 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-10 11:28:00.087109100 +0000 2024-09-10T11:29:58.907 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-10 11:28:00.087109100 +0000 2024-09-10T11:29:58.907 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-10 11:28:00.087109100 +0000 2024-09-10T11:29:58.907 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T11:29:58.969 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-10T11:29:58.970 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-10T11:29:58.970 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000218116 s, 2.3 MB/s 2024-09-10T11:29:58.971 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T11:29:59.026 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T11:29:59.027 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T11:29:59.027 INFO:tasks.ceph:Generating config... 2024-09-10T11:29:59.028 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-10T11:29:59.028 INFO:tasks.ceph:[client] debug client = 20 2024-09-10T11:29:59.028 INFO:tasks.ceph:[client] debug ms = 1 2024-09-10T11:29:59.028 INFO:tasks.ceph:[client] pid file = 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.0] debug client = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.1] debug client = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-10T11:29:59.029 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-10T11:29:59.029 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-10T11:29:59.029 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-09-10T11:29:59.030 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-09-10T11:29:59.031 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-09-10T11:29:59.031 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-09-10T11:29:59.031 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-09-10T11:29:59.032 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-10T11:29:59.032 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T11:29:59.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T11:29:59.098 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T11:29:59.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T11:29:59.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T11:29:59.170 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.99'), ('mon.b', '172.21.15.196'), ('mon.c', '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]')] 2024-09-10T11:29:59.171 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.99,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-10T11:29:59.172 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:59.172 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T11:29:59.226 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.99 --add b 172.21.15.196 --addv c '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: generated fsid fb5864c6-feee-49d0-8ee4-0b85b9c6d632 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = quincy 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 0 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:fsid fb5864c6-feee-49d0-8ee4-0b85b9c6d632 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:last_changed 2024-09-10T11:29:59.312709+0000 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-09-10T11:29:59.312709+0000 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:min_mon_release 17 (quincy) 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:election_strategy: 1 2024-09-10T11:29:59.313 INFO:teuthology.orchestra.run.smithi099.stdout:0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-09-10T11:29:59.314 INFO:teuthology.orchestra.run.smithi099.stdout:1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-09-10T11:29:59.314 INFO:teuthology.orchestra.run.smithi099.stdout:2: [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] mon.c 2024-09-10T11:29:59.314 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T11:29:59.315 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T11:29:59.330 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fb5864c6-feee-49d0-8ee4-0b85b9c6d632... 2024-09-10T11:29:59.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T11:29:59.373 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T11:29:59.413 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-09-10T11:29:59.414 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-09-10T11:29:59.415 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.416 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.99,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = fb5864c6-feee-49d0-8ee4-0b85b9c6d632 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.417 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug trim objects = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-09-10T11:29:59.418 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bdev async discard = True 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bdev enable discard = True 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore allocator = bitmap 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2024-09-10T11:29:59.419 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: memstore device bytes = 200000000 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 1.0 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: osd mon report interval = 5 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = memstore 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-09-10T11:29:59.420 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-10T11:29:59.421 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T11:29:59.422 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon down mkfs grace = 300 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull ratio = 0.6 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.7 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.6 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on pool no app = False 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-09-10T11:29:59.423 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = "" 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-09-10T11:29:59.424 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug objectcacher = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug objecter = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug objectcacher = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug objecter = 20 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2024-09-10T11:29:59.425 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds balancer = 20 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2024-09-10T11:29:59.426 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-09-10T11:29:59.428 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-09-10T11:29:59.429 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T11:29:59.430 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.99,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = fb5864c6-feee-49d0-8ee4-0b85b9c6d632 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.431 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-09-10T11:29:59.432 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: bdev async discard = True 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: bdev enable discard = True 2024-09-10T11:29:59.433 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore allocator = bitmap 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 1/20 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 1/20 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 4/10 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: memstore device bytes = 200000000 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 1.0 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: osd mon report interval = 5 2024-09-10T11:29:59.434 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = memstore 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-09-10T11:29:59.435 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.436 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon down mkfs grace = 300 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull ratio = 0.6 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.7 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.6 2024-09-10T11:29:59.437 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on pool no app = False 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-09-10T11:29:59.438 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug objectcacher = 20 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug objecter = 20 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug objectcacher = 20 2024-09-10T11:29:59.439 INFO:teuthology.orchestra.run.smithi196.stdout: debug objecter = 20 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-09-10T11:29:59.440 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-09-10T11:29:59.440 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T11:29:59.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T11:29:59.522 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T11:29:59.522 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:59.522 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T11:29:59.537 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:59.537 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T11:29:59.592 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:29:59.593 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:59.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T11:29:59.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T11:29:59.667 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:29:59.667 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:29:59.723 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:29:59.723 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:59.723 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T11:29:59.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T11:29:59.758 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:29:59.758 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:29:59.814 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T11:29:59.814 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T11:29:59.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T11:29:59.865 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T11:29:59.867 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T11:29:59.929 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T11:29:59.932 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T11:29:59.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T11:29:59.983 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T11:29:59.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T11:30:00.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T11:30:00.104 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T11:30:00.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T11:30:00.130 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T11:30:00.316 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T11:30:00.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T11:30:00.343 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T11:30:00.438 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T11:30:00.441 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T11:30:00.466 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T11:30:00.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T11:30:00.509 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T11:30:00.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T11:30:00.605 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T11:30:00.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T11:30:00.658 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T11:30:00.671 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T11:30:00.753 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T11:30:00.766 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T11:30:00.766 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T11:30:00.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T11:30:00.791 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:00.791 INFO:tasks.ceph:role: osd.0 2024-09-10T11:30:00.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:00.792 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:00.864 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:00.865 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:00.865 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:00.865 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:00.865 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:01.360 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-10T11:30:01.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-10T11:30:01.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T11:30:01.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T11:30:01.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T11:30:01.585 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:01.585 INFO:tasks.ceph:role: osd.1 2024-09-10T11:30:01.586 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:01.586 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T11:30:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:01.658 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:01.658 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:01.659 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:01.659 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:02.120 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-10T11:30:02.122 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-10T11:30:02.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T11:30:02.223 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T11:30:02.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T11:30:02.316 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:02.316 INFO:tasks.ceph:role: osd.2 2024-09-10T11:30:02.317 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:02.317 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:02.390 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:02.390 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:02.390 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:02.871 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-10T11:30:02.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-10T11:30:02.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T11:30:03.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T11:30:03.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T11:30:03.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:03.102 INFO:tasks.ceph:role: osd.3 2024-09-10T11:30:03.102 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:03.103 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T11:30:03.174 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:03.175 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:03.690 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-10T11:30:03.692 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-10T11:30:03.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T11:30:03.821 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T11:30:03.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:03.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:03.942+0000 7f43551fd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T11:30:03.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:03.942+0000 7f43551fd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T11:30:03.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:03.943+0000 7f43551fd640 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-09-10T11:30:03.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T11:30:03.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:04.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.058+0000 7f8dae687640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T11:30:04.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.058+0000 7f8dae687640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T11:30:04.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.059+0000 7f8dae687640 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-09-10T11:30:04.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T11:30:04.091 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.173+0000 7fc9e6b56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T11:30:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.173+0000 7fc9e6b56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T11:30:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.174+0000 7fc9e6b56640 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-09-10T11:30:04.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T11:30:04.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.289+0000 7f2a34e0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T11:30:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.289+0000 7f2a34e0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T11:30:04.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:04.290+0000 7f2a34e0e640 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-09-10T11:30:04.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T11:30:04.322 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T11:30:04.351 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:04.352 INFO:tasks.ceph:role: osd.4 2024-09-10T11:30:04.352 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:04.352 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:04.428 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:04.429 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:04.941 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-09-10T11:30:04.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-09-10T11:30:04.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T11:30:05.078 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T11:30:05.111 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T11:30:05.178 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:05.178 INFO:tasks.ceph:role: osd.5 2024-09-10T11:30:05.178 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:05.178 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T11:30:05.261 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:05.261 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:05.261 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:05.261 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:05.261 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:05.262 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:05.262 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:05.262 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:05.262 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:05.262 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:05.703 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-09-10T11:30:05.705 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-09-10T11:30:05.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T11:30:05.798 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T11:30:05.827 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T11:30:05.892 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:05.892 INFO:tasks.ceph:role: osd.6 2024-09-10T11:30:05.892 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:05.892 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T11:30:05.965 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:05.966 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:06.454 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-09-10T11:30:06.456 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-09-10T11:30:06.456 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T11:30:06.566 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T11:30:06.596 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T11:30:06.661 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T11:30:06.662 INFO:tasks.ceph:role: osd.7 2024-09-10T11:30:06.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:06.662 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T11:30:06.734 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T11:30:06.734 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T11:30:06.734 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T11:30:06.735 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T11:30:07.284 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-09-10T11:30:07.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-09-10T11:30:07.286 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T11:30:07.433 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T11:30:07.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:07.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.565+0000 7fa7e6c3b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T11:30:07.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.565+0000 7fa7e6c3b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T11:30:07.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.566+0000 7fa7e6c3b640 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-09-10T11:30:07.576 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T11:30:07.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:07.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.683+0000 7fdb19216640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T11:30:07.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.683+0000 7fdb19216640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T11:30:07.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.684+0000 7fdb19216640 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-09-10T11:30:07.692 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T11:30:07.716 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:07.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.799+0000 7f631360c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T11:30:07.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.799+0000 7f631360c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T11:30:07.801 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.800+0000 7f631360c640 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-09-10T11:30:07.808 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T11:30:07.833 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:07.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.915+0000 7fc0b0334640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T11:30:07.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.915+0000 7fc0b0334640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T11:30:07.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:07.916+0000 7fc0b0334640 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-09-10T11:30:07.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T11:30:07.948 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T11:30:07.948 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:07.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T11:30:07.978 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:07.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T11:30:08.047 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T11:30:08.115 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T11:30:08.182 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T11:30:08.248 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T11:30:08.313 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T11:30:08.377 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.378 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T11:30:08.403 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.404 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T11:30:08.469 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.469 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T11:30:08.535 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T11:30:08.599 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T11:30:08.664 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T11:30:08.729 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T11:30:08.794 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.794 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T11:30:08.810 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:08.810 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T11:30:08.866 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.867 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T11:30:08.882 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:08.882 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T11:30:08.938 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T11:30:08.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T11:30:08.940 DEBUG:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg== 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA== 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A== 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA== 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg== 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-09-10T11:30:08.981 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg== 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-09-10T11:30:08.982 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw== 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw== 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:[client.2] 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw== 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA== 2024-09-10T11:30:08.983 INFO:teuthology.orchestra.run.smithi099.stdout:[client.3] 2024-09-10T11:30:08.984 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA== 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg== 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA== 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A== 2024-09-10T11:30:09.000 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ== 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-09-10T11:30:09.001 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw== 2024-09-10T11:30:09.002 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-09-10T11:30:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA== 2024-09-10T11:30:09.003 INFO:teuthology.orchestra.run.smithi196.stdout:[client.3] 2024-09-10T11:30:09.003 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA== 2024-09-10T11:30:09.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T11:30:09.027 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T11:30:09.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.097 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.208 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.260 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.314 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.364 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.366 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T11:30:09.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T11:30:09.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.473 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.528 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T11:30:09.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.636 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.742 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T11:30:09.793 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.795 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.849 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.851 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.907 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.961 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:09.964 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T11:30:10.019 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T11:30:10.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T11:30:10.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T11:30:10.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T11:30:10.163 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T11:30:10.188 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T11:30:10.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T11:30:10.310 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T11:30:10.375 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T11:30:10.470 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T11:30:10.494 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:10.497 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T11:30:10.550 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T11:30:10.550 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T11:30:10.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T11:30:10.553 INFO:tasks.ceph.mon.a:Started 2024-09-10T11:30:10.553 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T11:30:10.553 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T11:30:10.592 INFO:tasks.ceph.mon.b:Started 2024-09-10T11:30:10.592 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T11:30:10.592 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T11:30:10.594 INFO:tasks.ceph.mon.c:Started 2024-09-10T11:30:10.594 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T11:30:10.595 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T11:30:10.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T11:30:10.597 INFO:tasks.ceph.mgr.y:Started 2024-09-10T11:30:10.597 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T11:30:10.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T11:30:10.599 INFO:tasks.ceph.mgr.x:Started 2024-09-10T11:30:10.599 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T11:30:10.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T11:30:10.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.698+0000 7faaf8615640 1 Processor -- start 2024-09-10T11:30:10.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 -- start start 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faaf00b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf015ca80 con 0x7faaf00752a0 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7faaf015bdb0 con 0x7faaf015e7e0 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.699+0000 7faaf8615640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faaf015ccf0 con 0x7faaf00b4350 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54016/0 (socket says 172.21.15.99:54016) 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 -- 172.21.15.99:0/3273634682 learned_addr learned my addr 172.21.15.99:0/3273634682 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf63b1640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faaf00b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 -- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 msgr2=0x7faaf0165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 -- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 msgr2=0x7faaf0165500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T11:30:10.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 --2- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T11:30:10.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf6bb2640 1 --2- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:30:10.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf63b1640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 msgr2=0x7faaf00b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T11:30:10.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf63b1640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 msgr2=0x7faaf00b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T11:30:10.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf63b1640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faaf00b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T11:30:10.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.700+0000 7faaf63b1640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faaf00b4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:30:10.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.863+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898953303 0 0) 0x7faaf015ca80 con 0x7faaf00752a0 2024-09-10T11:30:10.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.863+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faadc003520 con 0x7faaf00752a0 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2670350848 0 0) 0x7faadc003520 con 0x7faaf00752a0 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 msgr2=0x7faaf00b4730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faaf00b4730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 msgr2=0x7faaf0165500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 --2- 172.21.15.99:0/3273634682 >> v2:172.21.15.99:3300/0 conn(0x7faaf015e7e0 0x7faaf0165500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf00b5380 con 0x7faaf00752a0 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7faaec001670 con 0x7faaf00752a0 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 >> v1:172.21.15.196:6789/0 conn(0x7faaf00752a0 legacy=0x7faaf00b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc0039b0 0x7faadc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faadc004800 0x7faadc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faadc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faaf015bdb0 con 0x7faadc004800 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faaf015ccf0 con 0x7faadc0039b0 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faadc00bac0 con 0x7faaf00b4350 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faadc004800 0x7faadc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf6bb2640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faadc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf63b1640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc0039b0 0x7faadc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 msgr2=0x7faadc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf00b4350 0x7faadc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc0039b0 msgr2=0x7faadc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc0039b0 0x7faadc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.865+0000 7faaf5bb0640 1 -- 172.21.15.99:0/3273634682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faadc003520 con 0x7faadc004800 2024-09-10T11:30:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.866+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faadc004800 0x7faadc004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faaf00b39d0 tx=0x7faadc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae45c92a961fa5bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:10.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.866+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec036070 con 0x7faadc004800 2024-09-10T11:30:10.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.866+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec031620 con 0x7faadc004800 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.866+0000 7faaf53af640 1 -- 172.21.15.99:0/3273634682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec007b90 con 0x7faadc004800 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 -- 172.21.15.99:0/3273634682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faadc004800 msgr2=0x7faadc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 --2- 172.21.15.99:0/3273634682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faadc004800 0x7faadc004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faaf00b39d0 tx=0x7faadc005830 comp rx=0 tx=0).stop 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf6bb2640 1 -- 172.21.15.99:0/3273634682 reap_dead start 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 -- 172.21.15.99:0/3273634682 shutdown_connections 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 -- 172.21.15.99:0/3273634682 >> 172.21.15.99:0/3273634682 conn(0x7faaf0072310 msgr2=0x7faaf00b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 -- 172.21.15.99:0/3273634682 shutdown_connections 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.867+0000 7faaf8615640 1 -- 172.21.15.99:0/3273634682 wait complete. 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.868+0000 7faaf8615640 1 Processor -- start 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.868+0000 7faaf8615640 1 -- start start 2024-09-10T11:30:10.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc004800 0x7faaf0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf0156450 0x7faaf01568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faaf00b5770 con 0x7faaf006eff0 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faaf00b55f0 con 0x7faadc004800 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faaf00cd4d0 con 0x7faaf0156450 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54038/0 (socket says 172.21.15.99:54038) 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 -- 172.21.15.99:0/660061981 learned_addr learned my addr 172.21.15.99:0/660061981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf63b1640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc004800 0x7faaf0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf6bb2640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf0156450 0x7faaf01568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 -- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf0156450 msgr2=0x7faaf01568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf0156450 0x7faaf01568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 -- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc004800 msgr2=0x7faaf0147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:10.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc004800 0x7faaf0147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 -- 172.21.15.99:0/660061981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf0157060 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf5bb0640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faaec00c110 tx=0x7faaf015d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec041020 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaec037040 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.869+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf0157340 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.870+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec036070 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.870+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf0157900 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.870+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faaec044020 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.870+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faaec03d570 con 0x7faaf006eff0 2024-09-10T11:30:10.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.870+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf00b4730 con 0x7faaf006eff0 2024-09-10T11:30:10.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.873+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faaec04b030 con 0x7faaf006eff0 2024-09-10T11:30:10.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.936+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7faaf015ad40 con 0x7faaf006eff0 2024-09-10T11:30:10.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.937+0000 7faae6ffd640 1 -- 172.21.15.99:0/660061981 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7faaec003b10 con 0x7faaf006eff0 2024-09-10T11:30:10.938 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-09-10T11:30:10.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 msgr2=0x7faaf006f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:10.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faaec00c110 tx=0x7faaf015d2b0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 shutdown_connections 2024-09-10T11:30:10.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaf0156450 0x7faaf01568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaf006eff0 0x7faaf006f450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 --2- 172.21.15.99:0/660061981 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faadc004800 0x7faaf0147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 >> 172.21.15.99:0/660061981 conn(0x7faaf0072310 msgr2=0x7faaf00d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 shutdown_connections 2024-09-10T11:30:10.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:10.938+0000 7faaf8615640 1 -- 172.21.15.99:0/660061981 wait complete. 2024-09-10T11:30:10.951 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T11:30:10.951 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T11:30:10.951 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:10.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T11:30:10.967 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:10.966+0000 7f8f44b26040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T11:30:10.968 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:10.968+0000 7f6f6d45c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T11:30:10.978 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:10.978 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T11:30:11.007 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.006+0000 7f8f44b26040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T11:30:11.011 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.010+0000 7f6f6d45c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T11:30:11.039 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:11.039 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T11:30:11.108 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:11.108 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T11:30:11.153 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.151+0000 7f8f44b26040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T11:30:11.161 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.160+0000 7f6f6d45c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T11:30:11.177 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:11.177 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T11:30:11.203 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:11.203 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T11:30:11.204 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.202+0000 7f8f44b26040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T11:30:11.212 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.212+0000 7f6f6d45c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T11:30:11.228 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:11.228 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T11:30:11.295 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:11.295 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T11:30:11.361 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-09-10T11:30:11.368 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.368+0000 7f6f6d45c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T11:30:11.384 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.383+0000 7f8f44b26040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T11:30:11.478 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.477+0000 7f6f6d45c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T11:30:11.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.484+0000 7fd86832a640 1 Processor -- start 2024-09-10T11:30:11.486 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.485+0000 7fd86832a640 1 -- start start 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.485+0000 7fd86832a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 0x7fd8600733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.485+0000 7fd86832a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8600b5b90 0x7fd8600b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd86832a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd860145790 con 0x7fd86006f1b0 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd86832a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd8600b6540 con 0x7fd8601453b0 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd86832a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd8600ce9f0 con 0x7fd8600b5b90 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8600b5b90 0x7fd8600b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8600b5b90 0x7fd8600b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40000/0 (socket says 172.21.15.196:40000) 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 -- 172.21.15.196:0/4067520330 learned_addr learned my addr 172.21.15.196:0/4067520330 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8660c6640 1 --2- 172.21.15.196:0/4067520330 >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 0x7fd8600733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8650c4640 1 -- 172.21.15.196:0/4067520330 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 231121662 0 0) 0x7fd860145790 con 0x7fd86006f1b0 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 -- 172.21.15.196:0/4067520330 >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 msgr2=0x7fd8600733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 --2- 172.21.15.196:0/4067520330 >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 0x7fd8600733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 -- 172.21.15.196:0/4067520330 >> v1:172.21.15.99:6789/0 conn(0x7fd86006f1b0 legacy=0x7fd86006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8658c5640 1 -- 172.21.15.196:0/4067520330 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8600d15d0 con 0x7fd8600b5b90 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.486+0000 7fd8660c6640 1 --2- 172.21.15.196:0/4067520330 >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 0x7fd8600733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.487+0000 7fd8658c5640 1 --2- 172.21.15.196:0/4067520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8600b5b90 0x7fd8600b6000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8600b7620 tx=0x7fd85002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff9d88d980cb948 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.487+0000 7fd8650c4640 1 -- 172.21.15.196:0/4067520330 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850037070 con 0x7fd8600b5b90 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.487+0000 7fd8650c4640 1 -- 172.21.15.196:0/4067520330 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd850032440 con 0x7fd8600b5b90 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.487+0000 7fd8650c4640 1 -- 172.21.15.196:0/4067520330 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850032810 con 0x7fd8600b5b90 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 -- 172.21.15.196:0/4067520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8600b5b90 msgr2=0x7fd8600b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 --2- 172.21.15.196:0/4067520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8600b5b90 0x7fd8600b6000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd8600b7620 tx=0x7fd85002a270 comp rx=0 tx=0).stop 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 -- 172.21.15.196:0/4067520330 shutdown_connections 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 --2- 172.21.15.196:0/4067520330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8600b5b90 0x7fd8600b6000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 --2- 172.21.15.196:0/4067520330 >> v2:172.21.15.99:3300/0 conn(0x7fd8601453b0 0x7fd8600733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 -- 172.21.15.196:0/4067520330 >> 172.21.15.196:0/4067520330 conn(0x7fd860072680 msgr2=0x7fd860072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 -- 172.21.15.196:0/4067520330 shutdown_connections 2024-09-10T11:30:11.491 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.488+0000 7fd86832a640 1 -- 172.21.15.196:0/4067520330 wait complete. 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.489+0000 7fd86832a640 1 Processor -- start 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.489+0000 7fd86832a640 1 -- start start 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 0x7fd86013c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 0x7fd860141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8600ce430 con 0x7fd86006f1b0 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd860073910 con 0x7fd8601453b0 2024-09-10T11:30:11.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd86832a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8600ce710 con 0x7fd8600b5b90 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8658c5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 0x7fd860141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40012/0 (socket says 172.21.15.196:40012) 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 -- 172.21.15.196:0/2995337513 learned_addr learned my addr 172.21.15.196:0/2995337513 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8660c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 0x7fd86013c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 -- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 msgr2=0x7fd860141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 0x7fd860141e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 -- 172.21.15.196:0/2995337513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 msgr2=0x7fd86013c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 0x7fd86013c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 -- 172.21.15.196:0/2995337513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd860140680 con 0x7fd8601453b0 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8658c5640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 0x7fd860141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8660c6640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 0x7fd86013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.490+0000 7fd8668c7640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c760 tx=0x7fd85800cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858003da0 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd858005070 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd860140960 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858005460 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd860140f20 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd858005600 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.491+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd85801b950 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.492+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8600b6000 con 0x7fd8601453b0 2024-09-10T11:30:11.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.494+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd85800a640 con 0x7fd8601453b0 2024-09-10T11:30:11.501 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.500+0000 7f8f44b26040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T11:30:11.564 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.563+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) -- 0x7fd860143e30 con 0x7fd8601453b0 2024-09-10T11:30:11.567 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.567+0000 7fd856ffd640 1 -- 172.21.15.196:0/2995337513 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd85801bba0 con 0x7fd8601453b0 2024-09-10T11:30:11.568 INFO:teuthology.orchestra.run.smithi196.stdout:0 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 msgr2=0x7fd8600b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c760 tx=0x7fd85800cc20 comp rx=0 tx=0).stop 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 shutdown_connections 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8601453b0 0x7fd8600b15f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8600b5b90 0x7fd860141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 --2- 172.21.15.196:0/2995337513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd86006f1b0 0x7fd86013c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 >> 172.21.15.196:0/2995337513 conn(0x7fd860072680 msgr2=0x7fd8600b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.569 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 shutdown_connections 2024-09-10T11:30:11.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.568+0000 7fd86832a640 1 -- 172.21.15.196:0/2995337513 wait complete. 2024-09-10T11:30:11.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-09-10T11:30:11.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.676+0000 7fd90bc47640 1 Processor -- start 2024-09-10T11:30:11.678 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.676+0000 7fd90bc47640 1 -- start start 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90bc47640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd904145360 0x7fd9040d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90bc47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd9040b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90bc47640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd904145740 con 0x7fd9040b60d0 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90bc47640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd9040b6540 con 0x7fd904145360 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90bc47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd90406e5a0 con 0x7fd9040ce270 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd9091e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd9040b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90a1e4640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd9040b60d0 0x7fd9040cff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49292/0 (socket says 172.21.15.196:49292) 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd90a1e4640 1 -- 172.21.15.196:0/3849167722 learned_addr learned my addr 172.21.15.196:0/3849167722 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.677+0000 7fd9099e3640 1 --2- 172.21.15.196:0/3849167722 >> v2:172.21.15.99:3300/0 conn(0x7fd904145360 0x7fd9040d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.678+0000 7fd9089e1640 1 -- 172.21.15.196:0/3849167722 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781517306 0 0) 0x7fd904145740 con 0x7fd9040b60d0 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.678+0000 7fd9091e2640 1 -- 172.21.15.196:0/3849167722 >> v2:172.21.15.99:3300/0 conn(0x7fd904145360 msgr2=0x7fd9040d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.678+0000 7fd9091e2640 1 --2- 172.21.15.196:0/3849167722 >> v2:172.21.15.99:3300/0 conn(0x7fd904145360 0x7fd9040d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.678+0000 7fd9091e2640 1 -- 172.21.15.196:0/3849167722 >> v1:172.21.15.196:6789/0 conn(0x7fd9040b60d0 legacy=0x7fd9040cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.678+0000 7fd9091e2640 1 -- 172.21.15.196:0/3849167722 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90406e880 con 0x7fd9040ce270 2024-09-10T11:30:11.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd9091e2640 1 --2- 172.21.15.196:0/3849167722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd9040b5b90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40097a0 tx=0x7fd8f402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e61d83361eb04dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd9089e1640 1 -- 172.21.15.196:0/3849167722 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4037070 con 0x7fd9040ce270 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd9089e1640 1 -- 172.21.15.196:0/3849167722 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f4032440 con 0x7fd9040ce270 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd9089e1640 1 -- 172.21.15.196:0/3849167722 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4032810 con 0x7fd9040ce270 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 -- 172.21.15.196:0/3849167722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 msgr2=0x7fd9040b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 --2- 172.21.15.196:0/3849167722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd9040b5b90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40097a0 tx=0x7fd8f402a210 comp rx=0 tx=0).stop 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 -- 172.21.15.196:0/3849167722 shutdown_connections 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 --2- 172.21.15.196:0/3849167722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd9040b5b90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 --2- 172.21.15.196:0/3849167722 >> v2:172.21.15.99:3300/0 conn(0x7fd904145360 0x7fd9040d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 -- 172.21.15.196:0/3849167722 >> 172.21.15.196:0/3849167722 conn(0x7fd904072680 msgr2=0x7fd904072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 -- 172.21.15.196:0/3849167722 shutdown_connections 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.679+0000 7fd90bc47640 1 -- 172.21.15.196:0/3849167722 wait complete. 2024-09-10T11:30:11.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 Processor -- start 2024-09-10T11:30:11.683 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 -- start start 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9040b60d0 0x7fd90413c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd904145360 0x7fd9041394e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040ce6e0 con 0x7fd9040b60d0 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd90406ee40 con 0x7fd904145360 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd90bc47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9040ce860 con 0x7fd9040ce270 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd9091e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd9091e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47854/0 (socket says 172.21.15.196:47854) 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.681+0000 7fd9091e2640 1 -- 172.21.15.196:0/836292745 learned_addr learned my addr 172.21.15.196:0/836292745 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:11.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd90a1e4640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd904145360 0x7fd9041394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9099e3640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9040b60d0 0x7fd90413c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 -- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd904145360 msgr2=0x7fd9041394e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd904145360 0x7fd9041394e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 -- 172.21.15.196:0/836292745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9040b60d0 msgr2=0x7fd90413c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9040b60d0 0x7fd90413c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 -- 172.21.15.196:0/836292745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90413b290 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd9091e2640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40098c0 tx=0x7fd8f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4037040 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f4030500 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90413b570 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40308d0 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90413bb30 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd8f4044020 con 0x7fd9040ce270 2024-09-10T11:30:11.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.682+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd8f403bc40 con 0x7fd9040ce270 2024-09-10T11:30:11.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.683+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9040b5b90 con 0x7fd9040ce270 2024-09-10T11:30:11.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.685+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd8f403b430 con 0x7fd9040ce270 2024-09-10T11:30:11.755 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.754+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) -- 0x7fd904143fc0 con 0x7fd9040ce270 2024-09-10T11:30:11.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.760+0000 7fd8fa7fc640 1 -- 172.21.15.196:0/836292745 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd8f403b610 con 0x7fd9040ce270 2024-09-10T11:30:11.761 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:30:11.762 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 msgr2=0x7fd90413a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40098c0 tx=0x7fd8f40046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 shutdown_connections 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd904145360 0x7fd9041394e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9040ce270 0x7fd90413a960 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 --2- 172.21.15.196:0/836292745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9040b60d0 0x7fd90413c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 >> 172.21.15.196:0/836292745 conn(0x7fd904072680 msgr2=0x7fd9040d0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 shutdown_connections 2024-09-10T11:30:11.763 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.761+0000 7fd90bc47640 1 -- 172.21.15.196:0/836292745 wait complete. 2024-09-10T11:30:11.772 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-09-10T11:30:11.802 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.801+0000 7f6f6d45c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T11:30:11.853 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.853+0000 7f8f44b26040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T11:30:11.857 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:11.857+0000 7f6f6d45c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T11:30:11.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.872+0000 7f5806ad7640 1 Processor -- start 2024-09-10T11:30:11.874 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.872+0000 7f5806ad7640 1 -- start start 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5806ad7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f580015ee60 0x7f58000b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5806ad7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58000b3850 0x7f58000731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5806ad7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58000b3cc0 con 0x7f5800073700 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5806ad7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f58000b46c0 con 0x7f58000b3850 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5806ad7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5800073b50 con 0x7f580015ee60 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5804873640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f580015ee60 0x7f58000b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5804873640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f580015ee60 0x7f58000b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40028/0 (socket says 172.21.15.196:40028) 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f5804873640 1 -- 172.21.15.196:0/2560820246 learned_addr learned my addr 172.21.15.196:0/2560820246 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.873+0000 7f57fffff640 1 --2- 172.21.15.196:0/2560820246 >> v2:172.21.15.99:3300/0 conn(0x7f58000b3850 0x7f58000731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f5804873640 1 -- 172.21.15.196:0/2560820246 >> v2:172.21.15.99:3300/0 conn(0x7f58000b3850 msgr2=0x7f58000731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f5804873640 1 --2- 172.21.15.196:0/2560820246 >> v2:172.21.15.99:3300/0 conn(0x7f58000b3850 0x7f58000731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.876 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f5804873640 1 -- 172.21.15.196:0/2560820246 >> v1:172.21.15.99:6789/0 conn(0x7f5800073700 legacy=0x7f580015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f57ff7fe640 1 -- 172.21.15.196:0/2560820246 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918612992 0 0) 0x7f57f8001490 con 0x7f5800073700 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f5804873640 1 -- 172.21.15.196:0/2560820246 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001606a0 con 0x7f580015ee60 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.874+0000 7f5804873640 1 --2- 172.21.15.196:0/2560820246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f580015ee60 0x7f58000b40e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58000d20e0 tx=0x7f57f402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575f0276dfbafb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f57ff7fe640 1 -- 172.21.15.196:0/2560820246 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4037070 con 0x7f580015ee60 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f57ff7fe640 1 -- 172.21.15.196:0/2560820246 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4032440 con 0x7f580015ee60 2024-09-10T11:30:11.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f57ff7fe640 1 -- 172.21.15.196:0/2560820246 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4032810 con 0x7f580015ee60 2024-09-10T11:30:11.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 -- 172.21.15.196:0/2560820246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f580015ee60 msgr2=0x7f58000b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2560820246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f580015ee60 0x7f58000b40e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f58000d20e0 tx=0x7f57f402a270 comp rx=0 tx=0).stop 2024-09-10T11:30:11.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 -- 172.21.15.196:0/2560820246 shutdown_connections 2024-09-10T11:30:11.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2560820246 >> v2:172.21.15.99:3300/0 conn(0x7f58000b3850 0x7f58000731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2560820246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f580015ee60 0x7f58000b40e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 -- 172.21.15.196:0/2560820246 >> 172.21.15.196:0/2560820246 conn(0x7f5800072310 msgr2=0x7f58000b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 -- 172.21.15.196:0/2560820246 shutdown_connections 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.875+0000 7f5806ad7640 1 -- 172.21.15.196:0/2560820246 wait complete. 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.876+0000 7f5806ad7640 1 Processor -- start 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 -- start start 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58000b3850 0x7f58000cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f580015ee60 0x7f580013c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5800160f40 con 0x7f58000b3850 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5800160c60 con 0x7f5800073700 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.877+0000 7f5806ad7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5800162470 con 0x7f580015ee60 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40034/0 (socket says 172.21.15.196:40034) 2024-09-10T11:30:11.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 -- 172.21.15.196:0/2185175576 learned_addr learned my addr 172.21.15.196:0/2185175576 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fffff640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58000b3850 0x7f58000cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5805074640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f580015ee60 0x7f580013c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 -- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f580015ee60 msgr2=0x7f580013c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f580015ee60 0x7f580013c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 -- 172.21.15.196:0/2185175576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58000b3850 msgr2=0x7f58000cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58000b3850 0x7f58000cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 -- 172.21.15.196:0/2185175576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58000cfff0 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5804873640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4037040 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f4042070 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58000d02d0 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4030880 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580013bdd0 con 0x7f5800073700 2024-09-10T11:30:11.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f57f4030a20 con 0x7f5800073700 2024-09-10T11:30:11.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.878+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f57f403b430 con 0x7f5800073700 2024-09-10T11:30:11.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.879+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58000731c0 con 0x7f5800073700 2024-09-10T11:30:11.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.881+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f57f4047460 con 0x7f5800073700 2024-09-10T11:30:11.908 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:11.907+0000 7f8f44b26040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T11:30:11.952 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.950+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) -- 0x7f580013e580 con 0x7f5800073700 2024-09-10T11:30:11.957 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.953+0000 7f57fd7fa640 1 -- 172.21.15.196:0/2185175576 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f57f40476c0 con 0x7f5800073700 2024-09-10T11:30:11.958 INFO:teuthology.orchestra.run.smithi196.stdout:2 2024-09-10T11:30:11.958 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 msgr2=0x7f580013e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f40046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 shutdown_connections 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f580015ee60 0x7f580013c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58000b3850 0x7f58000cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 --2- 172.21.15.196:0/2185175576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5800073700 0x7f580013e040 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.954+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 >> 172.21.15.196:0/2185175576 conn(0x7f5800072310 msgr2=0x7f580015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.955+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 shutdown_connections 2024-09-10T11:30:11.959 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:11.955+0000 7f5806ad7640 1 -- 172.21.15.196:0/2185175576 wait complete. 2024-09-10T11:30:11.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-09-10T11:30:12.008 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:12.007+0000 7f6f6d45c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T11:30:12.065 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:12.064+0000 7f8f44b26040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T11:30:12.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.068+0000 7f53a0e23640 1 Processor -- start 2024-09-10T11:30:12.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.069+0000 7f53a0e23640 1 -- start start 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.069+0000 7f53a0e23640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f539c0701c0 0x7f539c0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.069+0000 7f53a0e23640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c145790 con 0x7f539c0b62a0 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f53a0e23640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539c0d15d0 con 0x7f539c1453b0 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f53a0e23640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f539c0ce9f0 con 0x7f539c0701c0 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539a59c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f539c1453b0 0x7f539c0b5b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49298/0 (socket says 172.21.15.196:49298) 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539a59c640 1 -- 172.21.15.196:0/886124698 learned_addr learned my addr 172.21.15.196:0/886124698 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539ad9d640 1 --2- 172.21.15.196:0/886124698 >> v2:172.21.15.99:3300/0 conn(0x7f539c0701c0 0x7f539c0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.071 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330033918 0 0) 0x7f539c0d15d0 con 0x7f539c1453b0 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5388003520 con 0x7f539c1453b0 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157696236 0 0) 0x7f539c145790 con 0x7f539c0b62a0 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f539c0d15d0 con 0x7f539c0b62a0 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539ad9d640 1 -- 172.21.15.196:0/886124698 >> v1:172.21.15.196:6789/0 conn(0x7f539c1453b0 legacy=0x7f539c0b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539ad9d640 1 -- 172.21.15.196:0/886124698 >> v1:172.21.15.99:6789/0 conn(0x7f539c0b62a0 legacy=0x7f539c0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.070+0000 7f539ad9d640 1 -- 172.21.15.196:0/886124698 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c0ce150 con 0x7f539c0701c0 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.071+0000 7f539ad9d640 1 --2- 172.21.15.196:0/886124698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c0701c0 0x7f539c0733d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5394009d90 tx=0x7f539402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4240b77e44f821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.071+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394007740 con 0x7f539c0701c0 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.071+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53940078e0 con 0x7f539c0701c0 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.071+0000 7f539959a640 1 -- 172.21.15.196:0/886124698 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394007cb0 con 0x7f539c0701c0 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 -- 172.21.15.196:0/886124698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c0701c0 msgr2=0x7f539c0733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.073 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 --2- 172.21.15.196:0/886124698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c0701c0 0x7f539c0733d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5394009d90 tx=0x7f539402a6f0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 -- 172.21.15.196:0/886124698 shutdown_connections 2024-09-10T11:30:12.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 --2- 172.21.15.196:0/886124698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c0701c0 0x7f539c0733d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 -- 172.21.15.196:0/886124698 >> 172.21.15.196:0/886124698 conn(0x7f539c072680 msgr2=0x7f539c072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 -- 172.21.15.196:0/886124698 shutdown_connections 2024-09-10T11:30:12.074 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.072+0000 7f53a0e23640 1 -- 172.21.15.196:0/886124698 wait complete. 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.073+0000 7f53a0e23640 1 Processor -- start 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.073+0000 7f53a0e23640 1 -- start start 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 0x7f539c1e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0ce710 con 0x7f539c1453b0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f539c073910 con 0x7f539c0b62a0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f53a0e23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0d0540 con 0x7f539c0701c0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f5399d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47874/0 (socket says 172.21.15.196:47874) 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f5399d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40040/0 (socket says 172.21.15.196:40040) 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 -- 172.21.15.196:0/2363014433 learned_addr learned my addr 172.21.15.196:0/2363014433 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539ad9d640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 0x7f539c1e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 -- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 msgr2=0x7f539c1e6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 -- 172.21.15.196:0/2363014433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 msgr2=0x7f539c1e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 0x7f539c1e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 -- 172.21.15.196:0/2363014433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c1d41d0 con 0x7f539c0701c0 2024-09-10T11:30:12.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f5399d9b640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539ad9d640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 0x7f539c1e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.074+0000 7f539a59c640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f538400c410 tx=0x7f538400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384017070 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384005ce0 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c13ec30 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53840072f0 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c13f1f0 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f538401a070 con 0x7f539c0701c0 2024-09-10T11:30:12.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.075+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f5384012890 con 0x7f539c0701c0 2024-09-10T11:30:12.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.076+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c0733d0 con 0x7f539c0701c0 2024-09-10T11:30:12.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.078+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5384021070 con 0x7f539c0701c0 2024-09-10T11:30:12.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.146+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) -- 0x7f539c1e6b40 con 0x7f539c0701c0 2024-09-10T11:30:12.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.149+0000 7f5392ffd640 1 -- 172.21.15.196:0/2363014433 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5384012400 con 0x7f539c0701c0 2024-09-10T11:30:12.150 INFO:teuthology.orchestra.run.smithi196.stdout:3 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 msgr2=0x7f539c1d3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f538400c410 tx=0x7f538400c8d0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 shutdown_connections 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f539c1453b0 0x7f539c1e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f539c0b62a0 0x7f539c1e6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 --2- 172.21.15.196:0/2363014433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f539c0701c0 0x7f539c1d3c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.151 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 >> 172.21.15.196:0/2363014433 conn(0x7f539c072680 msgr2=0x7f539c0b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 shutdown_connections 2024-09-10T11:30:12.152 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.150+0000 7f53a0e23640 1 -- 172.21.15.196:0/2363014433 wait complete. 2024-09-10T11:30:12.165 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-09-10T11:30:12.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.262+0000 7f6aa2be4640 1 Processor -- start 2024-09-10T11:30:12.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.262+0000 7f6aa2be4640 1 -- start start 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa2be4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa2be4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6a9c0733c0 0x7f6a9c072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa2be4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c15ccc0 con 0x7f6a9c1594a0 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa2be4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9c15c700 con 0x7f6a9c0733c0 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa2be4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c15c9e0 con 0x7f6a9c1590c0 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa0980640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa1181640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6a9c1594a0 0x7f6a9c0b04f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49304/0 (socket says 172.21.15.196:49304) 2024-09-10T11:30:12.266 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa0980640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47886/0 (socket says 172.21.15.196:47886) 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6aa1181640 1 -- 172.21.15.196:0/3621121077 learned_addr learned my addr 172.21.15.196:0/3621121077 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6a9bfff640 1 --2- 172.21.15.196:0/3621121077 >> v2:172.21.15.99:3300/0 conn(0x7f6a9c0733c0 0x7f6a9c072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6aa0980640 1 -- 172.21.15.196:0/3621121077 >> v2:172.21.15.99:3300/0 conn(0x7f6a9c0733c0 msgr2=0x7f6a9c072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.263+0000 7f6a9b7fe640 1 -- 172.21.15.196:0/3621121077 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557464842 0 0) 0x7f6a9c15ccc0 con 0x7f6a9c1594a0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6aa0980640 1 --2- 172.21.15.196:0/3621121077 >> v2:172.21.15.99:3300/0 conn(0x7f6a9c0733c0 0x7f6a9c072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6aa0980640 1 -- 172.21.15.196:0/3621121077 >> v1:172.21.15.196:6789/0 conn(0x7f6a9c1594a0 legacy=0x7f6a9c0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6aa0980640 1 -- 172.21.15.196:0/3621121077 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c0cf600 con 0x7f6a9c1590c0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6aa0980640 1 --2- 172.21.15.196:0/3621121077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a900097d0 tx=0x7f6a9002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31d14b1080142ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6a9b7fe640 1 -- 172.21.15.196:0/3621121077 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90037070 con 0x7f6a9c1590c0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6a9b7fe640 1 -- 172.21.15.196:0/3621121077 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9002ae40 con 0x7f6a9c1590c0 2024-09-10T11:30:12.267 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.264+0000 7f6a9b7fe640 1 -- 172.21.15.196:0/3621121077 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90032660 con 0x7f6a9c1590c0 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 -- 172.21.15.196:0/3621121077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 msgr2=0x7f6a9c15bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/3621121077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6a900097d0 tx=0x7f6a9002a130 comp rx=0 tx=0).stop 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 -- 172.21.15.196:0/3621121077 shutdown_connections 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/3621121077 >> v2:172.21.15.99:3300/0 conn(0x7f6a9c0733c0 0x7f6a9c072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/3621121077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c1590c0 0x7f6a9c15bd60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 -- 172.21.15.196:0/3621121077 >> 172.21.15.196:0/3621121077 conn(0x7f6a9c0d1900 msgr2=0x7f6a9c0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 -- 172.21.15.196:0/3621121077 shutdown_connections 2024-09-10T11:30:12.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.265+0000 7f6aa2be4640 1 -- 172.21.15.196:0/3621121077 wait complete. 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.266+0000 7f6aa2be4640 1 Processor -- start 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.266+0000 7f6aa2be4640 1 -- start start 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a9c1590c0 0x7f6a9c15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 0x7f6a9c142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c15c420 con 0x7f6a9c1590c0 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c15c2a0 con 0x7f6a9c1594a0 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa2be4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c072b60 con 0x7f6a9c0733c0 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa0980640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa0980640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47888/0 (socket says 172.21.15.196:47888) 2024-09-10T11:30:12.271 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa0980640 1 -- 172.21.15.196:0/1679659018 learned_addr learned my addr 172.21.15.196:0/1679659018 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6aa1181640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 0x7f6a9c142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.267+0000 7f6a9bfff640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a9c1590c0 0x7f6a9c15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 -- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 msgr2=0x7f6a9c142ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 0x7f6a9c142ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 -- 172.21.15.196:0/1679659018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a9c1590c0 msgr2=0x7f6a9c15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a9c1590c0 0x7f6a9c15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 -- 172.21.15.196:0/1679659018 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c15d890 con 0x7f6a9c0733c0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa1181640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 0x7f6a9c142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa0980640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a90035040 tx=0x7f6a900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90041070 con 0x7f6a9c0733c0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a9003c670 con 0x7f6a9c0733c0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90037040 con 0x7f6a9c0733c0 2024-09-10T11:30:12.272 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c15db30 con 0x7f6a9c0733c0 2024-09-10T11:30:12.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c15e0f0 con 0x7f6a9c0733c0 2024-09-10T11:30:12.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6a9003a070 con 0x7f6a9c0733c0 2024-09-10T11:30:12.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.268+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6a90045c80 con 0x7f6a9c0733c0 2024-09-10T11:30:12.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.269+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c072620 con 0x7f6a9c0733c0 2024-09-10T11:30:12.273 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.271+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6a90045770 con 0x7f6a9c0733c0 2024-09-10T11:30:12.341 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.340+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) -- 0x7f6a9c0b6490 con 0x7f6a9c0733c0 2024-09-10T11:30:12.347 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.346+0000 7f6a997fa640 1 -- 172.21.15.196:0/1679659018 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6a90045950 con 0x7f6a9c0733c0 2024-09-10T11:30:12.347 INFO:teuthology.orchestra.run.smithi196.stdout:4 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 msgr2=0x7f6a9c146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a90035040 tx=0x7f6a900046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 shutdown_connections 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9c1594a0 0x7f6a9c142ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6a9c1590c0 0x7f6a9c15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.348 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 --2- 172.21.15.196:0/1679659018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a9c0733c0 0x7f6a9c146310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 >> 172.21.15.196:0/1679659018 conn(0x7f6a9c0d1900 msgr2=0x7f6a9c0b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 shutdown_connections 2024-09-10T11:30:12.349 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.347+0000 7f6aa2be4640 1 -- 172.21.15.196:0/1679659018 wait complete. 2024-09-10T11:30:12.358 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-09-10T11:30:12.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.456+0000 7f5ab676d640 1 Processor -- start 2024-09-10T11:30:12.457 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.456+0000 7f5ab676d640 1 -- start start 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab676d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5ab01453b0 0x7f5ab00733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab676d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 0x7f5ab00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab676d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ab0145790 con 0x7f5ab006f1b0 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab676d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab00b6540 con 0x7f5ab00b5b90 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab676d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5ab00ce9f0 con 0x7f5ab01453b0 2024-09-10T11:30:12.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5ab01453b0 0x7f5ab00733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab4d0a640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5ab006f1b0 0x7f5ab006f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49320/0 (socket says 172.21.15.196:49320) 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5ab4d0a640 1 -- 172.21.15.196:0/4259571831 learned_addr learned my addr 172.21.15.196:0/4259571831 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaf7fe640 1 --2- 172.21.15.196:0/4259571831 >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 0x7f5ab00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaffff640 1 -- 172.21.15.196:0/4259571831 >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 msgr2=0x7f5ab00b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaffff640 1 --2- 172.21.15.196:0/4259571831 >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 0x7f5ab00b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaffff640 1 -- 172.21.15.196:0/4259571831 >> v1:172.21.15.196:6789/0 conn(0x7f5ab006f1b0 legacy=0x7f5ab006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaeffd640 1 -- 172.21.15.196:0/4259571831 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080852488 0 0) 0x7f5ab0145790 con 0x7f5ab006f1b0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaffff640 1 -- 172.21.15.196:0/4259571831 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab00d15d0 con 0x7f5ab01453b0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.457+0000 7f5aaf7fe640 1 --2- 172.21.15.196:0/4259571831 >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 0x7f5ab00b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.458+0000 7f5aaffff640 1 --2- 172.21.15.196:0/4259571831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab01453b0 0x7f5ab00733d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ab01459e0 tx=0x7f5aa002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85107890e4771f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.458+0000 7f5aaeffd640 1 -- 172.21.15.196:0/4259571831 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0037070 con 0x7f5ab01453b0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.458+0000 7f5aaeffd640 1 -- 172.21.15.196:0/4259571831 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa0032440 con 0x7f5ab01453b0 2024-09-10T11:30:12.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.458+0000 7f5aaeffd640 1 -- 172.21.15.196:0/4259571831 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0032810 con 0x7f5ab01453b0 2024-09-10T11:30:12.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 -- 172.21.15.196:0/4259571831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab01453b0 msgr2=0x7f5ab00733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 --2- 172.21.15.196:0/4259571831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab01453b0 0x7f5ab00733d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ab01459e0 tx=0x7f5aa002a1a0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 -- 172.21.15.196:0/4259571831 shutdown_connections 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 --2- 172.21.15.196:0/4259571831 >> v2:172.21.15.99:3300/0 conn(0x7f5ab00b5b90 0x7f5ab00b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 --2- 172.21.15.196:0/4259571831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab01453b0 0x7f5ab00733d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 -- 172.21.15.196:0/4259571831 >> 172.21.15.196:0/4259571831 conn(0x7f5ab0072680 msgr2=0x7f5ab0072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 -- 172.21.15.196:0/4259571831 shutdown_connections 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.459+0000 7f5ab676d640 1 -- 172.21.15.196:0/4259571831 wait complete. 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.460+0000 7f5ab676d640 1 Processor -- start 2024-09-10T11:30:12.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 -- start start 2024-09-10T11:30:12.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab00b5b90 0x7f5ab01d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ab01453b0 0x7f5ab006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab00ce430 con 0x7f5ab00b5b90 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0073910 con 0x7f5ab006f1b0 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5ab676d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ab00ce710 con 0x7f5ab01453b0 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.461+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40066/0 (socket says 172.21.15.196:40066) 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 -- 172.21.15.196:0/3668715446 learned_addr learned my addr 172.21.15.196:0/3668715446 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaf7fe640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab00b5b90 0x7f5ab01d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5ab4d0a640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ab01453b0 0x7f5ab006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 -- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ab01453b0 msgr2=0x7f5ab006fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ab01453b0 0x7f5ab006fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 -- 172.21.15.196:0/3668715446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab00b5b90 msgr2=0x7f5ab01d5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab00b5b90 0x7f5ab01d5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 -- 172.21.15.196:0/3668715446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab01d63f0 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aaffff640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000c00 tx=0x7f5aa00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0037040 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa0042070 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0030bd0 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab013d490 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01d38c0 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5aa0047460 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.462+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5aa0030dc0 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.463+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ab00b6000 con 0x7f5ab006f1b0 2024-09-10T11:30:12.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.465+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5aa0032e20 con 0x7f5ab006f1b0 2024-09-10T11:30:12.535 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.534+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) -- 0x7f5ab006be20 con 0x7f5ab006f1b0 2024-09-10T11:30:12.540 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.539+0000 7f5aacff9640 1 -- 172.21.15.196:0/3668715446 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5aa003d440 con 0x7f5ab006f1b0 2024-09-10T11:30:12.541 INFO:teuthology.orchestra.run.smithi196.stdout:5 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 msgr2=0x7f5ab013ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0000c00 tx=0x7f5aa00046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 shutdown_connections 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ab01453b0 0x7f5ab006fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ab00b5b90 0x7f5ab01d5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 --2- 172.21.15.196:0/3668715446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ab006f1b0 0x7f5ab013ceb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 >> 172.21.15.196:0/3668715446 conn(0x7f5ab0072680 msgr2=0x7f5ab00cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 shutdown_connections 2024-09-10T11:30:12.542 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.541+0000 7f5ab676d640 1 -- 172.21.15.196:0/3668715446 wait complete. 2024-09-10T11:30:12.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-09-10T11:30:12.651 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.650+0000 7f873df1f640 1 Processor -- start 2024-09-10T11:30:12.652 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.650+0000 7f873df1f640 1 -- start start 2024-09-10T11:30:12.653 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f873df1f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87381453b0 0x7f87380733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f873df1f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8738145790 con 0x7f87380b5b90 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f873df1f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87380d15d0 con 0x7f87380b6000 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f873df1f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f87380ce9f0 con 0x7f87381453b0 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f87377fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87381453b0 0x7f87380733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f8737fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f87380b6000 0x7f873806f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49326/0 (socket says 172.21.15.196:49326) 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f87377fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87381453b0 0x7f87380733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40070/0 (socket says 172.21.15.196:40070) 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.651+0000 7f8737fff640 1 -- 172.21.15.196:0/4227007052 learned_addr learned my addr 172.21.15.196:0/4227007052 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87377fe640 1 -- 172.21.15.196:0/4227007052 >> v1:172.21.15.196:6789/0 conn(0x7f87380b6000 legacy=0x7f873806f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87377fe640 1 -- 172.21.15.196:0/4227007052 >> v1:172.21.15.99:6789/0 conn(0x7f87380b5b90 legacy=0x7f873806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87377fe640 1 -- 172.21.15.196:0/4227007052 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87380ce150 con 0x7f87381453b0 2024-09-10T11:30:12.654 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87367fc640 1 -- 172.21.15.196:0/4227007052 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377382027 0 0) 0x7f8738145790 con 0x7f87380b5b90 2024-09-10T11:30:12.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87377fe640 1 --2- 172.21.15.196:0/4227007052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87381453b0 0x7f87380733d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f872002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ec0cfe4ee20a52b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87367fc640 1 -- 172.21.15.196:0/4227007052 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720037070 con 0x7f87381453b0 2024-09-10T11:30:12.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87367fc640 1 -- 172.21.15.196:0/4227007052 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8720032440 con 0x7f87381453b0 2024-09-10T11:30:12.655 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.652+0000 7f87367fc640 1 -- 172.21.15.196:0/4227007052 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8720032810 con 0x7f87381453b0 2024-09-10T11:30:12.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 -- 172.21.15.196:0/4227007052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87381453b0 msgr2=0x7f87380733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.658 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 --2- 172.21.15.196:0/4227007052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87381453b0 0x7f87380733d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f87200097a0 tx=0x7f872002a230 comp rx=0 tx=0).stop 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 -- 172.21.15.196:0/4227007052 shutdown_connections 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 --2- 172.21.15.196:0/4227007052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87381453b0 0x7f87380733d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 -- 172.21.15.196:0/4227007052 >> 172.21.15.196:0/4227007052 conn(0x7f8738072680 msgr2=0x7f8738072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 -- 172.21.15.196:0/4227007052 shutdown_connections 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.653+0000 7f873df1f640 1 -- 172.21.15.196:0/4227007052 wait complete. 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.654+0000 7f873df1f640 1 Processor -- start 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 -- start start 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 0x7f87381e0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87381453b0 0x7f87381dc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87380ce710 con 0x7f87381453b0 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8738073910 con 0x7f87380b6000 2024-09-10T11:30:12.659 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.655+0000 7f873df1f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87380b6430 con 0x7f87380b5b90 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f87377fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 0x7f87381e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f87377fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47908/0 (socket says 172.21.15.196:47908) 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f87377fe640 1 -- 172.21.15.196:0/37571976 learned_addr learned my addr 172.21.15.196:0/37571976 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8737fff640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87381453b0 0x7f87381dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 -- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 msgr2=0x7f8738142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.660 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 -- 172.21.15.196:0/37571976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87381453b0 msgr2=0x7f87381dc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87381453b0 0x7f87381dc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 -- 172.21.15.196:0/37571976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87381de1f0 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f87377fe640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f8736ffd640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 0x7f87381e0330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f872800c480 tx=0x7f872800c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728017070 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8728005480 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873813c910 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728003830 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87381de750 con 0x7f87380b6000 2024-09-10T11:30:12.661 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f872801a070 con 0x7f87380b6000 2024-09-10T11:30:12.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.656+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f87280129f0 con 0x7f87380b6000 2024-09-10T11:30:12.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.657+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87380733d0 con 0x7f87380b6000 2024-09-10T11:30:12.662 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.659+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8728021020 con 0x7f87380b6000 2024-09-10T11:30:12.729 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.728+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) -- 0x7f87381dc2f0 con 0x7f87380b6000 2024-09-10T11:30:12.732 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.731+0000 7f872ffff640 1 -- 172.21.15.196:0/37571976 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f87381de1f0 con 0x7f87380b6000 2024-09-10T11:30:12.732 INFO:teuthology.orchestra.run.smithi196.stdout:6 2024-09-10T11:30:12.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 msgr2=0x7f87381e0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 0x7f87381e0330 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f872800c480 tx=0x7f872800c940 comp rx=0 tx=0).stop 2024-09-10T11:30:12.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 shutdown_connections 2024-09-10T11:30:12.733 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87381453b0 0x7f87381dc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87380b6000 0x7f87381e0330 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 --2- 172.21.15.196:0/37571976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87380b5b90 0x7f8738142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 >> 172.21.15.196:0/37571976 conn(0x7f8738072680 msgr2=0x7f87380b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 shutdown_connections 2024-09-10T11:30:12.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.732+0000 7f873df1f640 1 -- 172.21.15.196:0/37571976 wait complete. 2024-09-10T11:30:12.744 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-09-10T11:30:12.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.842+0000 7ff7a2f31640 1 Processor -- start 2024-09-10T11:30:12.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.842+0000 7ff7a2f31640 1 -- start start 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.842+0000 7ff7a2f31640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff79c15ff30 0x7ff79c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.842+0000 7ff7a2f31640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff79c0b5b90 0x7ff79c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a2f31640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79c0b6020 con 0x7ff79c160310 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a2f31640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c0b6300 con 0x7ff79c0b5b90 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a2f31640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff79c0b65e0 con 0x7ff79c15ff30 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff79c15ff30 0x7ff79c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff79c15ff30 0x7ff79c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:40090/0 (socket says 172.21.15.196:40090) 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/774292748 learned_addr learned my addr 172.21.15.196:0/774292748 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff79bfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff79c160310 0x7ff79c070610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:49338/0 (socket says 172.21.15.196:49338) 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a14ce640 1 --2- 172.21.15.196:0/774292748 >> v2:172.21.15.99:3300/0 conn(0x7ff79c0b5b90 0x7ff79c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/774292748 >> v2:172.21.15.99:3300/0 conn(0x7ff79c0b5b90 msgr2=0x7ff79c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:12.845 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 --2- 172.21.15.196:0/774292748 >> v2:172.21.15.99:3300/0 conn(0x7ff79c0b5b90 0x7ff79c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/774292748 >> v1:172.21.15.196:6789/0 conn(0x7ff79c160310 legacy=0x7ff79c070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/774292748 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c0b68c0 con 0x7ff79c15ff30 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.843+0000 7ff79b7fe640 1 -- 172.21.15.196:0/774292748 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269121393 0 0) 0x7ff79c0b6020 con 0x7ff79c160310 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff7a0ccd640 1 --2- 172.21.15.196:0/774292748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c15ff30 0x7ff79c0d1100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff79c06cc50 tx=0x7ff79002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b3854119178e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff79b7fe640 1 -- 172.21.15.196:0/774292748 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790037070 con 0x7ff79c15ff30 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff79b7fe640 1 -- 172.21.15.196:0/774292748 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790032440 con 0x7ff79c15ff30 2024-09-10T11:30:12.846 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff79b7fe640 1 -- 172.21.15.196:0/774292748 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790032810 con 0x7ff79c15ff30 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff7a2f31640 1 -- 172.21.15.196:0/774292748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c15ff30 msgr2=0x7ff79c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.844+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/774292748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c15ff30 0x7ff79c0d1100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff79c06cc50 tx=0x7ff79002a270 comp rx=0 tx=0).stop 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 -- 172.21.15.196:0/774292748 shutdown_connections 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/774292748 >> v2:172.21.15.99:3300/0 conn(0x7ff79c0b5b90 0x7ff79c145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/774292748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c15ff30 0x7ff79c0d1100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 -- 172.21.15.196:0/774292748 >> 172.21.15.196:0/774292748 conn(0x7ff79c072680 msgr2=0x7ff79c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 -- 172.21.15.196:0/774292748 shutdown_connections 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.845+0000 7ff7a2f31640 1 -- 172.21.15.196:0/774292748 wait complete. 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.846+0000 7ff7a2f31640 1 Processor -- start 2024-09-10T11:30:12.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.846+0000 7ff7a2f31640 1 -- start start 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 0x7ff79c143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff79c141530 0x7ff79c145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c0d1640 con 0x7ff79c141530 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff79c070d20 con 0x7ff79c160310 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff79c1f6460 con 0x7ff79c15ff30 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff79bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 0x7ff79c143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:47920/0 (socket says 172.21.15.196:47920) 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/222551234 learned_addr learned my addr 172.21.15.196:0/222551234 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a14ce640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff79c141530 0x7ff79c145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 msgr2=0x7ff79c143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 0x7ff79c143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/222551234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff79c141530 msgr2=0x7ff79c145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff79c141530 0x7ff79c145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 -- 172.21.15.196:0/222551234 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79c143660 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff79bfff640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 0x7ff79c143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a0ccd640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff7900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790037040 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff790042070 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790030860 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.847+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79c143be0 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.848+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79c1f5e10 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.848+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff790030a00 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.848+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7ff790047b40 con 0x7ff79c15ff30 2024-09-10T11:30:12.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.849+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79c0d1100 con 0x7ff79c15ff30 2024-09-10T11:30:12.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.851+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff790047460 con 0x7ff79c15ff30 2024-09-10T11:30:12.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.921+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) -- 0x7ff79c0b60b0 con 0x7ff79c15ff30 2024-09-10T11:30:12.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.927+0000 7ff7997fa640 1 -- 172.21.15.196:0/222551234 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7ff790047640 con 0x7ff79c15ff30 2024-09-10T11:30:12.928 INFO:teuthology.orchestra.run.smithi196.stdout:7 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 msgr2=0x7ff79c1f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff7900046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 shutdown_connections 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff79c141530 0x7ff79c145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff79c160310 0x7ff79c143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 --2- 172.21.15.196:0/222551234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff79c15ff30 0x7ff79c1f68f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 >> 172.21.15.196:0/222551234 conn(0x7ff79c072680 msgr2=0x7ff79c0b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 shutdown_connections 2024-09-10T11:30:12.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:12.928+0000 7ff7a2f31640 1 -- 172.21.15.196:0/222551234 wait complete. 2024-09-10T11:30:12.939 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T11:30:12.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T11:30:12.942 INFO:tasks.ceph.osd.0:Started 2024-09-10T11:30:12.942 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T11:30:12.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T11:30:12.944 INFO:tasks.ceph.osd.1:Started 2024-09-10T11:30:12.944 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T11:30:12.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T11:30:12.946 INFO:tasks.ceph.osd.2:Started 2024-09-10T11:30:12.947 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T11:30:12.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T11:30:12.949 INFO:tasks.ceph.osd.3:Started 2024-09-10T11:30:12.949 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T11:30:12.949 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T11:30:12.952 INFO:tasks.ceph.osd.4:Started 2024-09-10T11:30:12.952 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T11:30:12.952 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T11:30:12.954 INFO:tasks.ceph.osd.5:Started 2024-09-10T11:30:12.954 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T11:30:12.954 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T11:30:12.956 INFO:tasks.ceph.osd.6:Started 2024-09-10T11:30:12.957 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T11:30:12.957 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T11:30:12.959 INFO:tasks.ceph.osd.7:Started 2024-09-10T11:30:12.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:12.971 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:12.970+0000 7f6f6d45c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T11:30:13.024 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-09-10T11:30:13.023+0000 7f42be419640 -1 Falling back to public interface 2024-09-10T11:30:13.033 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-09-10T11:30:13.032+0000 7fa31b343640 -1 Falling back to public interface 2024-09-10T11:30:13.034 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-09-10T11:30:13.033+0000 7fe5f1502640 -1 Falling back to public interface 2024-09-10T11:30:13.036 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-09-10T11:30:13.035+0000 7f8dfa222640 -1 Falling back to public interface 2024-09-10T11:30:13.038 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-09-10T11:30:13.038+0000 7f42be419640 -1 osd.0 0 log_to_monitors true 2024-09-10T11:30:13.049 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-09-10T11:30:13.049+0000 7fa3d6bae640 -1 Falling back to public interface 2024-09-10T11:30:13.051 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.050+0000 7f6f6d45c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T11:30:13.053 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-09-10T11:30:13.052+0000 7fe5f1502640 -1 osd.6 0 log_to_monitors true 2024-09-10T11:30:13.053 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-09-10T11:30:13.053+0000 7fa31b343640 -1 osd.4 0 log_to_monitors true 2024-09-10T11:30:13.055 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-09-10T11:30:13.054+0000 7f866f632640 -1 Falling back to public interface 2024-09-10T11:30:13.055 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-09-10T11:30:13.055+0000 7fa850fb9640 -1 Falling back to public interface 2024-09-10T11:30:13.056 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-09-10T11:30:13.055+0000 7f8dfa222640 -1 osd.7 0 log_to_monitors true 2024-09-10T11:30:13.056 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-09-10T11:30:13.055+0000 7f866db8b640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6817/0: (98) Address already in use 2024-09-10T11:30:13.063 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-09-10T11:30:13.062+0000 7f949820b640 -1 Falling back to public interface 2024-09-10T11:30:13.068 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-09-10T11:30:13.067+0000 7f866f632640 -1 osd.3 0 log_to_monitors true 2024-09-10T11:30:13.068 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.068+0000 7f8f44b26040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T11:30:13.069 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-09-10T11:30:13.068+0000 7fa850fb9640 -1 osd.2 0 log_to_monitors true 2024-09-10T11:30:13.069 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-09-10T11:30:13.068+0000 7fa3d6bae640 -1 osd.1 0 log_to_monitors true 2024-09-10T11:30:13.076 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-09-10T11:30:13.075+0000 7f949820b640 -1 osd.5 0 log_to_monitors true 2024-09-10T11:30:13.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.077+0000 7f34b18b7640 1 Processor -- start 2024-09-10T11:30:13.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.077+0000 7f34b18b7640 1 -- start start 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f34ac1590c0 0x7f34ac15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 0x7f34ac072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac15ccc0 con 0x7f34ac0733c0 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f34ac15c700 con 0x7f34ac1590c0 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34b18b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac15c9e0 con 0x7f34ac1594a0 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34aa7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 0x7f34ac072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:35088/0 (socket says 172.21.15.99:35088) 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34ab7fe640 1 -- 172.21.15.99:0/390764428 learned_addr learned my addr 172.21.15.99:0/390764428 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.078+0000 7f34aaffd640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.196:3300/0 conn(0x7f34ac1590c0 0x7f34ac15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 -- 172.21.15.99:0/390764428 >> v2:172.21.15.196:3300/0 conn(0x7f34ac1590c0 msgr2=0x7f34ac15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.196:3300/0 conn(0x7f34ac1590c0 0x7f34ac15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 -- 172.21.15.99:0/390764428 >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 msgr2=0x7f34ac072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 0x7f34ac072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 -- 172.21.15.99:0/390764428 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0cf600 con 0x7f34ac1594a0 2024-09-10T11:30:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34aa7fc640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 0x7f34ac072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.079+0000 7f34ab7fe640 1 --2- 172.21.15.99:0/390764428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0b4600 tx=0x7f34a402ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8203ca138582b50b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34a9ffb640 1 -- 172.21.15.99:0/390764428 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4037020 con 0x7f34ac1594a0 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34a9ffb640 1 -- 172.21.15.99:0/390764428 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a4033670 con 0x7f34ac1594a0 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34a9ffb640 1 -- 172.21.15.99:0/390764428 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4033a40 con 0x7f34ac1594a0 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 -- 172.21.15.99:0/390764428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 msgr2=0x7f34ac072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 --2- 172.21.15.99:0/390764428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0b4600 tx=0x7f34a402ea60 comp rx=0 tx=0).stop 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 -- 172.21.15.99:0/390764428 shutdown_connections 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 --2- 172.21.15.99:0/390764428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac072b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.99:3300/0 conn(0x7f34ac0733c0 0x7f34ac072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 --2- 172.21.15.99:0/390764428 >> v2:172.21.15.196:3300/0 conn(0x7f34ac1590c0 0x7f34ac15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 -- 172.21.15.99:0/390764428 >> 172.21.15.99:0/390764428 conn(0x7f34ac0d1900 msgr2=0x7f34ac0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 -- 172.21.15.99:0/390764428 shutdown_connections 2024-09-10T11:30:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.080+0000 7f34b18b7640 1 -- 172.21.15.99:0/390764428 wait complete. 2024-09-10T11:30:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.082+0000 7f34b18b7640 1 Processor -- start 2024-09-10T11:30:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.082+0000 7f34b18b7640 1 -- start start 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 0x7f34ac0b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac0b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac15c420 con 0x7f34ac1590c0 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac15c2a0 con 0x7f34ac0733c0 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34b18b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f34ac0b04f0 con 0x7f34ac1594a0 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aa7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 0x7f34ac0b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:53390/0 (socket says 172.21.15.99:53390) 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 -- 172.21.15.99:0/2406621185 learned_addr learned my addr 172.21.15.99:0/2406621185 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34ab7fe640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac0b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 -- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 msgr2=0x7f34ac0b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac0b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 -- 172.21.15.99:0/2406621185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 msgr2=0x7f34ac0b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 0x7f34ac0b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 -- 172.21.15.99:0/2406621185 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac0aa530 con 0x7f34ac0733c0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aa7fc640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 0x7f34ac0b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.083+0000 7f34aaffd640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9e0 tx=0x7f349c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c017070 con 0x7f34ac0733c0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c00c9d0 con 0x7f34ac0733c0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac0aa810 con 0x7f34ac0733c0 2024-09-10T11:30:13.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c00cdc0 con 0x7f34ac0733c0 2024-09-10T11:30:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac0aadd0 con 0x7f34ac0733c0 2024-09-10T11:30:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f349c01a070 con 0x7f34ac0733c0 2024-09-10T11:30:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.084+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f349c013b40 con 0x7f34ac0733c0 2024-09-10T11:30:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.085+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34ac0cf2c0 con 0x7f34ac0733c0 2024-09-10T11:30:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.087+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f349c013470 con 0x7f34ac0733c0 2024-09-10T11:30:13.123 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.122+0000 7f8f44b26040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T11:30:13.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.145+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34ac15bd60 con 0x7f34ac0733c0 2024-09-10T11:30:13.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.146+0000 7f3493fff640 1 -- 172.21.15.99:0/2406621185 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f349c013650 con 0x7f34ac0733c0 2024-09-10T11:30:13.147 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:13.147 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:12.922527+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:13.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 msgr2=0x7f34ac157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:13.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f349c00e9e0 tx=0x7f349c00eea0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 shutdown_connections 2024-09-10T11:30:13.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34ac1594a0 0x7f34ac0b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34ac1590c0 0x7f34ac0b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 --2- 172.21.15.99:0/2406621185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34ac0733c0 0x7f34ac157250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:13.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 >> 172.21.15.99:0/2406621185 conn(0x7f34ac0d1900 msgr2=0x7f34ac06fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:13.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 shutdown_connections 2024-09-10T11:30:13.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:13.147+0000 7f34b18b7640 1 -- 172.21.15.99:0/2406621185 wait complete. 2024-09-10T11:30:13.159 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T11:30:13.159 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T11:30:13.428 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.427+0000 7f6f6d45c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T11:30:13.489 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.488+0000 7f8f44b26040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T11:30:13.512 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.512+0000 7f6f6d45c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T11:30:13.574 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.573+0000 7f8f44b26040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T11:30:13.618 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.617+0000 7f6f6d45c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T11:30:13.664 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.663+0000 7f6f6d45c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T11:30:13.681 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.680+0000 7f8f44b26040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T11:30:13.728 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.727+0000 7f8f44b26040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T11:30:13.774 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.774+0000 7f6f6d45c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T11:30:13.826 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.826+0000 7f6f6d45c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T11:30:13.840 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.839+0000 7f8f44b26040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T11:30:13.871 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.871+0000 7f6f6d45c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T11:30:13.895 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.895+0000 7f8f44b26040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T11:30:13.942 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:13.941+0000 7f8f44b26040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T11:30:13.964 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:13.986 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:13.985+0000 7f6f6d45c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T11:30:14.038 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.037+0000 7f6f6d45c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T11:30:14.054 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.053+0000 7f8f44b26040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T11:30:14.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.054+0000 7fb89924c640 1 Processor -- start 2024-09-10T11:30:14.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.054+0000 7fb89924c640 1 -- start start 2024-09-10T11:30:14.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89924c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb8940cd040 0x7fb894070630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:14.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89924c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89406f710 0x7fb89415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:14.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89924c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb89415f2b0 con 0x7fb8940ceb60 2024-09-10T11:30:14.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89924c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb89406cda0 con 0x7fb89406f710 2024-09-10T11:30:14.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89924c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb89406d080 con 0x7fb8940cd040 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89406f710 0x7fb89415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb89406f710 0x7fb89415ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60042/0 (socket says 172.21.15.99:60042) 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 -- 172.21.15.99:0/4120276593 learned_addr learned my addr 172.21.15.99:0/4120276593 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89259c640 1 --2- 172.21.15.99:0/4120276593 >> v2:172.21.15.196:3300/0 conn(0x7fb8940cd040 0x7fb894070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 -- 172.21.15.99:0/4120276593 >> v2:172.21.15.196:3300/0 conn(0x7fb8940cd040 msgr2=0x7fb894070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 --2- 172.21.15.99:0/4120276593 >> v2:172.21.15.196:3300/0 conn(0x7fb8940cd040 0x7fb894070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 -- 172.21.15.99:0/4120276593 >> v1:172.21.15.196:6789/0 conn(0x7fb8940ceb60 legacy=0x7fb89415e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:14.057 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.055+0000 7fb89359e640 1 -- 172.21.15.99:0/4120276593 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89406d360 con 0x7fb89406f710 2024-09-10T11:30:14.058 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.056+0000 7fb89359e640 1 --2- 172.21.15.99:0/4120276593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb89415ed70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb894072900 tx=0x7fb88402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f814a0444411b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:14.058 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.056+0000 7fb891d9b640 1 -- 172.21.15.99:0/4120276593 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884037070 con 0x7fb89406f710 2024-09-10T11:30:14.058 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.056+0000 7fb891d9b640 1 -- 172.21.15.99:0/4120276593 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb884032440 con 0x7fb89406f710 2024-09-10T11:30:14.058 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.056+0000 7fb891d9b640 1 -- 172.21.15.99:0/4120276593 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884032810 con 0x7fb89406f710 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 -- 172.21.15.99:0/4120276593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 msgr2=0x7fb89415ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 --2- 172.21.15.99:0/4120276593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb89415ed70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb894072900 tx=0x7fb88402a1e0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 -- 172.21.15.99:0/4120276593 shutdown_connections 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 --2- 172.21.15.99:0/4120276593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb89415ed70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 --2- 172.21.15.99:0/4120276593 >> v2:172.21.15.196:3300/0 conn(0x7fb8940cd040 0x7fb894070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 -- 172.21.15.99:0/4120276593 >> 172.21.15.99:0/4120276593 conn(0x7fb8940d29f0 msgr2=0x7fb8940d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 -- 172.21.15.99:0/4120276593 shutdown_connections 2024-09-10T11:30:14.059 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.057+0000 7fb89924c640 1 -- 172.21.15.99:0/4120276593 wait complete. 2024-09-10T11:30:14.061 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.058+0000 7fb89924c640 1 Processor -- start 2024-09-10T11:30:14.061 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.058+0000 7fb89924c640 1 -- start start 2024-09-10T11:30:14.061 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:14.061 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8940cd040 0x7fb8940c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:14.061 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8940ceb60 0x7fb8940cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940cef40 con 0x7fb89406f710 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb89406d920 con 0x7fb8940cd040 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89924c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb8940cf0c0 con 0x7fb8940ceb60 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60048/0 (socket says 172.21.15.99:60048) 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 -- 172.21.15.99:0/2689377430 learned_addr learned my addr 172.21.15.99:0/2689377430 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89259c640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8940cd040 0x7fb8940c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb89359e640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8940ceb60 0x7fb8940cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 -- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8940ceb60 msgr2=0x7fb8940cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8940ceb60 0x7fb8940cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 -- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8940cd040 msgr2=0x7fb8940c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8940cd040 0x7fb8940c9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.062 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 -- 172.21.15.99:0/2689377430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8940ca310 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.059+0000 7fb892d9d640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c760 tx=0x7fb88c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c016020 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c006d50 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8940ca5f0 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c003bb0 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8940cae90 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb88c00ab80 con 0x7fb89406f710 2024-09-10T11:30:14.063 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.060+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1883+0+0 (secure 0 0 0) 0x7fb88c012be0 con 0x7fb89406f710 2024-09-10T11:30:14.064 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.061+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb894070630 con 0x7fb89406f710 2024-09-10T11:30:14.064 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.063+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb88c020020 con 0x7fb89406f710 2024-09-10T11:30:14.093 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.092+0000 7f6f6d45c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T11:30:14.103 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.103+0000 7f8f44b26040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T11:30:14.122 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.122+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8940ca7c0 con 0x7fb89406f710 2024-09-10T11:30:14.123 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.122+0000 7fb88b7fe640 1 -- 172.21.15.99:0/2689377430 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+7058 (secure 0 0 0) 0x7fb88c003d50 con 0x7fb89406f710 2024-09-10T11:30:14.123 INFO:teuthology.misc.health.smithi099.stdout: 2024-09-10T11:30:14.123 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":11,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:13.923711+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:14.124 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 msgr2=0x7fb8940b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:14.124 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c760 tx=0x7fb88c00cc20 comp rx=0 tx=0).stop 2024-09-10T11:30:14.124 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 shutdown_connections 2024-09-10T11:30:14.124 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8940ceb60 0x7fb8940cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.125 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8940cd040 0x7fb8940c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.125 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 --2- 172.21.15.99:0/2689377430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb89406f710 0x7fb8940b7a60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:14.125 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 >> 172.21.15.99:0/2689377430 conn(0x7fb8940d29f0 msgr2=0x7fb8940cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:14.125 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 shutdown_connections 2024-09-10T11:30:14.125 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:14.123+0000 7fb89924c640 1 -- 172.21.15.99:0/2689377430 wait complete. 2024-09-10T11:30:14.133 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T11:30:14.154 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.153+0000 7f8f44b26040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T11:30:14.163 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.162+0000 7f6f6d45c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T11:30:14.207 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.207+0000 7f6f6d45c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T11:30:14.222 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.221+0000 7f8f44b26040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T11:30:14.250 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.250+0000 7f6f6d45c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T11:30:14.266 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.265+0000 7f8f44b26040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T11:30:14.309 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.308+0000 7f8f44b26040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T11:30:14.898 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:14.897+0000 7f6f6d45c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T11:30:14.930 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-09-10T11:30:14.929+0000 7f8df5b14640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T11:30:14.930 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-09-10T11:30:14.930+0000 7fe5ed491640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T11:30:14.933 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-09-10T11:30:14.932+0000 7fa3172d2640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T11:30:14.933 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-09-10T11:30:14.932+0000 7f9493bf9640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T11:30:14.934 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-09-10T11:30:14.932+0000 7f42b9e01640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T11:30:14.934 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-09-10T11:30:14.932+0000 7fa3d2b3d640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T11:30:14.935 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-09-10T11:30:14.932+0000 7fa84cf48640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T11:30:14.935 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-09-10T11:30:14.932+0000 7f866ab72640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T11:30:14.936 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-09-10T11:30:14.934+0000 7f42b3435640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.936 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-09-10T11:30:14.934+0000 7f86641a6640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.936 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-09-10T11:30:14.934+0000 7fa3cc171640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.936 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-09-10T11:30:14.934+0000 7fa84657c640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.937 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-09-10T11:30:14.934+0000 7fa310906640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.937 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-09-10T11:30:14.934+0000 7f948d22d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.937 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-09-10T11:30:14.934+0000 7fe5e6ac5640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.938 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-09-10T11:30:14.934+0000 7f8def148640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:30:14.968 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-09-10T11:30:14.967+0000 7f8f44b26040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T11:30:15.915 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:15.914+0000 7f6f0dd28640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T11:30:15.915 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-09-10T11:30:15.915+0000 7f6f0dd28640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T11:30:20.941 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:21.043 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.042+0000 7efd21e5e640 1 Processor -- start 2024-09-10T11:30:21.043 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.042+0000 7efd21e5e640 1 -- start start 2024-09-10T11:30:21.046 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd21e5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd1c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd21e5e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd1c0cdd10 0x7efd1c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd21e5e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd1c0d1760 con 0x7efd1c15d050 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd21e5e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efd1c0d1a40 con 0x7efd1c0cdd10 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd21e5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c0b2bb0 con 0x7efd1c15cc70 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1affd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efd1c15d050 0x7efd1c0cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48386/0 (socket says 172.21.15.99:48386) 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1affd640 1 -- 172.21.15.99:0/3342871189 learned_addr learned my addr 172.21.15.99:0/3342871189 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1bfff640 1 --2- 172.21.15.99:0/3342871189 >> v2:172.21.15.196:3300/0 conn(0x7efd1c0cdd10 0x7efd1c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd1c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481742490 0 0) 0x7efd1c0d1760 con 0x7efd1c15d050 2024-09-10T11:30:21.047 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd04003520 con 0x7efd1c15d050 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.043+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3656274678 0 0) 0x7efd04003520 con 0x7efd1c15d050 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 msgr2=0x7efd1c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd1c0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 >> v2:172.21.15.196:3300/0 conn(0x7efd1c0cdd10 msgr2=0x7efd1c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 --2- 172.21.15.99:0/3342871189 >> v2:172.21.15.196:3300/0 conn(0x7efd1c0cdd10 0x7efd1c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c0b2560 con 0x7efd1c15d050 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7efd10001670 con 0x7efd1c15d050 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd1c0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1bfff640 1 --2- 172.21.15.99:0/3342871189 >> v2:172.21.15.196:3300/0 conn(0x7efd1c0cdd10 0x7efd1c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 >> v1:172.21.15.99:6789/0 conn(0x7efd1c15d050 legacy=0x7efd1c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.048 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd04004d40 0x7efd040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d1a40 con 0x7efd040039b0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0b2bb0 con 0x7efd04004d40 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd0400b9c0 con 0x7efd1c15cc70 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1affd640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1bfff640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd04004d40 0x7efd040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 -- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 msgr2=0x7efd04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c15cc70 0x7efd04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 -- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd04004d40 msgr2=0x7efd040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd04004d40 0x7efd040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.044+0000 7efd1b7fe640 1 -- 172.21.15.99:0/3342871189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd04003520 con 0x7efd040039b0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd04003dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009cc0 tx=0x7efd04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d81a9c5c188d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.049 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c036070 con 0x7efd040039b0 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c031400 con 0x7efd040039b0 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd1a7fc640 1 -- 172.21.15.99:0/3342871189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c009860 con 0x7efd040039b0 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd21e5e640 1 -- 172.21.15.99:0/3342871189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 msgr2=0x7efd04003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd21e5e640 1 --2- 172.21.15.99:0/3342871189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd04003dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009cc0 tx=0x7efd04005830 comp rx=0 tx=0).stop 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd1bfff640 1 -- 172.21.15.99:0/3342871189 reap_dead start 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd21e5e640 1 -- 172.21.15.99:0/3342871189 shutdown_connections 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.045+0000 7efd21e5e640 1 -- 172.21.15.99:0/3342871189 >> 172.21.15.99:0/3342871189 conn(0x7efd1c0d2af0 msgr2=0x7efd1c0d2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.046+0000 7efd21e5e640 1 -- 172.21.15.99:0/3342871189 shutdown_connections 2024-09-10T11:30:21.050 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.046+0000 7efd21e5e640 1 -- 172.21.15.99:0/3342871189 wait complete. 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.047+0000 7efd21e5e640 1 Processor -- start 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.047+0000 7efd21e5e640 1 -- start start 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd1c1574a0 0x7efd1c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c0b09d0 0x7efd1c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d10d0 con 0x7efd040039b0 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd1c0d0340 con 0x7efd1c1574a0 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd1c0d1250 con 0x7efd1c0b09d0 2024-09-10T11:30:21.053 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60164/0 (socket says 172.21.15.99:60164) 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 -- 172.21.15.99:0/1045306967 learned_addr learned my addr 172.21.15.99:0/1045306967 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1affd640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd1c1574a0 0x7efd1c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1bfff640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c0b09d0 0x7efd1c0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 -- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c0b09d0 msgr2=0x7efd1c0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c0b09d0 0x7efd1c0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 -- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd1c1574a0 msgr2=0x7efd1c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd1c1574a0 0x7efd1c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 -- 172.21.15.99:0/1045306967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1c1f46c0 con 0x7efd040039b0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd1b7fe640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009cc0 tx=0x7efd0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c040030 con 0x7efd040039b0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c0093e0 con 0x7efd040039b0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1c144ed0 con 0x7efd040039b0 2024-09-10T11:30:21.054 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.048+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c036040 con 0x7efd040039b0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.049+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd1c157d40 con 0x7efd040039b0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.050+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd1c0d1760 con 0x7efd040039b0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.050+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd0c043070 con 0x7efd040039b0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.050+0000 7efd03fff640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 0x7efd0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.051+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7efd0c0aa560 con 0x7efd040039b0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.051+0000 7efd1affd640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 0x7efd0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.055 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.051+0000 7efd1affd640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 0x7efd0806e2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd1c158a30 tx=0x7efd10001e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.056 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.055+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd0c078b80 con 0x7efd040039b0 2024-09-10T11:30:21.225 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.224+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd1c1450a0 con 0x7efd040039b0 2024-09-10T11:30:21.226 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.225+0000 7efd03fff640 1 -- 172.21.15.99:0/1045306967 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12896 (secure 0 0 0) 0x7efd0c07ce80 con 0x7efd040039b0 2024-09-10T11:30:21.226 INFO:teuthology.misc.health.smithi099.stdout: 2024-09-10T11:30:21.226 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":16,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:18.943731+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:21.228 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 msgr2=0x7efd0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.228 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 0x7efd0806e2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd1c158a30 tx=0x7efd10001e10 comp rx=0 tx=0).stop 2024-09-10T11:30:21.228 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 msgr2=0x7efd1c151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.228 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009cc0 tx=0x7efd0c004730 comp rx=0 tx=0).stop 2024-09-10T11:30:21.228 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 shutdown_connections 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0806be30 0x7efd0806e2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd1c0b09d0 0x7efd1c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd1c1574a0 0x7efd1c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 --2- 172.21.15.99:0/1045306967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd040039b0 0x7efd1c151e20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 >> 172.21.15.99:0/1045306967 conn(0x7efd1c0d2af0 msgr2=0x7efd1c06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 shutdown_connections 2024-09-10T11:30:21.229 INFO:teuthology.misc.health.smithi099.stderr:2024-09-10T11:30:21.227+0000 7efd21e5e640 1 -- 172.21.15.99:0/1045306967 wait complete. 2024-09-10T11:30:21.238 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T11:30:21.238 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T11:30:21.238 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:30:21.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:30:21.241 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:30:21.241 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:30:21.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:30:21.243 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:30:21.243 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:30:21.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:30:21.247 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:30:21.247 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:30:21.247 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:30:21.250 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:30:21.250 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T11:30:21.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:21.315 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:30:21.316 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:30:21.332 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:30:21.333 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:30:21.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.351+0000 7f3d6cde4640 1 Processor -- start 2024-09-10T11:30:21.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.351+0000 7f3d6cde4640 1 -- start start 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6cde4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d68073300 0x7f3d680b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6cde4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d680b5d70 con 0x7f3d68146490 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6cde4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d680b6050 con 0x7f3d68070610 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6cde4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680b6330 con 0x7f3d68073300 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d65d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d68073300 0x7f3d680b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6659c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3d68146490 0x7f3d680d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48400/0 (socket says 172.21.15.99:48400) 2024-09-10T11:30:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d65d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3d68073300 0x7f3d680b6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60208/0 (socket says 172.21.15.99:60208) 2024-09-10T11:30:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d6659c640 1 -- 172.21.15.99:0/4235951708 learned_addr learned my addr 172.21.15.99:0/4235951708 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d65d9b640 1 -- 172.21.15.99:0/4235951708 >> v1:172.21.15.196:6789/0 conn(0x7f3d68070610 legacy=0x7f3d680709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d65d9b640 1 -- 172.21.15.99:0/4235951708 >> v1:172.21.15.99:6789/0 conn(0x7f3d68146490 legacy=0x7f3d680d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.352+0000 7f3d65d9b640 1 -- 172.21.15.99:0/4235951708 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d68146870 con 0x7f3d68073300 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.353+0000 7f3d65d9b640 1 --2- 172.21.15.99:0/4235951708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d680b6b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0097a0 tx=0x7f3d5c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3a7f9fac3a3219 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.353+0000 7f3d6559a640 1 -- 172.21.15.99:0/4235951708 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c037070 con 0x7f3d68073300 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.353+0000 7f3d6559a640 1 -- 172.21.15.99:0/4235951708 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c032440 con 0x7f3d68073300 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.353+0000 7f3d6559a640 1 -- 172.21.15.99:0/4235951708 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c032810 con 0x7f3d68073300 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 -- 172.21.15.99:0/4235951708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 msgr2=0x7f3d680b6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/4235951708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d680b6b30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0097a0 tx=0x7f3d5c02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 -- 172.21.15.99:0/4235951708 shutdown_connections 2024-09-10T11:30:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/4235951708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d680b6b30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 -- 172.21.15.99:0/4235951708 >> 172.21.15.99:0/4235951708 conn(0x7f3d68072680 msgr2=0x7f3d68072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 -- 172.21.15.99:0/4235951708 shutdown_connections 2024-09-10T11:30:21.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.354+0000 7f3d6cde4640 1 -- 172.21.15.99:0/4235951708 wait complete. 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.355+0000 7f3d6cde4640 1 Processor -- start 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.355+0000 7f3d6cde4640 1 -- start start 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d68070610 0x7f3d681e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d68146490 0x7f3d681f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6815d630 con 0x7f3d68073300 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6815d4b0 con 0x7f3d68146490 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6cde4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6815d7b0 con 0x7f3d68070610 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60210/0 (socket says 172.21.15.99:60210) 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 -- 172.21.15.99:0/3616830616 learned_addr learned my addr 172.21.15.99:0/3616830616 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d6659c640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d68070610 0x7f3d681e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d66d9d640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d68146490 0x7f3d681f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 -- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d68070610 msgr2=0x7f3d681e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d68070610 0x7f3d681e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 -- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d68146490 msgr2=0x7f3d681f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d68146490 0x7f3d681f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 -- 172.21.15.99:0/3616830616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680ae940 con 0x7f3d68073300 2024-09-10T11:30:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d65d9b640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098c0 tx=0x7f3d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c037040 con 0x7f3d68073300 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.356+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c030770 con 0x7f3d68073300 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.357+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d680aebe0 con 0x7f3d68073300 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.357+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c030b40 con 0x7f3d68073300 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.357+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681f2540 con 0x7f3d68073300 2024-09-10T11:30:21.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.358+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6815dba0 con 0x7f3d68073300 2024-09-10T11:30:21.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.358+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d5c044020 con 0x7f3d68073300 2024-09-10T11:30:21.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.358+0000 7f3d56ffd640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 0x7f3d4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:21.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.358+0000 7f3d6659c640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 0x7f3d4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:21.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.358+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3d5c0aaf10 con 0x7f3d68073300 2024-09-10T11:30:21.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.359+0000 7f3d6659c640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 0x7f3d4806e2f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005880 tx=0x7f3d58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.363+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d5c032e20 con 0x7f3d68073300 2024-09-10T11:30:21.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.559+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d68070d70 con 0x7f3d68073300 2024-09-10T11:30:21.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.559+0000 7f3d56ffd640 1 -- 172.21.15.99:0/3616830616 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f3d5c079650 con 0x7f3d68073300 2024-09-10T11:30:21.560 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:21.560 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":2,"btime":"2024-09-10T11:30:21:319087+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T11:30:21.561 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 2 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 msgr2=0x7f3d4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 0x7f3d4806e2f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005880 tx=0x7f3d58009040 comp rx=0 tx=0).stop 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 msgr2=0x7f3d681f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0098c0 tx=0x7f3d5c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 shutdown_connections 2024-09-10T11:30:21.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3d4806be30 0x7f3d4806e2f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d68146490 0x7f3d681f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d68073300 0x7f3d681f56d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 --2- 172.21.15.99:0/3616830616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d68070610 0x7f3d681e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 >> 172.21.15.99:0/3616830616 conn(0x7f3d68072680 msgr2=0x7f3d680b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 shutdown_connections 2024-09-10T11:30:21.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:21.561+0000 7f3d6cde4640 1 -- 172.21.15.99:0/3616830616 wait complete. 2024-09-10T11:30:23.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:23.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 Processor -- start 2024-09-10T11:30:23.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 -- start start 2024-09-10T11:30:23.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9170163580 0x7f91700b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91700b3620 con 0x7f91700b3960 2024-09-10T11:30:23.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f917006d960 con 0x7f91700b3160 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.683+0000 7f917629b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f917006dc40 con 0x7f9170163580 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916f7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f91700b3960 0x7f91700d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33024/0 (socket says 172.21.15.99:33024) 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916f7fe640 1 -- 172.21.15.99:0/614940408 learned_addr learned my addr 172.21.15.99:0/614940408 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916ffff640 1 --2- 172.21.15.99:0/614940408 >> v2:172.21.15.196:3300/0 conn(0x7f9170163580 0x7f91700b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844304586 0 0) 0x7f91700b3620 con 0x7f91700b3960 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9158003520 con 0x7f91700b3960 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896934659 0 0) 0x7f917006d960 con 0x7f91700b3160 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91700b3620 con 0x7f91700b3160 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1563829466 0 0) 0x7f9158003520 con 0x7f91700b3960 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 >> v2:172.21.15.196:3300/0 conn(0x7f9170163580 msgr2=0x7f91700b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 --2- 172.21.15.99:0/614940408 >> v2:172.21.15.196:3300/0 conn(0x7f9170163580 0x7f91700b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 >> v1:172.21.15.196:6789/0 conn(0x7f91700b3160 legacy=0x7f917006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917006df20 con 0x7f91700b3960 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9164001670 con 0x7f91700b3960 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 >> v1:172.21.15.99:6789/0 conn(0x7f91700b3960 legacy=0x7f91700d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158003b40 0x7f9158003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9158004800 0x7f9158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 0x7f915800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f917006dc40 con 0x7f9158005230 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f915800c190 con 0x7f9158004800 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f915800c390 con 0x7f9158003b40 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f9174838640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 0x7f915800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916f7fe640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9158004800 0x7f9158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.684+0000 7f916ffff640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158003b40 0x7f9158003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 -- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158003b40 msgr2=0x7f9158003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158003b40 0x7f9158003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 -- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9158004800 msgr2=0x7f9158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9158004800 0x7f9158004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 -- 172.21.15.99:0/614940408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9158003520 con 0x7f9158005230 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 -- 172.21.15.99:0/614940408 reap_dead start 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f9174838640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 0x7f915800b9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9168001fd0 tx=0x7f9158005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ef24117e9923a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168001bf0 con 0x7f9158005230 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9168003b70 con 0x7f9158005230 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.685+0000 7f916effd640 1 -- 172.21.15.99:0/614940408 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680050d0 con 0x7f9158005230 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 -- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 msgr2=0x7f915800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 0x7f915800b9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9168001fd0 tx=0x7f9158005830 comp rx=0 tx=0).stop 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 -- 172.21.15.99:0/614940408 shutdown_connections 2024-09-10T11:30:23.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 --2- 172.21.15.99:0/614940408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9158005230 0x7f915800b9c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 -- 172.21.15.99:0/614940408 >> 172.21.15.99:0/614940408 conn(0x7f91700b4570 msgr2=0x7f91700b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:23.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 -- 172.21.15.99:0/614940408 shutdown_connections 2024-09-10T11:30:23.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.686+0000 7f917629b640 1 -- 172.21.15.99:0/614940408 wait complete. 2024-09-10T11:30:23.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.687+0000 7f917629b640 1 Processor -- start 2024-09-10T11:30:23.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 -- start start 2024-09-10T11:30:23.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158005230 0x7f91701f5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91701f8b60 0x7f91700cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91700b5210 con 0x7f917015b260 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91700b5090 con 0x7f91701f8b60 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f917629b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f91700b5390 con 0x7f9158005230 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f9174838640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f9174838640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56798/0 (socket says 172.21.15.99:56798) 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f9174838640 1 -- 172.21.15.99:0/2902890330 learned_addr learned my addr 172.21.15.99:0/2902890330 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.688+0000 7f916f7fe640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91701f8b60 0x7f91700cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f916ffff640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158005230 0x7f91701f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 -- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158005230 msgr2=0x7f91701f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158005230 0x7f91701f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 -- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91701f8b60 msgr2=0x7f91700cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91701f8b60 0x7f91700cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 -- 172.21.15.99:0/2902890330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91700cbbf0 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f9174838640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91680020c0 tx=0x7f9168001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168041070 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9168005480 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168036040 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91701f80c0 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.689+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701f8360 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.690+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91700761a0 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.691+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9168007040 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.691+0000 7f916cff9640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 0x7f915c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.691+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f91680a9fd0 con 0x7f917015b260 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.691+0000 7f916ffff640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 0x7f915c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:23.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.692+0000 7f916ffff640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 0x7f915c06e320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f917006c1c0 tx=0x7f9160009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:23.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.696+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9168078710 con 0x7f917015b260 2024-09-10T11:30:23.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.896+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91700b89a0 con 0x7f917015b260 2024-09-10T11:30:23.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.896+0000 7f916cff9640 1 -- 172.21.15.99:0/2902890330 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7f916807ca10 con 0x7f917015b260 2024-09-10T11:30:23.897 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:23.897 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"btime":"2024-09-10T11:30:22:320159+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4303,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T11:30:23.898 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2024-09-10T11:30:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 msgr2=0x7f915c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 0x7f915c06e320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f917006c1c0 tx=0x7f9160009190 comp rx=0 tx=0).stop 2024-09-10T11:30:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 msgr2=0x7f91700cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:23.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f91680020c0 tx=0x7f9168001650 comp rx=0 tx=0).stop 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 shutdown_connections 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f915c06be60 0x7f915c06e320 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f917015b260 0x7f91700cf660 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91701f8b60 0x7f91700cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 --2- 172.21.15.99:0/2902890330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9158005230 0x7f91701f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 >> 172.21.15.99:0/2902890330 conn(0x7f91700b4570 msgr2=0x7f91700c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 shutdown_connections 2024-09-10T11:30:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:23.898+0000 7f917629b640 1 -- 172.21.15.99:0/2902890330 wait complete. 2024-09-10T11:30:23.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:30:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.009+0000 7f9376c82640 1 Processor -- start 2024-09-10T11:30:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.009+0000 7f9376c82640 1 -- start start 2024-09-10T11:30:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9376c82640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f937015ee60 0x7f93700d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9376c82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d2d80 0x7f93700d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9376c82640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937015f240 con 0x7f93700d3770 2024-09-10T11:30:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9376c82640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f93700c87c0 con 0x7f937015ee60 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9376c82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93700d0430 con 0x7f93700d2d80 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f937015ee60 0x7f93700d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f937015ee60 0x7f93700d2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:36400/0 (socket says 172.21.15.99:36400) 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9367fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d2d80 0x7f93700d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f937521f640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f93700d3770 0x7f93700d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41084/0 (socket says 172.21.15.99:41084) 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 -- 172.21.15.99:0/289725727 learned_addr learned my addr 172.21.15.99:0/289725727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 -- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d2d80 msgr2=0x7f93700d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 --2- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d2d80 0x7f93700d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 -- 172.21.15.99:0/289725727 >> v1:172.21.15.196:6789/0 conn(0x7f93700d3770 legacy=0x7f93700d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f9374a1e640 1 -- 172.21.15.99:0/289725727 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93700d0710 con 0x7f937015ee60 2024-09-10T11:30:24.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.010+0000 7f93677fe640 1 -- 172.21.15.99:0/289725727 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197258953 0 0) 0x7f937015f240 con 0x7f93700d3770 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.011+0000 7f9374a1e640 1 --2- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700d2840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5403be25b363c5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.011+0000 7f93677fe640 1 -- 172.21.15.99:0/289725727 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c037070 con 0x7f937015ee60 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.011+0000 7f93677fe640 1 -- 172.21.15.99:0/289725727 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c032440 con 0x7f937015ee60 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.011+0000 7f93677fe640 1 -- 172.21.15.99:0/289725727 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c032810 con 0x7f937015ee60 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 -- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 msgr2=0x7f93700d2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 --2- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700d2840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c02a270 comp rx=0 tx=0).stop 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 -- 172.21.15.99:0/289725727 shutdown_connections 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 --2- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d2d80 0x7f93700d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 --2- 172.21.15.99:0/289725727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700d2840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 -- 172.21.15.99:0/289725727 >> 172.21.15.99:0/289725727 conn(0x7f93700d4c70 msgr2=0x7f93700d5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 -- 172.21.15.99:0/289725727 shutdown_connections 2024-09-10T11:30:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.012+0000 7f9376c82640 1 -- 172.21.15.99:0/289725727 wait complete. 2024-09-10T11:30:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.013+0000 7f9376c82640 1 Processor -- start 2024-09-10T11:30:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.013+0000 7f9376c82640 1 -- start start 2024-09-10T11:30:24.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d3770 0x7f93700b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93700d5a40 con 0x7f93700d2d80 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93700c8a10 con 0x7f937015ee60 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9376c82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93700c9f40 con 0x7f93700d3770 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56812/0 (socket says 172.21.15.99:56812) 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 -- 172.21.15.99:0/1832555069 learned_addr learned my addr 172.21.15.99:0/1832555069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9367fff640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d3770 0x7f93700b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f937521f640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 -- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d3770 msgr2=0x7f93700b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d3770 0x7f93700b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 -- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 msgr2=0x7f93700cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f9374a1e640 1 -- 172.21.15.99:0/1832555069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93700cd760 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.014+0000 7f937521f640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f9374a1e640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c037040 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936c004030 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93700b8bb0 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004400 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.015+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93700b9170 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.016+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93700d3230 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.016+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f936c03a070 con 0x7f93700d2d80 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.016+0000 7f93657fa640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 0x7f935c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.017+0000 7f9367fff640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 0x7f935c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.017+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f936c0aaf80 con 0x7f93700d2d80 2024-09-10T11:30:24.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.017+0000 7f9367fff640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 0x7f935c06e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f937015e500 tx=0x7f9360009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.021+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f936c041020 con 0x7f93700d2d80 2024-09-10T11:30:24.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.186+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9370068440 con 0x7f93700d2d80 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.187+0000 7f93657fa640 1 -- 172.21.15.99:0/1832555069 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f936c079630 con 0x7f93700d2d80 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:30:24.188 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 msgr2=0x7f935c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 0x7f935c06e2f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f937015e500 tx=0x7f9360009190 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 msgr2=0x7f9370150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f936c009870 tx=0x7f936c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 shutdown_connections 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f935c06be30 0x7f935c06e2f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f937015ee60 0x7f93700cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93700d3770 0x7f93700b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 --2- 172.21.15.99:0/1832555069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93700d2d80 0x7f9370150130 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 >> 172.21.15.99:0/1832555069 conn(0x7f93700d4c70 msgr2=0x7f93700cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:24.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 shutdown_connections 2024-09-10T11:30:24.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.189+0000 7f9376c82640 1 -- 172.21.15.99:0/1832555069 wait complete. 2024-09-10T11:30:24.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:30:24.200 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:30:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 Processor -- start 2024-09-10T11:30:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 -- start start 2024-09-10T11:30:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24a8146490 0x7f24a80b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24a8070610 0x7f24a80709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a8070f30 con 0x7f24a8073300 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f24a80b5d70 con 0x7f24a8070610 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.302+0000 7f24aeb81640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24a80b6050 con 0x7f24a8146490 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24a8070610 0x7f24a80709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24a8070610 0x7f24a80709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56824/0 (socket says 172.21.15.99:56824) 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 -- 172.21.15.99:0/1753835640 learned_addr learned my addr 172.21.15.99:0/1753835640 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ac91d640 1 --2- 172.21.15.99:0/1753835640 >> v2:172.21.15.196:3300/0 conn(0x7f24a8146490 0x7f24a80b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 -- 172.21.15.99:0/1753835640 >> v2:172.21.15.196:3300/0 conn(0x7f24a8146490 msgr2=0x7f24a80b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 --2- 172.21.15.99:0/1753835640 >> v2:172.21.15.196:3300/0 conn(0x7f24a8146490 0x7f24a80b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 -- 172.21.15.99:0/1753835640 >> v1:172.21.15.196:6789/0 conn(0x7f24a8073300 legacy=0x7f24a80d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.303+0000 7f24ad11e640 1 -- 172.21.15.99:0/1753835640 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80b6330 con 0x7f24a8070610 2024-09-10T11:30:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24ad11e640 1 --2- 172.21.15.99:0/1753835640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8070610 0x7f24a80709f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a806cc20 tx=0x7f24a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e85ec693ab26f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24a77fe640 1 -- 172.21.15.99:0/1753835640 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0037070 con 0x7f24a8070610 2024-09-10T11:30:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24a77fe640 1 -- 172.21.15.99:0/1753835640 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a0032440 con 0x7f24a8070610 2024-09-10T11:30:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24a77fe640 1 -- 172.21.15.99:0/1753835640 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0032810 con 0x7f24a8070610 2024-09-10T11:30:24.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 -- 172.21.15.99:0/1753835640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8070610 msgr2=0x7f24a80709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 --2- 172.21.15.99:0/1753835640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8070610 0x7f24a80709f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f24a806cc20 tx=0x7f24a002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 -- 172.21.15.99:0/1753835640 shutdown_connections 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 --2- 172.21.15.99:0/1753835640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8070610 0x7f24a80709f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 --2- 172.21.15.99:0/1753835640 >> v2:172.21.15.196:3300/0 conn(0x7f24a8146490 0x7f24a80b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 -- 172.21.15.99:0/1753835640 >> 172.21.15.99:0/1753835640 conn(0x7f24a8072680 msgr2=0x7f24a8072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 -- 172.21.15.99:0/1753835640 shutdown_connections 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.304+0000 7f24aeb81640 1 -- 172.21.15.99:0/1753835640 wait complete. 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.305+0000 7f24aeb81640 1 Processor -- start 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 -- start start 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8070610 0x7f24a81449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24a8073300 0x7f24a80c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24a815d3e0 con 0x7f24a8146490 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24a815d260 con 0x7f24a8070610 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.306+0000 7f24aeb81640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24a815d560 con 0x7f24a8073300 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56836/0 (socket says 172.21.15.99:56836) 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 -- 172.21.15.99:0/3280654430 learned_addr learned my addr 172.21.15.99:0/3280654430 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ac91d640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8070610 0x7f24a81449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24a7fff640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24a8073300 0x7f24a80c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 -- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24a8073300 msgr2=0x7f24a80c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24a8073300 0x7f24a80c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 -- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8070610 msgr2=0x7f24a81449c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8070610 0x7f24a81449c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 -- 172.21.15.99:0/3280654430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a80c9810 con 0x7f24a8146490 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24ad11e640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24a002a220 tx=0x7f24a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0041020 con 0x7f24a8146490 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a003d640 con 0x7f24a8146490 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0037040 con 0x7f24a8146490 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a80c9af0 con 0x7f24a8146490 2024-09-10T11:30:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.307+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a80adec0 con 0x7f24a8146490 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.308+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a80709f0 con 0x7f24a8146490 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.309+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24a0044020 con 0x7f24a8146490 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.309+0000 7f24a57fa640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.309+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f24a00ab0b0 con 0x7f24a8146490 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.309+0000 7f24ac91d640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.310+0000 7f24ac91d640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 0x7f248806e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f24a80afa10 tx=0x7f249800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:24.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.313+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24a004b030 con 0x7f24a8146490 2024-09-10T11:30:24.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:24.479+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f24a80b6e20 con 0x7f24a8146490 2024-09-10T11:30:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.335+0000 7f24a57fa640 1 -- 172.21.15.99:0/3280654430 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f24a0079770 con 0x7f24a8146490 2024-09-10T11:30:25.337 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:30:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 msgr2=0x7f248806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 0x7f248806e2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f24a80afa10 tx=0x7f249800b040 comp rx=0 tx=0).stop 2024-09-10T11:30:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 msgr2=0x7f24a80c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24a002a220 tx=0x7f24a00046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 shutdown_connections 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f248806be30 0x7f248806e2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a8146490 0x7f24a80c70b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24a8073300 0x7f24a80c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 --2- 172.21.15.99:0/3280654430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24a8070610 0x7f24a81449c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 >> 172.21.15.99:0/3280654430 conn(0x7f24a8072680 msgr2=0x7f24a815d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 shutdown_connections 2024-09-10T11:30:25.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.340+0000 7f24aeb81640 1 -- 172.21.15.99:0/3280654430 wait complete. 2024-09-10T11:30:25.356 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:30:25.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.466+0000 7f1d7fae4640 1 Processor -- start 2024-09-10T11:30:25.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.466+0000 7f1d7fae4640 1 -- start start 2024-09-10T11:30:25.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7fae4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d78160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7fae4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d780b5140 con 0x7f1d7806bc10 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7fae4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d78162b40 con 0x7f1d780b4930 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7fae4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d780b5420 con 0x7f1d78163620 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7d880640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1d7806bc10 0x7f1d7806bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33036/0 (socket says 172.21.15.99:33036) 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7d880640 1 -- 172.21.15.99:0/126914434 learned_addr learned my addr 172.21.15.99:0/126914434 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7d07f640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d78160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3685083120 0 0) 0x7f1d780b5140 con 0x7f1d7806bc10 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d64003520 con 0x7f1d7806bc10 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3787962130 0 0) 0x7f1d64003520 con 0x7f1d7806bc10 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 msgr2=0x7f1d78160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:25.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d78160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 >> v1:172.21.15.196:6789/0 conn(0x7f1d780b4930 legacy=0x7f1d7806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d78162e20 con 0x7f1d7806bc10 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.467+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1d74001670 con 0x7f1d7806bc10 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 >> v1:172.21.15.99:6789/0 conn(0x7f1d7806bc10 legacy=0x7f1d7806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d640039e0 0x7f1d64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d64004d40 0x7f1d640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780b5420 con 0x7f1d64004d40 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6400ba70 con 0x7f1d640039e0 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6400bc70 con 0x7f1d78163620 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d64004d40 0x7f1d640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7d880640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d640039e0 0x7f1d64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7d07f640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 -- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 msgr2=0x7f1d64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d78163620 0x7f1d64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 -- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d640039e0 msgr2=0x7f1d64003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d640039e0 0x7f1d64003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.468+0000 7f1d7e081640 1 -- 172.21.15.99:0/126914434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d64003520 con 0x7f1d64004d40 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7e081640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d64004d40 0x7f1d640051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002010 tx=0x7f1d64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dcdb3f3e099d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70036070 con 0x7f1d64004d40 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d70031600 con 0x7f1d64004d40 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7c87e640 1 -- 172.21.15.99:0/126914434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70001be0 con 0x7f1d64004d40 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 -- 172.21.15.99:0/126914434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d64004d40 msgr2=0x7f1d640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/126914434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d64004d40 0x7f1d640051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002010 tx=0x7f1d64005830 comp rx=0 tx=0).stop 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7e081640 1 -- 172.21.15.99:0/126914434 reap_dead start 2024-09-10T11:30:25.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 -- 172.21.15.99:0/126914434 shutdown_connections 2024-09-10T11:30:25.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 -- 172.21.15.99:0/126914434 >> 172.21.15.99:0/126914434 conn(0x7f1d780b3630 msgr2=0x7f1d780b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:25.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 -- 172.21.15.99:0/126914434 shutdown_connections 2024-09-10T11:30:25.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.469+0000 7f1d7fae4640 1 -- 172.21.15.99:0/126914434 wait complete. 2024-09-10T11:30:25.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 Processor -- start 2024-09-10T11:30:25.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 -- start start 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 0x7f1d781f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d781407b0 0x7f1d7815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780b4d90 con 0x7f1d78140c10 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780d1560 con 0x7f1d64004d40 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7fae4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7813e460 con 0x7f1d781407b0 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7e081640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7e081640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56852/0 (socket says 172.21.15.99:56852) 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7e081640 1 -- 172.21.15.99:0/3425244903 learned_addr learned my addr 172.21.15.99:0/3425244903 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.471+0000 7f1d7d07f640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d781407b0 0x7f1d7815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7d880640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 0x7f1d781f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 -- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d781407b0 msgr2=0x7f1d7815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d781407b0 0x7f1d7815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 -- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 msgr2=0x7f1d781f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 0x7f1d781f5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 -- 172.21.15.99:0/3425244903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7813e740 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7d880640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 0x7f1d781f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7e081640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002010 tx=0x7f1d70001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70042070 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d70003970 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7813ea20 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70036040 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.472+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7813efe0 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.473+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d78162b40 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.473+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d70045070 con 0x7f1d78140c10 2024-09-10T11:30:25.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.474+0000 7f1d6e7fc640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 0x7f1d5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:25.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.474+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f1d700aa4b0 con 0x7f1d78140c10 2024-09-10T11:30:25.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.474+0000 7f1d7d880640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 0x7f1d5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:25.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.474+0000 7f1d7d880640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 0x7f1d5c06e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d74000c00 tx=0x7f1d74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:25.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.478+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d700789f0 con 0x7f1d78140c10 2024-09-10T11:30:25.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:25.647+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1d78073d40 con 0x7f1d78140c10 2024-09-10T11:30:26.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.339+0000 7f1d6e7fc640 1 -- 172.21.15.99:0/3425244903 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f1d7007ccf0 con 0x7f1d78140c10 2024-09-10T11:30:26.341 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:30:26.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.346+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 msgr2=0x7f1d5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.346+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 0x7f1d5c06e320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1d74000c00 tx=0x7f1d74009040 comp rx=0 tx=0).stop 2024-09-10T11:30:26.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.346+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 msgr2=0x7f1d7815f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.346+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d70002010 tx=0x7f1d70001630 comp rx=0 tx=0).stop 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 shutdown_connections 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d5c06be60 0x7f1d5c06e320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d78140c10 0x7f1d7815f2a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d781407b0 0x7f1d7815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 --2- 172.21.15.99:0/3425244903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d64004d40 0x7f1d781f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.348+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 >> 172.21.15.99:0/3425244903 conn(0x7f1d780b3630 msgr2=0x7f1d7806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:26.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.350+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 shutdown_connections 2024-09-10T11:30:26.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.350+0000 7f1d7fae4640 1 -- 172.21.15.99:0/3425244903 wait complete. 2024-09-10T11:30:26.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:30:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.469+0000 7f958b4b7640 1 Processor -- start 2024-09-10T11:30:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 -- start start 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95840ce0e0 0x7f95840d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 0x7f9584070860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95840ce6c0 con 0x7f95841603c0 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f95840ce9a0 con 0x7f95840ce0e0 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.470+0000 7f958b4b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f958406f910 con 0x7f95840703f0 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9588a52640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95840ce0e0 0x7f95840d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9589a54640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 0x7f9584070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9589253640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f95841603c0 0x7f95840d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41098/0 (socket says 172.21.15.99:41098) 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9589253640 1 -- 172.21.15.99:0/2445097719 learned_addr learned my addr 172.21.15.99:0/2445097719 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9588a52640 1 -- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 msgr2=0x7f9584070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9588a52640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 0x7f9584070860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9588a52640 1 -- 172.21.15.99:0/2445097719 >> v1:172.21.15.196:6789/0 conn(0x7f95841603c0 legacy=0x7f95840d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9588a52640 1 -- 172.21.15.99:0/2445097719 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958406fbf0 con 0x7f95840ce0e0 2024-09-10T11:30:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.471+0000 7f9589a54640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 0x7f9584070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f9588a52640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95840ce0e0 0x7f95840d11c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95740097a0 tx=0x7f957402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaa9bdf331480415 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f957bfff640 1 -- 172.21.15.99:0/2445097719 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574037070 con 0x7f95840ce0e0 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f957bfff640 1 -- 172.21.15.99:0/2445097719 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957402adb0 con 0x7f95840ce0e0 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f957bfff640 1 -- 172.21.15.99:0/2445097719 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574032660 con 0x7f95840ce0e0 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 -- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95840ce0e0 msgr2=0x7f95840d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95840ce0e0 0x7f95840d11c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95740097a0 tx=0x7f957402a060 comp rx=0 tx=0).stop 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 -- 172.21.15.99:0/2445097719 shutdown_connections 2024-09-10T11:30:26.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840703f0 0x7f9584070860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 --2- 172.21.15.99:0/2445097719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95840ce0e0 0x7f95840d11c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.472+0000 7f958b4b7640 1 -- 172.21.15.99:0/2445097719 >> 172.21.15.99:0/2445097719 conn(0x7f95840b45c0 msgr2=0x7f95840b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:26.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.473+0000 7f958b4b7640 1 -- 172.21.15.99:0/2445097719 shutdown_connections 2024-09-10T11:30:26.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.473+0000 7f958b4b7640 1 -- 172.21.15.99:0/2445097719 wait complete. 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 Processor -- start 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 -- start start 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840ce0e0 0x7f95840b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95841603c0 0x7f95840b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d1880 con 0x7f95840703f0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d1700 con 0x7f95841603c0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.474+0000 7f958b4b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9584070da0 con 0x7f95840ce0e0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56864/0 (socket says 172.21.15.99:56864) 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 -- 172.21.15.99:0/3493427727 learned_addr learned my addr 172.21.15.99:0/3493427727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9588a52640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840ce0e0 0x7f95840b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589a54640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95841603c0 0x7f95840b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 -- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840ce0e0 msgr2=0x7f95840b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840ce0e0 0x7f95840b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 -- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95841603c0 msgr2=0x7f95840b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95841603c0 0x7f95840b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:26.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 -- 172.21.15.99:0/3493427727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840b5960 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9589253640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f958000c3c0 tx=0x7f958000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580017070 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9580005ce0 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95800072f0 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95840b7f20 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.475+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95840b84e0 con 0x7f95840703f0 2024-09-10T11:30:26.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.476+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9584070860 con 0x7f95840703f0 2024-09-10T11:30:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.477+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f958001a070 con 0x7f95840703f0 2024-09-10T11:30:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.477+0000 7f9579ffb640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 0x7f956806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.477+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f958008a070 con 0x7f95840703f0 2024-09-10T11:30:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.477+0000 7f9588a52640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 0x7f956806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:26.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.478+0000 7f9588a52640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 0x7f956806e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9574034000 tx=0x7f957402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:26.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.482+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95800583b0 con 0x7f95840703f0 2024-09-10T11:30:26.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:26.681+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f95840b5d60 con 0x7f95840703f0 2024-09-10T11:30:27.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.363+0000 7f9579ffb640 1 -- 172.21.15.99:0/3493427727 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f958005c6b0 con 0x7f95840703f0 2024-09-10T11:30:27.367 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T11:30:27.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.371+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 msgr2=0x7f956806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.371+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 0x7f956806e2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9574034000 tx=0x7f957402e000 comp rx=0 tx=0).stop 2024-09-10T11:30:27.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.371+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 msgr2=0x7f95840b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.371+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f958000c3c0 tx=0x7f958000c7c0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 shutdown_connections 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f956806be30 0x7f956806e2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95841603c0 0x7f95840b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95840ce0e0 0x7f95840b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 --2- 172.21.15.99:0/3493427727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95840703f0 0x7f95840b36e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.372+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 >> 172.21.15.99:0/3493427727 conn(0x7f95840b45c0 msgr2=0x7f95840b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:27.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.373+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 shutdown_connections 2024-09-10T11:30:27.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.373+0000 7f958b4b7640 1 -- 172.21.15.99:0/3493427727 wait complete. 2024-09-10T11:30:27.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:27.444 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:30:27.446 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:30:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.491+0000 7f4a24fef640 1 Processor -- start 2024-09-10T11:30:27.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.492+0000 7f4a24fef640 1 -- start start 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.492+0000 7f4a24fef640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a200d3a30 0x7f4a200d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.492+0000 7f4a24fef640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a200d2990 con 0x7f4a2015efc0 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.492+0000 7f4a24fef640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a200d2c70 con 0x7f4a200d1ae0 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.492+0000 7f4a24fef640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a200d2f50 con 0x7f4a200d3a30 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1e59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4a2015efc0 0x7f4a200d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33038/0 (socket says 172.21.15.99:33038) 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1e59c640 1 -- 172.21.15.99:0/273492199 learned_addr learned my addr 172.21.15.99:0/273492199 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/273492199 >> v2:172.21.15.196:3300/0 conn(0x7f4a200d3a30 0x7f4a200d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782431358 0 0) 0x7f4a200d2990 con 0x7f4a2015efc0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a08003520 con 0x7f4a2015efc0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3208322417 0 0) 0x7f4a200d2c70 con 0x7f4a200d1ae0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a200d2990 con 0x7f4a200d1ae0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1667370702 0 0) 0x7f4a08003520 con 0x7f4a2015efc0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 >> v2:172.21.15.196:3300/0 conn(0x7f4a200d3a30 msgr2=0x7f4a200d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 --2- 172.21.15.99:0/273492199 >> v2:172.21.15.196:3300/0 conn(0x7f4a200d3a30 0x7f4a200d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 >> v1:172.21.15.196:6789/0 conn(0x7f4a200d1ae0 legacy=0x7f4a200b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a200d3230 con 0x7f4a2015efc0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4a10001670 con 0x7f4a2015efc0 2024-09-10T11:30:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 >> v1:172.21.15.99:6789/0 conn(0x7f4a2015efc0 legacy=0x7f4a200d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a08003b40 0x7f4a08003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a080051c0 0x7f4a0800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a200d2f50 con 0x7f4a08004800 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0800c040 con 0x7f4a080051c0 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0800c240 con 0x7f4a08003b40 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1ed9d640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a080051c0 0x7f4a0800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1e59c640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a08003b40 0x7f4a08003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 -- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a08003b40 msgr2=0x7f4a08003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a08003b40 0x7f4a08003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 -- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a080051c0 msgr2=0x7f4a0800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a080051c0 0x7f4a0800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1dd9b640 1 -- 172.21.15.99:0/273492199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a08003520 con 0x7f4a08004800 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1ed9d640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a080051c0 0x7f4a0800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.493+0000 7f4a1ed9d640 1 -- 172.21.15.99:0/273492199 reap_dead start 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.494+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a08004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009a90 tx=0x7f4a08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5950139dcb30e7e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.494+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14036070 con 0x7f4a08004800 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.494+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a14031400 con 0x7f4a08004800 2024-09-10T11:30:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.494+0000 7f4a1d59a640 1 -- 172.21.15.99:0/273492199 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14009690 con 0x7f4a08004800 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 -- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 msgr2=0x7f4a08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a08004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009a90 tx=0x7f4a08005830 comp rx=0 tx=0).stop 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 -- 172.21.15.99:0/273492199 shutdown_connections 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 --2- 172.21.15.99:0/273492199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a08004c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 -- 172.21.15.99:0/273492199 >> 172.21.15.99:0/273492199 conn(0x7f4a200d4c20 msgr2=0x7f4a200d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 -- 172.21.15.99:0/273492199 shutdown_connections 2024-09-10T11:30:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.495+0000 7f4a24fef640 1 -- 172.21.15.99:0/273492199 wait complete. 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.496+0000 7f4a24fef640 1 Processor -- start 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.496+0000 7f4a24fef640 1 -- start start 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a200cc0e0 0x7f4a200ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 0x7f4a201f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a200d1930 con 0x7f4a08004800 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a200d17b0 con 0x7f4a201f88d0 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a24fef640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a200d03f0 con 0x7f4a200cc0e0 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56880/0 (socket says 172.21.15.99:56880) 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 -- 172.21.15.99:0/3504720167 learned_addr learned my addr 172.21.15.99:0/3504720167 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a200cc0e0 0x7f4a200ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1ed9d640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 0x7f4a201f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 -- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a200cc0e0 msgr2=0x7f4a200ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a200cc0e0 0x7f4a200ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 -- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 msgr2=0x7f4a201f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 0x7f4a201f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 -- 172.21.15.99:0/3504720167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a20156090 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1ed9d640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 0x7f4a201f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.497+0000 7f4a1e59c640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e7f0 tx=0x7f4a1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.498+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000c8d0 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.498+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a10018070 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.498+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a20156370 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.498+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000cc70 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.498+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a20156930 con 0x7f4a08004800 2024-09-10T11:30:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.499+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a200d07e0 con 0x7f4a08004800 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.499+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a1001d460 con 0x7f4a08004800 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.500+0000 7f4a0effd640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 0x7f4a0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.500+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 0x7f4a0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.500+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f4a1008aca0 con 0x7f4a08004800 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.500+0000 7f4a1dd9b640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 0x7f4a0006e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a14033000 tx=0x7f4a14008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.504+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a100596e0 con 0x7f4a08004800 2024-09-10T11:30:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.672+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a2008ec70 con 0x7f4a08004800 2024-09-10T11:30:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.673+0000 7f4a0effd640 1 -- 172.21.15.99:0/3504720167 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16682 (secure 0 0 0) 0x7f4a10058fe0 con 0x7f4a08004800 2024-09-10T11:30:27.675 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:27.675 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:26.683785+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:27.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 msgr2=0x7f4a0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 0x7f4a0006e2f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a14033000 tx=0x7f4a14008030 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 msgr2=0x7f4a20156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e7f0 tx=0x7f4a1000ecb0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 shutdown_connections 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a0006be30 0x7f4a0006e2f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a201f88d0 0x7f4a201f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a200cc0e0 0x7f4a200ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 --2- 172.21.15.99:0/3504720167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a08004800 0x7f4a20156e50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 >> 172.21.15.99:0/3504720167 conn(0x7f4a200d4c20 msgr2=0x7f4a200d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 shutdown_connections 2024-09-10T11:30:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.675+0000 7f4a24fef640 1 -- 172.21.15.99:0/3504720167 wait complete. 2024-09-10T11:30:27.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.785+0000 7ff00e68d640 1 Processor -- start 2024-09-10T11:30:27.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.785+0000 7ff00e68d640 1 -- start start 2024-09-10T11:30:27.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff00e68d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0080cee30 0x7ff0080d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff00e68d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0080cf240 con 0x7ff00815cc30 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff00e68d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0080cea00 con 0x7ff0080cf410 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff00e68d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff0080d1540 con 0x7ff0080cee30 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff00cc2a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0080cee30 0x7ff0080d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff0077fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff00815cc30 0x7ff0080ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33054/0 (socket says 172.21.15.99:33054) 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.786+0000 7ff0077fe640 1 -- 172.21.15.99:0/3379300329 learned_addr learned my addr 172.21.15.99:0/3379300329 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.787+0000 7ff00cc2a640 1 -- 172.21.15.99:0/3379300329 >> v1:172.21.15.196:6789/0 conn(0x7ff0080cf410 legacy=0x7ff0080cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.787+0000 7ff006ffd640 1 -- 172.21.15.99:0/3379300329 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455917488 0 0) 0x7ff0080cf240 con 0x7ff00815cc30 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.787+0000 7ff00cc2a640 1 -- 172.21.15.99:0/3379300329 >> v1:172.21.15.99:6789/0 conn(0x7ff00815cc30 legacy=0x7ff0080ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.787+0000 7ff00cc2a640 1 -- 172.21.15.99:0/3379300329 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080d1770 con 0x7ff0080cee30 2024-09-10T11:30:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.787+0000 7ff00cc2a640 1 --2- 172.21.15.99:0/3379300329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cee30 0x7ff0080d1000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff000009d90 tx=0x7ff00002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb89bfc8332b20c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff006ffd640 1 -- 172.21.15.99:0/3379300329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000031600 con 0x7ff0080cee30 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff006ffd640 1 -- 172.21.15.99:0/3379300329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0000317a0 con 0x7ff0080cee30 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff006ffd640 1 -- 172.21.15.99:0/3379300329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000031b70 con 0x7ff0080cee30 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 -- 172.21.15.99:0/3379300329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cee30 msgr2=0x7ff0080d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 --2- 172.21.15.99:0/3379300329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cee30 0x7ff0080d1000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff000009d90 tx=0x7ff00002a5d0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 -- 172.21.15.99:0/3379300329 shutdown_connections 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 --2- 172.21.15.99:0/3379300329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cee30 0x7ff0080d1000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 -- 172.21.15.99:0/3379300329 >> 172.21.15.99:0/3379300329 conn(0x7ff0080b4570 msgr2=0x7ff0080b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 -- 172.21.15.99:0/3379300329 shutdown_connections 2024-09-10T11:30:27.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.788+0000 7ff00e68d640 1 -- 172.21.15.99:0/3379300329 wait complete. 2024-09-10T11:30:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.789+0000 7ff00e68d640 1 Processor -- start 2024-09-10T11:30:27.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.789+0000 7ff00e68d640 1 -- start start 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 0x7ff0080cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff00815cc30 0x7ff0081f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080cd4c0 con 0x7ff0080cf410 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0080cdaf0 con 0x7ff0080cee30 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff00e68d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0080b5870 con 0x7ff00815cc30 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff0077fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56886/0 (socket says 172.21.15.99:56886) 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.790+0000 7ff0077fe640 1 -- 172.21.15.99:0/307280775 learned_addr learned my addr 172.21.15.99:0/307280775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff00cc2a640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff00815cc30 0x7ff0081f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff007fff640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 0x7ff0080cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 -- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff00815cc30 msgr2=0x7ff0081f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff00815cc30 0x7ff0081f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 -- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 msgr2=0x7ff0080cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 0x7ff0080cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 -- 172.21.15.99:0/307280775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0080ccc40 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff007fff640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 0x7ff0080cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff0077fe640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c390 tx=0x7feffc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc0074e0 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feffc0047e0 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feffc004bd0 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0080ca1e0 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.791+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff00815ff40 con 0x7ff0080cf410 2024-09-10T11:30:27.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.792+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0081e60e0 con 0x7ff0080cf410 2024-09-10T11:30:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.792+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feffc011070 con 0x7ff0080cf410 2024-09-10T11:30:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.793+0000 7ff004ff9640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 0x7feff406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.793+0000 7ff007fff640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 0x7feff406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.793+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7feffc08aa10 con 0x7ff0080cf410 2024-09-10T11:30:27.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.793+0000 7ff007fff640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 0x7feff406e320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feff8000f60 tx=0x7feff8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:27.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.797+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0081e60e0 con 0x7ff0080cf410 2024-09-10T11:30:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.965+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0080d1000 con 0x7ff0080cf410 2024-09-10T11:30:27.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.966+0000 7ff004ff9640 1 -- 172.21.15.99:0/307280775 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16682 (secure 0 0 0) 0x7feffc0593c0 con 0x7ff0080cf410 2024-09-10T11:30:27.968 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:27.968 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:26.683785+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 msgr2=0x7feff406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 0x7feff406e320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feff8000f60 tx=0x7feff8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 msgr2=0x7ff0081425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c390 tx=0x7feffc00c850 comp rx=0 tx=0).stop 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 shutdown_connections 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feff406be60 0x7feff406e320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff00815cc30 0x7ff0081f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0080cf410 0x7ff0081425d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 --2- 172.21.15.99:0/307280775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0080cee30 0x7ff0080cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:27.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 >> 172.21.15.99:0/307280775 conn(0x7ff0080b4570 msgr2=0x7ff0080b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:27.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 shutdown_connections 2024-09-10T11:30:27.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:27.968+0000 7ff00e68d640 1 -- 172.21.15.99:0/307280775 wait complete. 2024-09-10T11:30:27.980 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:30:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.121+0000 7f97b278d640 1 Processor -- start 2024-09-10T11:30:28.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.121+0000 7f97b278d640 1 -- start start 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b278d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97ac163580 0x7f97ac0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b278d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f97ac0b3160 0x7f97ac06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b278d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97ac0b3620 con 0x7f97ac0b3960 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b278d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac06d960 con 0x7f97ac163580 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b278d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f97ac06dc40 con 0x7f97ac0b3160 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97ac163580 0x7f97ac0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97ac163580 0x7f97ac0b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56892/0 (socket says 172.21.15.99:56892) 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 -- 172.21.15.99:0/3533836991 learned_addr learned my addr 172.21.15.99:0/3533836991 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97b0d2a640 1 --2- 172.21.15.99:0/3533836991 >> v2:172.21.15.196:3300/0 conn(0x7f97ac0b3160 0x7f97ac06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 -- 172.21.15.99:0/3533836991 >> v2:172.21.15.196:3300/0 conn(0x7f97ac0b3160 msgr2=0x7f97ac06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 --2- 172.21.15.99:0/3533836991 >> v2:172.21.15.196:3300/0 conn(0x7f97ac0b3160 0x7f97ac06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 -- 172.21.15.99:0/3533836991 >> v1:172.21.15.196:6789/0 conn(0x7f97ac0b3960 legacy=0x7f97ac0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.122+0000 7f97abfff640 1 -- 172.21.15.99:0/3533836991 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac06df20 con 0x7f97ac163580 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.123+0000 7f97abfff640 1 --2- 172.21.15.99:0/3533836991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac0b5090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97ac073250 tx=0x7f979c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b123534e47f46ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.123+0000 7f97aaffd640 1 -- 172.21.15.99:0/3533836991 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c037070 con 0x7f97ac163580 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.123+0000 7f97aaffd640 1 -- 172.21.15.99:0/3533836991 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979c032440 con 0x7f97ac163580 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.123+0000 7f97aaffd640 1 -- 172.21.15.99:0/3533836991 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c032810 con 0x7f97ac163580 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 -- 172.21.15.99:0/3533836991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 msgr2=0x7f97ac0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 --2- 172.21.15.99:0/3533836991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac0b5090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f97ac073250 tx=0x7f979c02a150 comp rx=0 tx=0).stop 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 -- 172.21.15.99:0/3533836991 shutdown_connections 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 --2- 172.21.15.99:0/3533836991 >> v2:172.21.15.196:3300/0 conn(0x7f97ac0b3160 0x7f97ac06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 --2- 172.21.15.99:0/3533836991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac0b5090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 -- 172.21.15.99:0/3533836991 >> 172.21.15.99:0/3533836991 conn(0x7f97ac0b4570 msgr2=0x7f97ac0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:28.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 -- 172.21.15.99:0/3533836991 shutdown_connections 2024-09-10T11:30:28.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.124+0000 7f97b278d640 1 -- 172.21.15.99:0/3533836991 wait complete. 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.125+0000 7f97b278d640 1 Processor -- start 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.125+0000 7f97b278d640 1 -- start start 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac0b3160 0x7f97ac0cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97ac0b3960 0x7f97ac0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac06d3e0 con 0x7f97ac163580 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac06d260 con 0x7f97ac0b3160 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b278d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97ac06d560 con 0x7f97ac0b3960 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56904/0 (socket says 172.21.15.99:56904) 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 -- 172.21.15.99:0/1646681573 learned_addr learned my addr 172.21.15.99:0/1646681573 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:28.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97abfff640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac0b3160 0x7f97ac0cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97ab7fe640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97ac0b3960 0x7f97ac0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 -- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97ac0b3960 msgr2=0x7f97ac0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97ac0b3960 0x7f97ac0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 -- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac0b3160 msgr2=0x7f97ac0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac0b3160 0x7f97ac0cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 -- 172.21.15.99:0/1646681573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac0cd7a0 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.126+0000 7f97b0d2a640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007e60 tx=0x7f97a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.127+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014420 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.127+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a4019070 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.127+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac0b71c0 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.127+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014820 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.127+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac0b7780 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.128+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac06cd20 con 0x7f97ac163580 2024-09-10T11:30:28.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.128+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97a40149c0 con 0x7f97ac163580 2024-09-10T11:30:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.128+0000 7f97a8ff9640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.129+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f97a408b450 con 0x7f97ac163580 2024-09-10T11:30:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.129+0000 7f97abfff640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.129+0000 7f97abfff640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 0x7f979806e320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.133+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97a4059840 con 0x7f97ac163580 2024-09-10T11:30:28.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.335+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f97ac0b89a0 con 0x7f97ac163580 2024-09-10T11:30:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.364+0000 7f97a8ff9640 1 -- 172.21.15.99:0/1646681573 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7f97a405db40 con 0x7f97ac163580 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 msgr2=0x7f979806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 0x7f979806e320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f979c034000 tx=0x7f979c02e000 comp rx=0 tx=0).stop 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 msgr2=0x7f97ac154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a4007e60 tx=0x7f97a400e4b0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 shutdown_connections 2024-09-10T11:30:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f979806be60 0x7f979806e320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97ac163580 0x7f97ac154e70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97ac0b3960 0x7f97ac0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 --2- 172.21.15.99:0/1646681573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac0b3160 0x7f97ac0cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 >> 172.21.15.99:0/1646681573 conn(0x7f97ac0b4570 msgr2=0x7f97ac0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 shutdown_connections 2024-09-10T11:30:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.366+0000 7f97b278d640 1 -- 172.21.15.99:0/1646681573 wait complete. 2024-09-10T11:30:28.379 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['disabled']} 2024-09-10T11:30:28.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:30:28.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.478+0000 7fa4fa1ed640 1 Processor -- start 2024-09-10T11:30:28.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.478+0000 7fa4fa1ed640 1 -- start start 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4fa1ed640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f40d1200 0x7fa4f4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4fa1ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa4f406e5d0 0x7fa4f406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4fa1ed640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f4152e60 con 0x7fa4f415c060 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4fa1ed640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f406ef80 con 0x7fa4f40d1200 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4fa1ed640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f40d15e0 con 0x7fa4f406e5d0 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f40d1200 0x7fa4f4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4f40d1200 0x7fa4f4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56914/0 (socket says 172.21.15.99:56914) 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 -- 172.21.15.99:0/534532409 learned_addr learned my addr 172.21.15.99:0/534532409 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:28.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f2ffd640 1 --2- 172.21.15.99:0/534532409 >> v2:172.21.15.196:3300/0 conn(0x7fa4f406e5d0 0x7fa4f406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 -- 172.21.15.99:0/534532409 >> v2:172.21.15.196:3300/0 conn(0x7fa4f406e5d0 msgr2=0x7fa4f406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 --2- 172.21.15.99:0/534532409 >> v2:172.21.15.196:3300/0 conn(0x7fa4f406e5d0 0x7fa4f406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 -- 172.21.15.99:0/534532409 >> v1:172.21.15.196:6789/0 conn(0x7fa4f415c060 legacy=0x7fa4f415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.479+0000 7fa4f37fe640 1 -- 172.21.15.99:0/534532409 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4072680 con 0x7fa4f40d1200 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.480+0000 7fa4f37fe640 1 --2- 172.21.15.99:0/534532409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f40d1200 0x7fa4f4073440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4071bb0 tx=0x7fa4e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eba481f14c20d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.480+0000 7fa4f27fc640 1 -- 172.21.15.99:0/534532409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4037070 con 0x7fa4f40d1200 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.480+0000 7fa4f27fc640 1 -- 172.21.15.99:0/534532409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e4032440 con 0x7fa4f40d1200 2024-09-10T11:30:28.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.480+0000 7fa4f27fc640 1 -- 172.21.15.99:0/534532409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4032810 con 0x7fa4f40d1200 2024-09-10T11:30:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/534532409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f40d1200 msgr2=0x7fa4f4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/534532409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f40d1200 0x7fa4f4073440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4071bb0 tx=0x7fa4e402a1e0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/534532409 shutdown_connections 2024-09-10T11:30:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/534532409 >> v2:172.21.15.196:3300/0 conn(0x7fa4f406e5d0 0x7fa4f406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/534532409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f40d1200 0x7fa4f4073440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/534532409 >> 172.21.15.99:0/534532409 conn(0x7fa4f406bc10 msgr2=0x7fa4f406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/534532409 shutdown_connections 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.481+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/534532409 wait complete. 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.482+0000 7fa4fa1ed640 1 Processor -- start 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.482+0000 7fa4fa1ed640 1 -- start start 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4f406e5d0 0x7fa4f40adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4f40d1200 0x7fa4f40c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4072f20 con 0x7fa4f415c060 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4072c40 con 0x7fa4f40d1200 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f415cd90 con 0x7fa4f406e5d0 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56924/0 (socket says 172.21.15.99:56924) 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 -- 172.21.15.99:0/1845367743 learned_addr learned my addr 172.21.15.99:0/1845367743 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f2ffd640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4f40d1200 0x7fa4f40c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f37fe640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4f406e5d0 0x7fa4f40adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 -- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4f406e5d0 msgr2=0x7fa4f40adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4f406e5d0 0x7fa4f40adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 -- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4f40d1200 msgr2=0x7fa4f40c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4f40d1200 0x7fa4f40c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 -- 172.21.15.99:0/1845367743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f4151920 con 0x7fa4f415c060 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4f3fff640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c7a0 tx=0x7fa4ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec003b70 con 0x7fa4f415c060 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec003d30 con 0x7fa4f415c060 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f4151c00 con 0x7fa4f415c060 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec00aa90 con 0x7fa4f415c060 2024-09-10T11:30:28.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.483+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41521c0 con 0x7fa4f415c060 2024-09-10T11:30:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.484+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f415ea90 con 0x7fa4f415c060 2024-09-10T11:30:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.485+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4ec00ac30 con 0x7fa4f415c060 2024-09-10T11:30:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.485+0000 7fa4dbfff640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.485+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4ec08abe0 con 0x7fa4f415c060 2024-09-10T11:30:28.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.485+0000 7fa4f37fe640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:28.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.486+0000 7fa4f37fe640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 0x7fa4e006e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4034000 tx=0x7fa4e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:28.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.489+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4f415ea90 con 0x7fa4f415c060 2024-09-10T11:30:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:28.688+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa4f415c490 con 0x7fa4f415c060 2024-09-10T11:30:29.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.369+0000 7fa4dbfff640 1 -- 172.21.15.99:0/1845367743 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fa4f415c490 con 0x7fa4f415c060 2024-09-10T11:30:29.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 msgr2=0x7fa4e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 0x7fa4e006e2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4034000 tx=0x7fa4e402e000 comp rx=0 tx=0).stop 2024-09-10T11:30:29.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 msgr2=0x7fa4f40c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00c7a0 tx=0x7fa4ec00cc60 comp rx=0 tx=0).stop 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 shutdown_connections 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4e006be30 0x7fa4e006e2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f415c060 0x7fa4f40c5cf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4f40d1200 0x7fa4f40c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 --2- 172.21.15.99:0/1845367743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4f406e5d0 0x7fa4f40adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 >> 172.21.15.99:0/1845367743 conn(0x7fa4f406bc10 msgr2=0x7fa4f40cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.371+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 shutdown_connections 2024-09-10T11:30:29.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.372+0000 7fa4fa1ed640 1 -- 172.21.15.99:0/1845367743 wait complete. 2024-09-10T11:30:29.385 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:30:29.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:29.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.483+0000 7fb2c44e6640 1 Processor -- start 2024-09-10T11:30:29.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.483+0000 7fb2c44e6640 1 -- start start 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c44e6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc0b3960 0x7fb2bc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c44e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 0x7fb2bc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c44e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2bc0b3620 con 0x7fb2bc163580 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c44e6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc06d960 con 0x7fb2bc0b3960 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c44e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc06dc40 con 0x7fb2bc0b3160 2024-09-10T11:30:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc0b3960 0x7fb2bc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc0b3960 0x7fb2bc0b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56926/0 (socket says 172.21.15.99:56926) 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 -- 172.21.15.99:0/1609417184 learned_addr learned my addr 172.21.15.99:0/1609417184 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c2a83640 1 --2- 172.21.15.99:0/1609417184 >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 0x7fb2bc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 -- 172.21.15.99:0/1609417184 >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 msgr2=0x7fb2bc06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 --2- 172.21.15.99:0/1609417184 >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 0x7fb2bc06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 -- 172.21.15.99:0/1609417184 >> v1:172.21.15.196:6789/0 conn(0x7fb2bc163580 legacy=0x7fb2bc0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c1a81640 1 -- 172.21.15.99:0/1609417184 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc06df20 con 0x7fb2bc0b3960 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.484+0000 7fb2c2a83640 1 --2- 172.21.15.99:0/1609417184 >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 0x7fb2bc06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c1a81640 1 --2- 172.21.15.99:0/1609417184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc0b5090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc073250 tx=0x7fb2ac02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26083c83b55972d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c1280640 1 -- 172.21.15.99:0/1609417184 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac037070 con 0x7fb2bc0b3960 2024-09-10T11:30:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c1280640 1 -- 172.21.15.99:0/1609417184 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac032440 con 0x7fb2bc0b3960 2024-09-10T11:30:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c1280640 1 -- 172.21.15.99:0/1609417184 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac032810 con 0x7fb2bc0b3960 2024-09-10T11:30:29.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 -- 172.21.15.99:0/1609417184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 msgr2=0x7fb2bc0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/1609417184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc0b5090 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc073250 tx=0x7fb2ac02a150 comp rx=0 tx=0).stop 2024-09-10T11:30:29.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 -- 172.21.15.99:0/1609417184 shutdown_connections 2024-09-10T11:30:29.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/1609417184 >> v2:172.21.15.196:3300/0 conn(0x7fb2bc0b3160 0x7fb2bc06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/1609417184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc0b5090 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.485+0000 7fb2c44e6640 1 -- 172.21.15.99:0/1609417184 >> 172.21.15.99:0/1609417184 conn(0x7fb2bc0b4570 msgr2=0x7fb2bc0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.486+0000 7fb2c44e6640 1 -- 172.21.15.99:0/1609417184 shutdown_connections 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.486+0000 7fb2c44e6640 1 -- 172.21.15.99:0/1609417184 wait complete. 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 Processor -- start 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 -- start start 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2bc0b3160 0x7fb2bc0b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2bc163580 0x7fb2bc0c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc06d3e0 con 0x7fb2bc0b3960 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc06d260 con 0x7fb2bc0b3160 2024-09-10T11:30:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.487+0000 7fb2c44e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc06d560 con 0x7fb2bc163580 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56936/0 (socket says 172.21.15.99:56936) 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 -- 172.21.15.99:0/754480652 learned_addr learned my addr 172.21.15.99:0/754480652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c2a83640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2bc163580 0x7fb2bc0c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c2282640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2bc0b3160 0x7fb2bc0b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 -- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2bc163580 msgr2=0x7fb2bc0c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2bc163580 0x7fb2bc0c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 -- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2bc0b3160 msgr2=0x7fb2bc0b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2bc0b3160 0x7fb2bc0b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 -- 172.21.15.99:0/754480652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc0c9880 con 0x7fb2bc0b3960 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c1a81640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac02a190 tx=0x7fb2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac041020 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0306c0 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc0c95a0 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac037040 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.488+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc0b7650 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.489+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc06cd20 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.490+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2ac044020 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.490+0000 7fb2b2ffd640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 0x7fb2a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.490+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb2ac0ab580 con 0x7fb2bc0b3960 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.490+0000 7fb2c2282640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 0x7fb2a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.491+0000 7fb2c2282640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 0x7fb2a006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8001780 tx=0x7fb2b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.495+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2ac04b030 con 0x7fb2bc0b3960 2024-09-10T11:30:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.692+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2bc0b89a0 con 0x7fb2bc0b3960 2024-09-10T11:30:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.692+0000 7fb2b2ffd640 1 -- 172.21.15.99:0/754480652 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7fb2ac0798e0 con 0x7fb2bc0b3960 2024-09-10T11:30:29.694 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:29.695 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:29.695 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:29.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 msgr2=0x7fb2a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 0x7fb2a006e2f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8001780 tx=0x7fb2b8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 msgr2=0x7fb2bc1505b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac02a190 tx=0x7fb2ac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 shutdown_connections 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2a006be30 0x7fb2a006e2f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2bc163580 0x7fb2bc0c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3960 0x7fb2bc1505b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 --2- 172.21.15.99:0/754480652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2bc0b3160 0x7fb2bc0b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 >> 172.21.15.99:0/754480652 conn(0x7fb2bc0b4570 msgr2=0x7fb2bc06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 shutdown_connections 2024-09-10T11:30:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.694+0000 7fb2c44e6640 1 -- 172.21.15.99:0/754480652 wait complete. 2024-09-10T11:30:29.707 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:29.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.803+0000 7f1314245640 1 Processor -- start 2024-09-10T11:30:29.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.803+0000 7f1314245640 1 -- start start 2024-09-10T11:30:29.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1314245640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f130c0b3960 0x7f130c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1314245640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c0b3160 0x7f130c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1314245640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f130c0b3620 con 0x7f130c163580 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1314245640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f130c06d960 con 0x7f130c0b3960 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1314245640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f130c06dc40 con 0x7f130c0b3160 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1311fe1640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f130c163580 0x7f130c0d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33062/0 (socket says 172.21.15.99:33062) 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1311fe1640 1 -- 172.21.15.99:0/681708943 learned_addr learned my addr 172.21.15.99:0/681708943 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123180945 0 0) 0x7f130c0b3620 con 0x7f130c163580 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12f8003520 con 0x7f130c163580 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.804+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c0b3160 0x7f130c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13117e0640 1 --2- 172.21.15.99:0/681708943 >> v2:172.21.15.196:3300/0 conn(0x7f130c0b3960 0x7f130c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3183970993 0 0) 0x7f12f8003520 con 0x7f130c163580 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c0b3160 msgr2=0x7f130c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:29.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c0b3160 0x7f130c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 >> v2:172.21.15.196:3300/0 conn(0x7f130c0b3960 msgr2=0x7f130c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 --2- 172.21.15.99:0/681708943 >> v2:172.21.15.196:3300/0 conn(0x7f130c0b3960 0x7f130c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c06df20 con 0x7f130c163580 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1308001670 con 0x7f130c163580 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13117e0640 1 --2- 172.21.15.99:0/681708943 >> v2:172.21.15.196:3300/0 conn(0x7f130c0b3960 0x7f130c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 >> v1:172.21.15.99:6789/0 conn(0x7f130c163580 legacy=0x7f130c0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f80039b0 0x7f12f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12f8004800 0x7f12f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f12f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06d960 con 0x7f12f80051c0 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06dc40 con 0x7f12f80039b0 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12f800c140 con 0x7f12f8004800 2024-09-10T11:30:29.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c0b3160 0x7f130c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1311fe1640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f80039b0 0x7f12f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f12f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13117e0640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12f8004800 0x7f12f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 -- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12f8004800 msgr2=0x7f12f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12f8004800 0x7f12f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 -- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f80039b0 msgr2=0x7f12f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f80039b0 0x7f12f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f13127e2640 1 -- 172.21.15.99:0/681708943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f8003520 con 0x7f12f80051c0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.805+0000 7f1311fe1640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12f80039b0 0x7f12f8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.806+0000 7f13127e2640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f12f800b9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1304009f00 tx=0x7f12f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd37a8c72d78b60b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.806+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304009b50 con 0x7f12f80051c0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.806+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1304003990 con 0x7f12f80051c0 2024-09-10T11:30:29.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.806+0000 7f1310fdf640 1 -- 172.21.15.99:0/681708943 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304003d60 con 0x7f12f80051c0 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 -- 172.21.15.99:0/681708943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 msgr2=0x7f12f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 --2- 172.21.15.99:0/681708943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f12f800b9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1304009f00 tx=0x7f12f8005830 comp rx=0 tx=0).stop 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f13127e2640 1 -- 172.21.15.99:0/681708943 reap_dead start 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 -- 172.21.15.99:0/681708943 shutdown_connections 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 -- 172.21.15.99:0/681708943 >> 172.21.15.99:0/681708943 conn(0x7f130c0b4570 msgr2=0x7f130c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 -- 172.21.15.99:0/681708943 shutdown_connections 2024-09-10T11:30:29.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.807+0000 7f1314245640 1 -- 172.21.15.99:0/681708943 wait complete. 2024-09-10T11:30:29.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.808+0000 7f1314245640 1 Processor -- start 2024-09-10T11:30:29.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.808+0000 7f1314245640 1 -- start start 2024-09-10T11:30:29.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c14e4d0 0x7f130c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f130c0b7ad0 0x7f130c0b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06cea0 con 0x7f12f80051c0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06cd20 con 0x7f130c0b7ad0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f130c06d020 con 0x7f130c14e4d0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56948/0 (socket says 172.21.15.99:56948) 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 -- 172.21.15.99:0/2019227405 learned_addr learned my addr 172.21.15.99:0/2019227405 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f13117e0640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c14e4d0 0x7f130c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f13127e2640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f130c0b7ad0 0x7f130c0b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 -- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c14e4d0 msgr2=0x7f130c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c14e4d0 0x7f130c14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 -- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f130c0b7ad0 msgr2=0x7f130c0b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f130c0b7ad0 0x7f130c0b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 -- 172.21.15.99:0/2019227405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c14b3f0 con 0x7f12f80051c0 2024-09-10T11:30:29.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1311fe1640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1308007340 tx=0x7f1308007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130801a070 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130800f920 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130800fd10 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c0b5a30 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.809+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c0b5ff0 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.810+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c0761a0 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.811+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f130801d070 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.811+0000 7f13027fc640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 0x7f12f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.811+0000 7f13117e0640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 0x7f12f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.811+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f130808a890 con 0x7f12f80051c0 2024-09-10T11:30:29.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.812+0000 7f13117e0640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 0x7f12f006e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f130c06d710 tx=0x7f12fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:29.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.815+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1308058bf0 con 0x7f12f80051c0 2024-09-10T11:30:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.982+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f130c06d400 con 0x7f12f80051c0 2024-09-10T11:30:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.983+0000 7f13027fc640 1 -- 172.21.15.99:0/2019227405 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7f130805cef0 con 0x7f12f80051c0 2024-09-10T11:30:29.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:29.985 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 msgr2=0x7f12f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 0x7f12f006e2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f130c06d710 tx=0x7f12fc008040 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 msgr2=0x7f130c06ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1308007340 tx=0x7f1308007800 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 shutdown_connections 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12f006be30 0x7f12f006e2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f130c0b7ad0 0x7f130c0b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f130c14e4d0 0x7f130c14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 --2- 172.21.15.99:0/2019227405 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12f80051c0 0x7f130c06ffc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 >> 172.21.15.99:0/2019227405 conn(0x7f130c0b4570 msgr2=0x7f130c15e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 shutdown_connections 2024-09-10T11:30:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:29.985+0000 7f1314245640 1 -- 172.21.15.99:0/2019227405 wait complete. 2024-09-10T11:30:29.998 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:30:29.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:30.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.095+0000 7f2797ca4640 1 Processor -- start 2024-09-10T11:30:30.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 -- start start 2024-09-10T11:30:30.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 0x7f27900d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f27900d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279015ff70 con 0x7f279015eb80 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f27900d30c0 con 0x7f27901626c0 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.096+0000 7f2797ca4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f27900d52d0 con 0x7f27900d4e40 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2796241640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f27900d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f279523f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 0x7f27900d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2795a40640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f279015eb80 0x7f27900d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41156/0 (socket says 172.21.15.99:41156) 2024-09-10T11:30:30.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2795a40640 1 -- 172.21.15.99:0/1376504894 learned_addr learned my addr 172.21.15.99:0/1376504894 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2796241640 1 -- 172.21.15.99:0/1376504894 >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 msgr2=0x7f27900d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2796241640 1 --2- 172.21.15.99:0/1376504894 >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 0x7f27900d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2796241640 1 -- 172.21.15.99:0/1376504894 >> v1:172.21.15.196:6789/0 conn(0x7f279015eb80 legacy=0x7f27900d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f2796241640 1 -- 172.21.15.99:0/1376504894 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27900d1620 con 0x7f27900d4e40 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.097+0000 7f279523f640 1 --2- 172.21.15.99:0/1376504894 >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 0x7f27900d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2796241640 1 --2- 172.21.15.99:0/1376504894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f27900d1b20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27880097d0 tx=0x7f278802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850da33f707d02e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2794a3e640 1 -- 172.21.15.99:0/1376504894 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788037070 con 0x7f27900d4e40 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2794a3e640 1 -- 172.21.15.99:0/1376504894 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278802ae40 con 0x7f27900d4e40 2024-09-10T11:30:30.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2794a3e640 1 -- 172.21.15.99:0/1376504894 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788032660 con 0x7f27900d4e40 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 -- 172.21.15.99:0/1376504894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 msgr2=0x7f27900d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 --2- 172.21.15.99:0/1376504894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f27900d1b20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f27880097d0 tx=0x7f278802a130 comp rx=0 tx=0).stop 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 -- 172.21.15.99:0/1376504894 shutdown_connections 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 --2- 172.21.15.99:0/1376504894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f27900d1b20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 --2- 172.21.15.99:0/1376504894 >> v2:172.21.15.196:3300/0 conn(0x7f27901626c0 0x7f27900d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.098+0000 7f2797ca4640 1 -- 172.21.15.99:0/1376504894 >> 172.21.15.99:0/1376504894 conn(0x7f279008e680 msgr2=0x7f279009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.099+0000 7f2797ca4640 1 -- 172.21.15.99:0/1376504894 shutdown_connections 2024-09-10T11:30:30.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.099+0000 7f2797ca4640 1 -- 172.21.15.99:0/1376504894 wait complete. 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 Processor -- start 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 -- start start 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f279014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27901626c0 0x7f27900b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2790163180 con 0x7f279015eb80 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2790162ea0 con 0x7f27901626c0 2024-09-10T11:30:30.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.100+0000 7f2797ca4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2790163460 con 0x7f27900d4e40 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56964/0 (socket says 172.21.15.99:56964) 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 -- 172.21.15.99:0/859590746 learned_addr learned my addr 172.21.15.99:0/859590746 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f2796241640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27901626c0 0x7f27900b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f2795a40640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f279014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 -- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 msgr2=0x7f279014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f279014f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 -- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27901626c0 msgr2=0x7f27900b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27901626c0 0x7f27900b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 -- 172.21.15.99:0/859590746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279015aa90 con 0x7f279015eb80 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f2795a40640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f279014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f279523f640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f278400e910 tx=0x7f278400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784017070 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278400ca00 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278400cdb0 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279015ad70 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.101+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279014f5d0 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.102+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27900d4900 con 0x7f279015eb80 2024-09-10T11:30:30.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.103+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f278401a070 con 0x7f279015eb80 2024-09-10T11:30:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.103+0000 7f278e7fc640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 0x7f277c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.103+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f278408b050 con 0x7f279015eb80 2024-09-10T11:30:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.103+0000 7f2795a40640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 0x7f277c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.104+0000 7f2795a40640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 0x7f277c06e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27800012e0 tx=0x7f2780009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.107+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2784059a90 con 0x7f279015eb80 2024-09-10T11:30:30.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.271+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27900b6af0 con 0x7f277c06be30 2024-09-10T11:30:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.277+0000 7f278e7fc640 1 -- 172.21.15.99:0/859590746 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242211 (secure 0 0 0) 0x7f27900b6af0 con 0x7f277c06be30 2024-09-10T11:30:30.282 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:30.283 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 msgr2=0x7f277c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 0x7f277c06e2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27800012e0 tx=0x7f2780009040 comp rx=0 tx=0).stop 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 msgr2=0x7f27900ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f278400e910 tx=0x7f278400edd0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 shutdown_connections 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f277c06be30 0x7f277c06e2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27901626c0 0x7f27900b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f279015eb80 0x7f27900ce5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 --2- 172.21.15.99:0/859590746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27900d4e40 0x7f279014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 >> 172.21.15.99:0/859590746 conn(0x7f279008e680 msgr2=0x7f27900d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 shutdown_connections 2024-09-10T11:30:30.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.280+0000 7f2797ca4640 1 -- 172.21.15.99:0/859590746 wait complete. 2024-09-10T11:30:30.293 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T11:30:28.907002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598407872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000925"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347078+0000","last_change":"2024-09-10T11:30:26.347078+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347078+0000","last_undegraded":"2024-09-10T11:30:26.347078+0000","last_fullsized":"2024-09-10T11:30:26.347078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349754+0000","last_change":"2024-09-10T11:30:26.349754+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349754+0000","last_undegraded":"2024-09-10T11:30:26.349754+0000","last_fullsized":"2024-09-10T11:30:26.349754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.351662+0000","last_change":"2024-09-10T11:30:26.351662+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.351662+0000","last_undegraded":"2024-09-10T11:30:26.351662+0000","last_fullsized":"2024-09-10T11:30:26.351662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343642+0000","last_change":"2024-09-10T11:30:26.343642+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343642+0000","last_undegraded":"2024-09-10T11:30:26.343642+0000","last_fullsized":"2024-09-10T11:30:26.343642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344216+0000","last_change":"2024-09-10T11:30:26.344216+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344216+0000","last_undegraded":"2024-09-10T11:30:26.344216+0000","last_fullsized":"2024-09-10T11:30:26.344216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342938+0000","last_change":"2024-09-10T11:30:26.342938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342938+0000","last_undegraded":"2024-09-10T11:30:26.342938+0000","last_fullsized":"2024-09-10T11:30:26.342938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348539+0000","last_change":"2024-09-10T11:30:26.348539+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348539+0000","last_undegraded":"2024-09-10T11:30:26.348539+0000","last_fullsized":"2024-09-10T11:30:26.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350001+0000","last_change":"2024-09-10T11:30:26.350001+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350001+0000","last_undegraded":"2024-09-10T11:30:26.350001+0000","last_fullsized":"2024-09-10T11:30:26.350001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348916+0000","last_change":"2024-09-10T11:30:26.348916+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348916+0000","last_undegraded":"2024-09-10T11:30:26.348916+0000","last_fullsized":"2024-09-10T11:30:26.348916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345319+0000","last_change":"2024-09-10T11:30:26.345319+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345319+0000","last_undegraded":"2024-09-10T11:30:26.345319+0000","last_fullsized":"2024-09-10T11:30:26.345319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346748+0000","last_change":"2024-09-10T11:30:26.346748+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346748+0000","last_undegraded":"2024-09-10T11:30:26.346748+0000","last_fullsized":"2024-09-10T11:30:26.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350070+0000","last_change":"2024-09-10T11:30:26.350070+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350070+0000","last_undegraded":"2024-09-10T11:30:26.350070+0000","last_fullsized":"2024-09-10T11:30:26.350070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350068+0000","last_change":"2024-09-10T11:30:26.350068+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350068+0000","last_undegraded":"2024-09-10T11:30:26.350068+0000","last_fullsized":"2024-09-10T11:30:26.350068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349848+0000","last_change":"2024-09-10T11:30:26.349848+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349848+0000","last_undegraded":"2024-09-10T11:30:26.349848+0000","last_fullsized":"2024-09-10T11:30:26.349848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353163+0000","last_change":"2024-09-10T11:30:26.353163+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353163+0000","last_undegraded":"2024-09-10T11:30:26.353163+0000","last_fullsized":"2024-09-10T11:30:26.353163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349515+0000","last_change":"2024-09-10T11:30:26.349515+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349515+0000","last_undegraded":"2024-09-10T11:30:26.349515+0000","last_fullsized":"2024-09-10T11:30:26.349515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344553+0000","last_change":"2024-09-10T11:30:26.344553+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344553+0000","last_undegraded":"2024-09-10T11:30:26.344553+0000","last_fullsized":"2024-09-10T11:30:26.344553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348768+0000","last_change":"2024-09-10T11:30:26.348768+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348768+0000","last_undegraded":"2024-09-10T11:30:26.348768+0000","last_fullsized":"2024-09-10T11:30:26.348768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345295+0000","last_change":"2024-09-10T11:30:26.345295+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345295+0000","last_undegraded":"2024-09-10T11:30:26.345295+0000","last_fullsized":"2024-09-10T11:30:26.345295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344859+0000","last_change":"2024-09-10T11:30:26.344859+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344859+0000","last_undegraded":"2024-09-10T11:30:26.344859+0000","last_fullsized":"2024-09-10T11:30:26.344859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344914+0000","last_change":"2024-09-10T11:30:26.344914+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344914+0000","last_undegraded":"2024-09-10T11:30:26.344914+0000","last_fullsized":"2024-09-10T11:30:26.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346106+0000","last_change":"2024-09-10T11:30:26.346106+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346106+0000","last_undegraded":"2024-09-10T11:30:26.346106+0000","last_fullsized":"2024-09-10T11:30:26.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348811+0000","last_change":"2024-09-10T11:30:26.348811+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348811+0000","last_undegraded":"2024-09-10T11:30:26.348811+0000","last_fullsized":"2024-09-10T11:30:26.348811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346246+0000","last_change":"2024-09-10T11:30:26.346246+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346246+0000","last_undegraded":"2024-09-10T11:30:26.346246+0000","last_fullsized":"2024-09-10T11:30:26.346246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352548+0000","last_change":"2024-09-10T11:30:26.352548+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352548+0000","last_undegraded":"2024-09-10T11:30:26.352548+0000","last_fullsized":"2024-09-10T11:30:26.352548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342369+0000","last_change":"2024-09-10T11:30:26.342369+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342369+0000","last_undegraded":"2024-09-10T11:30:26.342369+0000","last_fullsized":"2024-09-10T11:30:26.342369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349505+0000","last_change":"2024-09-10T11:30:26.349505+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349505+0000","last_undegraded":"2024-09-10T11:30:26.349505+0000","last_fullsized":"2024-09-10T11:30:26.349505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349202+0000","last_change":"2024-09-10T11:30:26.349202+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349202+0000","last_undegraded":"2024-09-10T11:30:26.349202+0000","last_fullsized":"2024-09-10T11:30:26.349202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353929+0000","last_change":"2024-09-10T11:30:26.353929+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353929+0000","last_undegraded":"2024-09-10T11:30:26.353929+0000","last_fullsized":"2024-09-10T11:30:26.353929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348630+0000","last_change":"2024-09-10T11:30:26.348630+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348630+0000","last_undegraded":"2024-09-10T11:30:26.348630+0000","last_fullsized":"2024-09-10T11:30:26.348630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347834+0000","last_change":"2024-09-10T11:30:26.347834+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347834+0000","last_undegraded":"2024-09-10T11:30:26.347834+0000","last_fullsized":"2024-09-10T11:30:26.347834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343398+0000","last_change":"2024-09-10T11:30:26.343398+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343398+0000","last_undegraded":"2024-09-10T11:30:26.343398+0000","last_fullsized":"2024-09-10T11:30:26.343398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347190+0000","last_change":"2024-09-10T11:30:26.347190+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347190+0000","last_undegraded":"2024-09-10T11:30:26.347190+0000","last_fullsized":"2024-09-10T11:30:26.347190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355648+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:26.355648+0000","last_peered":"2024-09-10T11:30:26.355648+0000","last_clean":"2024-09-10T11:30:26.355648+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:26.355648+0000","last_undegraded":"2024-09-10T11:30:26.355648+0000","last_fullsized":"2024-09-10T11:30:26.355648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:01.364851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344409+0000","last_change":"2024-09-10T11:30:26.344409+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344409+0000","last_undegraded":"2024-09-10T11:30:26.344409+0000","last_fullsized":"2024-09-10T11:30:26.344409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352706+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:26.352706+0000","last_peered":"2024-09-10T11:30:26.352706+0000","last_clean":"2024-09-10T11:30:26.352706+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:26.352706+0000","last_undegraded":"2024-09-10T11:30:26.352706+0000","last_fullsized":"2024-09-10T11:30:26.352706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:02.279667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350997+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:26.350997+0000","last_peered":"2024-09-10T11:30:26.350997+0000","last_clean":"2024-09-10T11:30:26.350997+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:26.350997+0000","last_undegraded":"2024-09-10T11:30:26.350997+0000","last_fullsized":"2024-09-10T11:30:26.350997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:20.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352083+0000","last_change":"2024-09-10T11:30:26.352083+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352083+0000","last_undegraded":"2024-09-10T11:30:26.352083+0000","last_fullsized":"2024-09-10T11:30:26.352083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353752+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:26.353752+0000","last_peered":"2024-09-10T11:30:26.353752+0000","last_clean":"2024-09-10T11:30:26.353752+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:26.353752+0000","last_undegraded":"2024-09-10T11:30:26.353752+0000","last_fullsized":"2024-09-10T11:30:26.353752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:16.268202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344235+0000","last_change":"2024-09-10T11:30:26.344235+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344235+0000","last_undegraded":"2024-09-10T11:30:26.344235+0000","last_fullsized":"2024-09-10T11:30:26.344235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352206+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:26.352206+0000","last_peered":"2024-09-10T11:30:26.352206+0000","last_clean":"2024-09-10T11:30:26.352206+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:26.352206+0000","last_undegraded":"2024-09-10T11:30:26.352206+0000","last_fullsized":"2024-09-10T11:30:26.352206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:45.803657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342684+0000","last_change":"2024-09-10T11:30:26.342684+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342684+0000","last_undegraded":"2024-09-10T11:30:26.342684+0000","last_fullsized":"2024-09-10T11:30:26.342684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356269+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:26.356269+0000","last_peered":"2024-09-10T11:30:26.356269+0000","last_clean":"2024-09-10T11:30:26.356269+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:26.356269+0000","last_undegraded":"2024-09-10T11:30:26.356269+0000","last_fullsized":"2024-09-10T11:30:26.356269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:02.709036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350537+0000","last_change":"2024-09-10T11:30:26.350537+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350537+0000","last_undegraded":"2024-09-10T11:30:26.350537+0000","last_fullsized":"2024-09-10T11:30:26.350537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347731+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:26.347731+0000","last_peered":"2024-09-10T11:30:26.347731+0000","last_clean":"2024-09-10T11:30:26.347731+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:26.347731+0000","last_undegraded":"2024-09-10T11:30:26.347731+0000","last_fullsized":"2024-09-10T11:30:26.347731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:41.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345938+0000","last_change":"2024-09-10T11:30:26.345938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345938+0000","last_undegraded":"2024-09-10T11:30:26.345938+0000","last_fullsized":"2024-09-10T11:30:26.345938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.348741+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:26.348741+0000","last_peered":"2024-09-10T11:30:26.348741+0000","last_clean":"2024-09-10T11:30:26.348741+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:26.348741+0000","last_undegraded":"2024-09-10T11:30:26.348741+0000","last_fullsized":"2024-09-10T11:30:26.348741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345128+0000","last_change":"2024-09-10T11:30:26.345128+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345128+0000","last_undegraded":"2024-09-10T11:30:26.345128+0000","last_fullsized":"2024-09-10T11:30:26.345128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351884+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:26.351884+0000","last_peered":"2024-09-10T11:30:26.351884+0000","last_clean":"2024-09-10T11:30:26.351884+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:26.351884+0000","last_undegraded":"2024-09-10T11:30:26.351884+0000","last_fullsized":"2024-09-10T11:30:26.351884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:14.416062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349865+0000","last_change":"2024-09-10T11:30:26.349865+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349865+0000","last_undegraded":"2024-09-10T11:30:26.349865+0000","last_fullsized":"2024-09-10T11:30:26.349865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.354977+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:26.354977+0000","last_peered":"2024-09-10T11:30:26.354977+0000","last_clean":"2024-09-10T11:30:26.354977+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:26.354977+0000","last_undegraded":"2024-09-10T11:30:26.354977+0000","last_fullsized":"2024-09-10T11:30:26.354977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:34.505783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347122+0000","last_change":"2024-09-10T11:30:26.347122+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347122+0000","last_undegraded":"2024-09-10T11:30:26.347122+0000","last_fullsized":"2024-09-10T11:30:26.347122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356474+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:26.356474+0000","last_peered":"2024-09-10T11:30:26.356474+0000","last_clean":"2024-09-10T11:30:26.356474+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:26.356474+0000","last_undegraded":"2024-09-10T11:30:26.356474+0000","last_fullsized":"2024-09-10T11:30:26.356474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:37.442331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345736+0000","last_change":"2024-09-10T11:30:26.345736+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345736+0000","last_undegraded":"2024-09-10T11:30:26.345736+0000","last_fullsized":"2024-09-10T11:30:26.345736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347100+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:26.347100+0000","last_peered":"2024-09-10T11:30:26.347100+0000","last_clean":"2024-09-10T11:30:26.347100+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:26.347100+0000","last_undegraded":"2024-09-10T11:30:26.347100+0000","last_fullsized":"2024-09-10T11:30:26.347100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:38.630516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344204+0000","last_change":"2024-09-10T11:30:26.344204+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344204+0000","last_undegraded":"2024-09-10T11:30:26.344204+0000","last_fullsized":"2024-09-10T11:30:26.344204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358330+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:26.358330+0000","last_peered":"2024-09-10T11:30:26.358330+0000","last_clean":"2024-09-10T11:30:26.358330+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:26.358330+0000","last_undegraded":"2024-09-10T11:30:26.358330+0000","last_fullsized":"2024-09-10T11:30:26.358330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:28.172249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342367+0000","last_change":"2024-09-10T11:30:26.342367+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342367+0000","last_undegraded":"2024-09-10T11:30:26.342367+0000","last_fullsized":"2024-09-10T11:30:26.342367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:44.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345905+0000","last_change":"2024-09-10T11:30:26.345905+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345905+0000","last_undegraded":"2024-09-10T11:30:26.345905+0000","last_fullsized":"2024-09-10T11:30:26.345905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357762+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:26.357762+0000","last_peered":"2024-09-10T11:30:26.357762+0000","last_clean":"2024-09-10T11:30:26.357762+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:26.357762+0000","last_undegraded":"2024-09-10T11:30:26.357762+0000","last_fullsized":"2024-09-10T11:30:26.357762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349562+0000","last_change":"2024-09-10T11:30:26.349562+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349562+0000","last_undegraded":"2024-09-10T11:30:26.349562+0000","last_fullsized":"2024-09-10T11:30:26.349562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357477+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:26.357477+0000","last_peered":"2024-09-10T11:30:26.357477+0000","last_clean":"2024-09-10T11:30:26.357477+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:26.357477+0000","last_undegraded":"2024-09-10T11:30:26.357477+0000","last_fullsized":"2024-09-10T11:30:26.357477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:28.358480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347691+0000","last_change":"2024-09-10T11:30:26.347691+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347691+0000","last_undegraded":"2024-09-10T11:30:26.347691+0000","last_fullsized":"2024-09-10T11:30:26.347691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.349405+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:26.349405+0000","last_peered":"2024-09-10T11:30:26.349405+0000","last_clean":"2024-09-10T11:30:26.349405+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:26.349405+0000","last_undegraded":"2024-09-10T11:30:26.349405+0000","last_fullsized":"2024-09-10T11:30:26.349405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352072+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:26.352072+0000","last_peered":"2024-09-10T11:30:26.352072+0000","last_clean":"2024-09-10T11:30:26.352072+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:26.352072+0000","last_undegraded":"2024-09-10T11:30:26.352072+0000","last_fullsized":"2024-09-10T11:30:26.352072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:52.596921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343276+0000","last_change":"2024-09-10T11:30:26.343276+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343276+0000","last_undegraded":"2024-09-10T11:30:26.343276+0000","last_fullsized":"2024-09-10T11:30:26.343276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357911+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:26.357911+0000","last_peered":"2024-09-10T11:30:26.357911+0000","last_clean":"2024-09-10T11:30:26.357911+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:26.357911+0000","last_undegraded":"2024-09-10T11:30:26.357911+0000","last_fullsized":"2024-09-10T11:30:26.357911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:53.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355632+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:26.355632+0000","last_peered":"2024-09-10T11:30:26.355632+0000","last_clean":"2024-09-10T11:30:26.355632+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:26.355632+0000","last_undegraded":"2024-09-10T11:30:26.355632+0000","last_fullsized":"2024-09-10T11:30:26.355632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.067158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353205+0000","last_change":"2024-09-10T11:30:26.353205+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353205+0000","last_undegraded":"2024-09-10T11:30:26.353205+0000","last_fullsized":"2024-09-10T11:30:26.353205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351869+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:26.351869+0000","last_peered":"2024-09-10T11:30:26.351869+0000","last_clean":"2024-09-10T11:30:26.351869+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:26.351869+0000","last_undegraded":"2024-09-10T11:30:26.351869+0000","last_fullsized":"2024-09-10T11:30:26.351869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:01.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346396+0000","last_change":"2024-09-10T11:30:26.346396+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346396+0000","last_undegraded":"2024-09-10T11:30:26.346396+0000","last_fullsized":"2024-09-10T11:30:26.346396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356683+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:26.356683+0000","last_peered":"2024-09-10T11:30:26.356683+0000","last_clean":"2024-09-10T11:30:26.356683+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:26.356683+0000","last_undegraded":"2024-09-10T11:30:26.356683+0000","last_fullsized":"2024-09-10T11:30:26.356683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:05.310043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342388+0000","last_change":"2024-09-10T11:30:26.342388+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342388+0000","last_undegraded":"2024-09-10T11:30:26.342388+0000","last_fullsized":"2024-09-10T11:30:26.342388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353658+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:26.353658+0000","last_peered":"2024-09-10T11:30:26.353658+0000","last_clean":"2024-09-10T11:30:26.353658+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:26.353658+0000","last_undegraded":"2024-09-10T11:30:26.353658+0000","last_fullsized":"2024-09-10T11:30:26.353658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:33.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349242+0000","last_change":"2024-09-10T11:30:26.349242+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349242+0000","last_undegraded":"2024-09-10T11:30:26.349242+0000","last_fullsized":"2024-09-10T11:30:26.349242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351809+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:26.351809+0000","last_peered":"2024-09-10T11:30:26.351809+0000","last_clean":"2024-09-10T11:30:26.351809+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:26.351809+0000","last_undegraded":"2024-09-10T11:30:26.351809+0000","last_fullsized":"2024-09-10T11:30:26.351809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:39.942357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344018+0000","last_change":"2024-09-10T11:30:26.344018+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344018+0000","last_undegraded":"2024-09-10T11:30:26.344018+0000","last_fullsized":"2024-09-10T11:30:26.344018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:04.011154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344195+0000","last_change":"2024-09-10T11:30:26.344195+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344195+0000","last_undegraded":"2024-09-10T11:30:26.344195+0000","last_fullsized":"2024-09-10T11:30:26.344195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352963+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:26.352963+0000","last_peered":"2024-09-10T11:30:26.352963+0000","last_clean":"2024-09-10T11:30:26.352963+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:26.352963+0000","last_undegraded":"2024-09-10T11:30:26.352963+0000","last_fullsized":"2024-09-10T11:30:26.352963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:12.773758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345247+0000","last_change":"2024-09-10T11:30:26.345247+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345247+0000","last_undegraded":"2024-09-10T11:30:26.345247+0000","last_fullsized":"2024-09-10T11:30:26.345247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350527+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:26.350527+0000","last_peered":"2024-09-10T11:30:26.350527+0000","last_clean":"2024-09-10T11:30:26.350527+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:26.350527+0000","last_undegraded":"2024-09-10T11:30:26.350527+0000","last_fullsized":"2024-09-10T11:30:26.350527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:21.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343602+0000","last_change":"2024-09-10T11:30:26.343602+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343602+0000","last_undegraded":"2024-09-10T11:30:26.343602+0000","last_fullsized":"2024-09-10T11:30:26.343602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358594+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:26.358594+0000","last_peered":"2024-09-10T11:30:26.358594+0000","last_clean":"2024-09-10T11:30:26.358594+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:26.358594+0000","last_undegraded":"2024-09-10T11:30:26.358594+0000","last_fullsized":"2024-09-10T11:30:26.358594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:01.475428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352810+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:26.352810+0000","last_peered":"2024-09-10T11:30:26.352810+0000","last_clean":"2024-09-10T11:30:26.352810+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:26.352810+0000","last_undegraded":"2024-09-10T11:30:26.352810+0000","last_fullsized":"2024-09-10T11:30:26.352810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:57.958396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345612+0000","last_change":"2024-09-10T11:30:26.345612+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345612+0000","last_undegraded":"2024-09-10T11:30:26.345612+0000","last_fullsized":"2024-09-10T11:30:26.345612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347450+0000","last_change":"2024-09-10T11:30:26.347450+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347450+0000","last_undegraded":"2024-09-10T11:30:26.347450+0000","last_fullsized":"2024-09-10T11:30:26.347450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355021+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:26.355021+0000","last_peered":"2024-09-10T11:30:26.355021+0000","last_clean":"2024-09-10T11:30:26.355021+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:26.355021+0000","last_undegraded":"2024-09-10T11:30:26.355021+0000","last_fullsized":"2024-09-10T11:30:26.355021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:11.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347966+0000","last_change":"2024-09-10T11:30:26.347966+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347966+0000","last_undegraded":"2024-09-10T11:30:26.347966+0000","last_fullsized":"2024-09-10T11:30:26.347966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352946+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:26.352946+0000","last_peered":"2024-09-10T11:30:26.352946+0000","last_clean":"2024-09-10T11:30:26.352946+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:26.352946+0000","last_undegraded":"2024-09-10T11:30:26.352946+0000","last_fullsized":"2024-09-10T11:30:26.352946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:30.209672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349430+0000","last_change":"2024-09-10T11:30:26.349430+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349430+0000","last_undegraded":"2024-09-10T11:30:26.349430+0000","last_fullsized":"2024-09-10T11:30:26.349430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356992+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:26.356992+0000","last_peered":"2024-09-10T11:30:26.356992+0000","last_clean":"2024-09-10T11:30:26.356992+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:26.356992+0000","last_undegraded":"2024-09-10T11:30:26.356992+0000","last_fullsized":"2024-09-10T11:30:26.356992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:05.275854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353527+0000","last_change":"2024-09-10T11:30:26.353527+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353527+0000","last_undegraded":"2024-09-10T11:30:26.353527+0000","last_fullsized":"2024-09-10T11:30:26.353527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357789+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:26.357789+0000","last_peered":"2024-09-10T11:30:26.357789+0000","last_clean":"2024-09-10T11:30:26.357789+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:26.357789+0000","last_undegraded":"2024-09-10T11:30:26.357789+0000","last_fullsized":"2024-09-10T11:30:26.357789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:16.906847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:30.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:30.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.429+0000 7fab05465640 1 Processor -- start 2024-09-10T11:30:30.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.429+0000 7fab05465640 1 -- start start 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7fab05465640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fab00163580 0x7fab000b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7fab05465640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7fab000d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7fab05465640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab000b3620 con 0x7fab0006cd20 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7fab05465640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fab0006d960 con 0x7fab00163580 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7fab05465640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab0006dc40 con 0x7fab000b3960 2024-09-10T11:30:30.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faaff7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fab0006cd20 0x7fab0006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33072/0 (socket says 172.21.15.99:33072) 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faaff7fe640 1 -- 172.21.15.99:0/1958860023 learned_addr learned my addr 172.21.15.99:0/1958860023 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafeffd640 1 --2- 172.21.15.99:0/1958860023 >> v2:172.21.15.196:3300/0 conn(0x7fab00163580 0x7fab000b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafe7fc640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7fab000d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522313970 0 0) 0x7fab000b3620 con 0x7fab0006cd20 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faae8003520 con 0x7fab0006cd20 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2860994088 0 0) 0x7faae8003520 con 0x7fab0006cd20 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 msgr2=0x7fab000d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7fab000d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 >> v2:172.21.15.196:3300/0 conn(0x7fab00163580 msgr2=0x7fab000b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 --2- 172.21.15.99:0/1958860023 >> v2:172.21.15.196:3300/0 conn(0x7fab00163580 0x7fab000b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0006df20 con 0x7fab0006cd20 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7faaf80019f0 con 0x7fab0006cd20 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.430+0000 7faafeffd640 1 --2- 172.21.15.99:0/1958860023 >> v2:172.21.15.196:3300/0 conn(0x7fab00163580 0x7fab000b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 >> v1:172.21.15.99:6789/0 conn(0x7fab0006cd20 legacy=0x7fab0006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7faae80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae8003f30 0x7faae8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8004d40 0x7faae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab0006d960 con 0x7faae8003f30 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab0006dc40 con 0x7faae8004d40 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faae800ba10 con 0x7fab000b3960 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae8003f30 0x7faae8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafeffd640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7faae80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faaff7fe640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8004d40 0x7faae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 -- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 msgr2=0x7faae80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab000b3960 0x7faae80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 -- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8004d40 msgr2=0x7faae80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8004d40 0x7faae80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faafe7fc640 1 -- 172.21.15.99:0/1958860023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae8003520 con 0x7faae8003f30 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.431+0000 7faaff7fe640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faae8004d40 0x7faae80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7faafe7fc640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae8003f30 0x7faae8004800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009ad0 tx=0x7faae8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9073a71b6ff3c02a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4036070 con 0x7faae8003f30 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf4031400 con 0x7faae8003f30 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7faafdffb640 1 -- 172.21.15.99:0/1958860023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4009670 con 0x7faae8003f30 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 -- 172.21.15.99:0/1958860023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae8003f30 msgr2=0x7faae8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 --2- 172.21.15.99:0/1958860023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faae8003f30 0x7faae8004800 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009ad0 tx=0x7faae8005830 comp rx=0 tx=0).stop 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7faaff7fe640 1 -- 172.21.15.99:0/1958860023 reap_dead start 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 -- 172.21.15.99:0/1958860023 shutdown_connections 2024-09-10T11:30:30.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 -- 172.21.15.99:0/1958860023 >> 172.21.15.99:0/1958860023 conn(0x7fab000b4570 msgr2=0x7fab000b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 -- 172.21.15.99:0/1958860023 shutdown_connections 2024-09-10T11:30:30.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.432+0000 7fab05465640 1 -- 172.21.15.99:0/1958860023 wait complete. 2024-09-10T11:30:30.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 Processor -- start 2024-09-10T11:30:30.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 -- start start 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faae8003f30 0x7fab0015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab0015fe70 0x7fab000c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab000b32e0 con 0x7fab000cdab0 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fab000b3160 con 0x7fab0015fe70 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7fab05465640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fab000d1be0 con 0x7faae8003f30 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7faafe7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7faafe7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56978/0 (socket says 172.21.15.99:56978) 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.434+0000 7faafe7fc640 1 -- 172.21.15.99:0/649988664 learned_addr learned my addr 172.21.15.99:0/649988664 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 -- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faae8003f30 msgr2=0x7fab0015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faae8003f30 0x7fab0015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 -- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab0015fe70 msgr2=0x7fab000c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab0015fe70 0x7fab000c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 -- 172.21.15.99:0/649988664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0015f900 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faafe7fc640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaf402abb0 tx=0x7faaf4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4036040 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf403d460 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4009670 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0015fbe0 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.435+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001e8bd0 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.436+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab000761a0 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.437+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faaf4039040 con 0x7fab000cdab0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.437+0000 7faae77fe640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 0x7faaec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.437+0000 7faafeffd640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 0x7faaec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.437+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faaf40aae60 con 0x7fab000cdab0 2024-09-10T11:30:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.437+0000 7faafeffd640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 0x7faaec06e320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009cc0 tx=0x7faaf0009780 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.441+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faaf4041020 con 0x7fab000cdab0 2024-09-10T11:30:30.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.606+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab000d1fc0 con 0x7faaec06be60 2024-09-10T11:30:30.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.612+0000 7faae77fe640 1 -- 172.21.15.99:0/649988664 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242211 (secure 0 0 0) 0x7fab000d1fc0 con 0x7faaec06be60 2024-09-10T11:30:30.616 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:30.617 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 msgr2=0x7faaec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 0x7faaec06e320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009cc0 tx=0x7faaf0009780 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 msgr2=0x7fab00153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faaf402abb0 tx=0x7faaf4004730 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 shutdown_connections 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faaec06be60 0x7faaec06e320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab0015fe70 0x7fab000c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab000cdab0 0x7fab00153fd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 --2- 172.21.15.99:0/649988664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faae8003f30 0x7fab0015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 >> 172.21.15.99:0/649988664 conn(0x7fab000b4570 msgr2=0x7fab000d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 shutdown_connections 2024-09-10T11:30:30.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.615+0000 7fab05465640 1 -- 172.21.15.99:0/649988664 wait complete. 2024-09-10T11:30:30.628 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-10T11:30:28.907002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598407872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000925"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347078+0000","last_change":"2024-09-10T11:30:26.347078+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347078+0000","last_undegraded":"2024-09-10T11:30:26.347078+0000","last_fullsized":"2024-09-10T11:30:26.347078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349754+0000","last_change":"2024-09-10T11:30:26.349754+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349754+0000","last_undegraded":"2024-09-10T11:30:26.349754+0000","last_fullsized":"2024-09-10T11:30:26.349754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.351662+0000","last_change":"2024-09-10T11:30:26.351662+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.351662+0000","last_undegraded":"2024-09-10T11:30:26.351662+0000","last_fullsized":"2024-09-10T11:30:26.351662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343642+0000","last_change":"2024-09-10T11:30:26.343642+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343642+0000","last_undegraded":"2024-09-10T11:30:26.343642+0000","last_fullsized":"2024-09-10T11:30:26.343642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344216+0000","last_change":"2024-09-10T11:30:26.344216+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344216+0000","last_undegraded":"2024-09-10T11:30:26.344216+0000","last_fullsized":"2024-09-10T11:30:26.344216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342938+0000","last_change":"2024-09-10T11:30:26.342938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342938+0000","last_undegraded":"2024-09-10T11:30:26.342938+0000","last_fullsized":"2024-09-10T11:30:26.342938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348539+0000","last_change":"2024-09-10T11:30:26.348539+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348539+0000","last_undegraded":"2024-09-10T11:30:26.348539+0000","last_fullsized":"2024-09-10T11:30:26.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350001+0000","last_change":"2024-09-10T11:30:26.350001+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350001+0000","last_undegraded":"2024-09-10T11:30:26.350001+0000","last_fullsized":"2024-09-10T11:30:26.350001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348916+0000","last_change":"2024-09-10T11:30:26.348916+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348916+0000","last_undegraded":"2024-09-10T11:30:26.348916+0000","last_fullsized":"2024-09-10T11:30:26.348916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345319+0000","last_change":"2024-09-10T11:30:26.345319+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345319+0000","last_undegraded":"2024-09-10T11:30:26.345319+0000","last_fullsized":"2024-09-10T11:30:26.345319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346748+0000","last_change":"2024-09-10T11:30:26.346748+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346748+0000","last_undegraded":"2024-09-10T11:30:26.346748+0000","last_fullsized":"2024-09-10T11:30:26.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350070+0000","last_change":"2024-09-10T11:30:26.350070+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350070+0000","last_undegraded":"2024-09-10T11:30:26.350070+0000","last_fullsized":"2024-09-10T11:30:26.350070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350068+0000","last_change":"2024-09-10T11:30:26.350068+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350068+0000","last_undegraded":"2024-09-10T11:30:26.350068+0000","last_fullsized":"2024-09-10T11:30:26.350068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349848+0000","last_change":"2024-09-10T11:30:26.349848+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349848+0000","last_undegraded":"2024-09-10T11:30:26.349848+0000","last_fullsized":"2024-09-10T11:30:26.349848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353163+0000","last_change":"2024-09-10T11:30:26.353163+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353163+0000","last_undegraded":"2024-09-10T11:30:26.353163+0000","last_fullsized":"2024-09-10T11:30:26.353163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349515+0000","last_change":"2024-09-10T11:30:26.349515+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349515+0000","last_undegraded":"2024-09-10T11:30:26.349515+0000","last_fullsized":"2024-09-10T11:30:26.349515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344553+0000","last_change":"2024-09-10T11:30:26.344553+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344553+0000","last_undegraded":"2024-09-10T11:30:26.344553+0000","last_fullsized":"2024-09-10T11:30:26.344553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348768+0000","last_change":"2024-09-10T11:30:26.348768+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348768+0000","last_undegraded":"2024-09-10T11:30:26.348768+0000","last_fullsized":"2024-09-10T11:30:26.348768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345295+0000","last_change":"2024-09-10T11:30:26.345295+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345295+0000","last_undegraded":"2024-09-10T11:30:26.345295+0000","last_fullsized":"2024-09-10T11:30:26.345295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344859+0000","last_change":"2024-09-10T11:30:26.344859+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344859+0000","last_undegraded":"2024-09-10T11:30:26.344859+0000","last_fullsized":"2024-09-10T11:30:26.344859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344914+0000","last_change":"2024-09-10T11:30:26.344914+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344914+0000","last_undegraded":"2024-09-10T11:30:26.344914+0000","last_fullsized":"2024-09-10T11:30:26.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346106+0000","last_change":"2024-09-10T11:30:26.346106+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346106+0000","last_undegraded":"2024-09-10T11:30:26.346106+0000","last_fullsized":"2024-09-10T11:30:26.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348811+0000","last_change":"2024-09-10T11:30:26.348811+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348811+0000","last_undegraded":"2024-09-10T11:30:26.348811+0000","last_fullsized":"2024-09-10T11:30:26.348811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346246+0000","last_change":"2024-09-10T11:30:26.346246+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346246+0000","last_undegraded":"2024-09-10T11:30:26.346246+0000","last_fullsized":"2024-09-10T11:30:26.346246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352548+0000","last_change":"2024-09-10T11:30:26.352548+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352548+0000","last_undegraded":"2024-09-10T11:30:26.352548+0000","last_fullsized":"2024-09-10T11:30:26.352548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342369+0000","last_change":"2024-09-10T11:30:26.342369+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342369+0000","last_undegraded":"2024-09-10T11:30:26.342369+0000","last_fullsized":"2024-09-10T11:30:26.342369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349505+0000","last_change":"2024-09-10T11:30:26.349505+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349505+0000","last_undegraded":"2024-09-10T11:30:26.349505+0000","last_fullsized":"2024-09-10T11:30:26.349505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349202+0000","last_change":"2024-09-10T11:30:26.349202+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349202+0000","last_undegraded":"2024-09-10T11:30:26.349202+0000","last_fullsized":"2024-09-10T11:30:26.349202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353929+0000","last_change":"2024-09-10T11:30:26.353929+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353929+0000","last_undegraded":"2024-09-10T11:30:26.353929+0000","last_fullsized":"2024-09-10T11:30:26.353929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348630+0000","last_change":"2024-09-10T11:30:26.348630+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348630+0000","last_undegraded":"2024-09-10T11:30:26.348630+0000","last_fullsized":"2024-09-10T11:30:26.348630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347834+0000","last_change":"2024-09-10T11:30:26.347834+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347834+0000","last_undegraded":"2024-09-10T11:30:26.347834+0000","last_fullsized":"2024-09-10T11:30:26.347834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343398+0000","last_change":"2024-09-10T11:30:26.343398+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343398+0000","last_undegraded":"2024-09-10T11:30:26.343398+0000","last_fullsized":"2024-09-10T11:30:26.343398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347190+0000","last_change":"2024-09-10T11:30:26.347190+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347190+0000","last_undegraded":"2024-09-10T11:30:26.347190+0000","last_fullsized":"2024-09-10T11:30:26.347190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355648+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:26.355648+0000","last_peered":"2024-09-10T11:30:26.355648+0000","last_clean":"2024-09-10T11:30:26.355648+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:26.355648+0000","last_undegraded":"2024-09-10T11:30:26.355648+0000","last_fullsized":"2024-09-10T11:30:26.355648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:01.364851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344409+0000","last_change":"2024-09-10T11:30:26.344409+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344409+0000","last_undegraded":"2024-09-10T11:30:26.344409+0000","last_fullsized":"2024-09-10T11:30:26.344409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352706+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:26.352706+0000","last_peered":"2024-09-10T11:30:26.352706+0000","last_clean":"2024-09-10T11:30:26.352706+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:26.352706+0000","last_undegraded":"2024-09-10T11:30:26.352706+0000","last_fullsized":"2024-09-10T11:30:26.352706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:02.279667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350997+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:26.350997+0000","last_peered":"2024-09-10T11:30:26.350997+0000","last_clean":"2024-09-10T11:30:26.350997+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:26.350997+0000","last_undegraded":"2024-09-10T11:30:26.350997+0000","last_fullsized":"2024-09-10T11:30:26.350997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:20.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352083+0000","last_change":"2024-09-10T11:30:26.352083+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352083+0000","last_undegraded":"2024-09-10T11:30:26.352083+0000","last_fullsized":"2024-09-10T11:30:26.352083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353752+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:26.353752+0000","last_peered":"2024-09-10T11:30:26.353752+0000","last_clean":"2024-09-10T11:30:26.353752+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:26.353752+0000","last_undegraded":"2024-09-10T11:30:26.353752+0000","last_fullsized":"2024-09-10T11:30:26.353752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:16.268202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344235+0000","last_change":"2024-09-10T11:30:26.344235+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344235+0000","last_undegraded":"2024-09-10T11:30:26.344235+0000","last_fullsized":"2024-09-10T11:30:26.344235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352206+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:26.352206+0000","last_peered":"2024-09-10T11:30:26.352206+0000","last_clean":"2024-09-10T11:30:26.352206+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:26.352206+0000","last_undegraded":"2024-09-10T11:30:26.352206+0000","last_fullsized":"2024-09-10T11:30:26.352206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:45.803657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342684+0000","last_change":"2024-09-10T11:30:26.342684+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342684+0000","last_undegraded":"2024-09-10T11:30:26.342684+0000","last_fullsized":"2024-09-10T11:30:26.342684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356269+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:26.356269+0000","last_peered":"2024-09-10T11:30:26.356269+0000","last_clean":"2024-09-10T11:30:26.356269+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:26.356269+0000","last_undegraded":"2024-09-10T11:30:26.356269+0000","last_fullsized":"2024-09-10T11:30:26.356269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:02.709036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350537+0000","last_change":"2024-09-10T11:30:26.350537+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350537+0000","last_undegraded":"2024-09-10T11:30:26.350537+0000","last_fullsized":"2024-09-10T11:30:26.350537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347731+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:26.347731+0000","last_peered":"2024-09-10T11:30:26.347731+0000","last_clean":"2024-09-10T11:30:26.347731+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:26.347731+0000","last_undegraded":"2024-09-10T11:30:26.347731+0000","last_fullsized":"2024-09-10T11:30:26.347731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:41.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345938+0000","last_change":"2024-09-10T11:30:26.345938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345938+0000","last_undegraded":"2024-09-10T11:30:26.345938+0000","last_fullsized":"2024-09-10T11:30:26.345938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.348741+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:26.348741+0000","last_peered":"2024-09-10T11:30:26.348741+0000","last_clean":"2024-09-10T11:30:26.348741+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:26.348741+0000","last_undegraded":"2024-09-10T11:30:26.348741+0000","last_fullsized":"2024-09-10T11:30:26.348741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345128+0000","last_change":"2024-09-10T11:30:26.345128+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345128+0000","last_undegraded":"2024-09-10T11:30:26.345128+0000","last_fullsized":"2024-09-10T11:30:26.345128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351884+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:26.351884+0000","last_peered":"2024-09-10T11:30:26.351884+0000","last_clean":"2024-09-10T11:30:26.351884+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:26.351884+0000","last_undegraded":"2024-09-10T11:30:26.351884+0000","last_fullsized":"2024-09-10T11:30:26.351884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:14.416062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349865+0000","last_change":"2024-09-10T11:30:26.349865+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349865+0000","last_undegraded":"2024-09-10T11:30:26.349865+0000","last_fullsized":"2024-09-10T11:30:26.349865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.354977+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:26.354977+0000","last_peered":"2024-09-10T11:30:26.354977+0000","last_clean":"2024-09-10T11:30:26.354977+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:26.354977+0000","last_undegraded":"2024-09-10T11:30:26.354977+0000","last_fullsized":"2024-09-10T11:30:26.354977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:34.505783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347122+0000","last_change":"2024-09-10T11:30:26.347122+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347122+0000","last_undegraded":"2024-09-10T11:30:26.347122+0000","last_fullsized":"2024-09-10T11:30:26.347122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356474+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:26.356474+0000","last_peered":"2024-09-10T11:30:26.356474+0000","last_clean":"2024-09-10T11:30:26.356474+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:26.356474+0000","last_undegraded":"2024-09-10T11:30:26.356474+0000","last_fullsized":"2024-09-10T11:30:26.356474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:37.442331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345736+0000","last_change":"2024-09-10T11:30:26.345736+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345736+0000","last_undegraded":"2024-09-10T11:30:26.345736+0000","last_fullsized":"2024-09-10T11:30:26.345736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347100+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:26.347100+0000","last_peered":"2024-09-10T11:30:26.347100+0000","last_clean":"2024-09-10T11:30:26.347100+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:26.347100+0000","last_undegraded":"2024-09-10T11:30:26.347100+0000","last_fullsized":"2024-09-10T11:30:26.347100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:38.630516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344204+0000","last_change":"2024-09-10T11:30:26.344204+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344204+0000","last_undegraded":"2024-09-10T11:30:26.344204+0000","last_fullsized":"2024-09-10T11:30:26.344204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358330+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:26.358330+0000","last_peered":"2024-09-10T11:30:26.358330+0000","last_clean":"2024-09-10T11:30:26.358330+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:26.358330+0000","last_undegraded":"2024-09-10T11:30:26.358330+0000","last_fullsized":"2024-09-10T11:30:26.358330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:28.172249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342367+0000","last_change":"2024-09-10T11:30:26.342367+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342367+0000","last_undegraded":"2024-09-10T11:30:26.342367+0000","last_fullsized":"2024-09-10T11:30:26.342367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:44.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345905+0000","last_change":"2024-09-10T11:30:26.345905+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345905+0000","last_undegraded":"2024-09-10T11:30:26.345905+0000","last_fullsized":"2024-09-10T11:30:26.345905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357762+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:26.357762+0000","last_peered":"2024-09-10T11:30:26.357762+0000","last_clean":"2024-09-10T11:30:26.357762+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:26.357762+0000","last_undegraded":"2024-09-10T11:30:26.357762+0000","last_fullsized":"2024-09-10T11:30:26.357762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349562+0000","last_change":"2024-09-10T11:30:26.349562+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349562+0000","last_undegraded":"2024-09-10T11:30:26.349562+0000","last_fullsized":"2024-09-10T11:30:26.349562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357477+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:26.357477+0000","last_peered":"2024-09-10T11:30:26.357477+0000","last_clean":"2024-09-10T11:30:26.357477+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:26.357477+0000","last_undegraded":"2024-09-10T11:30:26.357477+0000","last_fullsized":"2024-09-10T11:30:26.357477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:28.358480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347691+0000","last_change":"2024-09-10T11:30:26.347691+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347691+0000","last_undegraded":"2024-09-10T11:30:26.347691+0000","last_fullsized":"2024-09-10T11:30:26.347691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.349405+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:26.349405+0000","last_peered":"2024-09-10T11:30:26.349405+0000","last_clean":"2024-09-10T11:30:26.349405+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:26.349405+0000","last_undegraded":"2024-09-10T11:30:26.349405+0000","last_fullsized":"2024-09-10T11:30:26.349405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352072+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:26.352072+0000","last_peered":"2024-09-10T11:30:26.352072+0000","last_clean":"2024-09-10T11:30:26.352072+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:26.352072+0000","last_undegraded":"2024-09-10T11:30:26.352072+0000","last_fullsized":"2024-09-10T11:30:26.352072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:52.596921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343276+0000","last_change":"2024-09-10T11:30:26.343276+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343276+0000","last_undegraded":"2024-09-10T11:30:26.343276+0000","last_fullsized":"2024-09-10T11:30:26.343276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357911+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:26.357911+0000","last_peered":"2024-09-10T11:30:26.357911+0000","last_clean":"2024-09-10T11:30:26.357911+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:26.357911+0000","last_undegraded":"2024-09-10T11:30:26.357911+0000","last_fullsized":"2024-09-10T11:30:26.357911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:53.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355632+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:26.355632+0000","last_peered":"2024-09-10T11:30:26.355632+0000","last_clean":"2024-09-10T11:30:26.355632+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:26.355632+0000","last_undegraded":"2024-09-10T11:30:26.355632+0000","last_fullsized":"2024-09-10T11:30:26.355632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.067158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353205+0000","last_change":"2024-09-10T11:30:26.353205+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353205+0000","last_undegraded":"2024-09-10T11:30:26.353205+0000","last_fullsized":"2024-09-10T11:30:26.353205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351869+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:26.351869+0000","last_peered":"2024-09-10T11:30:26.351869+0000","last_clean":"2024-09-10T11:30:26.351869+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:26.351869+0000","last_undegraded":"2024-09-10T11:30:26.351869+0000","last_fullsized":"2024-09-10T11:30:26.351869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:01.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346396+0000","last_change":"2024-09-10T11:30:26.346396+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346396+0000","last_undegraded":"2024-09-10T11:30:26.346396+0000","last_fullsized":"2024-09-10T11:30:26.346396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356683+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:26.356683+0000","last_peered":"2024-09-10T11:30:26.356683+0000","last_clean":"2024-09-10T11:30:26.356683+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:26.356683+0000","last_undegraded":"2024-09-10T11:30:26.356683+0000","last_fullsized":"2024-09-10T11:30:26.356683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:05.310043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342388+0000","last_change":"2024-09-10T11:30:26.342388+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342388+0000","last_undegraded":"2024-09-10T11:30:26.342388+0000","last_fullsized":"2024-09-10T11:30:26.342388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353658+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:26.353658+0000","last_peered":"2024-09-10T11:30:26.353658+0000","last_clean":"2024-09-10T11:30:26.353658+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:26.353658+0000","last_undegraded":"2024-09-10T11:30:26.353658+0000","last_fullsized":"2024-09-10T11:30:26.353658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:33.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349242+0000","last_change":"2024-09-10T11:30:26.349242+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349242+0000","last_undegraded":"2024-09-10T11:30:26.349242+0000","last_fullsized":"2024-09-10T11:30:26.349242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351809+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:26.351809+0000","last_peered":"2024-09-10T11:30:26.351809+0000","last_clean":"2024-09-10T11:30:26.351809+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:26.351809+0000","last_undegraded":"2024-09-10T11:30:26.351809+0000","last_fullsized":"2024-09-10T11:30:26.351809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:39.942357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344018+0000","last_change":"2024-09-10T11:30:26.344018+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344018+0000","last_undegraded":"2024-09-10T11:30:26.344018+0000","last_fullsized":"2024-09-10T11:30:26.344018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:04.011154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344195+0000","last_change":"2024-09-10T11:30:26.344195+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344195+0000","last_undegraded":"2024-09-10T11:30:26.344195+0000","last_fullsized":"2024-09-10T11:30:26.344195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352963+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:26.352963+0000","last_peered":"2024-09-10T11:30:26.352963+0000","last_clean":"2024-09-10T11:30:26.352963+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:26.352963+0000","last_undegraded":"2024-09-10T11:30:26.352963+0000","last_fullsized":"2024-09-10T11:30:26.352963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:12.773758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345247+0000","last_change":"2024-09-10T11:30:26.345247+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345247+0000","last_undegraded":"2024-09-10T11:30:26.345247+0000","last_fullsized":"2024-09-10T11:30:26.345247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350527+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:26.350527+0000","last_peered":"2024-09-10T11:30:26.350527+0000","last_clean":"2024-09-10T11:30:26.350527+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:26.350527+0000","last_undegraded":"2024-09-10T11:30:26.350527+0000","last_fullsized":"2024-09-10T11:30:26.350527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:21.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343602+0000","last_change":"2024-09-10T11:30:26.343602+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343602+0000","last_undegraded":"2024-09-10T11:30:26.343602+0000","last_fullsized":"2024-09-10T11:30:26.343602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358594+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:26.358594+0000","last_peered":"2024-09-10T11:30:26.358594+0000","last_clean":"2024-09-10T11:30:26.358594+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:26.358594+0000","last_undegraded":"2024-09-10T11:30:26.358594+0000","last_fullsized":"2024-09-10T11:30:26.358594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:01.475428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352810+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:26.352810+0000","last_peered":"2024-09-10T11:30:26.352810+0000","last_clean":"2024-09-10T11:30:26.352810+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:26.352810+0000","last_undegraded":"2024-09-10T11:30:26.352810+0000","last_fullsized":"2024-09-10T11:30:26.352810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:57.958396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345612+0000","last_change":"2024-09-10T11:30:26.345612+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345612+0000","last_undegraded":"2024-09-10T11:30:26.345612+0000","last_fullsized":"2024-09-10T11:30:26.345612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347450+0000","last_change":"2024-09-10T11:30:26.347450+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347450+0000","last_undegraded":"2024-09-10T11:30:26.347450+0000","last_fullsized":"2024-09-10T11:30:26.347450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355021+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:26.355021+0000","last_peered":"2024-09-10T11:30:26.355021+0000","last_clean":"2024-09-10T11:30:26.355021+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:26.355021+0000","last_undegraded":"2024-09-10T11:30:26.355021+0000","last_fullsized":"2024-09-10T11:30:26.355021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:11.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347966+0000","last_change":"2024-09-10T11:30:26.347966+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347966+0000","last_undegraded":"2024-09-10T11:30:26.347966+0000","last_fullsized":"2024-09-10T11:30:26.347966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352946+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:26.352946+0000","last_peered":"2024-09-10T11:30:26.352946+0000","last_clean":"2024-09-10T11:30:26.352946+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:26.352946+0000","last_undegraded":"2024-09-10T11:30:26.352946+0000","last_fullsized":"2024-09-10T11:30:26.352946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:30.209672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349430+0000","last_change":"2024-09-10T11:30:26.349430+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349430+0000","last_undegraded":"2024-09-10T11:30:26.349430+0000","last_fullsized":"2024-09-10T11:30:26.349430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356992+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:26.356992+0000","last_peered":"2024-09-10T11:30:26.356992+0000","last_clean":"2024-09-10T11:30:26.356992+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:26.356992+0000","last_undegraded":"2024-09-10T11:30:26.356992+0000","last_fullsized":"2024-09-10T11:30:26.356992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:05.275854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353527+0000","last_change":"2024-09-10T11:30:26.353527+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353527+0000","last_undegraded":"2024-09-10T11:30:26.353527+0000","last_fullsized":"2024-09-10T11:30:26.353527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357789+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:26.357789+0000","last_peered":"2024-09-10T11:30:26.357789+0000","last_clean":"2024-09-10T11:30:26.357789+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:26.357789+0000","last_undegraded":"2024-09-10T11:30:26.357789+0000","last_fullsized":"2024-09-10T11:30:26.357789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:16.906847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:30.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.764+0000 7fe6b2da6640 1 Processor -- start 2024-09-10T11:30:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.764+0000 7fe6b2da6640 1 -- start start 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 0x7fe6ac0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac0d2990 con 0x7fe6ac15efc0 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac0d2c70 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b2da6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac0d2f50 con 0x7fe6ac0d15a0 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b0b42640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b0b42640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56980/0 (socket says 172.21.15.99:56980) 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b0b42640 1 -- 172.21.15.99:0/3610599001 learned_addr learned my addr 172.21.15.99:0/3610599001 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6abfff640 1 --2- 172.21.15.99:0/3610599001 >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 0x7fe6ac0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b1343640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b0b42640 1 -- 172.21.15.99:0/3610599001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 msgr2=0x7fe6ac0d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.765+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6b0b42640 1 -- 172.21.15.99:0/3610599001 >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 msgr2=0x7fe6ac0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/3610599001 >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 0x7fe6ac0d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6b0b42640 1 -- 172.21.15.99:0/3610599001 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac0d3230 con 0x7fe6ac15efc0 2024-09-10T11:30:30.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6abfff640 1 --2- 172.21.15.99:0/3610599001 >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 0x7fe6ac0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0d4400 tx=0x7fe6a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1510355ae1fd5490 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6ab7fe640 1 -- 172.21.15.99:0/3610599001 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0037070 con 0x7fe6ac15efc0 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6ab7fe640 1 -- 172.21.15.99:0/3610599001 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a0032440 con 0x7fe6ac15efc0 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.766+0000 7fe6ab7fe640 1 -- 172.21.15.99:0/3610599001 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0032810 con 0x7fe6ac15efc0 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 -- 172.21.15.99:0/3610599001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac15efc0 msgr2=0x7fe6ac0d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0d4400 tx=0x7fe6a002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 -- 172.21.15.99:0/3610599001 shutdown_connections 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac0d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/3610599001 >> v2:172.21.15.196:3300/0 conn(0x7fe6ac0d3a30 0x7fe6ac0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/3610599001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0d03f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 -- 172.21.15.99:0/3610599001 >> 172.21.15.99:0/3610599001 conn(0x7fe6ac0d4c20 msgr2=0x7fe6ac0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 -- 172.21.15.99:0/3610599001 shutdown_connections 2024-09-10T11:30:30.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.767+0000 7fe6b2da6640 1 -- 172.21.15.99:0/3610599001 wait complete. 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.768+0000 7fe6b2da6640 1 Processor -- start 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.768+0000 7fe6b2da6640 1 -- start start 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac162d90 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac162c10 con 0x7fe6ac15efc0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b2da6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac162f10 con 0x7fe6ac0d15a0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56982/0 (socket says 172.21.15.99:56982) 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 -- 172.21.15.99:0/4105165994 learned_addr learned my addr 172.21.15.99:0/4105165994 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6b1343640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 -- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 msgr2=0x7fe6ac1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 -- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6ac15efc0 msgr2=0x7fe6ac0b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.769+0000 7fe6abfff640 1 -- 172.21.15.99:0/4105165994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac15a9e0 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6abfff640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c7f0 tx=0x7fe69800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698004230 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6980043d0 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac0b7c80 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe698007a70 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.770+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac0b8240 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.771+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ac0b7a70 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.771+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe698007c10 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.772+0000 7fe6a97fa640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 0x7fe69006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.772+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe69808aed0 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.772+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 0x7fe69006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:30.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.772+0000 7fe6b0b42640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 0x7fe69006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0034000 tx=0x7fe6a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:30.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.776+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe698015020 con 0x7fe6ac0d3a30 2024-09-10T11:30:30.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.942+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6ac0d1a50 con 0x7fe69006be30 2024-09-10T11:30:30.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.947+0000 7fe6a97fa640 1 -- 172.21.15.99:0/4105165994 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242211 (secure 0 0 0) 0x7fe6ac0d1a50 con 0x7fe69006be30 2024-09-10T11:30:30.951 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:30.952 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.950+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 msgr2=0x7fe69006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.950+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 0x7fe69006e2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0034000 tx=0x7fe6a002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.950+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 msgr2=0x7fe6ac15a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.950+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe69800c7f0 tx=0x7fe69800ccb0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 shutdown_connections 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe69006be30 0x7fe69006e2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6ac15efc0 0x7fe6ac0b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe6ac0d3a30 0x7fe6ac15a2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 --2- 172.21.15.99:0/4105165994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6ac0d15a0 0x7fe6ac1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 >> 172.21.15.99:0/4105165994 conn(0x7fe6ac0d4c20 msgr2=0x7fe6ac0b53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 shutdown_connections 2024-09-10T11:30:30.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:30.951+0000 7fe6b2da6640 1 -- 172.21.15.99:0/4105165994 wait complete. 2024-09-10T11:30:30.963 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-10T11:30:30.907445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598407872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570122"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347078+0000","last_change":"2024-09-10T11:30:26.347078+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347078+0000","last_undegraded":"2024-09-10T11:30:26.347078+0000","last_fullsized":"2024-09-10T11:30:26.347078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349754+0000","last_change":"2024-09-10T11:30:26.349754+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349754+0000","last_undegraded":"2024-09-10T11:30:26.349754+0000","last_fullsized":"2024-09-10T11:30:26.349754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.351662+0000","last_change":"2024-09-10T11:30:26.351662+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.351662+0000","last_undegraded":"2024-09-10T11:30:26.351662+0000","last_fullsized":"2024-09-10T11:30:26.351662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343642+0000","last_change":"2024-09-10T11:30:26.343642+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343642+0000","last_undegraded":"2024-09-10T11:30:26.343642+0000","last_fullsized":"2024-09-10T11:30:26.343642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344216+0000","last_change":"2024-09-10T11:30:26.344216+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344216+0000","last_undegraded":"2024-09-10T11:30:26.344216+0000","last_fullsized":"2024-09-10T11:30:26.344216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342938+0000","last_change":"2024-09-10T11:30:26.342938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342938+0000","last_undegraded":"2024-09-10T11:30:26.342938+0000","last_fullsized":"2024-09-10T11:30:26.342938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348539+0000","last_change":"2024-09-10T11:30:26.348539+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348539+0000","last_undegraded":"2024-09-10T11:30:26.348539+0000","last_fullsized":"2024-09-10T11:30:26.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350001+0000","last_change":"2024-09-10T11:30:26.350001+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350001+0000","last_undegraded":"2024-09-10T11:30:26.350001+0000","last_fullsized":"2024-09-10T11:30:26.350001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348916+0000","last_change":"2024-09-10T11:30:26.348916+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348916+0000","last_undegraded":"2024-09-10T11:30:26.348916+0000","last_fullsized":"2024-09-10T11:30:26.348916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345319+0000","last_change":"2024-09-10T11:30:26.345319+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345319+0000","last_undegraded":"2024-09-10T11:30:26.345319+0000","last_fullsized":"2024-09-10T11:30:26.345319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346748+0000","last_change":"2024-09-10T11:30:26.346748+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346748+0000","last_undegraded":"2024-09-10T11:30:26.346748+0000","last_fullsized":"2024-09-10T11:30:26.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350070+0000","last_change":"2024-09-10T11:30:26.350070+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350070+0000","last_undegraded":"2024-09-10T11:30:26.350070+0000","last_fullsized":"2024-09-10T11:30:26.350070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350068+0000","last_change":"2024-09-10T11:30:26.350068+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350068+0000","last_undegraded":"2024-09-10T11:30:26.350068+0000","last_fullsized":"2024-09-10T11:30:26.350068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349848+0000","last_change":"2024-09-10T11:30:26.349848+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349848+0000","last_undegraded":"2024-09-10T11:30:26.349848+0000","last_fullsized":"2024-09-10T11:30:26.349848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353163+0000","last_change":"2024-09-10T11:30:26.353163+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353163+0000","last_undegraded":"2024-09-10T11:30:26.353163+0000","last_fullsized":"2024-09-10T11:30:26.353163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349515+0000","last_change":"2024-09-10T11:30:26.349515+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349515+0000","last_undegraded":"2024-09-10T11:30:26.349515+0000","last_fullsized":"2024-09-10T11:30:26.349515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344553+0000","last_change":"2024-09-10T11:30:26.344553+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344553+0000","last_undegraded":"2024-09-10T11:30:26.344553+0000","last_fullsized":"2024-09-10T11:30:26.344553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348768+0000","last_change":"2024-09-10T11:30:26.348768+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348768+0000","last_undegraded":"2024-09-10T11:30:26.348768+0000","last_fullsized":"2024-09-10T11:30:26.348768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345295+0000","last_change":"2024-09-10T11:30:26.345295+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345295+0000","last_undegraded":"2024-09-10T11:30:26.345295+0000","last_fullsized":"2024-09-10T11:30:26.345295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344859+0000","last_change":"2024-09-10T11:30:26.344859+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344859+0000","last_undegraded":"2024-09-10T11:30:26.344859+0000","last_fullsized":"2024-09-10T11:30:26.344859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344914+0000","last_change":"2024-09-10T11:30:26.344914+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344914+0000","last_undegraded":"2024-09-10T11:30:26.344914+0000","last_fullsized":"2024-09-10T11:30:26.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346106+0000","last_change":"2024-09-10T11:30:26.346106+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346106+0000","last_undegraded":"2024-09-10T11:30:26.346106+0000","last_fullsized":"2024-09-10T11:30:26.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348811+0000","last_change":"2024-09-10T11:30:26.348811+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348811+0000","last_undegraded":"2024-09-10T11:30:26.348811+0000","last_fullsized":"2024-09-10T11:30:26.348811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346246+0000","last_change":"2024-09-10T11:30:26.346246+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346246+0000","last_undegraded":"2024-09-10T11:30:26.346246+0000","last_fullsized":"2024-09-10T11:30:26.346246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352548+0000","last_change":"2024-09-10T11:30:26.352548+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352548+0000","last_undegraded":"2024-09-10T11:30:26.352548+0000","last_fullsized":"2024-09-10T11:30:26.352548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342369+0000","last_change":"2024-09-10T11:30:26.342369+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342369+0000","last_undegraded":"2024-09-10T11:30:26.342369+0000","last_fullsized":"2024-09-10T11:30:26.342369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349505+0000","last_change":"2024-09-10T11:30:26.349505+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349505+0000","last_undegraded":"2024-09-10T11:30:26.349505+0000","last_fullsized":"2024-09-10T11:30:26.349505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349202+0000","last_change":"2024-09-10T11:30:26.349202+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349202+0000","last_undegraded":"2024-09-10T11:30:26.349202+0000","last_fullsized":"2024-09-10T11:30:26.349202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353929+0000","last_change":"2024-09-10T11:30:26.353929+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353929+0000","last_undegraded":"2024-09-10T11:30:26.353929+0000","last_fullsized":"2024-09-10T11:30:26.353929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348630+0000","last_change":"2024-09-10T11:30:26.348630+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348630+0000","last_undegraded":"2024-09-10T11:30:26.348630+0000","last_fullsized":"2024-09-10T11:30:26.348630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347834+0000","last_change":"2024-09-10T11:30:26.347834+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347834+0000","last_undegraded":"2024-09-10T11:30:26.347834+0000","last_fullsized":"2024-09-10T11:30:26.347834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343398+0000","last_change":"2024-09-10T11:30:26.343398+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343398+0000","last_undegraded":"2024-09-10T11:30:26.343398+0000","last_fullsized":"2024-09-10T11:30:26.343398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347190+0000","last_change":"2024-09-10T11:30:26.347190+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347190+0000","last_undegraded":"2024-09-10T11:30:26.347190+0000","last_fullsized":"2024-09-10T11:30:26.347190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355648+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:26.355648+0000","last_peered":"2024-09-10T11:30:26.355648+0000","last_clean":"2024-09-10T11:30:26.355648+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:26.355648+0000","last_undegraded":"2024-09-10T11:30:26.355648+0000","last_fullsized":"2024-09-10T11:30:26.355648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:01.364851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344409+0000","last_change":"2024-09-10T11:30:26.344409+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344409+0000","last_undegraded":"2024-09-10T11:30:26.344409+0000","last_fullsized":"2024-09-10T11:30:26.344409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352706+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:26.352706+0000","last_peered":"2024-09-10T11:30:26.352706+0000","last_clean":"2024-09-10T11:30:26.352706+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:26.352706+0000","last_undegraded":"2024-09-10T11:30:26.352706+0000","last_fullsized":"2024-09-10T11:30:26.352706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:02.279667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350997+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:26.350997+0000","last_peered":"2024-09-10T11:30:26.350997+0000","last_clean":"2024-09-10T11:30:26.350997+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:26.350997+0000","last_undegraded":"2024-09-10T11:30:26.350997+0000","last_fullsized":"2024-09-10T11:30:26.350997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:20.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352083+0000","last_change":"2024-09-10T11:30:26.352083+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352083+0000","last_undegraded":"2024-09-10T11:30:26.352083+0000","last_fullsized":"2024-09-10T11:30:26.352083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353752+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:26.353752+0000","last_peered":"2024-09-10T11:30:26.353752+0000","last_clean":"2024-09-10T11:30:26.353752+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:26.353752+0000","last_undegraded":"2024-09-10T11:30:26.353752+0000","last_fullsized":"2024-09-10T11:30:26.353752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:16.268202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344235+0000","last_change":"2024-09-10T11:30:26.344235+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344235+0000","last_undegraded":"2024-09-10T11:30:26.344235+0000","last_fullsized":"2024-09-10T11:30:26.344235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352206+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:26.352206+0000","last_peered":"2024-09-10T11:30:26.352206+0000","last_clean":"2024-09-10T11:30:26.352206+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:26.352206+0000","last_undegraded":"2024-09-10T11:30:26.352206+0000","last_fullsized":"2024-09-10T11:30:26.352206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:45.803657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342684+0000","last_change":"2024-09-10T11:30:26.342684+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342684+0000","last_undegraded":"2024-09-10T11:30:26.342684+0000","last_fullsized":"2024-09-10T11:30:26.342684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356269+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:26.356269+0000","last_peered":"2024-09-10T11:30:26.356269+0000","last_clean":"2024-09-10T11:30:26.356269+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:26.356269+0000","last_undegraded":"2024-09-10T11:30:26.356269+0000","last_fullsized":"2024-09-10T11:30:26.356269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:02.709036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350537+0000","last_change":"2024-09-10T11:30:26.350537+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350537+0000","last_undegraded":"2024-09-10T11:30:26.350537+0000","last_fullsized":"2024-09-10T11:30:26.350537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347731+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:26.347731+0000","last_peered":"2024-09-10T11:30:26.347731+0000","last_clean":"2024-09-10T11:30:26.347731+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:26.347731+0000","last_undegraded":"2024-09-10T11:30:26.347731+0000","last_fullsized":"2024-09-10T11:30:26.347731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:41.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345938+0000","last_change":"2024-09-10T11:30:26.345938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345938+0000","last_undegraded":"2024-09-10T11:30:26.345938+0000","last_fullsized":"2024-09-10T11:30:26.345938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.348741+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:26.348741+0000","last_peered":"2024-09-10T11:30:26.348741+0000","last_clean":"2024-09-10T11:30:26.348741+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:26.348741+0000","last_undegraded":"2024-09-10T11:30:26.348741+0000","last_fullsized":"2024-09-10T11:30:26.348741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345128+0000","last_change":"2024-09-10T11:30:26.345128+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345128+0000","last_undegraded":"2024-09-10T11:30:26.345128+0000","last_fullsized":"2024-09-10T11:30:26.345128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351884+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:26.351884+0000","last_peered":"2024-09-10T11:30:26.351884+0000","last_clean":"2024-09-10T11:30:26.351884+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:26.351884+0000","last_undegraded":"2024-09-10T11:30:26.351884+0000","last_fullsized":"2024-09-10T11:30:26.351884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:14.416062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349865+0000","last_change":"2024-09-10T11:30:26.349865+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349865+0000","last_undegraded":"2024-09-10T11:30:26.349865+0000","last_fullsized":"2024-09-10T11:30:26.349865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.354977+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:26.354977+0000","last_peered":"2024-09-10T11:30:26.354977+0000","last_clean":"2024-09-10T11:30:26.354977+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:26.354977+0000","last_undegraded":"2024-09-10T11:30:26.354977+0000","last_fullsized":"2024-09-10T11:30:26.354977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:34.505783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347122+0000","last_change":"2024-09-10T11:30:26.347122+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347122+0000","last_undegraded":"2024-09-10T11:30:26.347122+0000","last_fullsized":"2024-09-10T11:30:26.347122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356474+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:26.356474+0000","last_peered":"2024-09-10T11:30:26.356474+0000","last_clean":"2024-09-10T11:30:26.356474+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:26.356474+0000","last_undegraded":"2024-09-10T11:30:26.356474+0000","last_fullsized":"2024-09-10T11:30:26.356474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:37.442331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345736+0000","last_change":"2024-09-10T11:30:26.345736+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345736+0000","last_undegraded":"2024-09-10T11:30:26.345736+0000","last_fullsized":"2024-09-10T11:30:26.345736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347100+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:26.347100+0000","last_peered":"2024-09-10T11:30:26.347100+0000","last_clean":"2024-09-10T11:30:26.347100+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:26.347100+0000","last_undegraded":"2024-09-10T11:30:26.347100+0000","last_fullsized":"2024-09-10T11:30:26.347100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:38.630516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344204+0000","last_change":"2024-09-10T11:30:26.344204+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344204+0000","last_undegraded":"2024-09-10T11:30:26.344204+0000","last_fullsized":"2024-09-10T11:30:26.344204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358330+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:26.358330+0000","last_peered":"2024-09-10T11:30:26.358330+0000","last_clean":"2024-09-10T11:30:26.358330+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:26.358330+0000","last_undegraded":"2024-09-10T11:30:26.358330+0000","last_fullsized":"2024-09-10T11:30:26.358330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:28.172249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342367+0000","last_change":"2024-09-10T11:30:26.342367+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342367+0000","last_undegraded":"2024-09-10T11:30:26.342367+0000","last_fullsized":"2024-09-10T11:30:26.342367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:44.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345905+0000","last_change":"2024-09-10T11:30:26.345905+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345905+0000","last_undegraded":"2024-09-10T11:30:26.345905+0000","last_fullsized":"2024-09-10T11:30:26.345905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357762+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:26.357762+0000","last_peered":"2024-09-10T11:30:26.357762+0000","last_clean":"2024-09-10T11:30:26.357762+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:26.357762+0000","last_undegraded":"2024-09-10T11:30:26.357762+0000","last_fullsized":"2024-09-10T11:30:26.357762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349562+0000","last_change":"2024-09-10T11:30:26.349562+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349562+0000","last_undegraded":"2024-09-10T11:30:26.349562+0000","last_fullsized":"2024-09-10T11:30:26.349562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357477+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:26.357477+0000","last_peered":"2024-09-10T11:30:26.357477+0000","last_clean":"2024-09-10T11:30:26.357477+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:26.357477+0000","last_undegraded":"2024-09-10T11:30:26.357477+0000","last_fullsized":"2024-09-10T11:30:26.357477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:28.358480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347691+0000","last_change":"2024-09-10T11:30:26.347691+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347691+0000","last_undegraded":"2024-09-10T11:30:26.347691+0000","last_fullsized":"2024-09-10T11:30:26.347691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.349405+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:26.349405+0000","last_peered":"2024-09-10T11:30:26.349405+0000","last_clean":"2024-09-10T11:30:26.349405+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:26.349405+0000","last_undegraded":"2024-09-10T11:30:26.349405+0000","last_fullsized":"2024-09-10T11:30:26.349405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352072+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:26.352072+0000","last_peered":"2024-09-10T11:30:26.352072+0000","last_clean":"2024-09-10T11:30:26.352072+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:26.352072+0000","last_undegraded":"2024-09-10T11:30:26.352072+0000","last_fullsized":"2024-09-10T11:30:26.352072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:52.596921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343276+0000","last_change":"2024-09-10T11:30:26.343276+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343276+0000","last_undegraded":"2024-09-10T11:30:26.343276+0000","last_fullsized":"2024-09-10T11:30:26.343276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357911+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:26.357911+0000","last_peered":"2024-09-10T11:30:26.357911+0000","last_clean":"2024-09-10T11:30:26.357911+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:26.357911+0000","last_undegraded":"2024-09-10T11:30:26.357911+0000","last_fullsized":"2024-09-10T11:30:26.357911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:53.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355632+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:26.355632+0000","last_peered":"2024-09-10T11:30:26.355632+0000","last_clean":"2024-09-10T11:30:26.355632+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:26.355632+0000","last_undegraded":"2024-09-10T11:30:26.355632+0000","last_fullsized":"2024-09-10T11:30:26.355632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.067158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353205+0000","last_change":"2024-09-10T11:30:26.353205+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353205+0000","last_undegraded":"2024-09-10T11:30:26.353205+0000","last_fullsized":"2024-09-10T11:30:26.353205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351869+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:26.351869+0000","last_peered":"2024-09-10T11:30:26.351869+0000","last_clean":"2024-09-10T11:30:26.351869+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:26.351869+0000","last_undegraded":"2024-09-10T11:30:26.351869+0000","last_fullsized":"2024-09-10T11:30:26.351869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:01.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346396+0000","last_change":"2024-09-10T11:30:26.346396+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346396+0000","last_undegraded":"2024-09-10T11:30:26.346396+0000","last_fullsized":"2024-09-10T11:30:26.346396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356683+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:26.356683+0000","last_peered":"2024-09-10T11:30:26.356683+0000","last_clean":"2024-09-10T11:30:26.356683+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:26.356683+0000","last_undegraded":"2024-09-10T11:30:26.356683+0000","last_fullsized":"2024-09-10T11:30:26.356683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:05.310043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342388+0000","last_change":"2024-09-10T11:30:26.342388+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342388+0000","last_undegraded":"2024-09-10T11:30:26.342388+0000","last_fullsized":"2024-09-10T11:30:26.342388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353658+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:26.353658+0000","last_peered":"2024-09-10T11:30:26.353658+0000","last_clean":"2024-09-10T11:30:26.353658+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:26.353658+0000","last_undegraded":"2024-09-10T11:30:26.353658+0000","last_fullsized":"2024-09-10T11:30:26.353658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:33.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349242+0000","last_change":"2024-09-10T11:30:26.349242+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349242+0000","last_undegraded":"2024-09-10T11:30:26.349242+0000","last_fullsized":"2024-09-10T11:30:26.349242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351809+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:26.351809+0000","last_peered":"2024-09-10T11:30:26.351809+0000","last_clean":"2024-09-10T11:30:26.351809+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:26.351809+0000","last_undegraded":"2024-09-10T11:30:26.351809+0000","last_fullsized":"2024-09-10T11:30:26.351809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:39.942357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344018+0000","last_change":"2024-09-10T11:30:26.344018+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344018+0000","last_undegraded":"2024-09-10T11:30:26.344018+0000","last_fullsized":"2024-09-10T11:30:26.344018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:04.011154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344195+0000","last_change":"2024-09-10T11:30:26.344195+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344195+0000","last_undegraded":"2024-09-10T11:30:26.344195+0000","last_fullsized":"2024-09-10T11:30:26.344195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352963+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:26.352963+0000","last_peered":"2024-09-10T11:30:26.352963+0000","last_clean":"2024-09-10T11:30:26.352963+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:26.352963+0000","last_undegraded":"2024-09-10T11:30:26.352963+0000","last_fullsized":"2024-09-10T11:30:26.352963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:12.773758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345247+0000","last_change":"2024-09-10T11:30:26.345247+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345247+0000","last_undegraded":"2024-09-10T11:30:26.345247+0000","last_fullsized":"2024-09-10T11:30:26.345247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350527+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:26.350527+0000","last_peered":"2024-09-10T11:30:26.350527+0000","last_clean":"2024-09-10T11:30:26.350527+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:26.350527+0000","last_undegraded":"2024-09-10T11:30:26.350527+0000","last_fullsized":"2024-09-10T11:30:26.350527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:21.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343602+0000","last_change":"2024-09-10T11:30:26.343602+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343602+0000","last_undegraded":"2024-09-10T11:30:26.343602+0000","last_fullsized":"2024-09-10T11:30:26.343602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358594+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:26.358594+0000","last_peered":"2024-09-10T11:30:26.358594+0000","last_clean":"2024-09-10T11:30:26.358594+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:26.358594+0000","last_undegraded":"2024-09-10T11:30:26.358594+0000","last_fullsized":"2024-09-10T11:30:26.358594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:01.475428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352810+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:26.352810+0000","last_peered":"2024-09-10T11:30:26.352810+0000","last_clean":"2024-09-10T11:30:26.352810+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:26.352810+0000","last_undegraded":"2024-09-10T11:30:26.352810+0000","last_fullsized":"2024-09-10T11:30:26.352810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:57.958396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345612+0000","last_change":"2024-09-10T11:30:26.345612+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345612+0000","last_undegraded":"2024-09-10T11:30:26.345612+0000","last_fullsized":"2024-09-10T11:30:26.345612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347450+0000","last_change":"2024-09-10T11:30:26.347450+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347450+0000","last_undegraded":"2024-09-10T11:30:26.347450+0000","last_fullsized":"2024-09-10T11:30:26.347450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355021+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:26.355021+0000","last_peered":"2024-09-10T11:30:26.355021+0000","last_clean":"2024-09-10T11:30:26.355021+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:26.355021+0000","last_undegraded":"2024-09-10T11:30:26.355021+0000","last_fullsized":"2024-09-10T11:30:26.355021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:11.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347966+0000","last_change":"2024-09-10T11:30:26.347966+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347966+0000","last_undegraded":"2024-09-10T11:30:26.347966+0000","last_fullsized":"2024-09-10T11:30:26.347966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352946+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:26.352946+0000","last_peered":"2024-09-10T11:30:26.352946+0000","last_clean":"2024-09-10T11:30:26.352946+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:26.352946+0000","last_undegraded":"2024-09-10T11:30:26.352946+0000","last_fullsized":"2024-09-10T11:30:26.352946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:30.209672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349430+0000","last_change":"2024-09-10T11:30:26.349430+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349430+0000","last_undegraded":"2024-09-10T11:30:26.349430+0000","last_fullsized":"2024-09-10T11:30:26.349430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356992+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:26.356992+0000","last_peered":"2024-09-10T11:30:26.356992+0000","last_clean":"2024-09-10T11:30:26.356992+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:26.356992+0000","last_undegraded":"2024-09-10T11:30:26.356992+0000","last_fullsized":"2024-09-10T11:30:26.356992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:05.275854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353527+0000","last_change":"2024-09-10T11:30:26.353527+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353527+0000","last_undegraded":"2024-09-10T11:30:26.353527+0000","last_fullsized":"2024-09-10T11:30:26.353527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357789+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:26.357789+0000","last_peered":"2024-09-10T11:30:26.357789+0000","last_clean":"2024-09-10T11:30:26.357789+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:26.357789+0000","last_undegraded":"2024-09-10T11:30:26.357789+0000","last_fullsized":"2024-09-10T11:30:26.357789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:16.906847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:30.966 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347078+0000', 'last_change': '2024-09-10T11:30:26.347078+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347078+0000', 'last_undegraded': '2024-09-10T11:30:26.347078+0000', 'last_fullsized': '2024-09-10T11:30:26.347078+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349754+0000', 'last_change': '2024-09-10T11:30:26.349754+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349754+0000', 'last_undegraded': '2024-09-10T11:30:26.349754+0000', 'last_fullsized': '2024-09-10T11:30:26.349754+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.351662+0000', 'last_change': '2024-09-10T11:30:26.351662+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.351662+0000', 'last_undegraded': '2024-09-10T11:30:26.351662+0000', 'last_fullsized': '2024-09-10T11:30:26.351662+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.343642+0000', 'last_change': '2024-09-10T11:30:26.343642+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.343642+0000', 'last_undegraded': '2024-09-10T11:30:26.343642+0000', 'last_fullsized': '2024-09-10T11:30:26.343642+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-10T11:30:30.967 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344216+0000', 'last_change': '2024-09-10T11:30:26.344216+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344216+0000', 'last_undegraded': '2024-09-10T11:30:26.344216+0000', 'last_fullsized': '2024-09-10T11:30:26.344216+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.342938+0000', 'last_change': '2024-09-10T11:30:26.342938+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.342938+0000', 'last_undegraded': '2024-09-10T11:30:26.342938+0000', 'last_fullsized': '2024-09-10T11:30:26.342938+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348539+0000', 'last_change': '2024-09-10T11:30:26.348539+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348539+0000', 'last_undegraded': '2024-09-10T11:30:26.348539+0000', 'last_fullsized': '2024-09-10T11:30:26.348539+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.350001+0000', 'last_change': '2024-09-10T11:30:26.350001+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.350001+0000', 'last_undegraded': '2024-09-10T11:30:26.350001+0000', 'last_fullsized': '2024-09-10T11:30:26.350001+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348916+0000', 'last_change': '2024-09-10T11:30:26.348916+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348916+0000', 'last_undegraded': '2024-09-10T11:30:26.348916+0000', 'last_fullsized': '2024-09-10T11:30:26.348916+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-10T11:30:30.968 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345319+0000', 'last_change': '2024-09-10T11:30:26.345319+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345319+0000', 'last_undegraded': '2024-09-10T11:30:26.345319+0000', 'last_fullsized': '2024-09-10T11:30:26.345319+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.346748+0000', 'last_change': '2024-09-10T11:30:26.346748+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.346748+0000', 'last_undegraded': '2024-09-10T11:30:26.346748+0000', 'last_fullsized': '2024-09-10T11:30:26.346748+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.350070+0000', 'last_change': '2024-09-10T11:30:26.350070+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.350070+0000', 'last_undegraded': '2024-09-10T11:30:26.350070+0000', 'last_fullsized': '2024-09-10T11:30:26.350070+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.350068+0000', 'last_change': '2024-09-10T11:30:26.350068+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.350068+0000', 'last_undegraded': '2024-09-10T11:30:26.350068+0000', 'last_fullsized': '2024-09-10T11:30:26.350068+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349848+0000', 'last_change': '2024-09-10T11:30:26.349848+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349848+0000', 'last_undegraded': '2024-09-10T11:30:26.349848+0000', 'last_fullsized': '2024-09-10T11:30:26.349848+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.353163+0000', 'last_change': '2024-09-10T11:30:26.353163+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.353163+0000', 'last_undegraded': '2024-09-10T11:30:26.353163+0000', 'last_fullsized': '2024-09-10T11:30:26.353163+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-10T11:30:30.969 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349515+0000', 'last_change': '2024-09-10T11:30:26.349515+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349515+0000', 'last_undegraded': '2024-09-10T11:30:26.349515+0000', 'last_fullsized': '2024-09-10T11:30:26.349515+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344553+0000', 'last_change': '2024-09-10T11:30:26.344553+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344553+0000', 'last_undegraded': '2024-09-10T11:30:26.344553+0000', 'last_fullsized': '2024-09-10T11:30:26.344553+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348768+0000', 'last_change': '2024-09-10T11:30:26.348768+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348768+0000', 'last_undegraded': '2024-09-10T11:30:26.348768+0000', 'last_fullsized': '2024-09-10T11:30:26.348768+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345295+0000', 'last_change': '2024-09-10T11:30:26.345295+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345295+0000', 'last_undegraded': '2024-09-10T11:30:26.345295+0000', 'last_fullsized': '2024-09-10T11:30:26.345295+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344859+0000', 'last_change': '2024-09-10T11:30:26.344859+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344859+0000', 'last_undegraded': '2024-09-10T11:30:26.344859+0000', 'last_fullsized': '2024-09-10T11:30:26.344859+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344914+0000', 'last_change': '2024-09-10T11:30:26.344914+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344914+0000', 'last_undegraded': '2024-09-10T11:30:26.344914+0000', 'last_fullsized': '2024-09-10T11:30:26.344914+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-10T11:30:30.970 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.346106+0000', 'last_change': '2024-09-10T11:30:26.346106+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.346106+0000', 'last_undegraded': '2024-09-10T11:30:26.346106+0000', 'last_fullsized': '2024-09-10T11:30:26.346106+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348811+0000', 'last_change': '2024-09-10T11:30:26.348811+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348811+0000', 'last_undegraded': '2024-09-10T11:30:26.348811+0000', 'last_fullsized': '2024-09-10T11:30:26.348811+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.346246+0000', 'last_change': '2024-09-10T11:30:26.346246+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.346246+0000', 'last_undegraded': '2024-09-10T11:30:26.346246+0000', 'last_fullsized': '2024-09-10T11:30:26.346246+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.352548+0000', 'last_change': '2024-09-10T11:30:26.352548+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.352548+0000', 'last_undegraded': '2024-09-10T11:30:26.352548+0000', 'last_fullsized': '2024-09-10T11:30:26.352548+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-10T11:30:30.971 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.342369+0000', 'last_change': '2024-09-10T11:30:26.342369+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.342369+0000', 'last_undegraded': '2024-09-10T11:30:26.342369+0000', 'last_fullsized': '2024-09-10T11:30:26.342369+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349505+0000', 'last_change': '2024-09-10T11:30:26.349505+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349505+0000', 'last_undegraded': '2024-09-10T11:30:26.349505+0000', 'last_fullsized': '2024-09-10T11:30:26.349505+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349202+0000', 'last_change': '2024-09-10T11:30:26.349202+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349202+0000', 'last_undegraded': '2024-09-10T11:30:26.349202+0000', 'last_fullsized': '2024-09-10T11:30:26.349202+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.353929+0000', 'last_change': '2024-09-10T11:30:26.353929+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.353929+0000', 'last_undegraded': '2024-09-10T11:30:26.353929+0000', 'last_fullsized': '2024-09-10T11:30:26.353929+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348630+0000', 'last_change': '2024-09-10T11:30:26.348630+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348630+0000', 'last_undegraded': '2024-09-10T11:30:26.348630+0000', 'last_fullsized': '2024-09-10T11:30:26.348630+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347834+0000', 'last_change': '2024-09-10T11:30:26.347834+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347834+0000', 'last_undegraded': '2024-09-10T11:30:26.347834+0000', 'last_fullsized': '2024-09-10T11:30:26.347834+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:30:30.972 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.343398+0000', 'last_change': '2024-09-10T11:30:26.343398+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.343398+0000', 'last_undegraded': '2024-09-10T11:30:26.343398+0000', 'last_fullsized': '2024-09-10T11:30:26.343398+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347190+0000', 'last_change': '2024-09-10T11:30:26.347190+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347190+0000', 'last_undegraded': '2024-09-10T11:30:26.347190+0000', 'last_fullsized': '2024-09-10T11:30:26.347190+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344409+0000', 'last_change': '2024-09-10T11:30:26.344409+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344409+0000', 'last_undegraded': '2024-09-10T11:30:26.344409+0000', 'last_fullsized': '2024-09-10T11:30:26.344409+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.352083+0000', 'last_change': '2024-09-10T11:30:26.352083+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.352083+0000', 'last_undegraded': '2024-09-10T11:30:26.352083+0000', 'last_fullsized': '2024-09-10T11:30:26.352083+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344235+0000', 'last_change': '2024-09-10T11:30:26.344235+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344235+0000', 'last_undegraded': '2024-09-10T11:30:26.344235+0000', 'last_fullsized': '2024-09-10T11:30:26.344235+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-10T11:30:30.973 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.342684+0000', 'last_change': '2024-09-10T11:30:26.342684+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.342684+0000', 'last_undegraded': '2024-09-10T11:30:26.342684+0000', 'last_fullsized': '2024-09-10T11:30:26.342684+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.350537+0000', 'last_change': '2024-09-10T11:30:26.350537+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.350537+0000', 'last_undegraded': '2024-09-10T11:30:26.350537+0000', 'last_fullsized': '2024-09-10T11:30:26.350537+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345938+0000', 'last_change': '2024-09-10T11:30:26.345938+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345938+0000', 'last_undegraded': '2024-09-10T11:30:26.345938+0000', 'last_fullsized': '2024-09-10T11:30:26.345938+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345128+0000', 'last_change': '2024-09-10T11:30:26.345128+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345128+0000', 'last_undegraded': '2024-09-10T11:30:26.345128+0000', 'last_fullsized': '2024-09-10T11:30:26.345128+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349865+0000', 'last_change': '2024-09-10T11:30:26.349865+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349865+0000', 'last_undegraded': '2024-09-10T11:30:26.349865+0000', 'last_fullsized': '2024-09-10T11:30:26.349865+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347122+0000', 'last_change': '2024-09-10T11:30:26.347122+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347122+0000', 'last_undegraded': '2024-09-10T11:30:26.347122+0000', 'last_fullsized': '2024-09-10T11:30:26.347122+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-10T11:30:30.974 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345736+0000', 'last_change': '2024-09-10T11:30:26.345736+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345736+0000', 'last_undegraded': '2024-09-10T11:30:26.345736+0000', 'last_fullsized': '2024-09-10T11:30:26.345736+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344204+0000', 'last_change': '2024-09-10T11:30:26.344204+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344204+0000', 'last_undegraded': '2024-09-10T11:30:26.344204+0000', 'last_fullsized': '2024-09-10T11:30:26.344204+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.342367+0000', 'last_change': '2024-09-10T11:30:26.342367+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.342367+0000', 'last_undegraded': '2024-09-10T11:30:26.342367+0000', 'last_fullsized': '2024-09-10T11:30:26.342367+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345905+0000', 'last_change': '2024-09-10T11:30:26.345905+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345905+0000', 'last_undegraded': '2024-09-10T11:30:26.345905+0000', 'last_fullsized': '2024-09-10T11:30:26.345905+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349562+0000', 'last_change': '2024-09-10T11:30:26.349562+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349562+0000', 'last_undegraded': '2024-09-10T11:30:26.349562+0000', 'last_fullsized': '2024-09-10T11:30:26.349562+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-10T11:30:30.975 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347691+0000', 'last_change': '2024-09-10T11:30:26.347691+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347691+0000', 'last_undegraded': '2024-09-10T11:30:26.347691+0000', 'last_fullsized': '2024-09-10T11:30:26.347691+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.343276+0000', 'last_change': '2024-09-10T11:30:26.343276+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.343276+0000', 'last_undegraded': '2024-09-10T11:30:26.343276+0000', 'last_fullsized': '2024-09-10T11:30:26.343276+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348194+0000', 'last_change': '2024-09-10T11:30:26.348194+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348194+0000', 'last_undegraded': '2024-09-10T11:30:26.348194+0000', 'last_fullsized': '2024-09-10T11:30:26.348194+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.353205+0000', 'last_change': '2024-09-10T11:30:26.353205+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.353205+0000', 'last_undegraded': '2024-09-10T11:30:26.353205+0000', 'last_fullsized': '2024-09-10T11:30:26.353205+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.346396+0000', 'last_change': '2024-09-10T11:30:26.346396+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.346396+0000', 'last_undegraded': '2024-09-10T11:30:26.346396+0000', 'last_fullsized': '2024-09-10T11:30:26.346396+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-10T11:30:30.976 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.342388+0000', 'last_change': '2024-09-10T11:30:26.342388+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.342388+0000', 'last_undegraded': '2024-09-10T11:30:26.342388+0000', 'last_fullsized': '2024-09-10T11:30:26.342388+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349242+0000', 'last_change': '2024-09-10T11:30:26.349242+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349242+0000', 'last_undegraded': '2024-09-10T11:30:26.349242+0000', 'last_fullsized': '2024-09-10T11:30:26.349242+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344018+0000', 'last_change': '2024-09-10T11:30:26.344018+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344018+0000', 'last_undegraded': '2024-09-10T11:30:26.344018+0000', 'last_fullsized': '2024-09-10T11:30:26.344018+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.344195+0000', 'last_change': '2024-09-10T11:30:26.344195+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.344195+0000', 'last_undegraded': '2024-09-10T11:30:26.344195+0000', 'last_fullsized': '2024-09-10T11:30:26.344195+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345247+0000', 'last_change': '2024-09-10T11:30:26.345247+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345247+0000', 'last_undegraded': '2024-09-10T11:30:26.345247+0000', 'last_fullsized': '2024-09-10T11:30:26.345247+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.343602+0000', 'last_change': '2024-09-10T11:30:26.343602+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.343602+0000', 'last_undegraded': '2024-09-10T11:30:26.343602+0000', 'last_fullsized': '2024-09-10T11:30:26.343602+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:30:30.977 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.348194+0000', 'last_change': '2024-09-10T11:30:26.348194+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.348194+0000', 'last_undegraded': '2024-09-10T11:30:26.348194+0000', 'last_fullsized': '2024-09-10T11:30:26.348194+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.345612+0000', 'last_change': '2024-09-10T11:30:26.345612+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.345612+0000', 'last_undegraded': '2024-09-10T11:30:26.345612+0000', 'last_fullsized': '2024-09-10T11:30:26.345612+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347450+0000', 'last_change': '2024-09-10T11:30:26.347450+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347450+0000', 'last_undegraded': '2024-09-10T11:30:26.347450+0000', 'last_fullsized': '2024-09-10T11:30:26.347450+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.347966+0000', 'last_change': '2024-09-10T11:30:26.347966+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.347966+0000', 'last_undegraded': '2024-09-10T11:30:26.347966+0000', 'last_fullsized': '2024-09-10T11:30:26.347966+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.349430+0000', 'last_change': '2024-09-10T11:30:26.349430+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.349430+0000', 'last_undegraded': '2024-09-10T11:30:26.349430+0000', 'last_fullsized': '2024-09-10T11:30:26.349430+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-10T11:30:30.978 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:30:26.353527+0000', 'last_change': '2024-09-10T11:30:26.353527+0000', 'last_active': '2024-09-10T11:30:26.336629+0000', 'last_peered': '2024-09-10T11:30:26.336629+0000', 'last_clean': '2024-09-10T11:30:26.336629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:30:26.353527+0000', 'last_undegraded': '2024-09-10T11:30:26.353527+0000', 'last_fullsized': '2024-09-10T11:30:26.353527+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'last_clean_scrub_stamp': '2024-09-10T11:30:26.336629+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:30:30.979 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T11:30:31.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.099+0000 7f9f77f6d640 1 Processor -- start 2024-09-10T11:30:31.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.099+0000 7f9f77f6d640 1 -- start start 2024-09-10T11:30:31.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f77f6d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f700b3960 0x7f9f700b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f77f6d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 0x7f9f7006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f77f6d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f700b3620 con 0x7f9f70163580 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f77f6d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9f7006d960 con 0x7f9f700b3960 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f77f6d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9f7006dc40 con 0x7f9f700b3160 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f75508640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f700b3960 0x7f9f700b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f75508640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f700b3960 0x7f9f700b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56994/0 (socket says 172.21.15.99:56994) 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.100+0000 7f9f75508640 1 -- 172.21.15.99:0/632449738 learned_addr learned my addr 172.21.15.99:0/632449738 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f7650a640 1 --2- 172.21.15.99:0/632449738 >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 0x7f9f7006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f75508640 1 -- 172.21.15.99:0/632449738 >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 msgr2=0x7f9f7006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f75508640 1 --2- 172.21.15.99:0/632449738 >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 0x7f9f7006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f75508640 1 -- 172.21.15.99:0/632449738 >> v1:172.21.15.196:6789/0 conn(0x7f9f70163580 legacy=0x7f9f700d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f75508640 1 -- 172.21.15.99:0/632449738 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7006df20 con 0x7f9f700b3960 2024-09-10T11:30:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f7650a640 1 --2- 172.21.15.99:0/632449738 >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 0x7f9f7006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f75508640 1 --2- 172.21.15.99:0/632449738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700b3960 0x7f9f700b5090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f70073250 tx=0x7f9f6002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e151acaf6cfd30d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f74d07640 1 -- 172.21.15.99:0/632449738 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60037070 con 0x7f9f700b3960 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.101+0000 7f9f74d07640 1 -- 172.21.15.99:0/632449738 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6002ae20 con 0x7f9f700b3960 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f74d07640 1 -- 172.21.15.99:0/632449738 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f60032660 con 0x7f9f700b3960 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 -- 172.21.15.99:0/632449738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700b3960 msgr2=0x7f9f700b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/632449738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700b3960 0x7f9f700b5090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f70073250 tx=0x7f9f6002a110 comp rx=0 tx=0).stop 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 -- 172.21.15.99:0/632449738 shutdown_connections 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/632449738 >> v2:172.21.15.196:3300/0 conn(0x7f9f700b3160 0x7f9f7006cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/632449738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f700b3960 0x7f9f700b5090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 -- 172.21.15.99:0/632449738 >> 172.21.15.99:0/632449738 conn(0x7f9f700b4570 msgr2=0x7f9f700b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 -- 172.21.15.99:0/632449738 shutdown_connections 2024-09-10T11:30:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.102+0000 7f9f77f6d640 1 -- 172.21.15.99:0/632449738 wait complete. 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.103+0000 7f9f77f6d640 1 Processor -- start 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.103+0000 7f9f77f6d640 1 -- start start 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 0x7f9f700b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 0x7f9f700cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7006d3e0 con 0x7f9f70163580 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7006d260 con 0x7f9f700b3160 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f77f6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7006d560 con 0x7f9f700b3960 2024-09-10T11:30:31.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57006/0 (socket says 172.21.15.99:57006) 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 -- 172.21.15.99:0/543756720 learned_addr learned my addr 172.21.15.99:0/543756720 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f75d09640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 0x7f9f700b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f75508640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 0x7f9f700cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 -- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 msgr2=0x7f9f700cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 0x7f9f700cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 -- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 msgr2=0x7f9f700b79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 0x7f9f700b79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f7650a640 1 -- 172.21.15.99:0/543756720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f700cc060 con 0x7f9f70163580 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f75508640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 0x7f9f700cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.104+0000 7f9f75d09640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 0x7f9f700b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:31.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f7650a640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800cd80 tx=0x7f9f6800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800f680 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6800f840 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f700cc340 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6800fc30 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.105+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f700cc900 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.106+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7006cd20 con 0x7f9f70163580 2024-09-10T11:30:31.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.106+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f680213f0 con 0x7f9f70163580 2024-09-10T11:30:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.106+0000 7f9f667fc640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 0x7f9f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.107+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9f6808bc50 con 0x7f9f70163580 2024-09-10T11:30:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.107+0000 7f9f75d09640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 0x7f9f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.107+0000 7f9f75d09640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 0x7f9f5406e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f700712a0 tx=0x7f9f6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.111+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f68091020 con 0x7f9f70163580 2024-09-10T11:30:31.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.342+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9f700b89a0 con 0x7f9f70163580 2024-09-10T11:30:31.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.343+0000 7f9f667fc640 1 -- 172.21.15.99:0/543756720 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1068 (secure 0 0 0) 0x7f9f68014030 con 0x7f9f70163580 2024-09-10T11:30:31.345 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:31.345 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":20,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1725967815,"num_in_osds":8,"osd_in_since":1725967812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":1592128,"bytes_avail":1598407872,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4303}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-10T11:30:10.854726+0000","services":{}},"progress_events":{}} 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 msgr2=0x7f9f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 0x7f9f5406e2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9f700712a0 tx=0x7f9f6c009040 comp rx=0 tx=0).stop 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 msgr2=0x7f9f700cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800cd80 tx=0x7f9f6800a650 comp rx=0 tx=0).stop 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 shutdown_connections 2024-09-10T11:30:31.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f5406be30 0x7f9f5406e2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f70163580 0x7f9f700cba30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f700b3960 0x7f9f700cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 --2- 172.21.15.99:0/543756720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f700b3160 0x7f9f700b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 >> 172.21.15.99:0/543756720 conn(0x7f9f700b4570 msgr2=0x7f9f7015fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 shutdown_connections 2024-09-10T11:30:31.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.345+0000 7f9f77f6d640 1 -- 172.21.15.99:0/543756720 wait complete. 2024-09-10T11:30:31.357 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T11:30:31.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:31.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 Processor -- start 2024-09-10T11:30:31.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 -- start start 2024-09-10T11:30:31.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83501605c0 0x7f83501609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f835015f210 0x7f8350160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83500d1bd0 con 0x7f83500ae020 2024-09-10T11:30:31.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8350075620 con 0x7f835015f210 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.511+0000 7f8356f8d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f83500752b0 con 0x7f83501605c0 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f835552a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f835015f210 0x7f8350160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f834ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f83500ae020 0x7f835015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33076/0 (socket says 172.21.15.99:33076) 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f834ffff640 1 -- 172.21.15.99:0/2465086064 learned_addr learned my addr 172.21.15.99:0/2465086064 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f8354d29640 1 --2- 172.21.15.99:0/2465086064 >> v2:172.21.15.196:3300/0 conn(0x7f83501605c0 0x7f83501609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f835552a640 1 -- 172.21.15.99:0/2465086064 >> v2:172.21.15.196:3300/0 conn(0x7f83501605c0 msgr2=0x7f83501609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f835552a640 1 --2- 172.21.15.99:0/2465086064 >> v2:172.21.15.196:3300/0 conn(0x7f83501605c0 0x7f83501609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f835552a640 1 -- 172.21.15.99:0/2465086064 >> v1:172.21.15.99:6789/0 conn(0x7f83500ae020 legacy=0x7f835015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f834f7fe640 1 -- 172.21.15.99:0/2465086064 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749496390 0 0) 0x7f83500d1bd0 con 0x7f83500ae020 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.512+0000 7f835552a640 1 -- 172.21.15.99:0/2465086064 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83500d1eb0 con 0x7f835015f210 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f835552a640 1 --2- 172.21.15.99:0/2465086064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835015f210 0x7f8350160ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8350070af0 tx=0x7f834802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f16627ad0dec047 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f834f7fe640 1 -- 172.21.15.99:0/2465086064 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348007740 con 0x7f835015f210 2024-09-10T11:30:31.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f834f7fe640 1 -- 172.21.15.99:0/2465086064 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83480078e0 con 0x7f835015f210 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f834f7fe640 1 -- 172.21.15.99:0/2465086064 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348007cb0 con 0x7f835015f210 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 -- 172.21.15.99:0/2465086064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835015f210 msgr2=0x7f8350160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 --2- 172.21.15.99:0/2465086064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835015f210 0x7f8350160ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8350070af0 tx=0x7f834802a680 comp rx=0 tx=0).stop 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 -- 172.21.15.99:0/2465086064 shutdown_connections 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 --2- 172.21.15.99:0/2465086064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835015f210 0x7f8350160ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 --2- 172.21.15.99:0/2465086064 >> v2:172.21.15.196:3300/0 conn(0x7f83501605c0 0x7f83501609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 -- 172.21.15.99:0/2465086064 >> 172.21.15.99:0/2465086064 conn(0x7f83500730c0 msgr2=0x7f83500734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 -- 172.21.15.99:0/2465086064 shutdown_connections 2024-09-10T11:30:31.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.513+0000 7f8356f8d640 1 -- 172.21.15.99:0/2465086064 wait complete. 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.514+0000 7f8356f8d640 1 Processor -- start 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.514+0000 7f8356f8d640 1 -- start start 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835015f210 0x7f835014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83501605c0 0x7f83500b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8350073d00 con 0x7f83500ae020 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f835015a9e0 con 0x7f83501605c0 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8356f8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8350073e80 con 0x7f835015f210 2024-09-10T11:30:31.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8354d29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8354d29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57028/0 (socket says 172.21.15.99:57028) 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.515+0000 7f8354d29640 1 -- 172.21.15.99:0/104650719 learned_addr learned my addr 172.21.15.99:0/104650719 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f835552a640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83501605c0 0x7f83500b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f834ffff640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835015f210 0x7f835014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 -- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835015f210 msgr2=0x7f835014d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835015f210 0x7f835014d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 -- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83501605c0 msgr2=0x7f83500b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83501605c0 0x7f83500b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 -- 172.21.15.99:0/104650719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835009dee0 con 0x7f83500ae020 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8354d29640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f834400d620 tx=0x7f834400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834400b620 con 0x7f83500ae020 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344019070 con 0x7f83500ae020 2024-09-10T11:30:31.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834400ba00 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835009e1c0 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.516+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835009e780 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.517+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83501609a0 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.517+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f834400bba0 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.518+0000 7f834d7fa640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 0x7f833406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.518+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f834408b1e0 con 0x7f83500ae020 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.518+0000 7f834ffff640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 0x7f833406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:31.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.518+0000 7f834ffff640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 0x7f833406e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8350157bc0 tx=0x7f833c007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:31.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.522+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8344059c20 con 0x7f83500ae020 2024-09-10T11:30:31.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.682+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83500d0e00 con 0x7f833406be30 2024-09-10T11:30:31.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.688+0000 7f834d7fa640 1 -- 172.21.15.99:0/104650719 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242211 (secure 0 0 0) 0x7f83500d0e00 con 0x7f833406be30 2024-09-10T11:30:31.692 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:31.693 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 msgr2=0x7f833406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 0x7f833406e2f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8350157bc0 tx=0x7f833c007c00 comp rx=0 tx=0).stop 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 msgr2=0x7f835009d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f834400d620 tx=0x7f834400dae0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 shutdown_connections 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f833406be30 0x7f833406e2f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83501605c0 0x7f83500b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f835015f210 0x7f835014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 --2- 172.21.15.99:0/104650719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83500ae020 0x7f835009d870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:31.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 >> 172.21.15.99:0/104650719 conn(0x7f83500730c0 msgr2=0x7f835006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:31.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 shutdown_connections 2024-09-10T11:30:31.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:31.691+0000 7f8356f8d640 1 -- 172.21.15.99:0/104650719 wait complete. 2024-09-10T11:30:31.703 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-10T11:30:30.907445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598407872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570122"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347078+0000","last_change":"2024-09-10T11:30:26.347078+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347078+0000","last_undegraded":"2024-09-10T11:30:26.347078+0000","last_fullsized":"2024-09-10T11:30:26.347078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349754+0000","last_change":"2024-09-10T11:30:26.349754+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349754+0000","last_undegraded":"2024-09-10T11:30:26.349754+0000","last_fullsized":"2024-09-10T11:30:26.349754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.351662+0000","last_change":"2024-09-10T11:30:26.351662+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.351662+0000","last_undegraded":"2024-09-10T11:30:26.351662+0000","last_fullsized":"2024-09-10T11:30:26.351662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343642+0000","last_change":"2024-09-10T11:30:26.343642+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343642+0000","last_undegraded":"2024-09-10T11:30:26.343642+0000","last_fullsized":"2024-09-10T11:30:26.343642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344216+0000","last_change":"2024-09-10T11:30:26.344216+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344216+0000","last_undegraded":"2024-09-10T11:30:26.344216+0000","last_fullsized":"2024-09-10T11:30:26.344216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342938+0000","last_change":"2024-09-10T11:30:26.342938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342938+0000","last_undegraded":"2024-09-10T11:30:26.342938+0000","last_fullsized":"2024-09-10T11:30:26.342938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348539+0000","last_change":"2024-09-10T11:30:26.348539+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348539+0000","last_undegraded":"2024-09-10T11:30:26.348539+0000","last_fullsized":"2024-09-10T11:30:26.348539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350001+0000","last_change":"2024-09-10T11:30:26.350001+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350001+0000","last_undegraded":"2024-09-10T11:30:26.350001+0000","last_fullsized":"2024-09-10T11:30:26.350001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348916+0000","last_change":"2024-09-10T11:30:26.348916+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348916+0000","last_undegraded":"2024-09-10T11:30:26.348916+0000","last_fullsized":"2024-09-10T11:30:26.348916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345319+0000","last_change":"2024-09-10T11:30:26.345319+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345319+0000","last_undegraded":"2024-09-10T11:30:26.345319+0000","last_fullsized":"2024-09-10T11:30:26.345319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346748+0000","last_change":"2024-09-10T11:30:26.346748+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346748+0000","last_undegraded":"2024-09-10T11:30:26.346748+0000","last_fullsized":"2024-09-10T11:30:26.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350070+0000","last_change":"2024-09-10T11:30:26.350070+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350070+0000","last_undegraded":"2024-09-10T11:30:26.350070+0000","last_fullsized":"2024-09-10T11:30:26.350070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350068+0000","last_change":"2024-09-10T11:30:26.350068+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350068+0000","last_undegraded":"2024-09-10T11:30:26.350068+0000","last_fullsized":"2024-09-10T11:30:26.350068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349848+0000","last_change":"2024-09-10T11:30:26.349848+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349848+0000","last_undegraded":"2024-09-10T11:30:26.349848+0000","last_fullsized":"2024-09-10T11:30:26.349848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353163+0000","last_change":"2024-09-10T11:30:26.353163+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353163+0000","last_undegraded":"2024-09-10T11:30:26.353163+0000","last_fullsized":"2024-09-10T11:30:26.353163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349515+0000","last_change":"2024-09-10T11:30:26.349515+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349515+0000","last_undegraded":"2024-09-10T11:30:26.349515+0000","last_fullsized":"2024-09-10T11:30:26.349515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344553+0000","last_change":"2024-09-10T11:30:26.344553+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344553+0000","last_undegraded":"2024-09-10T11:30:26.344553+0000","last_fullsized":"2024-09-10T11:30:26.344553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348768+0000","last_change":"2024-09-10T11:30:26.348768+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348768+0000","last_undegraded":"2024-09-10T11:30:26.348768+0000","last_fullsized":"2024-09-10T11:30:26.348768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345295+0000","last_change":"2024-09-10T11:30:26.345295+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345295+0000","last_undegraded":"2024-09-10T11:30:26.345295+0000","last_fullsized":"2024-09-10T11:30:26.345295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344859+0000","last_change":"2024-09-10T11:30:26.344859+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344859+0000","last_undegraded":"2024-09-10T11:30:26.344859+0000","last_fullsized":"2024-09-10T11:30:26.344859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344914+0000","last_change":"2024-09-10T11:30:26.344914+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344914+0000","last_undegraded":"2024-09-10T11:30:26.344914+0000","last_fullsized":"2024-09-10T11:30:26.344914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346106+0000","last_change":"2024-09-10T11:30:26.346106+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346106+0000","last_undegraded":"2024-09-10T11:30:26.346106+0000","last_fullsized":"2024-09-10T11:30:26.346106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348811+0000","last_change":"2024-09-10T11:30:26.348811+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348811+0000","last_undegraded":"2024-09-10T11:30:26.348811+0000","last_fullsized":"2024-09-10T11:30:26.348811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346246+0000","last_change":"2024-09-10T11:30:26.346246+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346246+0000","last_undegraded":"2024-09-10T11:30:26.346246+0000","last_fullsized":"2024-09-10T11:30:26.346246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352548+0000","last_change":"2024-09-10T11:30:26.352548+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352548+0000","last_undegraded":"2024-09-10T11:30:26.352548+0000","last_fullsized":"2024-09-10T11:30:26.352548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342369+0000","last_change":"2024-09-10T11:30:26.342369+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342369+0000","last_undegraded":"2024-09-10T11:30:26.342369+0000","last_fullsized":"2024-09-10T11:30:26.342369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349505+0000","last_change":"2024-09-10T11:30:26.349505+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349505+0000","last_undegraded":"2024-09-10T11:30:26.349505+0000","last_fullsized":"2024-09-10T11:30:26.349505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349202+0000","last_change":"2024-09-10T11:30:26.349202+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349202+0000","last_undegraded":"2024-09-10T11:30:26.349202+0000","last_fullsized":"2024-09-10T11:30:26.349202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353929+0000","last_change":"2024-09-10T11:30:26.353929+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353929+0000","last_undegraded":"2024-09-10T11:30:26.353929+0000","last_fullsized":"2024-09-10T11:30:26.353929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348630+0000","last_change":"2024-09-10T11:30:26.348630+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348630+0000","last_undegraded":"2024-09-10T11:30:26.348630+0000","last_fullsized":"2024-09-10T11:30:26.348630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347834+0000","last_change":"2024-09-10T11:30:26.347834+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347834+0000","last_undegraded":"2024-09-10T11:30:26.347834+0000","last_fullsized":"2024-09-10T11:30:26.347834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343398+0000","last_change":"2024-09-10T11:30:26.343398+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343398+0000","last_undegraded":"2024-09-10T11:30:26.343398+0000","last_fullsized":"2024-09-10T11:30:26.343398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347190+0000","last_change":"2024-09-10T11:30:26.347190+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347190+0000","last_undegraded":"2024-09-10T11:30:26.347190+0000","last_fullsized":"2024-09-10T11:30:26.347190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355648+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:26.355648+0000","last_peered":"2024-09-10T11:30:26.355648+0000","last_clean":"2024-09-10T11:30:26.355648+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:26.355648+0000","last_undegraded":"2024-09-10T11:30:26.355648+0000","last_fullsized":"2024-09-10T11:30:26.355648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:01.364851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344409+0000","last_change":"2024-09-10T11:30:26.344409+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344409+0000","last_undegraded":"2024-09-10T11:30:26.344409+0000","last_fullsized":"2024-09-10T11:30:26.344409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352706+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:26.352706+0000","last_peered":"2024-09-10T11:30:26.352706+0000","last_clean":"2024-09-10T11:30:26.352706+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:26.352706+0000","last_undegraded":"2024-09-10T11:30:26.352706+0000","last_fullsized":"2024-09-10T11:30:26.352706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:02.279667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350997+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:26.350997+0000","last_peered":"2024-09-10T11:30:26.350997+0000","last_clean":"2024-09-10T11:30:26.350997+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:26.350997+0000","last_undegraded":"2024-09-10T11:30:26.350997+0000","last_fullsized":"2024-09-10T11:30:26.350997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:20.397556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.352083+0000","last_change":"2024-09-10T11:30:26.352083+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.352083+0000","last_undegraded":"2024-09-10T11:30:26.352083+0000","last_fullsized":"2024-09-10T11:30:26.352083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353752+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:26.353752+0000","last_peered":"2024-09-10T11:30:26.353752+0000","last_clean":"2024-09-10T11:30:26.353752+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:26.353752+0000","last_undegraded":"2024-09-10T11:30:26.353752+0000","last_fullsized":"2024-09-10T11:30:26.353752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:16.268202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344235+0000","last_change":"2024-09-10T11:30:26.344235+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344235+0000","last_undegraded":"2024-09-10T11:30:26.344235+0000","last_fullsized":"2024-09-10T11:30:26.344235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352206+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:26.352206+0000","last_peered":"2024-09-10T11:30:26.352206+0000","last_clean":"2024-09-10T11:30:26.352206+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:26.352206+0000","last_undegraded":"2024-09-10T11:30:26.352206+0000","last_fullsized":"2024-09-10T11:30:26.352206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:45.803657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342684+0000","last_change":"2024-09-10T11:30:26.342684+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342684+0000","last_undegraded":"2024-09-10T11:30:26.342684+0000","last_fullsized":"2024-09-10T11:30:26.342684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356269+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:26.356269+0000","last_peered":"2024-09-10T11:30:26.356269+0000","last_clean":"2024-09-10T11:30:26.356269+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:26.356269+0000","last_undegraded":"2024-09-10T11:30:26.356269+0000","last_fullsized":"2024-09-10T11:30:26.356269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:02.709036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.350537+0000","last_change":"2024-09-10T11:30:26.350537+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.350537+0000","last_undegraded":"2024-09-10T11:30:26.350537+0000","last_fullsized":"2024-09-10T11:30:26.350537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347731+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:26.347731+0000","last_peered":"2024-09-10T11:30:26.347731+0000","last_clean":"2024-09-10T11:30:26.347731+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:26.347731+0000","last_undegraded":"2024-09-10T11:30:26.347731+0000","last_fullsized":"2024-09-10T11:30:26.347731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:41.013135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345938+0000","last_change":"2024-09-10T11:30:26.345938+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345938+0000","last_undegraded":"2024-09-10T11:30:26.345938+0000","last_fullsized":"2024-09-10T11:30:26.345938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.348741+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:26.348741+0000","last_peered":"2024-09-10T11:30:26.348741+0000","last_clean":"2024-09-10T11:30:26.348741+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:26.348741+0000","last_undegraded":"2024-09-10T11:30:26.348741+0000","last_fullsized":"2024-09-10T11:30:26.348741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:24.768217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345128+0000","last_change":"2024-09-10T11:30:26.345128+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345128+0000","last_undegraded":"2024-09-10T11:30:26.345128+0000","last_fullsized":"2024-09-10T11:30:26.345128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351884+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:26.351884+0000","last_peered":"2024-09-10T11:30:26.351884+0000","last_clean":"2024-09-10T11:30:26.351884+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:26.351884+0000","last_undegraded":"2024-09-10T11:30:26.351884+0000","last_fullsized":"2024-09-10T11:30:26.351884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:14.416062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349865+0000","last_change":"2024-09-10T11:30:26.349865+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349865+0000","last_undegraded":"2024-09-10T11:30:26.349865+0000","last_fullsized":"2024-09-10T11:30:26.349865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.354977+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:26.354977+0000","last_peered":"2024-09-10T11:30:26.354977+0000","last_clean":"2024-09-10T11:30:26.354977+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:26.354977+0000","last_undegraded":"2024-09-10T11:30:26.354977+0000","last_fullsized":"2024-09-10T11:30:26.354977+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:34.505783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347122+0000","last_change":"2024-09-10T11:30:26.347122+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347122+0000","last_undegraded":"2024-09-10T11:30:26.347122+0000","last_fullsized":"2024-09-10T11:30:26.347122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356474+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:26.356474+0000","last_peered":"2024-09-10T11:30:26.356474+0000","last_clean":"2024-09-10T11:30:26.356474+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:26.356474+0000","last_undegraded":"2024-09-10T11:30:26.356474+0000","last_fullsized":"2024-09-10T11:30:26.356474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:37.442331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345736+0000","last_change":"2024-09-10T11:30:26.345736+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345736+0000","last_undegraded":"2024-09-10T11:30:26.345736+0000","last_fullsized":"2024-09-10T11:30:26.345736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.347100+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:26.347100+0000","last_peered":"2024-09-10T11:30:26.347100+0000","last_clean":"2024-09-10T11:30:26.347100+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:26.347100+0000","last_undegraded":"2024-09-10T11:30:26.347100+0000","last_fullsized":"2024-09-10T11:30:26.347100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:38.630516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344204+0000","last_change":"2024-09-10T11:30:26.344204+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344204+0000","last_undegraded":"2024-09-10T11:30:26.344204+0000","last_fullsized":"2024-09-10T11:30:26.344204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358330+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:26.358330+0000","last_peered":"2024-09-10T11:30:26.358330+0000","last_clean":"2024-09-10T11:30:26.358330+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:26.358330+0000","last_undegraded":"2024-09-10T11:30:26.358330+0000","last_fullsized":"2024-09-10T11:30:26.358330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:28.172249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342367+0000","last_change":"2024-09-10T11:30:26.342367+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342367+0000","last_undegraded":"2024-09-10T11:30:26.342367+0000","last_fullsized":"2024-09-10T11:30:26.342367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:44.091730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345905+0000","last_change":"2024-09-10T11:30:26.345905+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345905+0000","last_undegraded":"2024-09-10T11:30:26.345905+0000","last_fullsized":"2024-09-10T11:30:26.345905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357762+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:26.357762+0000","last_peered":"2024-09-10T11:30:26.357762+0000","last_clean":"2024-09-10T11:30:26.357762+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:26.357762+0000","last_undegraded":"2024-09-10T11:30:26.357762+0000","last_fullsized":"2024-09-10T11:30:26.357762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.323563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349562+0000","last_change":"2024-09-10T11:30:26.349562+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349562+0000","last_undegraded":"2024-09-10T11:30:26.349562+0000","last_fullsized":"2024-09-10T11:30:26.349562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357477+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:26.357477+0000","last_peered":"2024-09-10T11:30:26.357477+0000","last_clean":"2024-09-10T11:30:26.357477+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:26.357477+0000","last_undegraded":"2024-09-10T11:30:26.357477+0000","last_fullsized":"2024-09-10T11:30:26.357477+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:28.358480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347691+0000","last_change":"2024-09-10T11:30:26.347691+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347691+0000","last_undegraded":"2024-09-10T11:30:26.347691+0000","last_fullsized":"2024-09-10T11:30:26.347691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.349405+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:26.349405+0000","last_peered":"2024-09-10T11:30:26.349405+0000","last_clean":"2024-09-10T11:30:26.349405+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:26.349405+0000","last_undegraded":"2024-09-10T11:30:26.349405+0000","last_fullsized":"2024-09-10T11:30:26.349405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352072+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:26.352072+0000","last_peered":"2024-09-10T11:30:26.352072+0000","last_clean":"2024-09-10T11:30:26.352072+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:26.352072+0000","last_undegraded":"2024-09-10T11:30:26.352072+0000","last_fullsized":"2024-09-10T11:30:26.352072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:52.596921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343276+0000","last_change":"2024-09-10T11:30:26.343276+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343276+0000","last_undegraded":"2024-09-10T11:30:26.343276+0000","last_fullsized":"2024-09-10T11:30:26.343276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357911+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:26.357911+0000","last_peered":"2024-09-10T11:30:26.357911+0000","last_clean":"2024-09-10T11:30:26.357911+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:26.357911+0000","last_undegraded":"2024-09-10T11:30:26.357911+0000","last_fullsized":"2024-09-10T11:30:26.357911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:53.332439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355632+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:26.355632+0000","last_peered":"2024-09-10T11:30:26.355632+0000","last_clean":"2024-09-10T11:30:26.355632+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:26.355632+0000","last_undegraded":"2024-09-10T11:30:26.355632+0000","last_fullsized":"2024-09-10T11:30:26.355632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:00.067158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353205+0000","last_change":"2024-09-10T11:30:26.353205+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353205+0000","last_undegraded":"2024-09-10T11:30:26.353205+0000","last_fullsized":"2024-09-10T11:30:26.353205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351869+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:26.351869+0000","last_peered":"2024-09-10T11:30:26.351869+0000","last_clean":"2024-09-10T11:30:26.351869+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:26.351869+0000","last_undegraded":"2024-09-10T11:30:26.351869+0000","last_fullsized":"2024-09-10T11:30:26.351869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:01.061136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.346396+0000","last_change":"2024-09-10T11:30:26.346396+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.346396+0000","last_undegraded":"2024-09-10T11:30:26.346396+0000","last_fullsized":"2024-09-10T11:30:26.346396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356683+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:26.356683+0000","last_peered":"2024-09-10T11:30:26.356683+0000","last_clean":"2024-09-10T11:30:26.356683+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:26.356683+0000","last_undegraded":"2024-09-10T11:30:26.356683+0000","last_fullsized":"2024-09-10T11:30:26.356683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:05.310043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.342388+0000","last_change":"2024-09-10T11:30:26.342388+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.342388+0000","last_undegraded":"2024-09-10T11:30:26.342388+0000","last_fullsized":"2024-09-10T11:30:26.342388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.353658+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:26.353658+0000","last_peered":"2024-09-10T11:30:26.353658+0000","last_clean":"2024-09-10T11:30:26.353658+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:26.353658+0000","last_undegraded":"2024-09-10T11:30:26.353658+0000","last_fullsized":"2024-09-10T11:30:26.353658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:33.564380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349242+0000","last_change":"2024-09-10T11:30:26.349242+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349242+0000","last_undegraded":"2024-09-10T11:30:26.349242+0000","last_fullsized":"2024-09-10T11:30:26.349242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.351809+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:26.351809+0000","last_peered":"2024-09-10T11:30:26.351809+0000","last_clean":"2024-09-10T11:30:26.351809+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:26.351809+0000","last_undegraded":"2024-09-10T11:30:26.351809+0000","last_fullsized":"2024-09-10T11:30:26.351809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:39.942357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344018+0000","last_change":"2024-09-10T11:30:26.344018+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344018+0000","last_undegraded":"2024-09-10T11:30:26.344018+0000","last_fullsized":"2024-09-10T11:30:26.344018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355891+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:26.355891+0000","last_peered":"2024-09-10T11:30:26.355891+0000","last_clean":"2024-09-10T11:30:26.355891+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:26.355891+0000","last_undegraded":"2024-09-10T11:30:26.355891+0000","last_fullsized":"2024-09-10T11:30:26.355891+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:04.011154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.344195+0000","last_change":"2024-09-10T11:30:26.344195+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.344195+0000","last_undegraded":"2024-09-10T11:30:26.344195+0000","last_fullsized":"2024-09-10T11:30:26.344195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352963+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:26.352963+0000","last_peered":"2024-09-10T11:30:26.352963+0000","last_clean":"2024-09-10T11:30:26.352963+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:26.352963+0000","last_undegraded":"2024-09-10T11:30:26.352963+0000","last_fullsized":"2024-09-10T11:30:26.352963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:12.773758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345247+0000","last_change":"2024-09-10T11:30:26.345247+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345247+0000","last_undegraded":"2024-09-10T11:30:26.345247+0000","last_fullsized":"2024-09-10T11:30:26.345247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.350527+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:26.350527+0000","last_peered":"2024-09-10T11:30:26.350527+0000","last_clean":"2024-09-10T11:30:26.350527+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:26.350527+0000","last_undegraded":"2024-09-10T11:30:26.350527+0000","last_fullsized":"2024-09-10T11:30:26.350527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:21.044180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.343602+0000","last_change":"2024-09-10T11:30:26.343602+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.343602+0000","last_undegraded":"2024-09-10T11:30:26.343602+0000","last_fullsized":"2024-09-10T11:30:26.343602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.358594+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:26.358594+0000","last_peered":"2024-09-10T11:30:26.358594+0000","last_clean":"2024-09-10T11:30:26.358594+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:26.358594+0000","last_undegraded":"2024-09-10T11:30:26.358594+0000","last_fullsized":"2024-09-10T11:30:26.358594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:01.475428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.348194+0000","last_change":"2024-09-10T11:30:26.348194+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.348194+0000","last_undegraded":"2024-09-10T11:30:26.348194+0000","last_fullsized":"2024-09-10T11:30:26.348194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352810+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:26.352810+0000","last_peered":"2024-09-10T11:30:26.352810+0000","last_clean":"2024-09-10T11:30:26.352810+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:26.352810+0000","last_undegraded":"2024-09-10T11:30:26.352810+0000","last_fullsized":"2024-09-10T11:30:26.352810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:57.958396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.345612+0000","last_change":"2024-09-10T11:30:26.345612+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.345612+0000","last_undegraded":"2024-09-10T11:30:26.345612+0000","last_fullsized":"2024-09-10T11:30:26.345612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347450+0000","last_change":"2024-09-10T11:30:26.347450+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347450+0000","last_undegraded":"2024-09-10T11:30:26.347450+0000","last_fullsized":"2024-09-10T11:30:26.347450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.355021+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:26.355021+0000","last_peered":"2024-09-10T11:30:26.355021+0000","last_clean":"2024-09-10T11:30:26.355021+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:26.355021+0000","last_undegraded":"2024-09-10T11:30:26.355021+0000","last_fullsized":"2024-09-10T11:30:26.355021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:11.775691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.347966+0000","last_change":"2024-09-10T11:30:26.347966+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.347966+0000","last_undegraded":"2024-09-10T11:30:26.347966+0000","last_fullsized":"2024-09-10T11:30:26.347966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.352946+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:26.352946+0000","last_peered":"2024-09-10T11:30:26.352946+0000","last_clean":"2024-09-10T11:30:26.352946+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:26.352946+0000","last_undegraded":"2024-09-10T11:30:26.352946+0000","last_fullsized":"2024-09-10T11:30:26.352946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:30.209672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.349430+0000","last_change":"2024-09-10T11:30:26.349430+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.349430+0000","last_undegraded":"2024-09-10T11:30:26.349430+0000","last_fullsized":"2024-09-10T11:30:26.349430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.356992+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:26.356992+0000","last_peered":"2024-09-10T11:30:26.356992+0000","last_clean":"2024-09-10T11:30:26.356992+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:26.356992+0000","last_undegraded":"2024-09-10T11:30:26.356992+0000","last_fullsized":"2024-09-10T11:30:26.356992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:05.275854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-10T11:30:26.353527+0000","last_change":"2024-09-10T11:30:26.353527+0000","last_active":"2024-09-10T11:30:26.336629+0000","last_peered":"2024-09-10T11:30:26.336629+0000","last_clean":"2024-09-10T11:30:26.336629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:30:26.353527+0000","last_undegraded":"2024-09-10T11:30:26.353527+0000","last_fullsized":"2024-09-10T11:30:26.353527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-10T11:30:26.357789+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:26.357789+0000","last_peered":"2024-09-10T11:30:26.357789+0000","last_clean":"2024-09-10T11:30:26.357789+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:26.357789+0000","last_undegraded":"2024-09-10T11:30:26.357789+0000","last_fullsized":"2024-09-10T11:30:26.357789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:16.906847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:32.330 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-09-10T11:30:32.330+0000 7f067ea6f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:30:34.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:34.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.818+0000 7f49ca447640 1 Processor -- start 2024-09-10T11:30:34.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.818+0000 7f49ca447640 1 -- start start 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49ca447640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 0x7f49c40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49ca447640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 0x7f49c40d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49ca447640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c40d2990 con 0x7f49c40d3a30 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49ca447640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f49c40d2c70 con 0x7f49c415efc0 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49ca447640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49c40d2f50 con 0x7f49c40d1cb0 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49c37fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f49c40d3a30 0x7f49c40d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44880/0 (socket says 172.21.15.99:44880) 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49c37fe640 1 -- 172.21.15.99:0/37937568 learned_addr learned my addr 172.21.15.99:0/37937568 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49c3fff640 1 --2- 172.21.15.99:0/37937568 >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 0x7f49c40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.819+0000 7f49c89e4640 1 --2- 172.21.15.99:0/37937568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 0x7f49c40d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2644140665 0 0) 0x7f49c40d2990 con 0x7f49c40d3a30 2024-09-10T11:30:34.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49ac003520 con 0x7f49c40d3a30 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c89e4640 1 -- 172.21.15.99:0/37937568 >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 msgr2=0x7f49c40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c89e4640 1 --2- 172.21.15.99:0/37937568 >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 0x7f49c40d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c89e4640 1 -- 172.21.15.99:0/37937568 >> v1:172.21.15.99:6789/0 conn(0x7f49c40d3a30 legacy=0x7f49c40d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729828873 0 0) 0x7f49ac003520 con 0x7f49c40d3a30 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c89e4640 1 -- 172.21.15.99:0/37937568 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c40d3230 con 0x7f49c40d1cb0 2024-09-10T11:30:34.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.820+0000 7f49c3fff640 1 --2- 172.21.15.99:0/37937568 >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 0x7f49c40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:34.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49c89e4640 1 --2- 172.21.15.99:0/37937568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 0x7f49c40d0930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009d90 tx=0x7f49bc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb90eadaa046a29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:34.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8001690 con 0x7f49c40d1cb0 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49bc007540 con 0x7f49c40d1cb0 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49c2ffd640 1 -- 172.21.15.99:0/37937568 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0078f0 con 0x7f49c40d1cb0 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 -- 172.21.15.99:0/37937568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 msgr2=0x7f49c40d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 --2- 172.21.15.99:0/37937568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 0x7f49c40d0930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009d90 tx=0x7f49bc02a5f0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 -- 172.21.15.99:0/37937568 shutdown_connections 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 --2- 172.21.15.99:0/37937568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d1cb0 0x7f49c40d0930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 --2- 172.21.15.99:0/37937568 >> v2:172.21.15.196:3300/0 conn(0x7f49c415efc0 0x7f49c40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 -- 172.21.15.99:0/37937568 >> 172.21.15.99:0/37937568 conn(0x7f49c40d4c20 msgr2=0x7f49c40d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 -- 172.21.15.99:0/37937568 shutdown_connections 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.821+0000 7f49ca447640 1 -- 172.21.15.99:0/37937568 wait complete. 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.822+0000 7f49ca447640 1 Processor -- start 2024-09-10T11:30:34.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 -- start start 2024-09-10T11:30:34.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 0x7f49c415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 0x7f49c40ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49c40b4930 con 0x7f49c40d1cb0 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49c40b47b0 con 0x7f49c415efc0 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49ca447640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49c40b4ab0 con 0x7f49c40d3a30 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.823+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54242/0 (socket says 172.21.15.99:54242) 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 -- 172.21.15.99:0/1103611973 learned_addr learned my addr 172.21.15.99:0/1103611973 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c89e4640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 0x7f49c40ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c37fe640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 0x7f49c415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 -- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 msgr2=0x7f49c415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 0x7f49c415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 -- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 msgr2=0x7f49c40ba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:34.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 0x7f49c40ba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 -- 172.21.15.99:0/1103611973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c40b8f80 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c89e4640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 0x7f49c40ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c37fe640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 0x7f49c415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c3fff640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b400e960 tx=0x7f49b400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4017070 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b400ca00 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c40b9260 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b400cdd0 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.824+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c40b9820 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.825+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c40d15a0 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.826+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f49b401a070 con 0x7f49c40d1cb0 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.826+0000 7f49c0ff9640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 0x7f49b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:34.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.826+0000 7f49c37fe640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 0x7f49b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:34.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.826+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f49b408b0d0 con 0x7f49c40d1cb0 2024-09-10T11:30:34.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.826+0000 7f49c37fe640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 0x7f49b006e320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001200 tx=0x7f49b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:34.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.830+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49b4059a80 con 0x7f49c40d1cb0 2024-09-10T11:30:34.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:34.995+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49c40b26a0 con 0x7f49b006be60 2024-09-10T11:30:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.001+0000 7f49c0ff9640 1 -- 172.21.15.99:0/1103611973 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247484 (secure 0 0 0) 0x7f49c40b26a0 con 0x7f49b006be60 2024-09-10T11:30:35.005 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 msgr2=0x7f49b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 0x7f49b006e320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001200 tx=0x7f49b8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 msgr2=0x7f49c40caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f49b400e960 tx=0x7f49b400ee20 comp rx=0 tx=0).stop 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 shutdown_connections 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f49b006be60 0x7f49b006e320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c415efc0 0x7f49c40ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c40d3a30 0x7f49c415ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 --2- 172.21.15.99:0/1103611973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49c40d1cb0 0x7f49c40caec0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 >> 172.21.15.99:0/1103611973 conn(0x7f49c40d4c20 msgr2=0x7f49c40b4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 shutdown_connections 2024-09-10T11:30:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.004+0000 7f49ca447640 1 -- 172.21.15.99:0/1103611973 wait complete. 2024-09-10T11:30:35.016 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-10T11:30:34.908370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001762"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370941+0000","last_change":"2024-09-10T11:30:27.366179+0000","last_active":"2024-09-10T11:30:28.370941+0000","last_peered":"2024-09-10T11:30:28.370941+0000","last_clean":"2024-09-10T11:30:28.370941+0000","last_became_active":"2024-09-10T11:30:27.365953+0000","last_became_peered":"2024-09-10T11:30:27.365953+0000","last_unstale":"2024-09-10T11:30:28.370941+0000","last_undegraded":"2024-09-10T11:30:28.370941+0000","last_fullsized":"2024-09-10T11:30:28.370941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:09.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359715+0000","last_change":"2024-09-10T11:30:27.368422+0000","last_active":"2024-09-10T11:30:28.359715+0000","last_peered":"2024-09-10T11:30:28.359715+0000","last_clean":"2024-09-10T11:30:28.359715+0000","last_became_active":"2024-09-10T11:30:27.366676+0000","last_became_peered":"2024-09-10T11:30:27.366676+0000","last_unstale":"2024-09-10T11:30:28.359715+0000","last_undegraded":"2024-09-10T11:30:28.359715+0000","last_fullsized":"2024-09-10T11:30:28.359715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.234005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359802+0000","last_change":"2024-09-10T11:30:27.370589+0000","last_active":"2024-09-10T11:30:28.359802+0000","last_peered":"2024-09-10T11:30:28.359802+0000","last_clean":"2024-09-10T11:30:28.359802+0000","last_became_active":"2024-09-10T11:30:27.369527+0000","last_became_peered":"2024-09-10T11:30:27.369527+0000","last_unstale":"2024-09-10T11:30:28.359802+0000","last_undegraded":"2024-09-10T11:30:28.359802+0000","last_fullsized":"2024-09-10T11:30:28.359802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:28.644939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659543+0000","last_change":"2024-09-10T11:30:27.368869+0000","last_active":"2024-09-10T11:30:28.659543+0000","last_peered":"2024-09-10T11:30:28.659543+0000","last_clean":"2024-09-10T11:30:28.659543+0000","last_became_active":"2024-09-10T11:30:27.367497+0000","last_became_peered":"2024-09-10T11:30:27.367497+0000","last_unstale":"2024-09-10T11:30:28.659543+0000","last_undegraded":"2024-09-10T11:30:28.659543+0000","last_fullsized":"2024-09-10T11:30:28.659543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:11.740953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371428+0000","last_change":"2024-09-10T11:30:27.372181+0000","last_active":"2024-09-10T11:30:28.371428+0000","last_peered":"2024-09-10T11:30:28.371428+0000","last_clean":"2024-09-10T11:30:28.371428+0000","last_became_active":"2024-09-10T11:30:27.371972+0000","last_became_peered":"2024-09-10T11:30:27.371972+0000","last_unstale":"2024-09-10T11:30:28.371428+0000","last_undegraded":"2024-09-10T11:30:28.371428+0000","last_fullsized":"2024-09-10T11:30:28.371428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:52.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659909+0000","last_change":"2024-09-10T11:30:27.368458+0000","last_active":"2024-09-10T11:30:28.659909+0000","last_peered":"2024-09-10T11:30:28.659909+0000","last_clean":"2024-09-10T11:30:28.659909+0000","last_became_active":"2024-09-10T11:30:27.368163+0000","last_became_peered":"2024-09-10T11:30:27.368163+0000","last_unstale":"2024-09-10T11:30:28.659909+0000","last_undegraded":"2024-09-10T11:30:28.659909+0000","last_fullsized":"2024-09-10T11:30:28.659909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:17.388777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371439+0000","last_change":"2024-09-10T11:30:27.370692+0000","last_active":"2024-09-10T11:30:28.371439+0000","last_peered":"2024-09-10T11:30:28.371439+0000","last_clean":"2024-09-10T11:30:28.371439+0000","last_became_active":"2024-09-10T11:30:27.370323+0000","last_became_peered":"2024-09-10T11:30:27.370323+0000","last_unstale":"2024-09-10T11:30:28.371439+0000","last_undegraded":"2024-09-10T11:30:28.371439+0000","last_fullsized":"2024-09-10T11:30:28.371439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:30.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359963+0000","last_change":"2024-09-10T11:30:27.371135+0000","last_active":"2024-09-10T11:30:28.359963+0000","last_peered":"2024-09-10T11:30:28.359963+0000","last_clean":"2024-09-10T11:30:28.359963+0000","last_became_active":"2024-09-10T11:30:27.370855+0000","last_became_peered":"2024-09-10T11:30:27.370855+0000","last_unstale":"2024-09-10T11:30:28.359963+0000","last_undegraded":"2024-09-10T11:30:28.359963+0000","last_fullsized":"2024-09-10T11:30:28.359963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:08.475065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356981+0000","last_change":"2024-09-10T11:30:27.368622+0000","last_active":"2024-09-10T11:30:28.356981+0000","last_peered":"2024-09-10T11:30:28.356981+0000","last_clean":"2024-09-10T11:30:28.356981+0000","last_became_active":"2024-09-10T11:30:27.368043+0000","last_became_peered":"2024-09-10T11:30:27.368043+0000","last_unstale":"2024-09-10T11:30:28.356981+0000","last_undegraded":"2024-09-10T11:30:28.356981+0000","last_fullsized":"2024-09-10T11:30:28.356981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:11.979652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371169+0000","last_change":"2024-09-10T11:30:27.370344+0000","last_active":"2024-09-10T11:30:28.371169+0000","last_peered":"2024-09-10T11:30:28.371169+0000","last_clean":"2024-09-10T11:30:28.371169+0000","last_became_active":"2024-09-10T11:30:27.370093+0000","last_became_peered":"2024-09-10T11:30:27.370093+0000","last_unstale":"2024-09-10T11:30:28.371169+0000","last_undegraded":"2024-09-10T11:30:28.371169+0000","last_fullsized":"2024-09-10T11:30:28.371169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:55.994388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371076+0000","last_change":"2024-09-10T11:30:27.364571+0000","last_active":"2024-09-10T11:30:28.371076+0000","last_peered":"2024-09-10T11:30:28.371076+0000","last_clean":"2024-09-10T11:30:28.371076+0000","last_became_active":"2024-09-10T11:30:27.364329+0000","last_became_peered":"2024-09-10T11:30:27.364329+0000","last_unstale":"2024-09-10T11:30:28.371076+0000","last_undegraded":"2024-09-10T11:30:28.371076+0000","last_fullsized":"2024-09-10T11:30:28.371076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:56.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371023+0000","last_change":"2024-09-10T11:30:27.362816+0000","last_active":"2024-09-10T11:30:28.371023+0000","last_peered":"2024-09-10T11:30:28.371023+0000","last_clean":"2024-09-10T11:30:28.371023+0000","last_became_active":"2024-09-10T11:30:27.362165+0000","last_became_peered":"2024-09-10T11:30:27.362165+0000","last_unstale":"2024-09-10T11:30:28.371023+0000","last_undegraded":"2024-09-10T11:30:28.371023+0000","last_fullsized":"2024-09-10T11:30:28.371023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:13.704596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356572+0000","last_change":"2024-09-10T11:30:27.361112+0000","last_active":"2024-09-10T11:30:28.356572+0000","last_peered":"2024-09-10T11:30:28.356572+0000","last_clean":"2024-09-10T11:30:28.356572+0000","last_became_active":"2024-09-10T11:30:27.360081+0000","last_became_peered":"2024-09-10T11:30:27.360081+0000","last_unstale":"2024-09-10T11:30:28.356572+0000","last_undegraded":"2024-09-10T11:30:28.356572+0000","last_fullsized":"2024-09-10T11:30:28.356572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:25.075435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356459+0000","last_change":"2024-09-10T11:30:27.360154+0000","last_active":"2024-09-10T11:30:28.356459+0000","last_peered":"2024-09-10T11:30:28.356459+0000","last_clean":"2024-09-10T11:30:28.356459+0000","last_became_active":"2024-09-10T11:30:27.359341+0000","last_became_peered":"2024-09-10T11:30:27.359341+0000","last_unstale":"2024-09-10T11:30:28.356459+0000","last_undegraded":"2024-09-10T11:30:28.356459+0000","last_fullsized":"2024-09-10T11:30:28.356459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.266260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359988+0000","last_change":"2024-09-10T11:30:27.371329+0000","last_active":"2024-09-10T11:30:28.359988+0000","last_peered":"2024-09-10T11:30:28.359988+0000","last_clean":"2024-09-10T11:30:28.359988+0000","last_became_active":"2024-09-10T11:30:27.370948+0000","last_became_peered":"2024-09-10T11:30:27.370948+0000","last_unstale":"2024-09-10T11:30:28.359988+0000","last_undegraded":"2024-09-10T11:30:28.359988+0000","last_fullsized":"2024-09-10T11:30:28.359988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.619700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659901+0000","last_change":"2024-09-10T11:30:27.368565+0000","last_active":"2024-09-10T11:30:28.659901+0000","last_peered":"2024-09-10T11:30:28.659901+0000","last_clean":"2024-09-10T11:30:28.659901+0000","last_became_active":"2024-09-10T11:30:27.368309+0000","last_became_peered":"2024-09-10T11:30:27.368309+0000","last_unstale":"2024-09-10T11:30:28.659901+0000","last_undegraded":"2024-09-10T11:30:28.659901+0000","last_fullsized":"2024-09-10T11:30:28.659901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:33.008256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659831+0000","last_change":"2024-09-10T11:30:27.371513+0000","last_active":"2024-09-10T11:30:28.659831+0000","last_peered":"2024-09-10T11:30:28.659831+0000","last_clean":"2024-09-10T11:30:28.659831+0000","last_became_active":"2024-09-10T11:30:27.371229+0000","last_became_peered":"2024-09-10T11:30:27.371229+0000","last_unstale":"2024-09-10T11:30:28.659831+0000","last_undegraded":"2024-09-10T11:30:28.659831+0000","last_fullsized":"2024-09-10T11:30:28.659831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:28.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359314+0000","last_change":"2024-09-10T11:30:27.366960+0000","last_active":"2024-09-10T11:30:28.359314+0000","last_peered":"2024-09-10T11:30:28.359314+0000","last_clean":"2024-09-10T11:30:28.359314+0000","last_became_active":"2024-09-10T11:30:27.366744+0000","last_became_peered":"2024-09-10T11:30:27.366744+0000","last_unstale":"2024-09-10T11:30:28.359314+0000","last_undegraded":"2024-09-10T11:30:28.359314+0000","last_fullsized":"2024-09-10T11:30:28.359314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:30.599189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660402+0000","last_change":"2024-09-10T11:30:27.369325+0000","last_active":"2024-09-10T11:30:28.660402+0000","last_peered":"2024-09-10T11:30:28.660402+0000","last_clean":"2024-09-10T11:30:28.660402+0000","last_became_active":"2024-09-10T11:30:27.368956+0000","last_became_peered":"2024-09-10T11:30:27.368956+0000","last_unstale":"2024-09-10T11:30:28.660402+0000","last_undegraded":"2024-09-10T11:30:28.660402+0000","last_fullsized":"2024-09-10T11:30:28.660402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.633447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371252+0000","last_change":"2024-09-10T11:30:27.360270+0000","last_active":"2024-09-10T11:30:28.371252+0000","last_peered":"2024-09-10T11:30:28.371252+0000","last_clean":"2024-09-10T11:30:28.371252+0000","last_became_active":"2024-09-10T11:30:27.360000+0000","last_became_peered":"2024-09-10T11:30:27.360000+0000","last_unstale":"2024-09-10T11:30:28.371252+0000","last_undegraded":"2024-09-10T11:30:28.371252+0000","last_fullsized":"2024-09-10T11:30:28.371252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:00.874011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357411+0000","last_change":"2024-09-10T11:30:27.369424+0000","last_active":"2024-09-10T11:30:28.357411+0000","last_peered":"2024-09-10T11:30:28.357411+0000","last_clean":"2024-09-10T11:30:28.357411+0000","last_became_active":"2024-09-10T11:30:27.367954+0000","last_became_peered":"2024-09-10T11:30:27.367954+0000","last_unstale":"2024-09-10T11:30:28.357411+0000","last_undegraded":"2024-09-10T11:30:28.357411+0000","last_fullsized":"2024-09-10T11:30:28.357411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:15.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356963+0000","last_change":"2024-09-10T11:30:27.364058+0000","last_active":"2024-09-10T11:30:28.356963+0000","last_peered":"2024-09-10T11:30:28.356963+0000","last_clean":"2024-09-10T11:30:28.356963+0000","last_became_active":"2024-09-10T11:30:27.363699+0000","last_became_peered":"2024-09-10T11:30:27.363699+0000","last_unstale":"2024-09-10T11:30:28.356963+0000","last_undegraded":"2024-09-10T11:30:28.356963+0000","last_fullsized":"2024-09-10T11:30:28.356963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:36.749159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360157+0000","last_change":"2024-09-10T11:30:27.372337+0000","last_active":"2024-09-10T11:30:28.360157+0000","last_peered":"2024-09-10T11:30:28.360157+0000","last_clean":"2024-09-10T11:30:28.360157+0000","last_became_active":"2024-09-10T11:30:27.372074+0000","last_became_peered":"2024-09-10T11:30:27.372074+0000","last_unstale":"2024-09-10T11:30:28.360157+0000","last_undegraded":"2024-09-10T11:30:28.360157+0000","last_fullsized":"2024-09-10T11:30:28.360157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:39.707264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659804+0000","last_change":"2024-09-10T11:30:27.372831+0000","last_active":"2024-09-10T11:30:28.659804+0000","last_peered":"2024-09-10T11:30:28.659804+0000","last_clean":"2024-09-10T11:30:28.659804+0000","last_became_active":"2024-09-10T11:30:27.371298+0000","last_became_peered":"2024-09-10T11:30:27.371298+0000","last_unstale":"2024-09-10T11:30:28.659804+0000","last_undegraded":"2024-09-10T11:30:28.659804+0000","last_fullsized":"2024-09-10T11:30:28.659804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:04.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360104+0000","last_change":"2024-09-10T11:30:27.370679+0000","last_active":"2024-09-10T11:30:28.360104+0000","last_peered":"2024-09-10T11:30:28.360104+0000","last_clean":"2024-09-10T11:30:28.360104+0000","last_became_active":"2024-09-10T11:30:27.370082+0000","last_became_peered":"2024-09-10T11:30:27.370082+0000","last_unstale":"2024-09-10T11:30:28.360104+0000","last_undegraded":"2024-09-10T11:30:28.360104+0000","last_fullsized":"2024-09-10T11:30:28.360104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:29.103851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660089+0000","last_change":"2024-09-10T11:30:27.366962+0000","last_active":"2024-09-10T11:30:28.660089+0000","last_peered":"2024-09-10T11:30:28.660089+0000","last_clean":"2024-09-10T11:30:28.660089+0000","last_became_active":"2024-09-10T11:30:27.366732+0000","last_became_peered":"2024-09-10T11:30:27.366732+0000","last_unstale":"2024-09-10T11:30:28.660089+0000","last_undegraded":"2024-09-10T11:30:28.660089+0000","last_fullsized":"2024-09-10T11:30:28.660089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.894297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357208+0000","last_change":"2024-09-10T11:30:27.359455+0000","last_active":"2024-09-10T11:30:28.357208+0000","last_peered":"2024-09-10T11:30:28.357208+0000","last_clean":"2024-09-10T11:30:28.357208+0000","last_became_active":"2024-09-10T11:30:27.359192+0000","last_became_peered":"2024-09-10T11:30:27.359192+0000","last_unstale":"2024-09-10T11:30:28.357208+0000","last_undegraded":"2024-09-10T11:30:28.357208+0000","last_fullsized":"2024-09-10T11:30:28.357208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:32.342908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356734+0000","last_change":"2024-09-10T11:30:27.361285+0000","last_active":"2024-09-10T11:30:28.356734+0000","last_peered":"2024-09-10T11:30:28.356734+0000","last_clean":"2024-09-10T11:30:28.356734+0000","last_became_active":"2024-09-10T11:30:27.360381+0000","last_became_peered":"2024-09-10T11:30:27.360381+0000","last_unstale":"2024-09-10T11:30:28.356734+0000","last_undegraded":"2024-09-10T11:30:28.356734+0000","last_fullsized":"2024-09-10T11:30:28.356734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:44.857099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359897+0000","last_change":"2024-09-10T11:30:27.365604+0000","last_active":"2024-09-10T11:30:28.359897+0000","last_peered":"2024-09-10T11:30:28.359897+0000","last_clean":"2024-09-10T11:30:28.359897+0000","last_became_active":"2024-09-10T11:30:27.365092+0000","last_became_peered":"2024-09-10T11:30:27.365092+0000","last_unstale":"2024-09-10T11:30:28.359897+0000","last_undegraded":"2024-09-10T11:30:28.359897+0000","last_fullsized":"2024-09-10T11:30:28.359897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.304449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659583+0000","last_change":"2024-09-10T11:30:27.372482+0000","last_active":"2024-09-10T11:30:28.659583+0000","last_peered":"2024-09-10T11:30:28.659583+0000","last_clean":"2024-09-10T11:30:28.659583+0000","last_became_active":"2024-09-10T11:30:27.371823+0000","last_became_peered":"2024-09-10T11:30:27.371823+0000","last_unstale":"2024-09-10T11:30:28.659583+0000","last_undegraded":"2024-09-10T11:30:28.659583+0000","last_fullsized":"2024-09-10T11:30:28.659583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:00.379890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359182+0000","last_change":"2024-09-10T11:30:27.361947+0000","last_active":"2024-09-10T11:30:28.359182+0000","last_peered":"2024-09-10T11:30:28.359182+0000","last_clean":"2024-09-10T11:30:28.359182+0000","last_became_active":"2024-09-10T11:30:27.361705+0000","last_became_peered":"2024-09-10T11:30:27.361705+0000","last_unstale":"2024-09-10T11:30:28.359182+0000","last_undegraded":"2024-09-10T11:30:28.359182+0000","last_fullsized":"2024-09-10T11:30:28.359182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:49.977081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660062+0000","last_change":"2024-09-10T11:30:27.374439+0000","last_active":"2024-09-10T11:30:28.660062+0000","last_peered":"2024-09-10T11:30:28.660062+0000","last_clean":"2024-09-10T11:30:28.660062+0000","last_became_active":"2024-09-10T11:30:27.373730+0000","last_became_peered":"2024-09-10T11:30:27.373730+0000","last_unstale":"2024-09-10T11:30:28.660062+0000","last_undegraded":"2024-09-10T11:30:28.660062+0000","last_fullsized":"2024-09-10T11:30:28.660062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:55.938451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356881+0000","last_change":"2024-09-10T11:30:27.371303+0000","last_active":"2024-09-10T11:30:28.356881+0000","last_peered":"2024-09-10T11:30:28.356881+0000","last_clean":"2024-09-10T11:30:28.356881+0000","last_became_active":"2024-09-10T11:30:27.370830+0000","last_became_peered":"2024-09-10T11:30:27.370830+0000","last_unstale":"2024-09-10T11:30:28.356881+0000","last_undegraded":"2024-09-10T11:30:28.356881+0000","last_fullsized":"2024-09-10T11:30:28.356881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:11.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359871+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:28.359871+0000","last_peered":"2024-09-10T11:30:28.359871+0000","last_clean":"2024-09-10T11:30:28.359871+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:28.359871+0000","last_undegraded":"2024-09-10T11:30:28.359871+0000","last_fullsized":"2024-09-10T11:30:28.359871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:21.094608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357704+0000","last_change":"2024-09-10T11:30:27.368638+0000","last_active":"2024-09-10T11:30:28.357704+0000","last_peered":"2024-09-10T11:30:28.357704+0000","last_clean":"2024-09-10T11:30:28.357704+0000","last_became_active":"2024-09-10T11:30:27.368113+0000","last_became_peered":"2024-09-10T11:30:27.368113+0000","last_unstale":"2024-09-10T11:30:28.357704+0000","last_undegraded":"2024-09-10T11:30:28.357704+0000","last_fullsized":"2024-09-10T11:30:28.357704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:09.131376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659494+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:28.659494+0000","last_peered":"2024-09-10T11:30:28.659494+0000","last_clean":"2024-09-10T11:30:28.659494+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:28.659494+0000","last_undegraded":"2024-09-10T11:30:28.659494+0000","last_fullsized":"2024-09-10T11:30:28.659494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:51.779467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356585+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:28.356585+0000","last_peered":"2024-09-10T11:30:28.356585+0000","last_clean":"2024-09-10T11:30:28.356585+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:28.356585+0000","last_undegraded":"2024-09-10T11:30:28.356585+0000","last_fullsized":"2024-09-10T11:30:28.356585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:34.623802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360215+0000","last_change":"2024-09-10T11:30:27.370405+0000","last_active":"2024-09-10T11:30:28.360215+0000","last_peered":"2024-09-10T11:30:28.360215+0000","last_clean":"2024-09-10T11:30:28.360215+0000","last_became_active":"2024-09-10T11:30:27.369467+0000","last_became_peered":"2024-09-10T11:30:27.369467+0000","last_unstale":"2024-09-10T11:30:28.360215+0000","last_undegraded":"2024-09-10T11:30:28.360215+0000","last_fullsized":"2024-09-10T11:30:28.360215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:09.083864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660711+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:28.660711+0000","last_peered":"2024-09-10T11:30:28.660711+0000","last_clean":"2024-09-10T11:30:28.660711+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:28.660711+0000","last_undegraded":"2024-09-10T11:30:28.660711+0000","last_fullsized":"2024-09-10T11:30:28.660711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:34.426397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357445+0000","last_change":"2024-09-10T11:30:27.367728+0000","last_active":"2024-09-10T11:30:28.357445+0000","last_peered":"2024-09-10T11:30:28.357445+0000","last_clean":"2024-09-10T11:30:28.357445+0000","last_became_active":"2024-09-10T11:30:27.367459+0000","last_became_peered":"2024-09-10T11:30:27.367459+0000","last_unstale":"2024-09-10T11:30:28.357445+0000","last_undegraded":"2024-09-10T11:30:28.357445+0000","last_fullsized":"2024-09-10T11:30:28.357445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:06.747060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357160+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:28.357160+0000","last_peered":"2024-09-10T11:30:28.357160+0000","last_clean":"2024-09-10T11:30:28.357160+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:28.357160+0000","last_undegraded":"2024-09-10T11:30:28.357160+0000","last_fullsized":"2024-09-10T11:30:28.357160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:44.240242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660192+0000","last_change":"2024-09-10T11:30:27.368579+0000","last_active":"2024-09-10T11:30:28.660192+0000","last_peered":"2024-09-10T11:30:28.660192+0000","last_clean":"2024-09-10T11:30:28.660192+0000","last_became_active":"2024-09-10T11:30:27.368354+0000","last_became_peered":"2024-09-10T11:30:27.368354+0000","last_unstale":"2024-09-10T11:30:28.660192+0000","last_undegraded":"2024-09-10T11:30:28.660192+0000","last_fullsized":"2024-09-10T11:30:28.660192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:38.076870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371291+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:28.371291+0000","last_peered":"2024-09-10T11:30:28.371291+0000","last_clean":"2024-09-10T11:30:28.371291+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:28.371291+0000","last_undegraded":"2024-09-10T11:30:28.371291+0000","last_fullsized":"2024-09-10T11:30:28.371291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:03.475850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359716+0000","last_change":"2024-09-10T11:30:27.366765+0000","last_active":"2024-09-10T11:30:28.359716+0000","last_peered":"2024-09-10T11:30:28.359716+0000","last_clean":"2024-09-10T11:30:28.359716+0000","last_became_active":"2024-09-10T11:30:27.366456+0000","last_became_peered":"2024-09-10T11:30:27.366456+0000","last_unstale":"2024-09-10T11:30:28.359716+0000","last_undegraded":"2024-09-10T11:30:28.359716+0000","last_fullsized":"2024-09-10T11:30:28.359716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:07.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359589+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:28.359589+0000","last_peered":"2024-09-10T11:30:28.359589+0000","last_clean":"2024-09-10T11:30:28.359589+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:28.359589+0000","last_undegraded":"2024-09-10T11:30:28.359589+0000","last_fullsized":"2024-09-10T11:30:28.359589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:49.274560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660264+0000","last_change":"2024-09-10T11:30:27.373963+0000","last_active":"2024-09-10T11:30:28.660264+0000","last_peered":"2024-09-10T11:30:28.660264+0000","last_clean":"2024-09-10T11:30:28.660264+0000","last_became_active":"2024-09-10T11:30:27.373716+0000","last_became_peered":"2024-09-10T11:30:27.373716+0000","last_unstale":"2024-09-10T11:30:28.660264+0000","last_undegraded":"2024-09-10T11:30:28.660264+0000","last_fullsized":"2024-09-10T11:30:28.660264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:42.183989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659777+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:28.659777+0000","last_peered":"2024-09-10T11:30:28.659777+0000","last_clean":"2024-09-10T11:30:28.659777+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:28.659777+0000","last_undegraded":"2024-09-10T11:30:28.659777+0000","last_fullsized":"2024-09-10T11:30:28.659777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:16.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371116+0000","last_change":"2024-09-10T11:30:27.362545+0000","last_active":"2024-09-10T11:30:28.371116+0000","last_peered":"2024-09-10T11:30:28.371116+0000","last_clean":"2024-09-10T11:30:28.371116+0000","last_became_active":"2024-09-10T11:30:27.362303+0000","last_became_peered":"2024-09-10T11:30:27.362303+0000","last_unstale":"2024-09-10T11:30:28.371116+0000","last_undegraded":"2024-09-10T11:30:28.371116+0000","last_fullsized":"2024-09-10T11:30:28.371116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359239+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:28.359239+0000","last_peered":"2024-09-10T11:30:28.359239+0000","last_clean":"2024-09-10T11:30:28.359239+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:28.359239+0000","last_undegraded":"2024-09-10T11:30:28.359239+0000","last_fullsized":"2024-09-10T11:30:28.359239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:17.526780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356863+0000","last_change":"2024-09-10T11:30:27.363967+0000","last_active":"2024-09-10T11:30:28.356863+0000","last_peered":"2024-09-10T11:30:28.356863+0000","last_clean":"2024-09-10T11:30:28.356863+0000","last_became_active":"2024-09-10T11:30:27.363715+0000","last_became_peered":"2024-09-10T11:30:27.363715+0000","last_unstale":"2024-09-10T11:30:28.356863+0000","last_undegraded":"2024-09-10T11:30:28.356863+0000","last_fullsized":"2024-09-10T11:30:28.356863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:30.062610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659978+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:28.659978+0000","last_peered":"2024-09-10T11:30:28.659978+0000","last_clean":"2024-09-10T11:30:28.659978+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:28.659978+0000","last_undegraded":"2024-09-10T11:30:28.659978+0000","last_fullsized":"2024-09-10T11:30:28.659978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:17.448052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660486+0000","last_change":"2024-09-10T11:30:27.371396+0000","last_active":"2024-09-10T11:30:28.660486+0000","last_peered":"2024-09-10T11:30:28.660486+0000","last_clean":"2024-09-10T11:30:28.660486+0000","last_became_active":"2024-09-10T11:30:27.370265+0000","last_became_peered":"2024-09-10T11:30:27.370265+0000","last_unstale":"2024-09-10T11:30:28.660486+0000","last_undegraded":"2024-09-10T11:30:28.660486+0000","last_fullsized":"2024-09-10T11:30:28.660486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:27.284946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371504+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:28.371504+0000","last_peered":"2024-09-10T11:30:28.371504+0000","last_clean":"2024-09-10T11:30:28.371504+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:28.371504+0000","last_undegraded":"2024-09-10T11:30:28.371504+0000","last_fullsized":"2024-09-10T11:30:28.371504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:59.588789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357034+0000","last_change":"2024-09-10T11:30:27.371097+0000","last_active":"2024-09-10T11:30:28.357034+0000","last_peered":"2024-09-10T11:30:28.357034+0000","last_clean":"2024-09-10T11:30:28.357034+0000","last_became_active":"2024-09-10T11:30:27.370403+0000","last_became_peered":"2024-09-10T11:30:27.370403+0000","last_unstale":"2024-09-10T11:30:28.357034+0000","last_undegraded":"2024-09-10T11:30:28.357034+0000","last_fullsized":"2024-09-10T11:30:28.357034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:39.981595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357926+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:28.357926+0000","last_peered":"2024-09-10T11:30:28.357926+0000","last_clean":"2024-09-10T11:30:28.357926+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:28.357926+0000","last_undegraded":"2024-09-10T11:30:28.357926+0000","last_fullsized":"2024-09-10T11:30:28.357926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.466472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659957+0000","last_change":"2024-09-10T11:30:27.370723+0000","last_active":"2024-09-10T11:30:28.659957+0000","last_peered":"2024-09-10T11:30:28.659957+0000","last_clean":"2024-09-10T11:30:28.659957+0000","last_became_active":"2024-09-10T11:30:27.369619+0000","last_became_peered":"2024-09-10T11:30:27.369619+0000","last_unstale":"2024-09-10T11:30:28.659957+0000","last_undegraded":"2024-09-10T11:30:28.659957+0000","last_fullsized":"2024-09-10T11:30:28.659957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:12.483235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370845+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:28.370845+0000","last_peered":"2024-09-10T11:30:28.370845+0000","last_clean":"2024-09-10T11:30:28.370845+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:28.370845+0000","last_undegraded":"2024-09-10T11:30:28.370845+0000","last_fullsized":"2024-09-10T11:30:28.370845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:06.667422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660299+0000","last_change":"2024-09-10T11:30:27.366991+0000","last_active":"2024-09-10T11:30:28.660299+0000","last_peered":"2024-09-10T11:30:28.660299+0000","last_clean":"2024-09-10T11:30:28.660299+0000","last_became_active":"2024-09-10T11:30:27.366752+0000","last_became_peered":"2024-09-10T11:30:27.366752+0000","last_unstale":"2024-09-10T11:30:28.660299+0000","last_undegraded":"2024-09-10T11:30:28.660299+0000","last_fullsized":"2024-09-10T11:30:28.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:47.959643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357478+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:28.357478+0000","last_peered":"2024-09-10T11:30:28.357478+0000","last_clean":"2024-09-10T11:30:28.357478+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:28.357478+0000","last_undegraded":"2024-09-10T11:30:28.357478+0000","last_fullsized":"2024-09-10T11:30:28.357478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:28.210484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370778+0000","last_change":"2024-09-10T11:30:27.363710+0000","last_active":"2024-09-10T11:30:28.370778+0000","last_peered":"2024-09-10T11:30:28.370778+0000","last_clean":"2024-09-10T11:30:28.370778+0000","last_became_active":"2024-09-10T11:30:27.363475+0000","last_became_peered":"2024-09-10T11:30:27.363475+0000","last_unstale":"2024-09-10T11:30:28.370778+0000","last_undegraded":"2024-09-10T11:30:28.370778+0000","last_fullsized":"2024-09-10T11:30:28.370778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:48.768550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359866+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:28.359866+0000","last_peered":"2024-09-10T11:30:28.359866+0000","last_clean":"2024-09-10T11:30:28.359866+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:28.359866+0000","last_undegraded":"2024-09-10T11:30:28.359866+0000","last_fullsized":"2024-09-10T11:30:28.359866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:14.325407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371216+0000","last_change":"2024-09-10T11:30:27.367212+0000","last_active":"2024-09-10T11:30:28.371216+0000","last_peered":"2024-09-10T11:30:28.371216+0000","last_clean":"2024-09-10T11:30:28.371216+0000","last_became_active":"2024-09-10T11:30:27.366516+0000","last_became_peered":"2024-09-10T11:30:27.366516+0000","last_unstale":"2024-09-10T11:30:28.371216+0000","last_undegraded":"2024-09-10T11:30:28.371216+0000","last_fullsized":"2024-09-10T11:30:28.371216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:43.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357377+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:28.357377+0000","last_peered":"2024-09-10T11:30:28.357377+0000","last_clean":"2024-09-10T11:30:28.357377+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:28.357377+0000","last_undegraded":"2024-09-10T11:30:28.357377+0000","last_fullsized":"2024-09-10T11:30:28.357377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:16.594543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660098+0000","last_change":"2024-09-10T11:30:27.363291+0000","last_active":"2024-09-10T11:30:28.660098+0000","last_peered":"2024-09-10T11:30:28.660098+0000","last_clean":"2024-09-10T11:30:28.660098+0000","last_became_active":"2024-09-10T11:30:27.362845+0000","last_became_peered":"2024-09-10T11:30:27.362845+0000","last_unstale":"2024-09-10T11:30:28.660098+0000","last_undegraded":"2024-09-10T11:30:28.660098+0000","last_fullsized":"2024-09-10T11:30:28.660098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.348250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:28.357044+0000","last_peered":"2024-09-10T11:30:28.357044+0000","last_clean":"2024-09-10T11:30:28.357044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:28.357044+0000","last_undegraded":"2024-09-10T11:30:28.357044+0000","last_fullsized":"2024-09-10T11:30:28.357044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660171+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:28.660171+0000","last_peered":"2024-09-10T11:30:28.660171+0000","last_clean":"2024-09-10T11:30:28.660171+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:28.660171+0000","last_undegraded":"2024-09-10T11:30:28.660171+0000","last_fullsized":"2024-09-10T11:30:28.660171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:07.847876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358022+0000","last_change":"2024-09-10T11:30:27.368957+0000","last_active":"2024-09-10T11:30:28.358022+0000","last_peered":"2024-09-10T11:30:28.358022+0000","last_clean":"2024-09-10T11:30:28.358022+0000","last_became_active":"2024-09-10T11:30:27.368412+0000","last_became_peered":"2024-09-10T11:30:27.368412+0000","last_unstale":"2024-09-10T11:30:28.358022+0000","last_undegraded":"2024-09-10T11:30:28.358022+0000","last_fullsized":"2024-09-10T11:30:28.358022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:15.612861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660500+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:28.660500+0000","last_peered":"2024-09-10T11:30:28.660500+0000","last_clean":"2024-09-10T11:30:28.660500+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:28.660500+0000","last_undegraded":"2024-09-10T11:30:28.660500+0000","last_fullsized":"2024-09-10T11:30:28.660500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:18.084482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356890+0000","last_change":"2024-09-10T11:30:27.370947+0000","last_active":"2024-09-10T11:30:28.356890+0000","last_peered":"2024-09-10T11:30:28.356890+0000","last_clean":"2024-09-10T11:30:28.356890+0000","last_became_active":"2024-09-10T11:30:27.370630+0000","last_became_peered":"2024-09-10T11:30:27.370630+0000","last_unstale":"2024-09-10T11:30:28.356890+0000","last_undegraded":"2024-09-10T11:30:28.356890+0000","last_fullsized":"2024-09-10T11:30:28.356890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:42.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359693+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:28.359693+0000","last_peered":"2024-09-10T11:30:28.359693+0000","last_clean":"2024-09-10T11:30:28.359693+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:28.359693+0000","last_undegraded":"2024-09-10T11:30:28.359693+0000","last_fullsized":"2024-09-10T11:30:28.359693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:39.031878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360140+0000","last_change":"2024-09-10T11:30:27.365334+0000","last_active":"2024-09-10T11:30:28.360140+0000","last_peered":"2024-09-10T11:30:28.360140+0000","last_clean":"2024-09-10T11:30:28.360140+0000","last_became_active":"2024-09-10T11:30:27.364781+0000","last_became_peered":"2024-09-10T11:30:27.364781+0000","last_unstale":"2024-09-10T11:30:28.360140+0000","last_undegraded":"2024-09-10T11:30:28.360140+0000","last_fullsized":"2024-09-10T11:30:28.360140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:22.952484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660118+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:28.660118+0000","last_peered":"2024-09-10T11:30:28.660118+0000","last_clean":"2024-09-10T11:30:28.660118+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:28.660118+0000","last_undegraded":"2024-09-10T11:30:28.660118+0000","last_fullsized":"2024-09-10T11:30:28.660118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:52.177357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371203+0000","last_change":"2024-09-10T11:30:27.369498+0000","last_active":"2024-09-10T11:30:28.371203+0000","last_peered":"2024-09-10T11:30:28.371203+0000","last_clean":"2024-09-10T11:30:28.371203+0000","last_became_active":"2024-09-10T11:30:27.369231+0000","last_became_peered":"2024-09-10T11:30:27.369231+0000","last_unstale":"2024-09-10T11:30:28.371203+0000","last_undegraded":"2024-09-10T11:30:28.371203+0000","last_fullsized":"2024-09-10T11:30:28.371203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:05.406568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360323+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:28.360323+0000","last_peered":"2024-09-10T11:30:28.360323+0000","last_clean":"2024-09-10T11:30:28.360323+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:28.360323+0000","last_undegraded":"2024-09-10T11:30:28.360323+0000","last_fullsized":"2024-09-10T11:30:28.360323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:27.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659688+0000","last_change":"2024-09-10T11:30:27.369830+0000","last_active":"2024-09-10T11:30:28.659688+0000","last_peered":"2024-09-10T11:30:28.659688+0000","last_clean":"2024-09-10T11:30:28.659688+0000","last_became_active":"2024-09-10T11:30:27.369147+0000","last_became_peered":"2024-09-10T11:30:27.369147+0000","last_unstale":"2024-09-10T11:30:28.659688+0000","last_undegraded":"2024-09-10T11:30:28.659688+0000","last_fullsized":"2024-09-10T11:30:28.659688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:00.145985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660445+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:28.660445+0000","last_peered":"2024-09-10T11:30:28.660445+0000","last_clean":"2024-09-10T11:30:28.660445+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:28.660445+0000","last_undegraded":"2024-09-10T11:30:28.660445+0000","last_fullsized":"2024-09-10T11:30:28.660445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:02.778492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357067+0000","last_change":"2024-09-10T11:30:27.370822+0000","last_active":"2024-09-10T11:30:28.357067+0000","last_peered":"2024-09-10T11:30:28.357067+0000","last_clean":"2024-09-10T11:30:28.357067+0000","last_became_active":"2024-09-10T11:30:27.370501+0000","last_became_peered":"2024-09-10T11:30:27.370501+0000","last_unstale":"2024-09-10T11:30:28.357067+0000","last_undegraded":"2024-09-10T11:30:28.357067+0000","last_fullsized":"2024-09-10T11:30:28.357067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:05.773002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358086+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:28.358086+0000","last_peered":"2024-09-10T11:30:28.358086+0000","last_clean":"2024-09-10T11:30:28.358086+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:28.358086+0000","last_undegraded":"2024-09-10T11:30:28.358086+0000","last_fullsized":"2024-09-10T11:30:28.358086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:39.722801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357861+0000","last_change":"2024-09-10T11:30:27.362871+0000","last_active":"2024-09-10T11:30:28.357861+0000","last_peered":"2024-09-10T11:30:28.357861+0000","last_clean":"2024-09-10T11:30:28.357861+0000","last_became_active":"2024-09-10T11:30:27.362533+0000","last_became_peered":"2024-09-10T11:30:27.362533+0000","last_unstale":"2024-09-10T11:30:28.357861+0000","last_undegraded":"2024-09-10T11:30:28.357861+0000","last_fullsized":"2024-09-10T11:30:28.357861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:34.967835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660031+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:28.660031+0000","last_peered":"2024-09-10T11:30:28.660031+0000","last_clean":"2024-09-10T11:30:28.660031+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:28.660031+0000","last_undegraded":"2024-09-10T11:30:28.660031+0000","last_fullsized":"2024-09-10T11:30:28.660031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:00.971732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357833+0000","last_change":"2024-09-10T11:30:27.365497+0000","last_active":"2024-09-10T11:30:28.357833+0000","last_peered":"2024-09-10T11:30:28.357833+0000","last_clean":"2024-09-10T11:30:28.357833+0000","last_became_active":"2024-09-10T11:30:27.363947+0000","last_became_peered":"2024-09-10T11:30:27.363947+0000","last_unstale":"2024-09-10T11:30:28.357833+0000","last_undegraded":"2024-09-10T11:30:28.357833+0000","last_fullsized":"2024-09-10T11:30:28.357833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:24.206828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371397+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:28.371397+0000","last_peered":"2024-09-10T11:30:28.371397+0000","last_clean":"2024-09-10T11:30:28.371397+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:28.371397+0000","last_undegraded":"2024-09-10T11:30:28.371397+0000","last_fullsized":"2024-09-10T11:30:28.371397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:26.569057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660619+0000","last_change":"2024-09-10T11:30:27.372245+0000","last_active":"2024-09-10T11:30:28.660619+0000","last_peered":"2024-09-10T11:30:28.660619+0000","last_clean":"2024-09-10T11:30:28.660619+0000","last_became_active":"2024-09-10T11:30:27.371099+0000","last_became_peered":"2024-09-10T11:30:27.371099+0000","last_unstale":"2024-09-10T11:30:28.660619+0000","last_undegraded":"2024-09-10T11:30:28.660619+0000","last_fullsized":"2024-09-10T11:30:28.660619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:39.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357638+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:28.357638+0000","last_peered":"2024-09-10T11:30:28.357638+0000","last_clean":"2024-09-10T11:30:28.357638+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:28.357638+0000","last_undegraded":"2024-09-10T11:30:28.357638+0000","last_fullsized":"2024-09-10T11:30:28.357638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:59.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357610+0000","last_change":"2024-09-10T11:30:27.361895+0000","last_active":"2024-09-10T11:30:28.357610+0000","last_peered":"2024-09-10T11:30:28.357610+0000","last_clean":"2024-09-10T11:30:28.357610+0000","last_became_active":"2024-09-10T11:30:27.361662+0000","last_became_peered":"2024-09-10T11:30:27.361662+0000","last_unstale":"2024-09-10T11:30:28.357610+0000","last_undegraded":"2024-09-10T11:30:28.357610+0000","last_fullsized":"2024-09-10T11:30:28.357610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:08.687699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356658+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:28.356658+0000","last_peered":"2024-09-10T11:30:28.356658+0000","last_clean":"2024-09-10T11:30:28.356658+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:28.356658+0000","last_undegraded":"2024-09-10T11:30:28.356658+0000","last_fullsized":"2024-09-10T11:30:28.356658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:49.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359485+0000","last_change":"2024-09-10T11:30:27.362943+0000","last_active":"2024-09-10T11:30:28.359485+0000","last_peered":"2024-09-10T11:30:28.359485+0000","last_clean":"2024-09-10T11:30:28.359485+0000","last_became_active":"2024-09-10T11:30:27.362709+0000","last_became_peered":"2024-09-10T11:30:27.362709+0000","last_unstale":"2024-09-10T11:30:28.359485+0000","last_undegraded":"2024-09-10T11:30:28.359485+0000","last_fullsized":"2024-09-10T11:30:28.359485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:05.763931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659605+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:28.659605+0000","last_peered":"2024-09-10T11:30:28.659605+0000","last_clean":"2024-09-10T11:30:28.659605+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:28.659605+0000","last_undegraded":"2024-09-10T11:30:28.659605+0000","last_fullsized":"2024-09-10T11:30:28.659605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:01.450636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660335+0000","last_change":"2024-09-10T11:30:27.372613+0000","last_active":"2024-09-10T11:30:28.660335+0000","last_peered":"2024-09-10T11:30:28.660335+0000","last_clean":"2024-09-10T11:30:28.660335+0000","last_became_active":"2024-09-10T11:30:27.370970+0000","last_became_peered":"2024-09-10T11:30:27.370970+0000","last_unstale":"2024-09-10T11:30:28.660335+0000","last_undegraded":"2024-09-10T11:30:28.660335+0000","last_fullsized":"2024-09-10T11:30:28.660335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:51.164359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660329+0000","last_change":"2024-09-10T11:30:27.372047+0000","last_active":"2024-09-10T11:30:28.660329+0000","last_peered":"2024-09-10T11:30:28.660329+0000","last_clean":"2024-09-10T11:30:28.660329+0000","last_became_active":"2024-09-10T11:30:27.370618+0000","last_became_peered":"2024-09-10T11:30:27.370618+0000","last_unstale":"2024-09-10T11:30:28.660329+0000","last_undegraded":"2024-09-10T11:30:28.660329+0000","last_fullsized":"2024-09-10T11:30:28.660329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:47.350659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371285+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:28.371285+0000","last_peered":"2024-09-10T11:30:28.371285+0000","last_clean":"2024-09-10T11:30:28.371285+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:28.371285+0000","last_undegraded":"2024-09-10T11:30:28.371285+0000","last_fullsized":"2024-09-10T11:30:28.371285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:31.153745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357070+0000","last_change":"2024-09-10T11:30:27.360961+0000","last_active":"2024-09-10T11:30:28.357070+0000","last_peered":"2024-09-10T11:30:28.357070+0000","last_clean":"2024-09-10T11:30:28.357070+0000","last_became_active":"2024-09-10T11:30:27.360702+0000","last_became_peered":"2024-09-10T11:30:27.360702+0000","last_unstale":"2024-09-10T11:30:28.357070+0000","last_undegraded":"2024-09-10T11:30:28.357070+0000","last_fullsized":"2024-09-10T11:30:28.357070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:54.000251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371486+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:28.371486+0000","last_peered":"2024-09-10T11:30:28.371486+0000","last_clean":"2024-09-10T11:30:28.371486+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:28.371486+0000","last_undegraded":"2024-09-10T11:30:28.371486+0000","last_fullsized":"2024-09-10T11:30:28.371486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:49.786219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360066+0000","last_change":"2024-09-10T11:30:27.368686+0000","last_active":"2024-09-10T11:30:28.360066+0000","last_peered":"2024-09-10T11:30:28.360066+0000","last_clean":"2024-09-10T11:30:28.360066+0000","last_became_active":"2024-09-10T11:30:27.368329+0000","last_became_peered":"2024-09-10T11:30:27.368329+0000","last_unstale":"2024-09-10T11:30:28.360066+0000","last_undegraded":"2024-09-10T11:30:28.360066+0000","last_fullsized":"2024-09-10T11:30:28.360066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:03.672443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360276+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:28.360276+0000","last_peered":"2024-09-10T11:30:28.360276+0000","last_clean":"2024-09-10T11:30:28.360276+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:28.360276+0000","last_undegraded":"2024-09-10T11:30:28.360276+0000","last_fullsized":"2024-09-10T11:30:28.360276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:06.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360033+0000","last_change":"2024-09-10T11:30:27.372498+0000","last_active":"2024-09-10T11:30:28.360033+0000","last_peered":"2024-09-10T11:30:28.360033+0000","last_clean":"2024-09-10T11:30:28.360033+0000","last_became_active":"2024-09-10T11:30:27.372251+0000","last_became_peered":"2024-09-10T11:30:27.372251+0000","last_unstale":"2024-09-10T11:30:28.360033+0000","last_undegraded":"2024-09-10T11:30:28.360033+0000","last_fullsized":"2024-09-10T11:30:28.360033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:44.924775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357462+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:28.357462+0000","last_peered":"2024-09-10T11:30:28.357462+0000","last_clean":"2024-09-10T11:30:28.357462+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:28.357462+0000","last_undegraded":"2024-09-10T11:30:28.357462+0000","last_fullsized":"2024-09-10T11:30:28.357462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:16.783289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338766,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199926942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338788,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:35.020 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:30:35.020 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T11:30:35.020 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T11:30:35.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T11:30:35.021 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T11:30:35.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.153+0000 7f2a7e50a640 1 Processor -- start 2024-09-10T11:30:35.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.153+0000 7f2a7e50a640 1 -- start start 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a7e50a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a7815ee60 0x7f2a780d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a7e50a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d2d80 0x7f2a780d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a7e50a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7815f240 con 0x7f2a780d3770 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a7e50a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2a780c87c0 con 0x7f2a7815ee60 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a7e50a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a780d0430 con 0x7f2a780d2d80 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a7815ee60 0x7f2a780d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a7815ee60 0x7f2a780d2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54248/0 (socket says 172.21.15.99:54248) 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 -- 172.21.15.99:0/2684283385 learned_addr learned my addr 172.21.15.99:0/2684283385 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a6ffff640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d2d80 0x7f2a780d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 -- 172.21.15.99:0/2684283385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d2d80 msgr2=0x7f2a780d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d2d80 0x7f2a780d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 -- 172.21.15.99:0/2684283385 >> v1:172.21.15.196:6789/0 conn(0x7f2a780d3770 legacy=0x7f2a780d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.154+0000 7f2a77fff640 1 -- 172.21.15.99:0/2684283385 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a780d0710 con 0x7f2a7815ee60 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.155+0000 7f2a77fff640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a780d2840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009870 tx=0x7f2a6802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6b67d1412fabb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.155+0000 7f2a777fe640 1 -- 172.21.15.99:0/2684283385 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68037070 con 0x7f2a7815ee60 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.155+0000 7f2a777fe640 1 -- 172.21.15.99:0/2684283385 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a68032440 con 0x7f2a7815ee60 2024-09-10T11:30:35.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.155+0000 7f2a777fe640 1 -- 172.21.15.99:0/2684283385 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68032810 con 0x7f2a7815ee60 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 -- 172.21.15.99:0/2684283385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 msgr2=0x7f2a780d2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a780d2840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009870 tx=0x7f2a6802a210 comp rx=0 tx=0).stop 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 -- 172.21.15.99:0/2684283385 shutdown_connections 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d2d80 0x7f2a780d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/2684283385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a780d2840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 -- 172.21.15.99:0/2684283385 >> 172.21.15.99:0/2684283385 conn(0x7f2a780d4c70 msgr2=0x7f2a780d5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 -- 172.21.15.99:0/2684283385 shutdown_connections 2024-09-10T11:30:35.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.156+0000 7f2a7e50a640 1 -- 172.21.15.99:0/2684283385 wait complete. 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.157+0000 7f2a7e50a640 1 Processor -- start 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.157+0000 7f2a7e50a640 1 -- start start 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 0x7f2a78162070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 0x7f2a7815fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a780d5a40 con 0x7f2a7815ee60 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a780c8a10 con 0x7f2a780d2d80 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7e50a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a780c9f40 con 0x7f2a780d3770 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54254/0 (socket says 172.21.15.99:54254) 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 -- 172.21.15.99:0/1447491767 learned_addr learned my addr 172.21.15.99:0/1447491767 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a6ffff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 0x7f2a7815fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a77fff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 0x7f2a78162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 -- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 msgr2=0x7f2a7815fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 0x7f2a7815fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 -- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 msgr2=0x7f2a78162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 0x7f2a78162070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 -- 172.21.15.99:0/1447491767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7815bd30 con 0x7f2a7815ee60 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a6ffff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 0x7f2a7815fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a77fff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 0x7f2a78162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.158+0000 7f2a7caa7640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000c780 tx=0x7f2a7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.159+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70004b60 con 0x7f2a7815ee60 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.159+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a70004d00 con 0x7f2a7815ee60 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.159+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a780cdb10 con 0x7f2a7815ee60 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.159+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000aa90 con 0x7f2a7815ee60 2024-09-10T11:30:35.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.159+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a780ce0d0 con 0x7f2a7815ee60 2024-09-10T11:30:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.160+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a78073ef0 con 0x7f2a7815ee60 2024-09-10T11:30:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.160+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a7000ac30 con 0x7f2a7815ee60 2024-09-10T11:30:35.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.161+0000 7f2a757fa640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.161+0000 7f2a77fff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.161+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2a7008b160 con 0x7f2a7815ee60 2024-09-10T11:30:35.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.161+0000 7f2a77fff640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 0x7f2a5806e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034000 tx=0x7f2a6802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.165+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a78073ef0 con 0x7f2a7815ee60 2024-09-10T11:30:35.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.380+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2a780d3230 con 0x7f2a7815ee60 2024-09-10T11:30:35.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.384+0000 7f2a757fa640 1 -- 172.21.15.99:0/1447491767 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172938 (secure 0 0 0) 0x7f2a70020070 con 0x7f2a7815ee60 2024-09-10T11:30:35.388 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 msgr2=0x7f2a5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 0x7f2a5806e2f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2a68034000 tx=0x7f2a6802e000 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 msgr2=0x7f2a78158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000c780 tx=0x7f2a7000cc40 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 shutdown_connections 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a5806be30 0x7f2a5806e2f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a7815ee60 0x7f2a78158a10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a780d3770 0x7f2a7815fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 --2- 172.21.15.99:0/1447491767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a780d2d80 0x7f2a78162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 >> 172.21.15.99:0/1447491767 conn(0x7f2a780d4c70 msgr2=0x7f2a780ba5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 shutdown_connections 2024-09-10T11:30:35.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.387+0000 7f2a7e50a640 1 -- 172.21.15.99:0/1447491767 wait complete. 2024-09-10T11:30:35.399 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2898440632},{"type":"v1","addr":"172.21.15.99:6833","nonce":2898440632}]},"active_addr":"172.21.15.99:6833/2898440632","active_change":"2024-09-10T11:30:14.898786+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2271432356}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1396130417}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3599824922}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":199775715}]}]} 2024-09-10T11:30:35.402 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T11:30:35.403 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T11:30:35.403 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:35.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.498+0000 7faaf212a640 1 Processor -- start 2024-09-10T11:30:35.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.498+0000 7faaf212a640 1 -- start start 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.498+0000 7faaf212a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faaec0d1200 0x7faaec073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.498+0000 7faaf212a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaec06e5d0 0x7faaec06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.498+0000 7faaf212a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaec152e60 con 0x7faaec15c060 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaf212a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7faaec06ef80 con 0x7faaec06e5d0 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaf212a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faaec0d15e0 con 0x7faaec0d1200 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaec06e5d0 0x7faaec06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faaec06e5d0 0x7faaec06ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54264/0 (socket says 172.21.15.99:54264) 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 -- 172.21.15.99:0/602087744 learned_addr learned my addr 172.21.15.99:0/602087744 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeb7fe640 1 --2- 172.21.15.99:0/602087744 >> v2:172.21.15.196:3300/0 conn(0x7faaec0d1200 0x7faaec073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 -- 172.21.15.99:0/602087744 >> v2:172.21.15.196:3300/0 conn(0x7faaec0d1200 msgr2=0x7faaec073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 --2- 172.21.15.99:0/602087744 >> v2:172.21.15.196:3300/0 conn(0x7faaec0d1200 0x7faaec073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 -- 172.21.15.99:0/602087744 >> v1:172.21.15.196:6789/0 conn(0x7faaec15c060 legacy=0x7faaec15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.499+0000 7faaeaffd640 1 -- 172.21.15.99:0/602087744 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec072680 con 0x7faaec06e5d0 2024-09-10T11:30:35.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaeaffd640 1 --2- 172.21.15.99:0/602087744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec06e5d0 0x7faaec06ea40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d1850 tx=0x7faadc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47b53bee7840a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaea7fc640 1 -- 172.21.15.99:0/602087744 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc037070 con 0x7faaec06e5d0 2024-09-10T11:30:35.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaea7fc640 1 -- 172.21.15.99:0/602087744 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faadc032440 con 0x7faaec06e5d0 2024-09-10T11:30:35.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaea7fc640 1 -- 172.21.15.99:0/602087744 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc032810 con 0x7faaec06e5d0 2024-09-10T11:30:35.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 -- 172.21.15.99:0/602087744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec06e5d0 msgr2=0x7faaec06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 --2- 172.21.15.99:0/602087744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec06e5d0 0x7faaec06ea40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d1850 tx=0x7faadc02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 -- 172.21.15.99:0/602087744 shutdown_connections 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 --2- 172.21.15.99:0/602087744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec06e5d0 0x7faaec06ea40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 --2- 172.21.15.99:0/602087744 >> v2:172.21.15.196:3300/0 conn(0x7faaec0d1200 0x7faaec073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 -- 172.21.15.99:0/602087744 >> 172.21.15.99:0/602087744 conn(0x7faaec06bc10 msgr2=0x7faaec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 -- 172.21.15.99:0/602087744 shutdown_connections 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.500+0000 7faaf212a640 1 -- 172.21.15.99:0/602087744 wait complete. 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.501+0000 7faaf212a640 1 Processor -- start 2024-09-10T11:30:35.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 -- start start 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaec06e5d0 0x7faaec1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaec0d1200 0x7faaec15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faaec072f20 con 0x7faaec15c060 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faaec072c40 con 0x7faaec0d1200 2024-09-10T11:30:35.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.502+0000 7faaf212a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faaec15cd90 con 0x7faaec06e5d0 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54268/0 (socket says 172.21.15.99:54268) 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 -- 172.21.15.99:0/4234690651 learned_addr learned my addr 172.21.15.99:0/4234690651 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaeaffd640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaec0d1200 0x7faaec15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaeb7fe640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaec06e5d0 0x7faaec1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 -- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaec06e5d0 msgr2=0x7faaec1f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaec06e5d0 0x7faaec1f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 -- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaec0d1200 msgr2=0x7faaec15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaec0d1200 0x7faaec15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 -- 172.21.15.99:0/4234690651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec1f47a0 con 0x7faaec15c060 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaebfff640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faae400c780 tx=0x7faae400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004b60 con 0x7faaec15c060 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae4004d00 con 0x7faaec15c060 2024-09-10T11:30:35.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1f4a60 con 0x7faaec15c060 2024-09-10T11:30:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae400aa90 con 0x7faaec15c060 2024-09-10T11:30:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.503+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1f5020 con 0x7faaec15c060 2024-09-10T11:30:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.504+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec06ea40 con 0x7faaec15c060 2024-09-10T11:30:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.505+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faae400ac30 con 0x7faaec15c060 2024-09-10T11:30:35.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.505+0000 7faae3fff640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 0x7faacc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.505+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faae408af40 con 0x7faaec15c060 2024-09-10T11:30:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.505+0000 7faaeb7fe640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 0x7faacc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.506+0000 7faaeb7fe640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 0x7faacc06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faaec0cde80 tx=0x7faad4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.510+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faae4020070 con 0x7faaec15c060 2024-09-10T11:30:35.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.676+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faaec0b4b80 con 0x7faaec15c060 2024-09-10T11:30:35.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.678+0000 7faae3fff640 1 -- 172.21.15.99:0/4234690651 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7faae40592a0 con 0x7faaec15c060 2024-09-10T11:30:35.680 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:35.680 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 msgr2=0x7faacc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 0x7faacc06e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faaec0cde80 tx=0x7faad4008040 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 msgr2=0x7faaec15af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faae400c780 tx=0x7faae400cc40 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 shutdown_connections 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faacc06be30 0x7faacc06e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faaec15c060 0x7faaec15af20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faaec0d1200 0x7faaec15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 --2- 172.21.15.99:0/4234690651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faaec06e5d0 0x7faaec1f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 >> 172.21.15.99:0/4234690651 conn(0x7faaec06bc10 msgr2=0x7faaec155040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 shutdown_connections 2024-09-10T11:30:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.680+0000 7faaf212a640 1 -- 172.21.15.99:0/4234690651 wait complete. 2024-09-10T11:30:35.693 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T11:30:35.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.789+0000 7fdaddc77640 1 Processor -- start 2024-09-10T11:30:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.789+0000 7fdaddc77640 1 -- start start 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.789+0000 7fdaddc77640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdad80b4150 0x7fdad80b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdaddc77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 0x7fdad815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdaddc77640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad8072650 con 0x7fdad815cb70 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdaddc77640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdad80b1240 con 0x7fdad80b4150 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdaddc77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdad80734f0 con 0x7fdad80b0df0 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdacffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdad80b4150 0x7fdad80b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 0x7fdad815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad77fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fdad815cb70 0x7fdad8072de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59092/0 (socket says 172.21.15.99:59092) 2024-09-10T11:30:35.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad77fe640 1 -- 172.21.15.99:0/3645736960 learned_addr learned my addr 172.21.15.99:0/3645736960 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:35.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad7fff640 1 -- 172.21.15.99:0/3645736960 >> v2:172.21.15.196:3300/0 conn(0x7fdad80b4150 msgr2=0x7fdad80b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:35.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad7fff640 1 --2- 172.21.15.99:0/3645736960 >> v2:172.21.15.196:3300/0 conn(0x7fdad80b4150 0x7fdad80b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad7fff640 1 -- 172.21.15.99:0/3645736960 >> v1:172.21.15.196:6789/0 conn(0x7fdad815cb70 legacy=0x7fdad8072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.790+0000 7fdad7fff640 1 -- 172.21.15.99:0/3645736960 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad815db70 con 0x7fdad80b0df0 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.791+0000 7fdad7fff640 1 --2- 172.21.15.99:0/3645736960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 0x7fdad815cf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009d90 tx=0x7fdad002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b52e90a3339d558 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.791+0000 7fdad6ffd640 1 -- 172.21.15.99:0/3645736960 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0007740 con 0x7fdad80b0df0 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.791+0000 7fdad6ffd640 1 -- 172.21.15.99:0/3645736960 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad00078e0 con 0x7fdad80b0df0 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.791+0000 7fdad6ffd640 1 -- 172.21.15.99:0/3645736960 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0007cb0 con 0x7fdad80b0df0 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 -- 172.21.15.99:0/3645736960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 msgr2=0x7fdad815cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 --2- 172.21.15.99:0/3645736960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 0x7fdad815cf50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009d90 tx=0x7fdad002a660 comp rx=0 tx=0).stop 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 -- 172.21.15.99:0/3645736960 shutdown_connections 2024-09-10T11:30:35.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 --2- 172.21.15.99:0/3645736960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b0df0 0x7fdad815cf50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 --2- 172.21.15.99:0/3645736960 >> v2:172.21.15.196:3300/0 conn(0x7fdad80b4150 0x7fdad80b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 -- 172.21.15.99:0/3645736960 >> 172.21.15.99:0/3645736960 conn(0x7fdad80b3630 msgr2=0x7fdad80b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 -- 172.21.15.99:0/3645736960 shutdown_connections 2024-09-10T11:30:35.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.792+0000 7fdaddc77640 1 -- 172.21.15.99:0/3645736960 wait complete. 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.793+0000 7fdaddc77640 1 Processor -- start 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.793+0000 7fdaddc77640 1 -- start start 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdad80b0df0 0x7fdad81f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 0x7fdad81f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdad815e130 con 0x7fdad815cb70 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdad80728c0 con 0x7fdad80b0df0 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdaddc77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdad815d490 con 0x7fdad80b4150 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54272/0 (socket says 172.21.15.99:54272) 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 -- 172.21.15.99:0/1595768791 learned_addr learned my addr 172.21.15.99:0/1595768791 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdacffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 0x7fdad81f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad77fe640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdad80b0df0 0x7fdad81f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 -- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 msgr2=0x7fdad81f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 0x7fdad81f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 -- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdad80b0df0 msgr2=0x7fdad81f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdad80b0df0 0x7fdad81f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 -- 172.21.15.99:0/1595768791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad80b4b60 con 0x7fdad815cb70 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdacffff640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 0x7fdad81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.794+0000 7fdad7fff640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009eb0 tx=0x7fdad0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.795+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0040070 con 0x7fdad815cb70 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.795+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad0031bc0 con 0x7fdad815cb70 2024-09-10T11:30:35.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.795+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad80b4e40 con 0x7fdad815cb70 2024-09-10T11:30:35.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.795+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0009450 con 0x7fdad815cb70 2024-09-10T11:30:35.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.795+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad80b5400 con 0x7fdad815cb70 2024-09-10T11:30:35.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.796+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad815d870 con 0x7fdad815cb70 2024-09-10T11:30:35.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.796+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdad0039040 con 0x7fdad815cb70 2024-09-10T11:30:35.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.796+0000 7fdad4ff9640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 0x7fdab806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:35.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.796+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdad00ac4f0 con 0x7fdad815cb70 2024-09-10T11:30:35.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.796+0000 7fdad77fe640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 0x7fdab806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:35.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.797+0000 7fdad77fe640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 0x7fdab806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdad81573f0 tx=0x7fdac8007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:35.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.801+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdad007a8e0 con 0x7fdad815cb70 2024-09-10T11:30:35.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.967+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdad806c9f0 con 0x7fdad815cb70 2024-09-10T11:30:35.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.968+0000 7fdad4ff9640 1 -- 172.21.15.99:0/1595768791 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7fdad007ebe0 con 0x7fdad815cb70 2024-09-10T11:30:35.970 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:35.970 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:35.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 msgr2=0x7fdab806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 0x7fdab806e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdad81573f0 tx=0x7fdac8007e80 comp rx=0 tx=0).stop 2024-09-10T11:30:35.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 msgr2=0x7fdad8160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009eb0 tx=0x7fdad0004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 shutdown_connections 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdab806be30 0x7fdab806e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdad815cb70 0x7fdad8160970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdad80b4150 0x7fdad81f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 --2- 172.21.15.99:0/1595768791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdad80b0df0 0x7fdad81f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 >> 172.21.15.99:0/1595768791 conn(0x7fdad80b3630 msgr2=0x7fdad80b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 shutdown_connections 2024-09-10T11:30:35.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:35.970+0000 7fdaddc77640 1 -- 172.21.15.99:0/1595768791 wait complete. 2024-09-10T11:30:35.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T11:30:35.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T11:30:35.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T11:30:35.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T11:30:35.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T11:30:35.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T11:30:35.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T11:30:35.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T11:30:36.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7ff5232f1640 1 Processor -- start 2024-09-10T11:30:36.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7f53a5311640 1 Processor -- start 2024-09-10T11:30:36.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.150+0000 7f78d4a5e640 1 Processor -- start 2024-09-10T11:30:36.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7f412c9f5640 1 Processor -- start 2024-09-10T11:30:36.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7f53a5311640 1 -- start start 2024-09-10T11:30:36.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7ff5232f1640 1 -- start start 2024-09-10T11:30:36.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7f78d4a5e640 1 -- start start 2024-09-10T11:30:36.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.151+0000 7f412c9f5640 1 -- start start 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f53a5311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a0075210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78d4a5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 0x7f78d01673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78d4a5e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78d00b8280 0x7f78d00b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78d4a5e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78d00d4730 con 0x7f78d00b4150 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78d4a5e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f78d0163640 con 0x7f78d00b8280 2024-09-10T11:30:36.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78d4a5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78d006dbf0 con 0x7f78d006d4f0 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f53a5311640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53a00cd230 0x7f53a00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f53a5311640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a00b2790 con 0x7f53a015b410 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f53a5311640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53a00b2a70 con 0x7f53a00cd230 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f53a5311640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53a00ae7a0 con 0x7f53a0073c40 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f539effd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78ced9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78d00b8280 0x7f78d00b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f539f7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f53a015b410 0x7f53a00b0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59116/0 (socket says 172.21.15.99:59116) 2024-09-10T11:30:36.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78ced9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78d00b8280 0x7f78d00b86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35214/0 (socket says 172.21.15.99:35214) 2024-09-10T11:30:36.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f78ced9d640 1 -- 172.21.15.99:0/241187046 learned_addr learned my addr 172.21.15.99:0/241187046 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f539f7fe640 1 -- 172.21.15.99:0/20522254 learned_addr learned my addr 172.21.15.99:0/20522254 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f539e7fc640 1 --2- 172.21.15.99:0/20522254 >> v2:172.21.15.196:3300/0 conn(0x7f53a00cd230 0x7f53a00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.152+0000 7f412c9f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 0x7f4120053f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f78c5d9b640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 0x7f78d01673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f87d3eba640 1 Processor -- start 2024-09-10T11:30:36.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f78ced9d640 1 -- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 msgr2=0x7f78d01673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f2de48ac640 1 Processor -- start 2024-09-10T11:30:36.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412c9f5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f412004eb50 0x7f412001d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f4126d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 0x7f4120053f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7fd523fff640 1 Processor -- start 2024-09-10T11:30:36.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412659c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f41200e7110 0x7f4120005e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59124/0 (socket says 172.21.15.99:59124) 2024-09-10T11:30:36.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539dffb640 1 -- 172.21.15.99:0/20522254 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382258335 0 0) 0x7f53a00b2790 con 0x7f53a015b410 2024-09-10T11:30:36.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f4126d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 0x7f4120053f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56782/0 (socket says 172.21.15.99:56782) 2024-09-10T11:30:36.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539dffb640 1 -- 172.21.15.99:0/20522254 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5388003520 con 0x7f53a015b410 2024-09-10T11:30:36.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412659c640 1 -- 172.21.15.99:0/4261663145 learned_addr learned my addr 172.21.15.99:0/4261663145 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539e7fc640 1 -- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 msgr2=0x7f53a0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412759e640 1 --2- 172.21.15.99:0/4261663145 >> v2:172.21.15.196:3300/0 conn(0x7f412004eb50 0x7f412001d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539e7fc640 1 --2- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4120052320 con 0x7f41200e7110 2024-09-10T11:30:36.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7ff5232f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c0733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539e7fc640 1 -- 172.21.15.99:0/20522254 >> v1:172.21.15.196:6789/0 conn(0x7f53a015b410 legacy=0x7f53a00b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f41200520d0 con 0x7f412004eb50 2024-09-10T11:30:36.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4120006c70 con 0x7f4120005a50 2024-09-10T11:30:36.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f539e7fc640 1 -- 172.21.15.99:0/20522254 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a00ae150 con 0x7f53a00cd230 2024-09-10T11:30:36.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7ff5232f1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff51c0d0e40 0x7ff51c0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52088c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff51c0b5b90 0x7ff51c06f1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59128/0 (socket says 172.21.15.99:59128) 2024-09-10T11:30:36.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f4125d9b640 1 -- 172.21.15.99:0/4261663145 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420833146 0 0) 0x7f4120052320 con 0x7f41200e7110 2024-09-10T11:30:36.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f87d3eba640 1 -- start start 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff5232f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff51c145790 con 0x7ff51c0b5b90 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff5232f1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff51c0b6510 con 0x7ff51c0d0e40 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f4125d9b640 1 -- 172.21.15.99:0/4261663145 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4110003520 con 0x7f41200e7110 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f412759e640 1 -- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 msgr2=0x7f4120053f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f412759e640 1 --2- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 0x7f4120053f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f412759e640 1 -- 172.21.15.99:0/4261663145 >> v1:172.21.15.196:6789/0 conn(0x7f41200e7110 legacy=0x7f4120005e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f412759e640 1 -- 172.21.15.99:0/4261663145 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4120006f50 con 0x7f412004eb50 2024-09-10T11:30:36.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff5232f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c0ce9f0 con 0x7ff51c1453b0 2024-09-10T11:30:36.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f78ced9d640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 0x7f78d01673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f78ced9d640 1 -- 172.21.15.99:0/241187046 >> v1:172.21.15.196:6789/0 conn(0x7f78d00b4150 legacy=0x7f78d0166cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f78ced9d640 1 -- 172.21.15.99:0/241187046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d006d980 con 0x7f78d00b8280 2024-09-10T11:30:36.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52088c640 1 -- 172.21.15.99:0/3455300454 learned_addr learned my addr 172.21.15.99:0/3455300454 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f78c5d9b640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 0x7f78d01673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:36.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52188e640 1 --2- 172.21.15.99:0/3455300454 >> v2:172.21.15.99:3300/0 conn(0x7ff51c0d0e40 0x7ff51c0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f78ced9d640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00b86d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009e50 tx=0x7f78c802a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a933d5446f2b74f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7f2de48ac640 1 -- start start 2024-09-10T11:30:36.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52108d640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c0733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52188e640 1 -- 172.21.15.99:0/3455300454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 msgr2=0x7ff51c0733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f78cdd9b640 1 -- 172.21.15.99:0/241187046 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8036070 con 0x7f78d00b8280 2024-09-10T11:30:36.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52188e640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c0733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52188e640 1 -- 172.21.15.99:0/3455300454 >> v1:172.21.15.196:6789/0 conn(0x7ff51c0b5b90 legacy=0x7ff51c06f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7f78cdd9b640 1 -- 172.21.15.99:0/241187046 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8031600 con 0x7f78d00b8280 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.153+0000 7fd523fff640 1 -- start start 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fd523fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd51c036460 0x7fd51c033880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fd523fff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c037370 con 0x7fd51c055620 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fd523fff640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51c047ae0 con 0x7fd51c00abc0 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fd523fff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd51c006a80 con 0x7fd51c036460 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff52188e640 1 -- 172.21.15.99:0/3455300454 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c0d15d0 con 0x7ff51c0d0e40 2024-09-10T11:30:36.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fe0ea10a640 1 Processor -- start 2024-09-10T11:30:36.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7fe0ea10a640 1 -- start start 2024-09-10T11:30:36.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.154+0000 7ff513fff640 1 -- 172.21.15.99:0/3455300454 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003625329 0 0) 0x7ff51c145790 con 0x7ff51c0b5b90 2024-09-10T11:30:36.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2de48ac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2de0163140 0x7f2de0163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f539e7fc640 1 --2- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a00cd230 0x7f53a00b4040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53900097a0 tx=0x7f539002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a48f4c0ae5641e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f412759e640 1 --2- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f412004eb50 0x7f412001d520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41280518b0 tx=0x7f41280a6910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba5574bd977e073 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fd5237fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd51c00abc0 0x7fd51c00afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59144/0 (socket says 172.21.15.99:59144) 2024-09-10T11:30:36.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fd5237fe640 1 -- 172.21.15.99:0/2770971065 learned_addr learned my addr 172.21.15.99:0/2770971065 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fd5227fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd51c055620 0x7fd51c036840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44886/0 (socket says 172.21.15.99:44886) 2024-09-10T11:30:36.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2de48ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de00d0740 0x7f2de00d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2dde59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2de0163140 0x7f2de0163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2dde59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2de0163140 0x7f2de0163520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54288/0 (socket says 172.21.15.99:54288) 2024-09-10T11:30:36.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f539dffb640 1 -- 172.21.15.99:0/20522254 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390037070 con 0x7f53a00cd230 2024-09-10T11:30:36.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2de48ac640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de00b5d30 con 0x7f2de00714a0 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2dde59c640 1 -- 172.21.15.99:0/2093445452 learned_addr learned my addr 172.21.15.99:0/2093445452 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2de48ac640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2de00d10d0 con 0x7f2de0163140 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2de0071980 con 0x7f2de00d0740 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f539dffb640 1 -- 172.21.15.99:0/20522254 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539002ae40 con 0x7f53a00cd230 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f539dffb640 1 -- 172.21.15.99:0/20522254 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390032660 con 0x7f53a00cd230 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0ea10a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe0e40b49f0 0x7fe0e415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0ea10a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e40b3c30 con 0x7fe0e4071330 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0ea10a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e40b3f10 con 0x7fe0e406f420 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0ea10a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e40b57e0 con 0x7fe0e40b49f0 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f4125d9b640 1 -- 172.21.15.99:0/4261663145 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280b0020 con 0x7f412004eb50 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f4125d9b640 1 -- 172.21.15.99:0/4261663145 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41280ac910 con 0x7f412004eb50 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f4125d9b640 1 -- 172.21.15.99:0/4261663145 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280acd80 con 0x7f412004eb50 2024-09-10T11:30:36.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f2dded9d640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de00d0740 0x7f2de00d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f87d3eba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 0x7f87cc0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f87d3eba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87cc167300 0x7f87cc06d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0e3fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe0e4071330 0x7fe0e4071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44902/0 (socket says 172.21.15.99:44902) 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2283453402 learned_addr learned my addr 172.21.15.99:0/2283453402 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f78d4a5e640 1 -- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 msgr2=0x7f78d00b86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7f78d4a5e640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00b86d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009e50 tx=0x7f78c802a630 comp rx=0 tx=0).stop 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f2dde59c640 1 -- 172.21.15.99:0/2093445452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de00d0740 msgr2=0x7f2de00d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f2dde59c640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de00d0740 0x7f2de00d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f2ddd59a640 1 -- 172.21.15.99:0/2093445452 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2732020929 0 0) 0x7f2de00b5d30 con 0x7f2de00714a0 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f2dde59c640 1 -- 172.21.15.99:0/2093445452 >> v1:172.21.15.196:6789/0 conn(0x7f2de00714a0 legacy=0x7f2de00b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f2dde59c640 1 -- 172.21.15.99:0/2093445452 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de0071c60 con 0x7f2de0163140 2024-09-10T11:30:36.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873925138 0 0) 0x7fe0e40b3c30 con 0x7fe0e4071330 2024-09-10T11:30:36.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 -- 172.21.15.99:0/241187046 shutdown_connections 2024-09-10T11:30:36.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00b86d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 --2- 172.21.15.99:0/241187046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d006d4f0 0x7f78d01673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 -- 172.21.15.99:0/241187046 >> 172.21.15.99:0/241187046 conn(0x7f78d00d4e10 msgr2=0x7f78d00d5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0cc003520 con 0x7fe0e4071330 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 -- 172.21.15.99:0/241187046 shutdown_connections 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196889825 0 0) 0x7fd51c037370 con 0x7fd51c055620 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.155+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd50c003520 con 0x7fd51c055620 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4005388102 0 0) 0x7fd50c003520 con 0x7fd51c055620 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 >> v2:172.21.15.196:3300/0 conn(0x7fd51c036460 msgr2=0x7fd51c033880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 --2- 172.21.15.99:0/2770971065 >> v2:172.21.15.196:3300/0 conn(0x7fd51c036460 0x7fd51c033880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 >> v1:172.21.15.196:6789/0 conn(0x7fd51c00abc0 legacy=0x7fd51c00afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d1c56640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 0x7f87cc0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c047dc0 con 0x7fd51c055620 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fd518001670 con 0x7fd51c055620 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 >> v1:172.21.15.99:6789/0 conn(0x7fd51c055620 legacy=0x7fd51c036840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd50c0039b0 0x7fd50c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 0x7fd50c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c0053c0 0x7fd50c00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c006a80 con 0x7fd50c004800 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd50c01d2f0 con 0x7fd50c0053c0 2024-09-10T11:30:36.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd50c01d4f0 con 0x7fd50c0039b0 2024-09-10T11:30:36.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f78d4a5e640 1 -- 172.21.15.99:0/241187046 wait complete. 2024-09-10T11:30:36.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 -- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a00cd230 msgr2=0x7f53a00b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d2457640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87cc167300 0x7f87cc06d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d2457640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87cc167300 0x7f87cc06d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54310/0 (socket says 172.21.15.99:54310) 2024-09-10T11:30:36.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d2457640 1 -- 172.21.15.99:0/2578658653 learned_addr learned my addr 172.21.15.99:0/2578658653 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 --2- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a00cd230 0x7f53a00b4040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f53900097a0 tx=0x7f539002a0f0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd5227fc640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 0x7fd50c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 -- 172.21.15.99:0/20522254 shutdown_connections 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 --2- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a00cd230 0x7f53a00b4040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 --2- 172.21.15.99:0/20522254 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f53a5311640 1 -- 172.21.15.99:0/20522254 >> 172.21.15.99:0/20522254 conn(0x7f53a00730c0 msgr2=0x7f53a00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e2ffd640 1 --2- 172.21.15.99:0/2283453402 >> v2:172.21.15.196:3300/0 conn(0x7fe0e40b49f0 0x7fe0e415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2484733196 0 0) 0x7fe0e40b3f10 con 0x7fe0e406f420 2024-09-10T11:30:36.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0e40b3c30 con 0x7fe0e406f420 2024-09-10T11:30:36.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd5237fe640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c0053c0 0x7fd50c00ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fd522ffd640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd50c0039b0 0x7fd50c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87cc1634d0 con 0x7f87cc167790 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f87cc0b7860 con 0x7f87cc167300 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f87cc06cda0 con 0x7f87cc1630f0 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f53a5311640 1 -- 172.21.15.99:0/20522254 shutdown_connections 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 335112120 0 0) 0x7fe0cc003520 con 0x7fe0e4071330 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 >> v2:172.21.15.196:3300/0 conn(0x7fe0e40b49f0 msgr2=0x7fe0e415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 --2- 172.21.15.99:0/2283453402 >> v2:172.21.15.196:3300/0 conn(0x7fe0e40b49f0 0x7fe0e415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.156+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 >> v1:172.21.15.196:6789/0 conn(0x7fe0e406f420 legacy=0x7fe0e40b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e40b41f0 con 0x7fe0e4071330 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5227fc640 1 -- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd50c0039b0 msgr2=0x7fd50c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5227fc640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd50c0039b0 0x7fd50c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5227fc640 1 -- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c0053c0 msgr2=0x7fd50c00ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5227fc640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c0053c0 0x7fd50c00ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5227fc640 1 -- 172.21.15.99:0/2770971065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c047dc0 con 0x7fd50c004800 2024-09-10T11:30:36.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d0c54640 1 -- 172.21.15.99:0/2578658653 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 160649712 0 0) 0x7f87cc1634d0 con 0x7f87cc167790 2024-09-10T11:30:36.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5237fe640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c0053c0 0x7fd50c00ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7fd5237fe640 1 -- 172.21.15.99:0/2770971065 reap_dead start 2024-09-10T11:30:36.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f2dde59c640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de0163140 0x7f2de0163520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009ad0 tx=0x7f2dd402a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96bb7a1201aae55c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7ff52188e640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c0b5fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff51400bfa0 tx=0x7ff51402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e28ef780a60f775 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f412004eb50 msgr2=0x7f412001d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 --2- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f412004eb50 0x7f412001d520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f41280518b0 tx=0x7f41280a6910 comp rx=0 tx=0).stop 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 shutdown_connections 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 --2- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f412004eb50 0x7f412001d520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 --2- 172.21.15.99:0/4261663145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4120005a50 0x7f4120053f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 >> 172.21.15.99:0/4261663145 conn(0x7f4120036080 msgr2=0x7f4120036490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d2457640 1 -- 172.21.15.99:0/2578658653 >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 msgr2=0x7f87cc0b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d2457640 1 --2- 172.21.15.99:0/2578658653 >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 0x7f87cc0b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d2457640 1 -- 172.21.15.99:0/2578658653 >> v1:172.21.15.99:6789/0 conn(0x7f87cc167790 legacy=0x7f87cc166bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d2457640 1 -- 172.21.15.99:0/2578658653 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc06d080 con 0x7f87cc167300 2024-09-10T11:30:36.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 shutdown_connections 2024-09-10T11:30:36.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f87d1c56640 1 --2- 172.21.15.99:0/2578658653 >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 0x7f87cc0b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:36.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f412c9f5640 1 -- 172.21.15.99:0/4261663145 wait complete. 2024-09-10T11:30:36.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7ff513fff640 1 -- 172.21.15.99:0/3455300454 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514039070 con 0x7ff51c0d0e40 2024-09-10T11:30:36.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f2ddd59a640 1 -- 172.21.15.99:0/2093445452 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4036070 con 0x7f2de0163140 2024-09-10T11:30:36.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7ff513fff640 1 -- 172.21.15.99:0/3455300454 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514009540 con 0x7ff51c0d0e40 2024-09-10T11:30:36.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7ff513fff640 1 -- 172.21.15.99:0/3455300454 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514009910 con 0x7ff51c0d0e40 2024-09-10T11:30:36.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.157+0000 7f53a5311640 1 -- 172.21.15.99:0/20522254 wait complete. 2024-09-10T11:30:36.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fe0d4001670 con 0x7fe0e4071330 2024-09-10T11:30:36.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 >> v1:172.21.15.99:6789/0 conn(0x7fe0e4071330 legacy=0x7fe0e4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 0x7fe0cc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc004800 0x7fe0cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0cc0051c0 0x7fe0cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e40b57e0 con 0x7fe0cc004800 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc00c0f0 con 0x7fe0cc003bb0 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc00c2f0 con 0x7fe0cc0051c0 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7f2ddd59a640 1 -- 172.21.15.99:0/2093445452 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd4031600 con 0x7f2de0163140 2024-09-10T11:30:36.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7f78d4a5e640 1 Processor -- start 2024-09-10T11:30:36.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7f2ddd59a640 1 -- 172.21.15.99:0/2093445452 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40319d0 con 0x7f2de0163140 2024-09-10T11:30:36.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7f78d4a5e640 1 -- start start 2024-09-10T11:30:36.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fd5227fc640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 0x7fd50c004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c1e0 tx=0x7fd50c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de9dfdfd346692 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7f87d2457640 1 --2- 172.21.15.99:0/2578658653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc167300 0x7f87cc06d880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87cc072570 tx=0x7f87c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6e0304b958cc3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518007cc0 con 0x7fd50c004800 2024-09-10T11:30:36.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de0163140 msgr2=0x7f2de0163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de0163140 0x7f2de0163520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009ad0 tx=0x7f2dd402a4c0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f53a5311640 1 Processor -- start 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518007270 con 0x7fd50c004800 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fd521ffb640 1 -- 172.21.15.99:0/2770971065 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518007640 con 0x7fd50c004800 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f87d0c54640 1 -- 172.21.15.99:0/2578658653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8037070 con 0x7f87cc167300 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f87d0c54640 1 -- 172.21.15.99:0/2578658653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c8032440 con 0x7f87cc167300 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f87d0c54640 1 -- 172.21.15.99:0/2578658653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8032810 con 0x7f87cc167300 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 -- 172.21.15.99:0/3455300454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 msgr2=0x7ff51c0b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c0b5fd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff51400bfa0 tx=0x7ff51402c880 comp rx=0 tx=0).stop 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e3fff640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0cc0051c0 0x7fe0cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e2ffd640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc004800 0x7fe0cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.158+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 0x7fe0cc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e37fe640 1 -- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0cc0051c0 msgr2=0x7fe0cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0cc0051c0 0x7fe0cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e37fe640 1 -- 172.21.15.99:0/2283453402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc004800 msgr2=0x7fe0cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc004800 0x7fe0cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e37fe640 1 -- 172.21.15.99:0/2283453402 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc003520 con 0x7fe0cc003bb0 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2283453402 reap_dead start 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f412c9f5640 1 Processor -- start 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 shutdown_connections 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de00d0740 0x7f2de00d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 --2- 172.21.15.99:0/2093445452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de0163140 0x7f2de0163520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 >> 172.21.15.99:0/2093445452 conn(0x7f2de006bc10 msgr2=0x7f2de006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 -- 172.21.15.99:0/3455300454 shutdown_connections 2024-09-10T11:30:36.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c0b5fd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 --2- 172.21.15.99:0/3455300454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c0733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f78d4a5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d006d4f0 0x7f78d00cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7ff5232f1640 1 -- 172.21.15.99:0/3455300454 >> 172.21.15.99:0/3455300454 conn(0x7ff51c072680 msgr2=0x7ff51c072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f412c9f5640 1 -- start start 2024-09-10T11:30:36.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7ff5232f1640 1 -- 172.21.15.99:0/3455300454 shutdown_connections 2024-09-10T11:30:36.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7ff5232f1640 1 -- 172.21.15.99:0/3455300454 wait complete. 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78d4a5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d00b4150 0x7f78d00cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ce59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d006d4f0 0x7f78d00cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ce59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d006d4f0 0x7f78d00cfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54336/0 (socket says 172.21.15.99:54336) 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78d4a5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ce59c640 1 -- 172.21.15.99:0/195179763 learned_addr learned my addr 172.21.15.99:0/195179763 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78d00b7d60 con 0x7f78d006d4f0 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78d006e1b0 con 0x7f78d00b8280 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78d00b7a60 con 0x7f78d00b4150 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ced9d640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc167300 msgr2=0x7f87cc06d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f87d3eba640 1 --2- 172.21.15.99:0/2578658653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc167300 0x7f87cc06d880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f87cc072570 tx=0x7f87c802a270 comp rx=0 tx=0).stop 2024-09-10T11:30:36.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 -- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 msgr2=0x7fd50c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 0x7fd50c004c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c1e0 tx=0x7fd50c005830 comp rx=0 tx=0).stop 2024-09-10T11:30:36.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 -- 172.21.15.99:0/2770971065 shutdown_connections 2024-09-10T11:30:36.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 --2- 172.21.15.99:0/2770971065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c004800 0x7fd50c004c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.159+0000 7f53a5311640 1 -- start start 2024-09-10T11:30:36.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f53a5311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a006f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 -- 172.21.15.99:0/2770971065 >> 172.21.15.99:0/2770971065 conn(0x7fd51c01dd70 msgr2=0x7fd51c0549e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 -- 172.21.15.99:0/2770971065 shutdown_connections 2024-09-10T11:30:36.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7fd523fff640 1 -- 172.21.15.99:0/2770971065 wait complete. 2024-09-10T11:30:36.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f53a5311640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a00cd230 0x7f53a0150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78c5d9b640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d00b4150 0x7f78d00cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f412c9f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4120005a50 0x7f4120039c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f412c9f5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412004eb50 0x7f4120048030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 shutdown_connections 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ced9d640 1 -- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d00b4150 msgr2=0x7f78d00cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f78ced9d640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d00b4150 0x7f78d00cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78ced9d640 1 -- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d006d4f0 msgr2=0x7f78d00cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78ced9d640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78d006d4f0 0x7f78d00cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78ced9d640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d00cdef0 con 0x7f78d00b8280 2024-09-10T11:30:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.160+0000 7f412c9f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 0x7f41200dd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412c9f5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4120047b40 con 0x7f412004eb50 2024-09-10T11:30:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412c9f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4120053a00 con 0x7f4120005a50 2024-09-10T11:30:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412c9f5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4120047cc0 con 0x7f41200e7110 2024-09-10T11:30:36.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 shutdown_connections 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f87d3eba640 1 --2- 172.21.15.99:0/2578658653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc167300 0x7f87cc06d880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f87d3eba640 1 --2- 172.21.15.99:0/2578658653 >> v2:172.21.15.196:3300/0 conn(0x7f87cc1630f0 0x7f87cc0b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 >> 172.21.15.99:0/2578658653 conn(0x7f87cc09ba30 msgr2=0x7f87cc09be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 0x7f41200dd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f4126d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4120005a50 0x7f4120039c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 0x7f41200dd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56828/0 (socket says 172.21.15.99:56828) 2024-09-10T11:30:36.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 -- 172.21.15.99:0/3167938572 learned_addr learned my addr 172.21.15.99:0/3167938572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412659c640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412004eb50 0x7f4120048030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f539effd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78ced9d640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00c76b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78d008d530 tx=0x7f78c8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f539effd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a006f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56812/0 (socket says 172.21.15.99:56812) 2024-09-10T11:30:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f2de48ac640 1 -- 172.21.15.99:0/2093445452 wait complete. 2024-09-10T11:30:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f539effd640 1 -- 172.21.15.99:0/3414999339 learned_addr learned my addr 172.21.15.99:0/3414999339 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 0x7fe0cc003f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc002010 tx=0x7fe0cc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8789ba02a80162cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f53a5311640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a015b410 0x7f53a00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53a00758d0 con 0x7f53a00cd230 2024-09-10T11:30:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0075750 con 0x7f53a015b410 2024-09-10T11:30:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53a00b2340 con 0x7f53a0073c40 2024-09-10T11:30:36.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f539e7fc640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a00cd230 0x7f53a0150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8041020 con 0x7f78d00b8280 2024-09-10T11:30:36.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f539f7fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a015b410 0x7f53a00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c8008c30 con 0x7f78d00b8280 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c8036040 con 0x7f78d00b8280 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d0156390 con 0x7f78d00b8280 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d0155450 con 0x7f78d00b8280 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 -- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 msgr2=0x7f53a006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53a0073c40 0x7f53a006f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 -- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a015b410 msgr2=0x7f53a00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a015b410 0x7f53a00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a014fbd0 con 0x7f53a00cd230 2024-09-10T11:30:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539f7fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a015b410 0x7f53a00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7ff5232f1640 1 Processor -- start 2024-09-10T11:30:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7ff5232f1640 1 -- start start 2024-09-10T11:30:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f539e7fc640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a00cd230 0x7f53a0150990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5390031000 tx=0x7f53900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390042020 con 0x7f53a00cd230 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc036070 con 0x7fe0cc003bb0 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc031600 con 0x7fe0cc003bb0 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7fd523fff640 1 Processor -- start 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7fd523fff640 1 -- start start 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 -- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4120005a50 msgr2=0x7f4120039c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4120005a50 0x7f4120039c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.161+0000 7f412759e640 1 -- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412004eb50 msgr2=0x7f4120048030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a014feb0 con 0x7f53a00cd230 2024-09-10T11:30:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.162+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 shutdown_connections 2024-09-10T11:30:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f2de48ac640 1 Processor -- start 2024-09-10T11:30:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f2de48ac640 1 -- start start 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a0150ed0 con 0x7f53a00cd230 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0e27fc640 1 -- 172.21.15.99:0/2283453402 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc001be0 con 0x7fe0cc003bb0 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 msgr2=0x7fe0cc003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0ea10a640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 0x7fe0cc003f90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc002010 tx=0x7fe0cc005830 comp rx=0 tx=0).stop 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2283453402 shutdown_connections 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0ea10a640 1 --2- 172.21.15.99:0/2283453402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0cc003bb0 0x7fe0cc003f90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2283453402 >> 172.21.15.99:0/2283453402 conn(0x7fe0e40b23d0 msgr2=0x7fe0e40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f87d3eba640 1 -- 172.21.15.99:0/2578658653 wait complete. 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78d0156140 con 0x7f78d00b8280 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7ff5232f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 0x7ff51c13e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff5232f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff5232f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c13ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff52108d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 0x7ff51c13e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff52108d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 0x7ff51c13e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35312/0 (socket says 172.21.15.99:35312) 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff52108d640 1 -- 172.21.15.99:0/1680015747 learned_addr learned my addr 172.21.15.99:0/1680015747 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c0ce430 con 0x7ff51c0d0e40 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c073910 con 0x7ff51c0b5b90 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c0ce710 con 0x7ff51c1453b0 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 0x7f2de01ed4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 0x7f2de01fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de0163140 0x7f2de01eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dde59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 0x7f2de01ed4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dde59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 0x7f2de01ed4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35324/0 (socket says 172.21.15.99:35324) 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dddd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 0x7f2de01fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dddd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 0x7f2de01fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54360/0 (socket says 172.21.15.99:54360) 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f539003d690 con 0x7f53a00cd230 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390037040 con 0x7f53a00cd230 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7fd523fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c004800 0x7fd51c0d9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd523fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 0x7fd51c0d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd523fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0dfa30 0x7fd51c0dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd5227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 0x7fd51c0d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd5227fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 0x7fd51c0d55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54358/0 (socket says 172.21.15.99:54358) 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd5227fc640 1 -- 172.21.15.99:0/205349677 learned_addr learned my addr 172.21.15.99:0/205349677 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c033880 con 0x7fd51c0def00 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c007120 con 0x7fd50c004800 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd51c033a00 con 0x7fd51c0dfa30 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7fd5237fe640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0dfa30 0x7fd51c0dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f53857fa640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5380001030 con 0x7f53a00cd230 2024-09-10T11:30:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0072500 con 0x7f2de00d0740 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2de0072220 con 0x7f2de00714a0 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2de48ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2de0072a20 con 0x7f2de0163140 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dde59c640 1 -- 172.21.15.99:0/1980210000 learned_addr learned my addr 172.21.15.99:0/1980210000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f87d3eba640 1 Processor -- start 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f87d3eba640 1 -- start start 2024-09-10T11:30:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7ff52188e640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c13ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78c8044020 con 0x7f78d00b8280 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f78c77fe640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78b406be30 0x7f78b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78c80ac060 con 0x7f78d00b8280 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78c77fe640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f78b40740b0 0x7f78b40764d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78b4076b80 con 0x7f78b40740b0 2024-09-10T11:30:36.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f78c80ac450 con 0x7f78d00b8280 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f412759e640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412004eb50 0x7f4120048030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 -- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0dfa30 msgr2=0x7fd51c0dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0dfa30 0x7fd51c0dfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 -- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c004800 msgr2=0x7fd51c0d9320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd50c004800 0x7fd51c0d9320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c048e50 con 0x7fd51c0def00 2024-09-10T11:30:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7ff52188e640 1 -- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 msgr2=0x7ff51c13e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7ff52188e640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 0x7ff51c13e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7ff52188e640 1 -- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 msgr2=0x7ff51c13fb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:30:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f412759e640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412003a2b0 con 0x7f41200e7110 2024-09-10T11:30:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78c5d9b640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f78b40740b0 0x7f78b40764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f2dded9d640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de0163140 0x7f2de01eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dde59c640 1 -- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de0163140 msgr2=0x7f2de01eb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dde59c640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2de0163140 0x7f2de01eb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dde59c640 1 -- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 msgr2=0x7f2de01fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.163+0000 7f412659c640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f412004eb50 0x7f4120048030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f412759e640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 0x7f41200dd670 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4128069470 tx=0x7f412806bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280bc070 con 0x7f41200e7110 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41280b5c10 con 0x7f41200e7110 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280b0040 con 0x7f41200e7110 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f412c9f5640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412000a150 con 0x7f41200e7110 2024-09-10T11:30:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f412c9f5640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41200ddbb0 con 0x7f41200e7110 2024-09-10T11:30:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd5227fc640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 0x7fd51c0d55a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c1e0 tx=0x7fd518001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.164+0000 7f4126d9d640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4120005a50 0x7f4120039c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dde59c640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 0x7f2de01fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dde59c640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de01fd070 con 0x7f2de00714a0 2024-09-10T11:30:36.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f78ce59c640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78b406be30 0x7f78b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1630f0 0x7f87cc1ee470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f2dddd9b640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2de00d0740 0x7f2de01fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518001640 con 0x7fd51c0def00 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd518040070 con 0x7fd51c0def00 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 0x7f87cc0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc167790 0x7f87cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0b74a0 con 0x7f87cc1630f0 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87cc0b7b40 con 0x7f87cc167790 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7f87d3eba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87cc0b7620 con 0x7f87cc167300 2024-09-10T11:30:36.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7ff52088c640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518007620 con 0x7fd51c0def00 2024-09-10T11:30:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c049110 con 0x7fd51c0def00 2024-09-10T11:30:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7ff52188e640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff51c0d0e40 0x7ff51c13fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7ff52188e640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c13d600 con 0x7ff51c1453b0 2024-09-10T11:30:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7ff52108d640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff51c0b5b90 0x7ff51c13e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c04ae30 con 0x7fd51c0def00 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f2dde59c640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 0x7f2de01ed4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2dd402a9e0 tx=0x7f2dd40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.165+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2283453402 shutdown_connections 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 0x7f87cc0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d2457640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc167790 0x7f87cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 0x7f87cc0b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56874/0 (socket says 172.21.15.99:56874) 2024-09-10T11:30:36.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 -- 172.21.15.99:0/3859980434 learned_addr learned my addr 172.21.15.99:0/3859980434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f412c9f5640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41200475f0 con 0x7f41200e7110 2024-09-10T11:30:36.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7ff52188e640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c13ceb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff514033000 tx=0x7ff514004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2283453402 wait complete. 2024-09-10T11:30:36.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f78c5d9b640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f78b40740b0 0x7f78b40764d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1c56640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1630f0 0x7f87cc1ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 -- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc167790 msgr2=0x7f87cc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87cc167790 0x7f87cc15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f87d1455640 1 -- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1630f0 msgr2=0x7f87cc1ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f87d1455640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87cc1630f0 0x7f87cc1ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41280c7060 con 0x7f41200e7110 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4040070 con 0x7f2de00714a0 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd403b870 con 0x7f2de00714a0 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.166+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4036040 con 0x7f2de00714a0 2024-09-10T11:30:36.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f87d1455640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87cc1ef3a0 con 0x7f87cc167300 2024-09-10T11:30:36.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de01fd350 con 0x7f2de00714a0 2024-09-10T11:30:36.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de015c1a0 con 0x7f2de00714a0 2024-09-10T11:30:36.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f78ce59c640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78b406be30 0x7f78b406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78c0007fd0 tx=0x7f78c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5390045020 con 0x7f53a00cd230 2024-09-10T11:30:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f53877fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f538c06bf00 0x7f538c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== osd.4 v2:172.21.15.196:6800/3885371275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f78b4076b80 con 0x7f78b40740b0 2024-09-10T11:30:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd504001030 con 0x7fd51c0def00 2024-09-10T11:30:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f410f7fe640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f411406bf00 0x7f411406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4128126f70 con 0x7f41200e7110 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f410f7fe640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7f41140740a0 0x7f41140764c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41280c1ec0 con 0x7f41140740a0 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41280f1240 con 0x7f41200e7110 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.167+0000 7f87d1455640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 0x7f87cc0b25a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c7b0 tx=0x7f87c400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4003da0 con 0x7f87cc167300 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c4005070 con 0x7f87cc167300 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4005420 con 0x7f87cc167300 2024-09-10T11:30:36.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87cc1eeb00 con 0x7f87cc167300 2024-09-10T11:30:36.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f53900aba70 con 0x7f53a00cd230 2024-09-10T11:30:36.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87cc1ef0c0 con 0x7f87cc167300 2024-09-10T11:30:36.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f53877fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f538c074120 0x7f538c076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd51803a070 con 0x7fd51c0def00 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7fe0ea10a640 1 Processor -- start 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f539effd640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f538c06bf00 0x7f538c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514039050 con 0x7ff51c1453b0 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514045070 con 0x7ff51c1453b0 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514040450 con 0x7ff51c1453b0 2024-09-10T11:30:36.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c13d8e0 con 0x7ff51c1453b0 2024-09-10T11:30:36.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f539f7fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f538c074120 0x7f538c076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7fd50bfff640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd51006bf30 0x7fd51006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5180aa2a0 con 0x7fd51c0def00 2024-09-10T11:30:36.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c13dea0 con 0x7ff51c1453b0 2024-09-10T11:30:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7fd50bfff640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7fd510074130 0x7fd510076550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd518007270 con 0x7fd510074130 2024-09-10T11:30:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f539f7fe640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f538c074120 0x7f538c076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5390001d30 con 0x7f538c074120 2024-09-10T11:30:36.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f539effd640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f538c06bf00 0x7f538c06e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f4126d9d640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f411406bf00 0x7f411406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.168+0000 7f412659c640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7f41140740a0 0x7f41140764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f412659c640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7f41140740a0 0x7f41140764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f4126d9d640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f411406bf00 0x7f411406e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f411c009f20 tx=0x7f411c009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f53900abe60 con 0x7f53a00cd230 2024-09-10T11:30:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87cc15f380 con 0x7f87cc167300 2024-09-10T11:30:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dd4043070 con 0x7f2de00714a0 2024-09-10T11:30:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2de00d0b90 con 0x7f2de00714a0 2024-09-10T11:30:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f2dceffd640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2dc006beb0 0x7f2dc006e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7fe0ea10a640 1 -- start start 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== osd.2 v2:172.21.15.99:6817/1090025270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f5390001d30 con 0x7f538c074120 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.169+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd5180b1050 con 0x7fd51c0def00 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7fd522ffd640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd51006bf30 0x7fd51006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7fd522ffd640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd51006bf30 0x7fd51006e3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b700 tx=0x7fd514008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7fd5237fe640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7fd510074130 0x7fd510076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== osd.6 v2:172.21.15.196:6806/4190591731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f41280c1ec0 con 0x7f41140740a0 2024-09-10T11:30:36.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87c40055c0 con 0x7f87cc167300 2024-09-10T11:30:36.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f2dddd9b640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2dc006beb0 0x7f2dc006e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f2dddd9b640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2dc006beb0 0x7f2dc006e370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2de0162180 tx=0x7f2dd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f87c27fc640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f87b006be60 0x7f87b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f87c408ac90 con 0x7f87cc167300 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.170+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff51c070350 con 0x7ff51c1453b0 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7fe0ea10a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc003bb0 0x7fe0e4143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd4077a00 con 0x7f2de00714a0 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f2dceffd640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2dc0074cd0 0x7f2dc0077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dc0077840 con 0x7f2dc0074cd0 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2dd4076390 con 0x7f2de00714a0 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7fe0ea10a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0e41448e0 0x7fe0e414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0ea10a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 0x7fe0e4144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0ea10a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e40b5460 con 0x7fe0cc003bb0 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0ea10a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e40b52e0 con 0x7fe0e4149bf0 2024-09-10T11:30:36.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0ea10a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e40b55e0 con 0x7fe0e41448e0 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f87c27fc640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f87b0074000 0x7f87b0076420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87b0076ad0 con 0x7f87b0074000 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f87c4055000 con 0x7f87cc167300 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f87d1c56640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f87b006be60 0x7f87b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f2dded9d640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2dc0074cd0 0x7f2dc0077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 0x7fe0e4144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 0x7fe0e4144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35360/0 (socket says 172.21.15.99:35360) 2024-09-10T11:30:36.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2724595758 learned_addr learned my addr 172.21.15.99:0/2724595758 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.171+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff514009300 con 0x7ff51c1453b0 2024-09-10T11:30:36.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7f87d1c56640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f87b006be60 0x7f87b006e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87bc006fd0 tx=0x7f87bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2724595758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0e41448e0 msgr2=0x7fe0e414d0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0e41448e0 0x7fe0e414d0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc003bb0 msgr2=0x7fe0e4143160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0cc003bb0 0x7fe0e4143160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e4147790 con 0x7fe0e4149bf0 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7ff511ffb640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff500070310 0x7ff5000727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7ff52108d640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff500070310 0x7ff5000727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.172+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5140ac060 con 0x7ff51c1453b0 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7ff511ffb640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7ff500078530 0x7ff50007a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff51400ce60 con 0x7ff500078530 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff5140b3050 con 0x7ff51c1453b0 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7f2dded9d640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2dc0074cd0 0x7f2dc0077170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7fe0e3fff640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 0x7fe0e4144f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4008fd0 tx=0x7fe0d4008790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7f87d2457640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f87b0074000 0x7f87b0076420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d401a070 con 0x7fe0e4149bf0 2024-09-10T11:30:36.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7ff52108d640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff500070310 0x7ff5000727d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5180088b0 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d400f840 con 0x7fe0e4149bf0 2024-09-10T11:30:36.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e4147a30 con 0x7fe0e4149bf0 2024-09-10T11:30:36.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7fd5237fe640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7fd510074130 0x7fd510076550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.173+0000 7ff52088c640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7ff500078530 0x7ff50007a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== osd.1 v2:172.21.15.99:6808/2120836022 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f2dc0077840 con 0x7f2dc0074cd0 2024-09-10T11:30:36.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7f87d2457640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f87b0074000 0x7f87b0076420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400fc10 con 0x7fe0e4149bf0 2024-09-10T11:30:36.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e4147ff0 con 0x7fe0e4149bf0 2024-09-10T11:30:36.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== osd.5 v2:172.21.15.196:6824/3797029994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fd518007270 con 0x7fd510074130 2024-09-10T11:30:36.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.174+0000 7ff52088c640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7ff500078530 0x7ff50007a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.175+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0e4148230 con 0x7fe0e4149bf0 2024-09-10T11:30:36.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.175+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== osd.7 v2:172.21.15.196:6816/106140719 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f87b0076ad0 con 0x7f87b0074000 2024-09-10T11:30:36.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.175+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== osd.0 v2:172.21.15.99:6800/291490777 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7ff51400ce60 con 0x7ff500078530 2024-09-10T11:30:36.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0d401d070 con 0x7fe0e4149bf0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0d006bf30 0x7fe0d006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0d408aa90 con 0x7fe0e4149bf0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7fe0d0074150 0x7fe0d0076570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0d4006e50 con 0x7fe0d0074150 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.176+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe0d408ae80 con 0x7fe0e4149bf0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.177+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0d006bf30 0x7fe0d006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.177+0000 7fe0e2ffd640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7fe0d0074150 0x7fe0d0076570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.177+0000 7fe0e2ffd640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7fe0d0074150 0x7fe0d0076570 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.177+0000 7fe0e37fe640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0d006bf30 0x7fe0d006e3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc033000 tx=0x7fe0dc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.178+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== osd.3 v2:172.21.15.99:6816/1027928425 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fe0d4006e50 con 0x7fe0d0074150 2024-09-10T11:30:36.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.194+0000 7f78d4a5e640 1 -- 172.21.15.99:0/195179763 --> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78d00b6500 con 0x7f78b40740b0 2024-09-10T11:30:36.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.195+0000 7f78c77fe640 1 -- 172.21.15.99:0/195179763 <== osd.4 v2:172.21.15.196:6800/3885371275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78d00b6500 con 0x7f78b40740b0 2024-09-10T11:30:36.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.196+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f78b40740b0 msgr2=0x7f78b40764d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.196+0000 7f78c4d99640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f78b40740b0 0x7f78b40764d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.196+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78b406be30 msgr2=0x7f78b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.196+0000 7f78c4d99640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78b406be30 0x7f78b406e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78c0007fd0 tx=0x7f78c0009040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 msgr2=0x7f78d00c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.196+0000 7f412c9f5640 1 -- 172.21.15.99:0/3167938572 --> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f412000a760 con 0x7f41140740a0 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 --2- 172.21.15.99:0/195179763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d00b8280 0x7f78d00c76b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78d008d530 tx=0x7f78c8004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78ced9d640 1 -- 172.21.15.99:0/195179763 reap_dead start 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 shutdown_connections 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 >> 172.21.15.99:0/195179763 conn(0x7f78d00d4e10 msgr2=0x7f78d00b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f410f7fe640 1 -- 172.21.15.99:0/3167938572 <== osd.6 v2:172.21.15.196:6806/4190591731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f412000a760 con 0x7f41140740a0 2024-09-10T11:30:36.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 shutdown_connections 2024-09-10T11:30:36.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.197+0000 7f78c4d99640 1 -- 172.21.15.99:0/195179763 wait complete. 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7f41140740a0 msgr2=0x7f41140764c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7f41140740a0 0x7f41140764c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f411406bf00 msgr2=0x7f411406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f411406bf00 0x7f411406e3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f411c009f20 tx=0x7f411c009110 comp rx=0 tx=0).stop 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 msgr2=0x7f41200dd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f410d7fa640 1 --2- 172.21.15.99:0/3167938572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41200e7110 0x7f41200dd670 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4128069470 tx=0x7f412806bb50 comp rx=0 tx=0).stop 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.199+0000 7f412759e640 1 -- 172.21.15.99:0/3167938572 reap_dead start 2024-09-10T11:30:36.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.200+0000 7f53857fa640 1 -- 172.21.15.99:0/3414999339 --> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5380002dd0 con 0x7f538c074120 2024-09-10T11:30:36.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.200+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 --> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2de00cff40 con 0x7f2dc0074cd0 2024-09-10T11:30:36.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.200+0000 7f53877fe640 1 -- 172.21.15.99:0/3414999339 <== osd.2 v2:172.21.15.99:6817/1090025270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5380002dd0 con 0x7f538c074120 2024-09-10T11:30:36.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.200+0000 7f2dceffd640 1 -- 172.21.15.99:0/1980210000 <== osd.1 v2:172.21.15.99:6808/2120836022 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2de00cff40 con 0x7f2dc0074cd0 2024-09-10T11:30:36.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f538c074120 msgr2=0x7f538c076540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f538c074120 0x7f538c076540 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 shutdown_connections 2024-09-10T11:30:36.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 >> 172.21.15.99:0/3167938572 conn(0x7f4120036080 msgr2=0x7f4120031180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2dc0074cd0 msgr2=0x7f2dc0077170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2dc0074cd0 0x7f2dc0077170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 shutdown_connections 2024-09-10T11:30:36.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f410d7fa640 1 -- 172.21.15.99:0/3167938572 wait complete. 2024-09-10T11:30:36.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2dc006beb0 msgr2=0x7f2dc006e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2dc006beb0 0x7f2dc006e370 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2de0162180 tx=0x7f2dd0009040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 msgr2=0x7f2de01ed4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 --2- 172.21.15.99:0/1980210000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2de00714a0 0x7f2de01ed4c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2dd402a9e0 tx=0x7f2dd40046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2dded9d640 1 -- 172.21.15.99:0/1980210000 reap_dead start 2024-09-10T11:30:36.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 shutdown_connections 2024-09-10T11:30:36.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 >> 172.21.15.99:0/1980210000 conn(0x7f2de006bc10 msgr2=0x7f2de01622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7ff5232f1640 1 -- 172.21.15.99:0/1680015747 --> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff51c13e330 con 0x7ff500078530 2024-09-10T11:30:36.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 shutdown_connections 2024-09-10T11:30:36.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f2de48ac640 1 -- 172.21.15.99:0/1980210000 wait complete. 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f538c06bf00 msgr2=0x7f538c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f538c06bf00 0x7f538c06e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5394006fd0 tx=0x7f5394008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a00cd230 msgr2=0x7f53a0150990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.201+0000 7f53a5311640 1 --2- 172.21.15.99:0/3414999339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53a00cd230 0x7f53a0150990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5390031000 tx=0x7f53900046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f539f7fe640 1 -- 172.21.15.99:0/3414999339 reap_dead start 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 shutdown_connections 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 >> 172.21.15.99:0/3414999339 conn(0x7f53a00730c0 msgr2=0x7f53a015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 --> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87cc09e500 con 0x7f87b0074000 2024-09-10T11:30:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 shutdown_connections 2024-09-10T11:30:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7f53a5311640 1 -- 172.21.15.99:0/3414999339 wait complete. 2024-09-10T11:30:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 --> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd504002dd0 con 0x7fd510074130 2024-09-10T11:30:36.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.202+0000 7ff511ffb640 1 -- 172.21.15.99:0/1680015747 <== osd.0 v2:172.21.15.99:6800/291490777 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff51c13e330 con 0x7ff500078530 2024-09-10T11:30:36.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7f87c27fc640 1 -- 172.21.15.99:0/3859980434 <== osd.7 v2:172.21.15.196:6816/106140719 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f87cc09e500 con 0x7f87b0074000 2024-09-10T11:30:36.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7fd50bfff640 1 -- 172.21.15.99:0/205349677 <== osd.5 v2:172.21.15.196:6824/3797029994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd504002dd0 con 0x7fd510074130 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7ff500078530 msgr2=0x7ff50007a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7ff500078530 0x7ff50007a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff500070310 msgr2=0x7ff5000727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff500070310 0x7ff5000727d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5180088b0 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 msgr2=0x7ff51c13ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff5077fe640 1 --2- 172.21.15.99:0/1680015747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff51c1453b0 0x7ff51c13ceb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff514033000 tx=0x7ff514004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7fd510074130 msgr2=0x7fd510076550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7fd523fff640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7fd510074130 0x7fd510076550 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.203+0000 7ff52188e640 1 -- 172.21.15.99:0/1680015747 reap_dead start 2024-09-10T11:30:36.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd51006bf30 msgr2=0x7fd51006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd51006bf30 0x7fd51006e3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00b700 tx=0x7fd514008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f87b0074000 msgr2=0x7f87b0076420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f87b0074000 0x7f87b0076420 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 msgr2=0x7fd51c0d55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 --2- 172.21.15.99:0/205349677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd51c0def00 0x7fd51c0d55a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c1e0 tx=0x7fd518001970 comp rx=0 tx=0).stop 2024-09-10T11:30:36.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f87b006be60 msgr2=0x7f87b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd5237fe640 1 -- 172.21.15.99:0/205349677 reap_dead start 2024-09-10T11:30:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f87b006be60 0x7f87b006e320 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87bc006fd0 tx=0x7f87bc008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 msgr2=0x7f87cc0b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 --2- 172.21.15.99:0/3859980434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87cc167300 0x7f87cc0b25a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f87c400c7b0 tx=0x7f87c400cc70 comp rx=0 tx=0).stop 2024-09-10T11:30:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d2457640 1 -- 172.21.15.99:0/3859980434 reap_dead start 2024-09-10T11:30:36.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 shutdown_connections 2024-09-10T11:30:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 >> 172.21.15.99:0/1680015747 conn(0x7ff51c072680 msgr2=0x7ff51c1e1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 shutdown_connections 2024-09-10T11:30:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 >> 172.21.15.99:0/205349677 conn(0x7fd51c01dd70 msgr2=0x7fd51c0318d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 shutdown_connections 2024-09-10T11:30:36.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 >> 172.21.15.99:0/3859980434 conn(0x7f87cc09ba30 msgr2=0x7f87cc0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 shutdown_connections 2024-09-10T11:30:36.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7ff5077fe640 1 -- 172.21.15.99:0/1680015747 wait complete. 2024-09-10T11:30:36.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 shutdown_connections 2024-09-10T11:30:36.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.204+0000 7fd523fff640 1 -- 172.21.15.99:0/205349677 wait complete. 2024-09-10T11:30:36.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.205+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 shutdown_connections 2024-09-10T11:30:36.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.205+0000 7f87d3eba640 1 -- 172.21.15.99:0/3859980434 wait complete. 2024-09-10T11:30:36.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.206+0000 7fe0ea10a640 1 -- 172.21.15.99:0/2724595758 --> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0e4071740 con 0x7fe0d0074150 2024-09-10T11:30:36.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.207+0000 7fe0cbfff640 1 -- 172.21.15.99:0/2724595758 <== osd.3 v2:172.21.15.99:6816/1027928425 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0e4071740 con 0x7fe0d0074150 2024-09-10T11:30:36.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.208+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7fe0d0074150 msgr2=0x7fe0d0076570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.208+0000 7fe0c9ffb640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7fe0d0074150 0x7fe0d0076570 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0d006bf30 msgr2=0x7fe0d006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0d006bf30 0x7fe0d006e3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc033000 tx=0x7fe0dc009030 comp rx=0 tx=0).stop 2024-09-10T11:30:36.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 msgr2=0x7fe0e4144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 --2- 172.21.15.99:0/2724595758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0e4149bf0 0x7fe0e4144f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4008fd0 tx=0x7fe0d4008790 comp rx=0 tx=0).stop 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0e3fff640 1 -- 172.21.15.99:0/2724595758 reap_dead start 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 shutdown_connections 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.209+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 >> 172.21.15.99:0/2724595758 conn(0x7fe0e40b23d0 msgr2=0x7fe0e40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.210+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 shutdown_connections 2024-09-10T11:30:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.210+0000 7fe0c9ffb640 1 -- 172.21.15.99:0/2724595758 wait complete. 2024-09-10T11:30:36.279 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T11:30:36.280 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:36.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T11:30:36.281 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.281 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T11:30:36.282 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T11:30:36.283 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:36.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T11:30:36.283 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T11:30:36.285 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:36.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T11:30:36.286 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:36.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T11:30:36.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.483+0000 7ffaf5336640 1 Processor -- start 2024-09-10T11:30:36.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.484+0000 7ffaf5336640 1 -- start start 2024-09-10T11:30:36.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7ffaf5336640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffaf006c700 0x7ffaf015b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7ffaf5336640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf00b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7ffaf5336640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf006d4a0 con 0x7ffaf00b5000 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7ffaf5336640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ffaf00b5ac0 con 0x7ffaf006c700 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7ffaf5336640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf006ddd0 con 0x7ffaf0071a20 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7ffaf4b35640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf00b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7ffae7fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ffaf00b5000 0x7ffaf0071310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44908/0 (socket says 172.21.15.99:44908) 2024-09-10T11:30:36.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7ffae7fff640 1 -- 172.21.15.99:0/3417788816 learned_addr learned my addr 172.21.15.99:0/3417788816 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7fdcd2065640 1 Processor -- start 2024-09-10T11:30:36.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7fdcd2065640 1 -- start start 2024-09-10T11:30:36.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7fdcd2065640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdccc163040 0x7fdccc0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7fdcd2065640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851152313 0 0) 0x7ffaf006d4a0 con 0x7ffaf00b5000 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffadc003520 con 0x7ffaf00b5000 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7f7784f2a640 1 Processor -- start 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.485+0000 7f7784f2a640 1 -- start start 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7f7784f2a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f778016ed50 0x7f778016bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7f7784f2a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7f7784f2a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77800718f0 con 0x7f77800b5a40 2024-09-10T11:30:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7f7784f2a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f778006eef0 con 0x7f778016ed50 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.486+0000 7f7784f2a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7780072500 con 0x7f77800b5620 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaeffff640 1 --2- 172.21.15.99:0/3417788816 >> v2:172.21.15.196:3300/0 conn(0x7ffaf006c700 0x7ffaf015b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaf4b35640 1 -- 172.21.15.99:0/3417788816 >> v2:172.21.15.196:3300/0 conn(0x7ffaf006c700 msgr2=0x7ffaf015b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaf4b35640 1 --2- 172.21.15.99:0/3417788816 >> v2:172.21.15.196:3300/0 conn(0x7ffaf006c700 0x7ffaf015b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56882/0 (socket says 172.21.15.99:56882) 2024-09-10T11:30:36.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 -- 172.21.15.99:0/930022746 learned_addr learned my addr 172.21.15.99:0/930022746 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777e59c640 1 --2- 172.21.15.99:0/930022746 >> v2:172.21.15.196:3300/0 conn(0x7f778016ed50 0x7f778016bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc0720e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56896/0 (socket says 172.21.15.99:56896) 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdccb7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdccc163040 0x7fdccc0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdccb7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdccc163040 0x7fdccc0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54398/0 (socket says 172.21.15.99:54398) 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdccaffd640 1 -- 172.21.15.99:0/4150187438 learned_addr learned my addr 172.21.15.99:0/4150187438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdccc06ed80 con 0x7fdccc072620 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdccc071920 con 0x7fdccc163040 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc06f020 con 0x7fdccc0aff70 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fdccb7fe640 1 -- 172.21.15.99:0/4150187438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 msgr2=0x7fdccc0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fdccb7fe640 1 --2- 172.21.15.99:0/4150187438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc0720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaf4b35640 1 -- 172.21.15.99:0/3417788816 >> v1:172.21.15.99:6789/0 conn(0x7ffaf00b5000 legacy=0x7ffaf0071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaf4b35640 1 -- 172.21.15.99:0/3417788816 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf006d780 con 0x7ffaf0071a20 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3205410069 0 0) 0x7ffadc003520 con 0x7ffaf00b5000 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 -- 172.21.15.99:0/930022746 >> v2:172.21.15.196:3300/0 conn(0x7f778016ed50 msgr2=0x7f778016bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 --2- 172.21.15.99:0/930022746 >> v2:172.21.15.196:3300/0 conn(0x7f778016ed50 0x7f778016bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 -- 172.21.15.99:0/930022746 >> v1:172.21.15.196:6789/0 conn(0x7f77800b5a40 legacy=0x7f7780072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.487+0000 7f777dd9b640 1 -- 172.21.15.99:0/930022746 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7780071bd0 con 0x7f77800b5620 2024-09-10T11:30:36.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fdccb7fe640 1 -- 172.21.15.99:0/4150187438 >> v1:172.21.15.196:6789/0 conn(0x7fdccc072620 legacy=0x7fdccc164690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fae8ffff640 1 Processor -- start 2024-09-10T11:30:36.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fdcca7fc640 1 -- 172.21.15.99:0/4150187438 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089502692 0 0) 0x7fdccc06ed80 con 0x7fdccc072620 2024-09-10T11:30:36.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7f777dd9b640 1 --2- 172.21.15.99:0/930022746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7778007630 tx=0x7f777802ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b6ce6cd479754c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7ffaf4b35640 1 --2- 172.21.15.99:0/3417788816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf00b5470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009d90 tx=0x7ffae802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d2167e8e3d86c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fae8ffff640 1 -- start start 2024-09-10T11:30:36.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7fdccb7fe640 1 -- 172.21.15.99:0/4150187438 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc071bc0 con 0x7fdccc163040 2024-09-10T11:30:36.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8001670 con 0x7ffaf0071a20 2024-09-10T11:30:36.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.488+0000 7f777d59a640 1 -- 172.21.15.99:0/930022746 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77780094e0 con 0x7f77800b5620 2024-09-10T11:30:36.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7f777d59a640 1 -- 172.21.15.99:0/930022746 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7778003990 con 0x7f77800b5620 2024-09-10T11:30:36.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7f777d59a640 1 -- 172.21.15.99:0/930022746 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778003da0 con 0x7f77800b5620 2024-09-10T11:30:36.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae8007540 con 0x7ffaf0071a20 2024-09-10T11:30:36.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7fae8ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae88053080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7fae8ffff640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae88057d80 con 0x7fae8803bc60 2024-09-10T11:30:36.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7fae8ffff640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae88058060 con 0x7fae88057830 2024-09-10T11:30:36.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7fae8ffff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae88005eb0 con 0x7fae880e28f0 2024-09-10T11:30:36.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.489+0000 7fdccb7fe640 1 --2- 172.21.15.99:0/4150187438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc0b5620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b050 tx=0x7fdcbc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227032285e199e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 -- 172.21.15.99:0/930022746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 msgr2=0x7f77800d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaef7fe640 1 -- 172.21.15.99:0/3417788816 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae8007910 con 0x7ffaf0071a20 2024-09-10T11:30:36.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 --2- 172.21.15.99:0/930022746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7778007630 tx=0x7f777802ad30 comp rx=0 tx=0).stop 2024-09-10T11:30:36.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8e7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fae8803bc60 0x7fae880535c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44914/0 (socket says 172.21.15.99:44914) 2024-09-10T11:30:36.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8e7fc640 1 -- 172.21.15.99:0/1841271241 learned_addr learned my addr 172.21.15.99:0/1841271241 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8effd640 1 --2- 172.21.15.99:0/1841271241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae88053080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fdcca7fc640 1 -- 172.21.15.99:0/4150187438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031650 con 0x7fdccc163040 2024-09-10T11:30:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fdcca7fc640 1 -- 172.21.15.99:0/4150187438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc036070 con 0x7fdccc163040 2024-09-10T11:30:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fdcca7fc640 1 -- 172.21.15.99:0/4150187438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031a10 con 0x7fdccc163040 2024-09-10T11:30:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 -- 172.21.15.99:0/930022746 shutdown_connections 2024-09-10T11:30:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 --2- 172.21.15.99:0/930022746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5620 0x7f77800d0740 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 --2- 172.21.15.99:0/930022746 >> v2:172.21.15.196:3300/0 conn(0x7f778016ed50 0x7f778016bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8dffb640 1 -- 172.21.15.99:0/1841271241 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456761361 0 0) 0x7fae88057d80 con 0x7fae8803bc60 2024-09-10T11:30:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7f7784f2a640 1 -- 172.21.15.99:0/930022746 >> 172.21.15.99:0/930022746 conn(0x7f7780075870 msgr2=0x7f778006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8dffb640 1 -- 172.21.15.99:0/1841271241 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae78003520 con 0x7fae8803bc60 2024-09-10T11:30:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 -- 172.21.15.99:0/3417788816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 msgr2=0x7ffaf00b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 --2- 172.21.15.99:0/3417788816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf00b5470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009d90 tx=0x7ffae802a5f0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8effd640 1 -- 172.21.15.99:0/1841271241 >> v1:172.21.15.196:6789/0 conn(0x7fae88057830 legacy=0x7fae880066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8effd640 1 -- 172.21.15.99:0/1841271241 >> v1:172.21.15.99:6789/0 conn(0x7fae8803bc60 legacy=0x7fae880535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 -- 172.21.15.99:0/3417788816 shutdown_connections 2024-09-10T11:30:36.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 --2- 172.21.15.99:0/3417788816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf00b5470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 --2- 172.21.15.99:0/3417788816 >> v2:172.21.15.196:3300/0 conn(0x7ffaf006c700 0x7ffaf015b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 -- 172.21.15.99:0/3417788816 >> 172.21.15.99:0/3417788816 conn(0x7ffaf006bc10 msgr2=0x7ffaf006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7fae8effd640 1 -- 172.21.15.99:0/1841271241 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae88006190 con 0x7fae880e28f0 2024-09-10T11:30:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 -- 172.21.15.99:0/3417788816 shutdown_connections 2024-09-10T11:30:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7f7784f2a640 1 -- 172.21.15.99:0/930022746 shutdown_connections 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7f7784f2a640 1 -- 172.21.15.99:0/930022746 wait complete. 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fc6ac5f6640 1 Processor -- start 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 msgr2=0x7fdccc0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 --2- 172.21.15.99:0/4150187438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc0b5620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00b050 tx=0x7fdcbc02a8c0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 shutdown_connections 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 --2- 172.21.15.99:0/4150187438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc0720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 --2- 172.21.15.99:0/4150187438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc0b5620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 >> 172.21.15.99:0/4150187438 conn(0x7fdccc075870 msgr2=0x7fdccc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fc6ac5f6640 1 -- start start 2024-09-10T11:30:36.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fae8effd640 1 --2- 172.21.15.99:0/1841271241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae88053080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae900518b0 tx=0x7fae900a67d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e18172806d08a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 shutdown_connections 2024-09-10T11:30:36.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fdcd2065640 1 -- 172.21.15.99:0/4150187438 wait complete. 2024-09-10T11:30:36.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fb718acf640 1 Processor -- start 2024-09-10T11:30:36.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.490+0000 7ffaf5336640 1 -- 172.21.15.99:0/3417788816 wait complete. 2024-09-10T11:30:36.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fb718acf640 1 -- start start 2024-09-10T11:30:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc801e58640 1 Processor -- start 2024-09-10T11:30:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7ffaf5336640 1 Processor -- start 2024-09-10T11:30:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc801e58640 1 -- start start 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc6ac5f6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6a40d0e50 0x7fc6a4072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc6ac5f6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a4158b50 con 0x7fc6a4158770 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc6ac5f6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a415b280 con 0x7fc6a406c320 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fc6ac5f6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc6a415b560 con 0x7fc6a40d0e50 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7f7784f2a640 1 Processor -- start 2024-09-10T11:30:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6a9b91640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6a40d0e50 0x7fc6a4072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6aa392640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc6a4158770 0x7fc6a40d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44922/0 (socket says 172.21.15.99:44922) 2024-09-10T11:30:36.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6aa392640 1 -- 172.21.15.99:0/2017929134 learned_addr learned my addr 172.21.15.99:0/2017929134 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7f7784f2a640 1 -- start start 2024-09-10T11:30:36.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fb718acf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb70c005a60 0x7fb70c053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fb718acf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb70c0e7120 0x7fb70c005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fb718acf640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70c006380 con 0x7fb70c04eb60 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6a9390640 1 -- 172.21.15.99:0/2017929134 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773341756 0 0) 0x7fc6a4158b50 con 0x7fc6a4158770 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6a9b91640 1 -- 172.21.15.99:0/2017929134 >> v1:172.21.15.196:6789/0 conn(0x7fc6a406c320 legacy=0x7fc6a4073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6a9b91640 1 -- 172.21.15.99:0/2017929134 >> v1:172.21.15.99:6789/0 conn(0x7fc6a4158770 legacy=0x7fc6a40d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801e58640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc7f4005a60 0x7fc7f4053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801e58640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f4052330 con 0x7fc7f404eb60 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801e58640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7f4006c80 con 0x7fc7f40e7120 2024-09-10T11:30:36.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801e58640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc7f40520e0 con 0x7fc7f4005a60 2024-09-10T11:30:36.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc6a9b91640 1 -- 172.21.15.99:0/2017929134 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a415b840 con 0x7fc6a40d0e50 2024-09-10T11:30:36.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801657640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc7f404eb60 0x7fc7f401d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44926/0 (socket says 172.21.15.99:44926) 2024-09-10T11:30:36.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fc801657640 1 -- 172.21.15.99:0/280095771 learned_addr learned my addr 172.21.15.99:0/280095771 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fb718acf640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c0520e0 con 0x7fb70c005a60 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fb718acf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb70c052330 con 0x7fb70c0e7120 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fc800e56640 1 --2- 172.21.15.99:0/280095771 >> v2:172.21.15.99:3300/0 conn(0x7fc7f4005a60 0x7fc7f4053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.491+0000 7fae8dffb640 1 -- 172.21.15.99:0/1841271241 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900b4070 con 0x7fae880e28f0 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fae8dffb640 1 -- 172.21.15.99:0/1841271241 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae900ab070 con 0x7fae880e28f0 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.492+0000 7fae8ffff640 1 -- 172.21.15.99:0/1841271241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 msgr2=0x7fae88053080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fae8dffb640 1 -- 172.21.15.99:0/1841271241 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae900b4070 con 0x7fae880e28f0 2024-09-10T11:30:36.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1841271241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae88053080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fae900518b0 tx=0x7fae900a67d0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fae8ffff640 1 -- 172.21.15.99:0/1841271241 shutdown_connections 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1841271241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae88053080 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fae8ffff640 1 -- 172.21.15.99:0/1841271241 >> 172.21.15.99:0/1841271241 conn(0x7fae8803a520 msgr2=0x7fae8803a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fdcd2065640 1 Processor -- start 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7fdcd2065640 1 -- start start 2024-09-10T11:30:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fbef4118640 1 Processor -- start 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.493+0000 7ffaf5336640 1 -- start start 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fb713fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb70c04eb60 0x7fb70c01c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59226/0 (socket says 172.21.15.99:59226) 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fb713fff640 1 -- 172.21.15.99:0/4292152389 learned_addr learned my addr 172.21.15.99:0/4292152389 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fb7137fe640 1 --2- 172.21.15.99:0/4292152389 >> v2:172.21.15.99:3300/0 conn(0x7fb70c005a60 0x7fb70c053f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fb712ffd640 1 --2- 172.21.15.99:0/4292152389 >> v2:172.21.15.196:3300/0 conn(0x7fb70c0e7120 0x7fb70c005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fae8ffff640 1 -- 172.21.15.99:0/1841271241 shutdown_connections 2024-09-10T11:30:36.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564269540 0 0) 0x7fc7f4052330 con 0x7fc7f404eb60 2024-09-10T11:30:36.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7ec003520 con 0x7fc7f404eb60 2024-09-10T11:30:36.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fbef4118640 1 -- start start 2024-09-10T11:30:36.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf006c700 0x7ffaf014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 0x7ffaf00b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf006e260 con 0x7ffaf00b5000 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf006eea0 con 0x7ffaf006c700 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffaf5336640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf015bef0 con 0x7ffaf0071a20 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77800b5620 0x7f7780165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f778016ed50 0x7f778008b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7780072190 con 0x7f77800b5620 2024-09-10T11:30:36.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f778016f130 con 0x7f778016ed50 2024-09-10T11:30:36.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7f7784f2a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77800714a0 con 0x7f77800b5a40 2024-09-10T11:30:36.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7fae8ffff640 1 -- 172.21.15.99:0/1841271241 wait complete. 2024-09-10T11:30:36.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.494+0000 7ffae7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56922/0 (socket says 172.21.15.99:56922) 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 -- 172.21.15.99:0/470350209 learned_addr learned my addr 172.21.15.99:0/470350209 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56924/0 (socket says 172.21.15.99:56924) 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 -- 172.21.15.99:0/2279693571 learned_addr learned my addr 172.21.15.99:0/2279693571 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1914352920 0 0) 0x7fc7f4006c80 con 0x7fc7f40e7120 2024-09-10T11:30:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f4052330 con 0x7fc7f40e7120 2024-09-10T11:30:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777ed9d640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f778016ed50 0x7f778008b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdccb7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35424/0 (socket says 172.21.15.99:35424) 2024-09-10T11:30:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc6a9b91640 1 --2- 172.21.15.99:0/2017929134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a40d0e50 0x7fc6a4072cd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc694009720 tx=0x7fc69402a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a31c2bb0910ce0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffaf4b35640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 0x7ffaf00b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdccaffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0b5ce0 con 0x7fdccc163040 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0b5b60 con 0x7fdccc072620 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdcd2065640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc164da0 con 0x7fdccc0aff70 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fdccb7fe640 1 -- 172.21.15.99:0/3999762403 learned_addr learned my addr 172.21.15.99:0/3999762403 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 -- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f778016ed50 msgr2=0x7f778008b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 -- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf006c700 msgr2=0x7ffaf014cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf006c700 0x7ffaf014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 -- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 msgr2=0x7ffaf00b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 0x7ffaf00b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 -- 172.21.15.99:0/470350209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf014d070 con 0x7ffaf0071a20 2024-09-10T11:30:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f778016ed50 0x7f778008b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1786674989 0 0) 0x7fc7f4052330 con 0x7fc7f40e7120 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 >> v2:172.21.15.99:3300/0 conn(0x7fc7f4005a60 msgr2=0x7fc7f4053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 --2- 172.21.15.99:0/280095771 >> v2:172.21.15.99:3300/0 conn(0x7fc7f4005a60 0x7fc7f4053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 >> v1:172.21.15.99:6789/0 conn(0x7fc7f404eb60 legacy=0x7fc7f401d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffaf4b35640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 0x7ffaf00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f4006f60 con 0x7fc7f40e7120 2024-09-10T11:30:36.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 -- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77800b5620 msgr2=0x7f7780165c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77800b5620 0x7f7780165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7f777dd9b640 1 -- 172.21.15.99:0/2279693571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7780166480 con 0x7f77800b5a40 2024-09-10T11:30:36.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7ffae7fff640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c780 tx=0x7ffad800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fc7f0001670 con 0x7fc7f40e7120 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 >> v1:172.21.15.196:6789/0 conn(0x7fc7f40e7120 legacy=0x7fc7f4005e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fb712ffd640 1 -- 172.21.15.99:0/4292152389 >> v2:172.21.15.99:3300/0 conn(0x7fb70c005a60 msgr2=0x7fb70c053f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fb712ffd640 1 --2- 172.21.15.99:0/4292152389 >> v2:172.21.15.99:3300/0 conn(0x7fb70c005a60 0x7fb70c053f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fb712ffd640 1 -- 172.21.15.99:0/4292152389 >> v1:172.21.15.196:6789/0 conn(0x7fb70c04eb60 legacy=0x7fb70c01c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fbef4118640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 0x7fbeec072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fbef4118640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbeec06c320 0x7fbeec073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fbef4118640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeec073750 con 0x7fbeec158770 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fbef4118640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec158b50 con 0x7fbeec0d0e50 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fbef4118640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec15b280 con 0x7fbeec06c320 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fb712ffd640 1 -- 172.21.15.99:0/4292152389 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c006c80 con 0x7fb70c0e7120 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 0x7fc7ec003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.495+0000 7fc7fb7fe640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7ec0051c0 0x7fc7ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f40520e0 con 0x7fc7ec0051c0 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec00bf30 con 0x7fc7ec003b40 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec00c130 con 0x7fc7ec004800 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7f777dd9b640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7778000f80 tx=0x7f7778004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef16b3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 0x7fbeec072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbeec06c320 0x7fbeec073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8017070 con 0x7ffaf0071a20 2024-09-10T11:30:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef16b3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 0x7fbeec072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54444/0 (socket says 172.21.15.99:54444) 2024-09-10T11:30:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef16b3640 1 -- 172.21.15.99:0/2928091679 learned_addr learned my addr 172.21.15.99:0/2928091679 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbeec06c320 0x7fbeec073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35434/0 (socket says 172.21.15.99:35434) 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc6a9390640 1 -- 172.21.15.99:0/2017929134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694037070 con 0x7fc6a40d0e50 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc6a9390640 1 -- 172.21.15.99:0/2017929134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69402ad40 con 0x7fc6a40d0e50 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc6a9390640 1 -- 172.21.15.99:0/2017929134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc694037070 con 0x7fc6a40d0e50 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad8005ad0 con 0x7ffaf0071a20 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8007070 con 0x7ffaf0071a20 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7ffaf5336640 1 -- 172.21.15.99:0/470350209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf01455c0 con 0x7ffaf0071a20 2024-09-10T11:30:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7ffaf5336640 1 -- 172.21.15.99:0/470350209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf0145b80 con 0x7ffaf0071a20 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 -- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 msgr2=0x7fdccc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 -- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 msgr2=0x7fdccc15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 -- 172.21.15.99:0/3999762403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc15a6d0 con 0x7fdccc072620 2024-09-10T11:30:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778008cc0 con 0x7f77800b5a40 2024-09-10T11:30:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fb712ffd640 1 --2- 172.21.15.99:0/4292152389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c005e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00d7c0 tx=0x7fb70402a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6587dee960595abd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fb7127fc640 1 -- 172.21.15.99:0/4292152389 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704037070 con 0x7fb70c0e7120 2024-09-10T11:30:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fae8ffff640 1 Processor -- start 2024-09-10T11:30:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc800e56640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 0x7fc7ec003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc7fbfff640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fc801657640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7ec0051c0 0x7fc7ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc800e56640 1 -- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec004800 msgr2=0x7fc7ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc800e56640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc800e56640 1 -- 172.21.15.99:0/280095771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7ec0051c0 msgr2=0x7fc7ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc800e56640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7ec0051c0 0x7fc7ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc800e56640 1 -- 172.21.15.99:0/280095771 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f4006f60 con 0x7fc7ec003b40 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 -- 172.21.15.99:0/2928091679 >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 msgr2=0x7fbeec072cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 --2- 172.21.15.99:0/2928091679 >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 0x7fbeec072cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 -- 172.21.15.99:0/2928091679 >> v1:172.21.15.196:6789/0 conn(0x7fbeec158770 legacy=0x7fbeec0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fbef26b5640 1 -- 172.21.15.99:0/2928091679 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec15b560 con 0x7fbeec06c320 2024-09-10T11:30:36.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fae8ffff640 1 -- start start 2024-09-10T11:30:36.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7778042070 con 0x7f77800b5a40 2024-09-10T11:30:36.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/2017929134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a40d0e50 msgr2=0x7fc6a4072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/2017929134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a40d0e50 0x7fc6a4072cd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc694009720 tx=0x7fc69402a030 comp rx=0 tx=0).stop 2024-09-10T11:30:36.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7780166760 con 0x7f77800b5a40 2024-09-10T11:30:36.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7780166d20 con 0x7f77800b5a40 2024-09-10T11:30:36.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778008cc0 con 0x7f77800b5a40 2024-09-10T11:30:36.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.496+0000 7fdccb7fe640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc033040 tx=0x7fdcbc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc03a810 con 0x7fdccc072620 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc03a9b0 con 0x7fdccc072620 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc03ad80 con 0x7fdccc072620 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fdcd2065640 1 -- 172.21.15.99:0/3999762403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc15a9b0 con 0x7fdccc072620 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fdcd2065640 1 -- 172.21.15.99:0/3999762403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc15af70 con 0x7fdccc072620 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fb7127fc640 1 -- 172.21.15.99:0/4292152389 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70402adc0 con 0x7fb70c0e7120 2024-09-10T11:30:36.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 0x7fae8804c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae880c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae88053e50 con 0x7fae8803bc60 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae88053cd0 con 0x7fae88057830 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8ffff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae880e2cd0 con 0x7fae880e28f0 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.497+0000 7fc801657640 1 -- 172.21.15.99:0/280095771 reap_dead start 2024-09-10T11:30:36.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fbef26b5640 1 --2- 172.21.15.99:0/2928091679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec06c320 0x7fbeec073210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbeec072a00 tx=0x7fbee802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb0fb5421c4f4063 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fc800e56640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 0x7fc7ec003f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009ab0 tx=0x7fc7ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d51eba5456117f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fb7127fc640 1 -- 172.21.15.99:0/4292152389 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704032660 con 0x7fb70c0e7120 2024-09-10T11:30:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fbef0eb2640 1 -- 172.21.15.99:0/2928091679 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbeec06c320 2024-09-10T11:30:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fbef0eb2640 1 -- 172.21.15.99:0/2928091679 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbeec06c320 2024-09-10T11:30:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fbef0eb2640 1 -- 172.21.15.99:0/2928091679 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee802ae60 con 0x7fbeec06c320 2024-09-10T11:30:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/2017929134 shutdown_connections 2024-09-10T11:30:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/2017929134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a40d0e50 0x7fc6a4072cd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/2017929134 >> 172.21.15.99:0/2017929134 conn(0x7fc6a4075870 msgr2=0x7fc6a406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8e7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 0x7fae8804c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7ffaf5336640 1 -- 172.21.15.99:0/470350209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf00b5410 con 0x7ffaf0071a20 2024-09-10T11:30:36.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778016bdf0 con 0x7f77800b5a40 2024-09-10T11:30:36.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54456/0 (socket says 172.21.15.99:54456) 2024-09-10T11:30:36.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.498+0000 7fae8effd640 1 -- 172.21.15.99:0/1558598898 learned_addr learned my addr 172.21.15.99:0/1558598898 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 -- 172.21.15.99:0/4292152389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 msgr2=0x7fb70c005e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/2017929134 shutdown_connections 2024-09-10T11:30:36.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8036070 con 0x7fc7ec003b40 2024-09-10T11:30:36.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8f7fe640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae880c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 --2- 172.21.15.99:0/4292152389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c005e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00d7c0 tx=0x7fb70402a130 comp rx=0 tx=0).stop 2024-09-10T11:30:36.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e8005990 con 0x7fc7ec003b40 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fc7fb7fe640 1 -- 172.21.15.99:0/280095771 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8036070 con 0x7fc7ec003b40 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 -- 172.21.15.99:0/4292152389 shutdown_connections 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 -- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 msgr2=0x7fae880c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae880c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 -- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 msgr2=0x7fae8803d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 -- 172.21.15.99:0/1558598898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae880c79b0 con 0x7fae88057830 2024-09-10T11:30:36.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 --2- 172.21.15.99:0/4292152389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c005e40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 --2- 172.21.15.99:0/4292152389 >> v2:172.21.15.99:3300/0 conn(0x7fb70c005a60 0x7fb70c053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/2017929134 wait complete. 2024-09-10T11:30:36.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8effd640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 -- 172.21.15.99:0/4292152389 >> 172.21.15.99:0/4292152389 conn(0x7fb70c036090 msgr2=0x7fb70c0364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8f7fe640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae880c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:36.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffad801a070 con 0x7ffaf0071a20 2024-09-10T11:30:36.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fae8e7fc640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 0x7fae8804c540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae8803e210 tx=0x7fae8000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fbef4118640 1 -- 172.21.15.99:0/2928091679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec06c320 msgr2=0x7fbeec073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fbef4118640 1 --2- 172.21.15.99:0/2928091679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec06c320 0x7fbeec073210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbeec072a00 tx=0x7fbee802a110 comp rx=0 tx=0).stop 2024-09-10T11:30:36.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fdcd2065640 1 -- 172.21.15.99:0/3999762403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc15b170 con 0x7fdccc072620 2024-09-10T11:30:36.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7fb718acf640 1 -- 172.21.15.99:0/4292152389 shutdown_connections 2024-09-10T11:30:36.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f777802b630 con 0x7f77800b5a40 2024-09-10T11:30:36.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7f776effd640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 0x7f776006e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcbc046480 con 0x7fdccc072620 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7778038040 con 0x7f77800b5a40 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7f777e59c640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 0x7f776006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fc801e58640 1 -- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 msgr2=0x7fc7ec003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fc801e58640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 0x7fc7ec003f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009ab0 tx=0x7fc7ec005830 comp rx=0 tx=0).stop 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fc801e58640 1 -- 172.21.15.99:0/280095771 shutdown_connections 2024-09-10T11:30:36.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fc801e58640 1 --2- 172.21.15.99:0/280095771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7ec003b40 0x7fc7ec003f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fc801e58640 1 -- 172.21.15.99:0/280095771 >> 172.21.15.99:0/280095771 conn(0x7fc7f4036090 msgr2=0x7fc7f40364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fdcc3fff640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 0x7fdcac06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fc801e58640 1 -- 172.21.15.99:0/280095771 shutdown_connections 2024-09-10T11:30:36.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fdccaffd640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 0x7fdcac06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdcbc0ac7a0 con 0x7fdccc072620 2024-09-10T11:30:36.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.499+0000 7ffaed7fa640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 0x7ffad006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ffad808a300 con 0x7ffaf0071a20 2024-09-10T11:30:36.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7ffaeffff640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 0x7ffad006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.500+0000 7fb718acf640 1 -- 172.21.15.99:0/4292152389 wait complete. 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fc6ac5f6640 1 Processor -- start 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fc6ac5f6640 1 -- start start 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800074e0 con 0x7fae88057830 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae800047e0 con 0x7fae88057830 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae80004b90 con 0x7fae88057830 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae880c9ce0 con 0x7fae88057830 2024-09-10T11:30:36.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae880ca2a0 con 0x7fae88057830 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7ffaeffff640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 0x7ffad006e3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffae0006fd0 tx=0x7ffae0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 -- 172.21.15.99:0/2928091679 shutdown_connections 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 --2- 172.21.15.99:0/2928091679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec06c320 0x7fbeec073210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 --2- 172.21.15.99:0/2928091679 >> v2:172.21.15.99:3300/0 conn(0x7fbeec0d0e50 0x7fbeec072cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 -- 172.21.15.99:0/2928091679 >> 172.21.15.99:0/2928091679 conn(0x7fbeec075870 msgr2=0x7fbeec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7f777e59c640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 0x7f776006e720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f777400de70 tx=0x7f777400f040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fc801e58640 1 -- 172.21.15.99:0/280095771 wait complete. 2024-09-10T11:30:36.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fdccaffd640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 0x7fdcac06e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdccc1595d0 tx=0x7fdcb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fb718acf640 1 Processor -- start 2024-09-10T11:30:36.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 -- 172.21.15.99:0/2928091679 shutdown_connections 2024-09-10T11:30:36.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.501+0000 7fbef4118640 1 -- 172.21.15.99:0/2928091679 wait complete. 2024-09-10T11:30:36.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fb718acf640 1 -- start start 2024-09-10T11:30:36.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fc801e58640 1 Processor -- start 2024-09-10T11:30:36.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae88053080 con 0x7fae88057830 2024-09-10T11:30:36.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fc801e58640 1 -- start start 2024-09-10T11:30:36.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fc6ac5f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fc6ac5f6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6a40d0e50 0x7fc6a415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.502+0000 7fc6ac5f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a4158770 0x7fc6a4147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6ac5f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a4071620 con 0x7fc6a4158770 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35448/0 (socket says 172.21.15.99:35448) 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 -- 172.21.15.99:0/3341419520 learned_addr learned my addr 172.21.15.99:0/3341419520 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fbef4118640 1 Processor -- start 2024-09-10T11:30:36.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aab93640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a4158770 0x7fc6a4147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6a9b91640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6a40d0e50 0x7fc6a415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fbef4118640 1 -- start start 2024-09-10T11:30:36.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a40714a0 con 0x7fc6a406c320 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 0x7fb70c171460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c04eb60 0x7fb70c035160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c053a10 con 0x7fb70c005a60 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c007240 con 0x7fb70c0e7120 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb718acf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c01dc40 con 0x7fb70c04eb60 2024-09-10T11:30:36.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a40717a0 con 0x7fc6a40d0e50 2024-09-10T11:30:36.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb713fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb7137fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 0x7fb70c171460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb713fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35458/0 (socket says 172.21.15.99:35458) 2024-09-10T11:30:36.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb713fff640 1 -- 172.21.15.99:0/1574669718 learned_addr learned my addr 172.21.15.99:0/1574669718 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fb712ffd640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c04eb60 0x7fb70c035160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec003b40 0x7fc7f40dda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f417d020 0x7fc7f40e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4047b50 con 0x7fc7f403a0d0 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f4053a10 con 0x7fc7f417d020 2024-09-10T11:30:36.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc801e58640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f4047cd0 con 0x7fc7ec003b40 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 -- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6a40d0e50 msgr2=0x7fc6a415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6a40d0e50 0x7fc6a415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 -- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a4158770 msgr2=0x7fc6a4147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc6aa392640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a4158770 0x7fc6a4147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc6aa392640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a4148340 con 0x7fc6a406c320 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc7fbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.503+0000 7fc7fbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54470/0 (socket says 172.21.15.99:54470) 2024-09-10T11:30:36.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 -- 172.21.15.99:0/1151729674 learned_addr learned my addr 172.21.15.99:0/1151729674 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc800e56640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec003b40 0x7fc7f40dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 0x7fbeec14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc801657640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f417d020 0x7fc7f40e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 -- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c04eb60 msgr2=0x7fb70c035160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c04eb60 0x7fb70c035160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 -- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 msgr2=0x7fb70c171460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 0x7fb70c171460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 -- 172.21.15.99:0/1574669718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c16ce90 con 0x7fb70c0e7120 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 -- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec003b40 msgr2=0x7fc7f40dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec003b40 0x7fc7f40dda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 0x7fbeec162b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb7137fe640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 0x7fb70c171460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 -- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f417d020 msgr2=0x7fc7f40e28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec163790 con 0x7fbeec06c320 2024-09-10T11:30:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f417d020 0x7fc7f40e28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc6aa392640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c470 tx=0x7fc6a000c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 -- 172.21.15.99:0/1151729674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f40e1b10 con 0x7fc7f403a0d0 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:56990/0 (socket says 172.21.15.99:56990) 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 -- 172.21.15.99:0/2182230286 learned_addr learned my addr 172.21.15.99:0/2182230286 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec15bb20 con 0x7fbeec158770 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec163910 con 0x7fbeec0d0e50 2024-09-10T11:30:36.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef1eb4640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 0x7fbeec14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffad8058640 con 0x7ffaf0071a20 2024-09-10T11:30:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fb713fff640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c550 tx=0x7fb70800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef26b5640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 0x7fbeec162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7fbfff640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000fb90 tx=0x7fc7f000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 -- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 msgr2=0x7fbeec162b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 0x7fbeec162b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 -- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 msgr2=0x7fbeec14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 0x7fbeec14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fbef16b3640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec166df0 con 0x7fbeec0d0e50 2024-09-10T11:30:36.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.504+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000db50 con 0x7fc7f403a0d0 2024-09-10T11:30:36.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fbef26b5640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 0x7fbeec162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fbef1eb4640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 0x7fbeec14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f40e1df0 con 0x7fc7f403a0d0 2024-09-10T11:30:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f40e2e10 con 0x7fc7f403a0d0 2024-09-10T11:30:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fbef16b3640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009fd0 tx=0x7fbedc00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f0018070 con 0x7fc7f403a0d0 2024-09-10T11:30:36.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.505+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000db50 con 0x7fc7f403a0d0 2024-09-10T11:30:36.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc005290 con 0x7fc7f403a0d0 2024-09-10T11:30:36.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdcbc07b230 con 0x7fdccc072620 2024-09-10T11:30:36.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0018070 con 0x7fc6a406c320 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a00074e0 con 0x7fc6a406c320 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0004a20 con 0x7fc6a406c320 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a4148620 con 0x7fc6a406c320 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.506+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a414ba70 con 0x7fc6a406c320 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708018070 con 0x7fb70c0e7120 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb708007500 con 0x7fb70c0e7120 2024-09-10T11:30:36.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c188e90 con 0x7fb70c0e7120 2024-09-10T11:30:36.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb708004a20 con 0x7fb70c0e7120 2024-09-10T11:30:36.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc7f000dcf0 con 0x7fc7f403a0d0 2024-09-10T11:30:36.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c189450 con 0x7fb70c0e7120 2024-09-10T11:30:36.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.507+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a40d0740 con 0x7fc6a406c320 2024-09-10T11:30:36.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae80011070 con 0x7fae88057830 2024-09-10T11:30:36.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fc7f97fa640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 0x7fc7e006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc7f008aa80 con 0x7fc7f403a0d0 2024-09-10T11:30:36.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fae77fff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 0x7fae7c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae8008b040 con 0x7fae88057830 2024-09-10T11:30:36.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fc800e56640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 0x7fc7e006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.508+0000 7fae8effd640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 0x7fae7c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c047b50 con 0x7fb70c0e7120 2024-09-10T11:30:36.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fc800e56640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 0x7fc7e006e4f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8033000 tx=0x7fc7e8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fae8effd640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 0x7fae7c06e3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae9006b610 tx=0x7fae9006f030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbeec0d0e50 2024-09-10T11:30:36.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc00c940 con 0x7fbeec0d0e50 2024-09-10T11:30:36.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec15c490 con 0x7fbeec0d0e50 2024-09-10T11:30:36.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f777807a230 con 0x7f77800b5a40 2024-09-10T11:30:36.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00cd10 con 0x7fbeec0d0e50 2024-09-10T11:30:36.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.509+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec167370 con 0x7fbeec0d0e50 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.511+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec0d0740 con 0x7fbeec0d0e50 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.511+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae800599d0 con 0x7fae88057830 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.512+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbedc01a070 con 0x7fbeec0d0e50 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.513+0000 7fbee27fc640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 0x7fbed006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.513+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbedc08ad50 con 0x7fbeec0d0e50 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.513+0000 7fbef1eb4640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 0x7fbed006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.514+0000 7fbef1eb4640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 0x7fbed006e3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbee400f1d0 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.514+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc6a0004bc0 con 0x7fc6a406c320 2024-09-10T11:30:36.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.515+0000 7fc69affd640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 0x7fc68806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.515+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc6a008a440 con 0x7fc6a406c320 2024-09-10T11:30:36.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fc6a9b91640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 0x7fc68806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbedc059120 con 0x7fbeec0d0e50 2024-09-10T11:30:36.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb70801b360 con 0x7fb70c0e7120 2024-09-10T11:30:36.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc6a0058790 con 0x7fc6a406c320 2024-09-10T11:30:36.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fb6fbfff640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 0x7fb70006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:36.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.516+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb70808a4b0 con 0x7fb70c0e7120 2024-09-10T11:30:36.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.517+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc7f0058d40 con 0x7fc7f403a0d0 2024-09-10T11:30:36.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.519+0000 7fc6a9b91640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 0x7fc68806e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc694034000 tx=0x7fc69402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.521+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb708058880 con 0x7fb70c0e7120 2024-09-10T11:30:36.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.524+0000 7fb7137fe640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 0x7fb70006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:36.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.525+0000 7fb7137fe640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 0x7fb70006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb714074aa0 tx=0x7fb71408f040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.897+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7780169810 con 0x7f77800b5a40 2024-09-10T11:30:36.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.898+0000 7f776effd640 1 -- 172.21.15.99:0/2279693571 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7778079b30 con 0x7f77800b5a40 2024-09-10T11:30:36.900 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2024-09-10T11:30:36.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.901+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 msgr2=0x7f776006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.901+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 0x7f776006e720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f777400de70 tx=0x7f777400f040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 msgr2=0x7f77801680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7778000f80 tx=0x7f7778004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 shutdown_connections 2024-09-10T11:30:36.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f776006c260 0x7f776006e720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f778016ed50 0x7f778008b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77800b5a40 0x7f77801680e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 --2- 172.21.15.99:0/2279693571 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77800b5620 0x7f7780165c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 >> 172.21.15.99:0/2279693571 conn(0x7f7780075870 msgr2=0x7f77800b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 shutdown_connections 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7fdcd2065640 1 -- 172.21.15.99:0/3999762403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdccc0720e0 con 0x7fdccc072620 2024-09-10T11:30:36.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.902+0000 7f7784f2a640 1 -- 172.21.15.99:0/2279693571 wait complete. 2024-09-10T11:30:36.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.903+0000 7fdcc3fff640 1 -- 172.21.15.99:0/3999762403 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdcbc07ab30 con 0x7fdccc072620 2024-09-10T11:30:36.906 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.905+0000 7ffaf5336640 1 -- 172.21.15.99:0/470350209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffaf0076060 con 0x7ffaf0071a20 2024-09-10T11:30:36.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.905+0000 7ffaed7fa640 1 -- 172.21.15.99:0/470350209 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffad805c940 con 0x7ffaf0071a20 2024-09-10T11:30:36.907 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2024-09-10T11:30:36.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.906+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 msgr2=0x7fdcac06e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 0x7fdcac06e340 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdccc1595d0 tx=0x7fdcb4008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 msgr2=0x7fdccc0d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc033040 tx=0x7fdcbc004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 shutdown_connections 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdcac06be80 0x7fdcac06e340 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdccc163040 0x7fdccc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdccc0aff70 0x7fdccc159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 --2- 172.21.15.99:0/3999762403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdccc072620 0x7fdccc0d0c20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.907+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 >> 172.21.15.99:0/3999762403 conn(0x7fdccc075870 msgr2=0x7fdccc0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.908+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 shutdown_connections 2024-09-10T11:30:36.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.908+0000 7fdcc1ffb640 1 -- 172.21.15.99:0/3999762403 wait complete. 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.909+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 msgr2=0x7ffad006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.909+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 0x7ffad006e3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffae0006fd0 tx=0x7ffae0008040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 msgr2=0x7ffaf0146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c780 tx=0x7ffad800cc40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 shutdown_connections 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ffad006bf00 0x7ffad006e3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffaf00b5000 0x7ffaf00b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf0071a20 0x7ffaf0146380 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 --2- 172.21.15.99:0/470350209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf006c700 0x7ffaf014cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 >> 172.21.15.99:0/470350209 conn(0x7ffaf006bc10 msgr2=0x7ffaf0071310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 shutdown_connections 2024-09-10T11:30:36.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.910+0000 7ffae67fc640 1 -- 172.21.15.99:0/470350209 wait complete. 2024-09-10T11:30:36.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.914+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fae880ddd00 con 0x7fae88057830 2024-09-10T11:30:36.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.915+0000 7fae77fff640 1 -- 172.21.15.99:0/1558598898 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae800592d0 con 0x7fae88057830 2024-09-10T11:30:36.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.915+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbeec0afbf0 con 0x7fbeec0d0e50 2024-09-10T11:30:36.918 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2024-09-10T11:30:36.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.916+0000 7fbee27fc640 1 -- 172.21.15.99:0/2182230286 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbedc05d420 con 0x7fbeec0d0e50 2024-09-10T11:30:36.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.917+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fb70c0e4ee0 con 0x7fb70c0e7120 2024-09-10T11:30:36.919 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.918+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc6a4157cb0 con 0x7fc6a406c320 2024-09-10T11:30:36.920 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.918+0000 7fb6fbfff640 1 -- 172.21.15.99:0/1574669718 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb70805cb80 con 0x7fb70c0e7120 2024-09-10T11:30:36.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.918+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc7dc005580 con 0x7fc7f403a0d0 2024-09-10T11:30:36.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.918+0000 7fc69affd640 1 -- 172.21.15.99:0/3341419520 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc6a005ca90 con 0x7fc6a406c320 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stdout:55834574852 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fc7f97fa640 1 -- 172.21.15.99:0/1151729674 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7f005d040 con 0x7fc7f403a0d0 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 msgr2=0x7fae7c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 0x7fae7c06e3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fae9006b610 tx=0x7fae9006f030 comp rx=0 tx=0).stop 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 msgr2=0x7fae8804c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 0x7fae8804c540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae8803e210 tx=0x7fae8000c8e0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 shutdown_connections 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7c06bf00 0x7fae7c06e3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae880e28f0 0x7fae880c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.919+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae88057830 0x7fae8804c540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fae8ffff640 1 --2- 172.21.15.99:0/1558598898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae8803bc60 0x7fae8803d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 >> 172.21.15.99:0/1558598898 conn(0x7fae8803a520 msgr2=0x7fae88038b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 shutdown_connections 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fae8ffff640 1 -- 172.21.15.99:0/1558598898 wait complete. 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 msgr2=0x7fbed006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 0x7fbed006e3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbee400f1d0 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 msgr2=0x7fbeec1668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbedc009fd0 tx=0x7fbedc00ed40 comp rx=0 tx=0).stop 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 shutdown_connections 2024-09-10T11:30:36.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed006bf00 0x7fbed006e3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec158770 0x7fbeec162b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0d0e50 0x7fbeec1668b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 --2- 172.21.15.99:0/2182230286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec06c320 0x7fbeec14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 >> 172.21.15.99:0/2182230286 conn(0x7fbeec075870 msgr2=0x7fbeec156d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 shutdown_connections 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.920+0000 7fbef4118640 1 -- 172.21.15.99:0/2182230286 wait complete. 2024-09-10T11:30:36.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.921+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 msgr2=0x7fb70006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.921+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 0x7fb70006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb714074aa0 tx=0x7fb71408f040 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 msgr2=0x7fb70c0356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c550 tx=0x7fb70800ca10 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 shutdown_connections 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb70006be30 0x7fb70006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb70c0e7120 0x7fb70c0356a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb70c04eb60 0x7fb70c035160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 --2- 172.21.15.99:0/1574669718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb70c005a60 0x7fb70c171460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 >> 172.21.15.99:0/1574669718 conn(0x7fb70c036090 msgr2=0x7fb70c04f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 shutdown_connections 2024-09-10T11:30:36.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.922+0000 7fb718acf640 1 -- 172.21.15.99:0/1574669718 wait complete. 2024-09-10T11:30:36.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.923+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 msgr2=0x7fc7e006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 0x7fc7e006e4f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8033000 tx=0x7fc7e8008030 comp rx=0 tx=0).stop 2024-09-10T11:30:36.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 msgr2=0x7fc7f417cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000fb90 tx=0x7fc7f000d520 comp rx=0 tx=0).stop 2024-09-10T11:30:36.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 shutdown_connections 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7e006c030 0x7fc7e006e4f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f417d020 0x7fc7f40e28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f403a0d0 0x7fc7f417cae0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 --2- 172.21.15.99:0/1151729674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7ec003b40 0x7fc7f40dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 >> 172.21.15.99:0/1151729674 conn(0x7fc7f4036090 msgr2=0x7fc7f4050360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 msgr2=0x7fc68806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 shutdown_connections 2024-09-10T11:30:36.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc801e58640 1 -- 172.21.15.99:0/1151729674 wait complete. 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 0x7fc68806e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc694034000 tx=0x7fc69402e000 comp rx=0 tx=0).stop 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 msgr2=0x7fc6a415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c470 tx=0x7fc6a000c930 comp rx=0 tx=0).stop 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 shutdown_connections 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc68806be30 0x7fc68806e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6a4158770 0x7fc6a4147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6a40d0e50 0x7fc6a415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 --2- 172.21.15.99:0/3341419520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6a406c320 0x7fc6a415a270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:36.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.924+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 >> 172.21.15.99:0/3341419520 conn(0x7fc6a4075870 msgr2=0x7fc6a40cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:36.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.925+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 shutdown_connections 2024-09-10T11:30:36.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:36.925+0000 7fc6ac5f6640 1 -- 172.21.15.99:0/3341419520 wait complete. 2024-09-10T11:30:36.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-09-10T11:30:36.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-09-10T11:30:36.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-09-10T11:30:36.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2024-09-10T11:30:36.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-09-10T11:30:36.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-09-10T11:30:36.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-09-10T11:30:37.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T11:30:37.934 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T11:30:37.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T11:30:37.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T11:30:37.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T11:30:37.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T11:30:37.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T11:30:37.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T11:30:38.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.082+0000 7fcf127c4640 1 Processor -- start 2024-09-10T11:30:38.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.082+0000 7fcf127c4640 1 -- start start 2024-09-10T11:30:38.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.083+0000 7fcf127c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcf0c06c320 0x7fcf0c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.083+0000 7fcf127c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c0d0740 0x7fcf0c0d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.083+0000 7fcf127c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c15b280 con 0x7fcf0c072cd0 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.083+0000 7fcf127c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0d10c0 con 0x7fcf0c06c320 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.083+0000 7fcf127c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c15b560 con 0x7fcf0c0d0740 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.084+0000 7fcf10d61640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf0c072cd0 0x7fcf0c073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44942/0 (socket says 172.21.15.99:44942) 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.084+0000 7fcf10d61640 1 -- 172.21.15.99:0/502380614 learned_addr learned my addr 172.21.15.99:0/502380614 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.084+0000 7fcf0bfff640 1 --2- 172.21.15.99:0/502380614 >> v2:172.21.15.99:3300/0 conn(0x7fcf0c06c320 0x7fcf0c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.085+0000 7fcf0bfff640 1 -- 172.21.15.99:0/502380614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c0d0740 msgr2=0x7fcf0c0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.085+0000 7fcf0bfff640 1 --2- 172.21.15.99:0/502380614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c0d0740 0x7fcf0c0d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.085+0000 7fcf0bfff640 1 -- 172.21.15.99:0/502380614 >> v1:172.21.15.99:6789/0 conn(0x7fcf0c072cd0 legacy=0x7fcf0c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.085+0000 7fcf0bfff640 1 -- 172.21.15.99:0/502380614 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c15b840 con 0x7fcf0c06c320 2024-09-10T11:30:38.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.086+0000 7fcf0bfff640 1 --2- 172.21.15.99:0/502380614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c06c320 0x7fcf0c1586e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf000097a0 tx=0x7fcf0002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96a902040916857 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.087+0000 7fcf0affd640 1 -- 172.21.15.99:0/502380614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00037070 con 0x7fcf0c06c320 2024-09-10T11:30:38.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.087+0000 7fcf0affd640 1 -- 172.21.15.99:0/502380614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0002ad30 con 0x7fcf0c06c320 2024-09-10T11:30:38.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.088+0000 7fcf127c4640 1 -- 172.21.15.99:0/502380614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c06c320 msgr2=0x7fcf0c1586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.088+0000 7fcf127c4640 1 --2- 172.21.15.99:0/502380614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c06c320 0x7fcf0c1586e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf000097a0 tx=0x7fcf0002a0a0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 -- 172.21.15.99:0/502380614 shutdown_connections 2024-09-10T11:30:38.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 --2- 172.21.15.99:0/502380614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c0d0740 0x7fcf0c0d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 --2- 172.21.15.99:0/502380614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c06c320 0x7fcf0c1586e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 -- 172.21.15.99:0/502380614 >> 172.21.15.99:0/502380614 conn(0x7fcf0c075870 msgr2=0x7fcf0c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 -- 172.21.15.99:0/502380614 shutdown_connections 2024-09-10T11:30:38.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.089+0000 7fcf127c4640 1 -- 172.21.15.99:0/502380614 wait complete. 2024-09-10T11:30:38.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.091+0000 7fcf127c4640 1 Processor -- start 2024-09-10T11:30:38.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.092+0000 7fcf127c4640 1 -- start start 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.093+0000 7fcf127c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c06c320 0x7fcf0c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.093+0000 7fcf127c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.093+0000 7fcf127c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 0x7fcf0c0b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf0b7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf0b7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57018/0 (socket says 172.21.15.99:57018) 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf0b7fe640 1 -- 172.21.15.99:0/2469510423 learned_addr learned my addr 172.21.15.99:0/2469510423 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf10d61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 0x7fcf0c0b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c071620 con 0x7fcf0c0d0740 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0714a0 con 0x7fcf0c06c320 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.094+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c0717a0 con 0x7fcf0c072cd0 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.095+0000 7fcf0b7fe640 1 -- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c06c320 msgr2=0x7fcf0c155f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:30:38.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.095+0000 7fcf0b7fe640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c06c320 0x7fcf0c155f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.095+0000 7fcf0b7fe640 1 -- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 msgr2=0x7fcf0c0b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.095+0000 7fcf0b7fe640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 0x7fcf0c0b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.095+0000 7fcf0b7fe640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0aa920 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.096+0000 7fcf10d61640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 0x7fcf0c0b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.096+0000 7fcf0b7fe640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b9f0 tx=0x7fcf0400beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.096+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04019070 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.096+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c0aac00 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.096+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c153ba0 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.097+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0400cbd0 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.097+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04014440 con 0x7fcf0c072cd0 2024-09-10T11:30:38.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.098+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c0d0b60 con 0x7fcf0c072cd0 2024-09-10T11:30:38.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.101+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf0401c070 con 0x7fcf0c072cd0 2024-09-10T11:30:38.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.101+0000 7fcf08ff9640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 0x7fcef806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.101+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf0408ade0 con 0x7fcf0c072cd0 2024-09-10T11:30:38.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.102+0000 7fcf0bfff640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 0x7fcef806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.102+0000 7fcf0bfff640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 0x7fcef806e3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf00034000 tx=0x7fcf0002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.104+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf04059120 con 0x7fcf0c072cd0 2024-09-10T11:30:38.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.141+0000 7fce7505e640 1 Processor -- start 2024-09-10T11:30:38.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.141+0000 7fce7505e640 1 -- start start 2024-09-10T11:30:38.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.141+0000 7f795f084640 1 Processor -- start 2024-09-10T11:30:38.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.142+0000 7f9965c65640 1 Processor -- start 2024-09-10T11:30:38.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.141+0000 7f795f084640 1 -- start start 2024-09-10T11:30:38.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.142+0000 7f9965c65640 1 -- start start 2024-09-10T11:30:38.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7fce7505e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce7015eb80 0x7fce700d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7fce6ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce7015eb80 0x7fce700d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795f084640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 0x7f79500eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795f084640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7950053ea0 con 0x7f79500e28f0 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795f084640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7950057d80 con 0x7f795003bc60 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795f084640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f79500076c0 con 0x7f7950057830 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795e082640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f79500e28f0 0x7f7950053080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44950/0 (socket says 172.21.15.99:44950) 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795e082640 1 -- 172.21.15.99:0/1633486359 learned_addr learned my addr 172.21.15.99:0/1633486359 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f9965c65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99600c82e0 0x7f9960163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f9965c65640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9960163b40 con 0x7f99600c86c0 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f9965c65640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9960163e20 con 0x7f99600705d0 2024-09-10T11:30:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f9965c65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9960164100 con 0x7f99600c82e0 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995ffff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f99600705d0 0x7f9960070a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59232/0 (socket says 172.21.15.99:59232) 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995ffff640 1 -- 172.21.15.99:0/3157254169 learned_addr learned my addr 172.21.15.99:0/3157254169 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7fce6ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce7015eb80 0x7fce700d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54508/0 (socket says 172.21.15.99:54508) 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7fce6ed9d640 1 -- 172.21.15.99:0/1184684099 learned_addr learned my addr 172.21.15.99:0/1184684099 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce7015ff70 con 0x7fce700d5010 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.142+0000 7f40fff3a640 1 Processor -- start 2024-09-10T11:30:38.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.142+0000 7f40fff3a640 1 -- start start 2024-09-10T11:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f795e883640 1 --2- 172.21.15.99:0/1633486359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 0x7f79500eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995f7fe640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99600c82e0 0x7f9960163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3028169702 0 0) 0x7f9960163e20 con 0x7f99600705d0 2024-09-10T11:30:38.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f994c003520 con 0x7f99600705d0 2024-09-10T11:30:38.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fff3a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40f806c320 0x7f40f80b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fdcd6640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f40f81630c0 0x7f40f806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44982/0 (socket says 172.21.15.99:44982) 2024-09-10T11:30:38.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f795e883640 1 -- 172.21.15.99:0/1633486359 >> v1:172.21.15.196:6789/0 conn(0x7f795003bc60 legacy=0x7f7950053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f795e883640 1 -- 172.21.15.99:0/1633486359 >> v1:172.21.15.99:6789/0 conn(0x7f79500e28f0 legacy=0x7f7950053080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f795e883640 1 -- 172.21.15.99:0/1633486359 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7950058060 con 0x7f7950057830 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fd4d5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40f806c320 0x7f40f80b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fd4d5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40f806c320 0x7f40f80b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54522/0 (socket says 172.21.15.99:54522) 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fd4d5640 1 -- 172.21.15.99:0/1245835111 learned_addr learned my addr 172.21.15.99:0/1245835111 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f80cfe90 con 0x7f40f81630c0 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f80d0170 con 0x7f40f80cf990 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f40f80b5830 con 0x7f40f806c320 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce700d1620 con 0x7fce701626c0 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fce70162bc0 con 0x7fce7015eb80 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101945715 0 0) 0x7fce700d1620 con 0x7fce701626c0 2024-09-10T11:30:38.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce58003520 con 0x7fce701626c0 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f691d7ea640 1 Processor -- start 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.143+0000 7f691d7ea640 1 -- start start 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3352905388 0 0) 0x7f994c003520 con 0x7f99600705d0 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99600c82e0 msgr2=0x7f9960163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99600c82e0 0x7f9960163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 >> v1:172.21.15.99:6789/0 conn(0x7f99600c86c0 legacy=0x7f99600c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99601643e0 con 0x7f99600705d0 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f99580019f0 con 0x7f99600705d0 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 >> v1:172.21.15.196:6789/0 conn(0x7f99600705d0 legacy=0x7f9960070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0039b0 0x7f994c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f994c004800 0x7f994c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f994c0051c0 0x7f994c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9960164100 con 0x7f994c0039b0 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f994c00c090 con 0x7f994c0051c0 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f994c00c260 con 0x7f994c004800 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2217486531 0 0) 0x7fce58003520 con 0x7fce701626c0 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 >> v2:172.21.15.99:3300/0 conn(0x7fce7015eb80 msgr2=0x7fce700d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 --2- 172.21.15.99:0/1184684099 >> v2:172.21.15.99:3300/0 conn(0x7fce7015eb80 0x7fce700d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 >> v1:172.21.15.99:6789/0 conn(0x7fce700d5010 legacy=0x7fce700d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70162ea0 con 0x7fce701626c0 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fce60001690 con 0x7fce701626c0 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 >> v1:172.21.15.196:6789/0 conn(0x7fce701626c0 legacy=0x7fce700d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce58003a00 0x7fce58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fd139227640 1 Processor -- start 2024-09-10T11:30:38.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fd139227640 1 -- start start 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f691d7ea640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6910053080 0x7f69100534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f691d7ea640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6910005e40 con 0x7f6910057010 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f691d7ea640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6910006120 con 0x7f69100ea390 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f691d7ea640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6910038020 con 0x7f6910053080 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f6917fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f69100ea390 0x7f691003b880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59256/0 (socket says 172.21.15.99:59256) 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f6917fff640 1 -- 172.21.15.99:0/3829042098 learned_addr learned my addr 172.21.15.99:0/3829042098 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.144+0000 7f69177fe640 1 --2- 172.21.15.99:0/3829042098 >> v2:172.21.15.196:3300/0 conn(0x7f6910053080 0x7f69100534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f795e883640 1 --2- 172.21.15.99:0/1633486359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 0x7f79500eda40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79540097d0 tx=0x7f795402a0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d8339394e4beb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58005210 0x7fce5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce70162bc0 con 0x7fce58005210 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce5800c160 con 0x7fce58004800 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce5800c360 con 0x7fce58003a00 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f795d080640 1 -- 172.21.15.99:0/1633486359 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954037070 con 0x7f7950057830 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f795d080640 1 -- 172.21.15.99:0/1633486359 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795402ad30 con 0x7f7950057830 2024-09-10T11:30:38.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f795d080640 1 -- 172.21.15.99:0/1633486359 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954032660 con 0x7f7950057830 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995ffff640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f994c0051c0 0x7f994c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995effd640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f994c004800 0x7f994c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6ed9d640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce58003a00 0x7fce58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f40fd4d5640 1 -- 172.21.15.99:0/1245835111 >> v1:172.21.15.196:6789/0 conn(0x7f40f80cf990 legacy=0x7f40f806e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f40fd4d5640 1 -- 172.21.15.99:0/1245835111 >> v1:172.21.15.99:6789/0 conn(0x7f40f81630c0 legacy=0x7f40f806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f40fd4d5640 1 -- 172.21.15.99:0/1245835111 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f80d0450 con 0x7f40f806c320 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f84233bc640 1 Processor -- start 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f69177fe640 1 -- 172.21.15.99:0/3829042098 >> v1:172.21.15.196:6789/0 conn(0x7f69100ea390 legacy=0x7f691003b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f69177fe640 1 -- 172.21.15.99:0/3829042098 >> v1:172.21.15.99:6789/0 conn(0x7f6910057010 legacy=0x7f6910057420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.145+0000 7f69177fe640 1 -- 172.21.15.99:0/3829042098 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910006400 con 0x7f6910053080 2024-09-10T11:30:38.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f69177fe640 1 --2- 172.21.15.99:0/3829042098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100534f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6904009720 tx=0x7f6904029fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9363c1e527109dec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995f7fe640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0039b0 0x7f994c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fd139227640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd134072680 0x7fd134162620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995ffff640 1 -- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f994c004800 msgr2=0x7f994c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995ffff640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f994c004800 0x7f994c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995ffff640 1 -- 172.21.15.99:0/3157254169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0039b0 msgr2=0x7f994c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f995ffff640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0039b0 0x7f994c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f995ffff640 1 -- 172.21.15.99:0/3157254169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994c003520 con 0x7f994c0051c0 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f40fd4d5640 1 --2- 172.21.15.99:0/1245835111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f806c320 0x7f40f80b52f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4994aab0587cc75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fd139227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 0x7fd134164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fd139227640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd134073140 con 0x7fd13406d7a0 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fd139227640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd13415af80 con 0x7fd134072680 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fd139227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd134072b80 con 0x7fd134071760 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f795f084640 1 -- 172.21.15.99:0/1633486359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 msgr2=0x7f79500eda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f795f084640 1 --2- 172.21.15.99:0/1633486359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 0x7f79500eda40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79540097d0 tx=0x7f795402a0a0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f795f084640 1 -- 172.21.15.99:0/1633486359 shutdown_connections 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f795f084640 1 --2- 172.21.15.99:0/1633486359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950057830 0x7f79500eda40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f795f084640 1 -- 172.21.15.99:0/1633486359 >> 172.21.15.99:0/1633486359 conn(0x7f795003a520 msgr2=0x7f795003a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd132d9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd13406d7a0 0x7fd13406db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59284/0 (socket says 172.21.15.99:59284) 2024-09-10T11:30:38.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd132d9d640 1 -- 172.21.15.99:0/836647926 learned_addr learned my addr 172.21.15.99:0/836647926 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd13359e640 1 --2- 172.21.15.99:0/836647926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 0x7fd134164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7f84233bc640 1 -- start start 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f6916ffd640 1 -- 172.21.15.99:0/3829042098 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904037070 con 0x7f6910053080 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f6916ffd640 1 -- 172.21.15.99:0/3829042098 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690402acc0 con 0x7f6910053080 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f6916ffd640 1 -- 172.21.15.99:0/3829042098 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904032660 con 0x7f6910053080 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f40fccd4640 1 -- 172.21.15.99:0/1245835111 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8037070 con 0x7f40f806c320 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f40fccd4640 1 -- 172.21.15.99:0/1245835111 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e8032440 con 0x7f40f806c320 2024-09-10T11:30:38.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f40fccd4640 1 -- 172.21.15.99:0/1245835111 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8032810 con 0x7f40f806c320 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 -- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce58003a00 msgr2=0x7fce58003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce58003a00 0x7fce58003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 -- 172.21.15.99:0/1184684099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58005210 msgr2=0x7fce5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce58005210 0x7fce5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6e59c640 1 -- 172.21.15.99:0/1184684099 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce58003520 con 0x7fce58004800 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.146+0000 7fce6f59e640 1 -- 172.21.15.99:0/1184684099 reap_dead start 2024-09-10T11:30:38.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f795f084640 1 -- 172.21.15.99:0/1633486359 shutdown_connections 2024-09-10T11:30:38.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd131d9b640 1 -- 172.21.15.99:0/836647926 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191365139 0 0) 0x7fd134073140 con 0x7fd13406d7a0 2024-09-10T11:30:38.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd131d9b640 1 -- 172.21.15.99:0/836647926 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd11c003520 con 0x7fd13406d7a0 2024-09-10T11:30:38.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fce6e59c640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce58004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce70162af0 tx=0x7fce58005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ace9720770eb4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd13359e640 1 -- 172.21.15.99:0/836647926 >> v2:172.21.15.196:3300/0 conn(0x7fd134072680 msgr2=0x7fd134162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f995effd640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f994c004800 0x7f994c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd13359e640 1 --2- 172.21.15.99:0/836647926 >> v2:172.21.15.196:3300/0 conn(0x7fd134072680 0x7fd134162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7fd13359e640 1 -- 172.21.15.99:0/836647926 >> v1:172.21.15.196:6789/0 conn(0x7fd13406d7a0 legacy=0x7fd13406db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7fd13359e640 1 -- 172.21.15.99:0/836647926 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd134072e60 con 0x7fd134071760 2024-09-10T11:30:38.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f995ffff640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f994c0051c0 0x7f994c00b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9958002230 tx=0x7f994c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a99e7bb121362a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84233bc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f841c06e120 0x7f841c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84233bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c068dc0 0x7f841c15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84233bc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841c15bb10 con 0x7f841c072750 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84233bc640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f841c0b45f0 con 0x7f841c06e120 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84233bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f841c0b48d0 con 0x7f841c068dc0 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f691d7ea640 1 -- 172.21.15.99:0/3829042098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 msgr2=0x7f69100534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f691d7ea640 1 --2- 172.21.15.99:0/3829042098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100534f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6904009720 tx=0x7f6904029fb0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.147+0000 7f795f084640 1 -- 172.21.15.99:0/1633486359 wait complete. 2024-09-10T11:30:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84223ba640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f841c072750 0x7f841c072b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59296/0 (socket says 172.21.15.99:59296) 2024-09-10T11:30:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f84223ba640 1 -- 172.21.15.99:0/4021115893 learned_addr learned my addr 172.21.15.99:0/4021115893 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8421bb9640 1 --2- 172.21.15.99:0/4021115893 >> v2:172.21.15.99:3300/0 conn(0x7f841c06e120 0x7f841c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f806c320 msgr2=0x7f40f80b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f40fff3a640 1 --2- 172.21.15.99:0/1245835111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f806c320 0x7f40f80b52f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802a2a0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8422bbb640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c068dc0 0x7f841c15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958005610 con 0x7f994c0051c0 2024-09-10T11:30:38.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99580082f0 con 0x7f994c0051c0 2024-09-10T11:30:38.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f995e7fc640 1 -- 172.21.15.99:0/3157254169 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958008720 con 0x7f994c0051c0 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60007b90 con 0x7fce58004800 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60031620 con 0x7fce58004800 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7fce6dd9b640 1 -- 172.21.15.99:0/1184684099 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60031a10 con 0x7fce58004800 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8421bb9640 1 -- 172.21.15.99:0/4021115893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c068dc0 msgr2=0x7f841c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8421bb9640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c068dc0 0x7f841c15b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8421bb9640 1 -- 172.21.15.99:0/4021115893 >> v1:172.21.15.196:6789/0 conn(0x7f841c072750 legacy=0x7f841c072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f8421bb9640 1 -- 172.21.15.99:0/4021115893 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c0b4bb0 con 0x7f841c06e120 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f691d7ea640 1 -- 172.21.15.99:0/3829042098 shutdown_connections 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f691d7ea640 1 --2- 172.21.15.99:0/3829042098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100534f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f691d7ea640 1 -- 172.21.15.99:0/3829042098 >> 172.21.15.99:0/3829042098 conn(0x7f691003a520 msgr2=0x7f691003a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f691d7ea640 1 -- 172.21.15.99:0/3829042098 shutdown_connections 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f691d7ea640 1 -- 172.21.15.99:0/3829042098 wait complete. 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 shutdown_connections 2024-09-10T11:30:38.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f40fff3a640 1 --2- 172.21.15.99:0/1245835111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f806c320 0x7f40f80b52f0 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802a2a0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.148+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 >> 172.21.15.99:0/1245835111 conn(0x7f40f806cd20 msgr2=0x7f40f806d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 shutdown_connections 2024-09-10T11:30:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f40fff3a640 1 -- 172.21.15.99:0/1245835111 wait complete. 2024-09-10T11:30:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f795f084640 1 Processor -- start 2024-09-10T11:30:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 msgr2=0x7fce58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f9965c65640 1 -- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f994c0051c0 msgr2=0x7f994c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f9965c65640 1 --2- 172.21.15.99:0/3157254169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f994c0051c0 0x7f994c00b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9958002230 tx=0x7f994c005830 comp rx=0 tx=0).stop 2024-09-10T11:30:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fce7505e640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce58004c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fce70162af0 tx=0x7fce58005830 comp rx=0 tx=0).stop 2024-09-10T11:30:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f995ffff640 1 -- 172.21.15.99:0/3157254169 reap_dead start 2024-09-10T11:30:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fd13359e640 1 --2- 172.21.15.99:0/836647926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 0x7fd134164b70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009d90 tx=0x7fd12c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1820e2274d3bf052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fd131d9b640 1 -- 172.21.15.99:0/836647926 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c007740 con 0x7fd134071760 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fd131d9b640 1 -- 172.21.15.99:0/836647926 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c0078e0 con 0x7fd134071760 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7fd131d9b640 1 -- 172.21.15.99:0/836647926 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c007cb0 con 0x7fd134071760 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f795f084640 1 -- start start 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f9965c65640 1 -- 172.21.15.99:0/3157254169 shutdown_connections 2024-09-10T11:30:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.149+0000 7f9965c65640 1 -- 172.21.15.99:0/3157254169 >> 172.21.15.99:0/3157254169 conn(0x7f99600b4660 msgr2=0x7f99600b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f9965c65640 1 -- 172.21.15.99:0/3157254169 shutdown_connections 2024-09-10T11:30:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f8421bb9640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c0b5970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097a0 tx=0x7f840c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39770bc48797f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f84213b8640 1 -- 172.21.15.99:0/4021115893 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037070 con 0x7f841c06e120 2024-09-10T11:30:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 shutdown_connections 2024-09-10T11:30:38.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f84213b8640 1 -- 172.21.15.99:0/4021115893 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c032440 con 0x7f841c06e120 2024-09-10T11:30:38.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fce7505e640 1 --2- 172.21.15.99:0/1184684099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f84213b8640 1 -- 172.21.15.99:0/4021115893 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037070 con 0x7f841c06e120 2024-09-10T11:30:38.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 >> 172.21.15.99:0/1184684099 conn(0x7fce7008e680 msgr2=0x7fce7009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 0x7f79500e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fd139227640 1 -- 172.21.15.99:0/836647926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 msgr2=0x7fd134164b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fd139227640 1 --2- 172.21.15.99:0/836647926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 0x7fd134164b70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009d90 tx=0x7fd12c02a660 comp rx=0 tx=0).stop 2024-09-10T11:30:38.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 shutdown_connections 2024-09-10T11:30:38.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f9965c65640 1 -- 172.21.15.99:0/3157254169 wait complete. 2024-09-10T11:30:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7fce7505e640 1 -- 172.21.15.99:0/1184684099 wait complete. 2024-09-10T11:30:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7950057830 0x7f79500d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f40fff3a640 1 Processor -- start 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7fd139227640 1 -- 172.21.15.99:0/836647926 shutdown_connections 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79500568c0 con 0x7f7950057830 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7950006190 con 0x7f795003bc60 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795f084640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f79500e2cd0 con 0x7f79500e28f0 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f795e883640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57072/0 (socket says 172.21.15.99:57072) 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 -- 172.21.15.99:0/80173066 learned_addr learned my addr 172.21.15.99:0/80173066 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e082640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 0x7f79500e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795d881640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7950057830 0x7f79500d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.150+0000 7f691d7ea640 1 Processor -- start 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f691d7ea640 1 -- start start 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 -- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 msgr2=0x7f79500e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 0x7f79500e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7fd139227640 1 --2- 172.21.15.99:0/836647926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd134071760 0x7fd134164b70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 -- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7950057830 msgr2=0x7f79500d93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7950057830 0x7f79500d93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 -- 172.21.15.99:0/80173066 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500da550 con 0x7f79500e28f0 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e082640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 0x7f79500e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795e883640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f79540097d0 tx=0x7f79540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7fd139227640 1 --2- 172.21.15.99:0/836647926 >> v2:172.21.15.196:3300/0 conn(0x7fd134072680 0x7fd134162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954037040 con 0x7f79500e28f0 2024-09-10T11:30:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7fd139227640 1 -- 172.21.15.99:0/836647926 >> 172.21.15.99:0/836647926 conn(0x7fd1340b2790 msgr2=0x7fd1340b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7fd139227640 1 -- 172.21.15.99:0/836647926 shutdown_connections 2024-09-10T11:30:38.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f9965c65640 1 Processor -- start 2024-09-10T11:30:38.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f9965c65640 1 -- start start 2024-09-10T11:30:38.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7fd139227640 1 -- 172.21.15.99:0/836647926 wait complete. 2024-09-10T11:30:38.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f40fff3a640 1 -- start start 2024-09-10T11:30:38.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f81630c0 0x7f40f8153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fdcd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f8153ab0 0x7f40f80ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fdcd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57078/0 (socket says 172.21.15.99:57078) 2024-09-10T11:30:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6910057010 0x7f69100e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f6917fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fdcd6640 1 -- 172.21.15.99:0/1489139598 learned_addr learned my addr 172.21.15.99:0/1489139598 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80b0e90 con 0x7f40f8153ab0 2024-09-10T11:30:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40f80d0a10 con 0x7f40f81630c0 2024-09-10T11:30:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fff3a640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f40f806eaf0 con 0x7f40f80cf990 2024-09-10T11:30:38.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fe4d7640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f8153ab0 0x7f40f80ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f6917fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35504/0 (socket says 172.21.15.99:35504) 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f6917fff640 1 -- 172.21.15.99:0/290371380 learned_addr learned my addr 172.21.15.99:0/290371380 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 0x7f69100e9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f691003bf90 con 0x7f6910057010 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6910053a30 con 0x7f6910053080 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6910057c40 con 0x7f69100ea390 2024-09-10T11:30:38.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f691cfe9640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 0x7f69100e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f40fd4d5640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f81630c0 0x7f40f8153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 -- 172.21.15.99:0/4021115893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 msgr2=0x7f841c0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c0b5970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f840c0097a0 tx=0x7f840c02a180 comp rx=0 tx=0).stop 2024-09-10T11:30:38.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f9965c65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 0x7f996015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 -- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 msgr2=0x7f69100e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 0x7f69100e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 -- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6910057010 msgr2=0x7f69100e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6910057010 0x7f69100e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69100e64a0 con 0x7f6910053080 2024-09-10T11:30:38.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f691cfe9640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 0x7f69100e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7954004230 con 0x7f79500e28f0 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.151+0000 7f795f084640 1 -- 172.21.15.99:0/80173066 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79500d9f90 con 0x7f79500e28f0 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 -- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f81630c0 msgr2=0x7f40f8153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f81630c0 0x7f40f8153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 -- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f8153ab0 msgr2=0x7f40f80ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f8153ab0 0x7f40f80ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f80aa6a0 con 0x7f40f80cf990 2024-09-10T11:30:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7fce7505e640 1 Processor -- start 2024-09-10T11:30:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954003610 con 0x7f79500e28f0 2024-09-10T11:30:38.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.152+0000 7f795f084640 1 -- 172.21.15.99:0/80173066 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f795004d7c0 con 0x7f79500e28f0 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f40fdcd6640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c3c0 tx=0x7f40f400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 -- 172.21.15.99:0/4021115893 shutdown_connections 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c068dc0 0x7f841c15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 --2- 172.21.15.99:0/4021115893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c0b5970 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f84233bc640 1 -- 172.21.15.99:0/4021115893 >> 172.21.15.99:0/4021115893 conn(0x7f841c0d0b30 msgr2=0x7f841c0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7fce7505e640 1 -- start start 2024-09-10T11:30:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f6917fff640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c750 tx=0x7f690c00cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.153+0000 7f9965c65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 0x7f996015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c018070 con 0x7f6910053080 2024-09-10T11:30:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f9965c65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996015d2b0 0x7f99600ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f9965c65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9960163430 con 0x7f994c0051c0 2024-09-10T11:30:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c0072d0 con 0x7f6910053080 2024-09-10T11:30:38.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c0047e0 con 0x7f6910053080 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f9965c65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99600c8b20 con 0x7f9960160ca0 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f9965c65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f99601635b0 con 0x7f996015d2b0 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f995f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 0x7f996015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f995f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 0x7f996015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54576/0 (socket says 172.21.15.99:54576) 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f995f7fe640 1 -- 172.21.15.99:0/659703434 learned_addr learned my addr 172.21.15.99:0/659703434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f995effd640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 0x7f996015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f995ffff640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996015d2b0 0x7f99600ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691004fd50 con 0x7f6910053080 2024-09-10T11:30:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6910050310 con 0x7f6910053080 2024-09-10T11:30:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40074e0 con 0x7f40f80cf990 2024-09-10T11:30:38.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.154+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f4007680 con 0x7f40f80cf990 2024-09-10T11:30:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f995effd640 1 -- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996015d2b0 msgr2=0x7f99600ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 0x7fce700cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7950007a90 con 0x7f79500e28f0 2024-09-10T11:30:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fd139227640 1 Processor -- start 2024-09-10T11:30:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f84233bc640 1 -- 172.21.15.99:0/4021115893 shutdown_connections 2024-09-10T11:30:38.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40049e0 con 0x7f40f80cf990 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f40fff3a640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f80aa980 con 0x7f40f80cf990 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f40fff3a640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f80aaf40 con 0x7f40f80cf990 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fd139227640 1 -- start start 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f995effd640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996015d2b0 0x7f99600ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f995effd640 1 -- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 msgr2=0x7f996015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f995effd640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 0x7f996015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7f995effd640 1 -- 172.21.15.99:0/659703434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99600afb90 con 0x7f9960160ca0 2024-09-10T11:30:38.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6ed9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:35528/0 (socket says 172.21.15.99:35528) 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6e59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 0x7fce700cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6e59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 0x7fce700cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57118/0 (socket says 172.21.15.99:57118) 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54590/0 (socket says 172.21.15.99:54590) 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce6ed9d640 1 -- 172.21.15.99:0/2270774282 learned_addr learned my addr 172.21.15.99:0/2270774282 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce700d4900 con 0x7fce700ce180 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce70163460 con 0x7fce58004800 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.155+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce700d4a80 con 0x7fce700cdd00 2024-09-10T11:30:38.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 -- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 msgr2=0x7fce700cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 0x7fce700cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 -- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 msgr2=0x7fce700cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce700cec10 con 0x7fce58004800 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6f59e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f84233bc640 1 -- 172.21.15.99:0/4021115893 wait complete. 2024-09-10T11:30:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7fce6ed9d640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce64011a50 tx=0x7fce64011f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f995effd640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 0x7f996015cd70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f995000f7f0 tx=0x7f995000fcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f79540037b0 con 0x7f79500e28f0 2024-09-10T11:30:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f794effd640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 0x7f793c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79540ab910 con 0x7f79500e28f0 2024-09-10T11:30:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.156+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f690c004980 con 0x7f6910053080 2024-09-10T11:30:38.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f806d9d0 con 0x7f40f80cf990 2024-09-10T11:30:38.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd139227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9950018020 con 0x7f9960160ca0 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400fb90 con 0x7fce58004800 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6400fd30 con 0x7fce58004800 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64014660 con 0x7fce58004800 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce700ccf40 con 0x7fce58004800 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99500050d0 con 0x7f9960160ca0 2024-09-10T11:30:38.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99600afe70 con 0x7f9960160ca0 2024-09-10T11:30:38.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99600b0430 con 0x7f9960160ca0 2024-09-10T11:30:38.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd139227640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce700cd500 con 0x7fce58004800 2024-09-10T11:30:38.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd132d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd132d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57134/0 (socket says 172.21.15.99:57134) 2024-09-10T11:30:38.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd13259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f6914ff9640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 0x7f68fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd13259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54600/0 (socket says 172.21.15.99:54600) 2024-09-10T11:30:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7fd132d9d640 1 -- 172.21.15.99:0/1908229098 learned_addr learned my addr 172.21.15.99:0/1908229098 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.157+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f690c08a220 con 0x7f6910053080 2024-09-10T11:30:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd134072680 0x7fd134146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd13406fff0 con 0x7fd134071760 2024-09-10T11:30:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99500054a0 con 0x7f9960160ca0 2024-09-10T11:30:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1340b4890 con 0x7fd134072680 2024-09-10T11:30:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd134070170 con 0x7fd13406d7a0 2024-09-10T11:30:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fd13359e640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd134072680 0x7fd134146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944005290 con 0x7f9960160ca0 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce700d29b0 con 0x7fce58004800 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 -- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd134072680 msgr2=0x7fd134146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd134072680 0x7fd134146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 -- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 msgr2=0x7fd1340b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13415c7a0 con 0x7fd13406d7a0 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f69177fe640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 0x7f68fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691004fae0 con 0x7f6910053080 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f69177fe640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 0x7f68fc06e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6904034000 tx=0x7f690402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f84233bc640 1 Processor -- start 2024-09-10T11:30:38.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f84233bc640 1 -- start start 2024-09-10T11:30:38.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.158+0000 7f795e082640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 0x7f793c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f795e082640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 0x7f793c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f795802dd50 tx=0x7f7958069b40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd13259c640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:38.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fd132d9d640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c400 tx=0x7fd12400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f841c068dc0 0x7f841c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c072750 0x7f841c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0b5450 con 0x7f841c06e120 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f841c0b5170 con 0x7f841c068dc0 2024-09-10T11:30:38.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7f84233bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f841c070090 con 0x7f841c072750 2024-09-10T11:30:38.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.159+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce64012040 con 0x7fce58004800 2024-09-10T11:30:38.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54608/0 (socket says 172.21.15.99:54608) 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 -- 172.21.15.99:0/3863369932 learned_addr learned my addr 172.21.15.99:0/3863369932 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8422bbb640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c072750 0x7f841c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 -- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c072750 msgr2=0x7f841c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c072750 0x7f841c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 -- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f841c068dc0 msgr2=0x7f841c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f841c068dc0 0x7f841c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 -- 172.21.15.99:0/3863369932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c14ec80 con 0x7f841c06e120 2024-09-10T11:30:38.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f995000d8e0 con 0x7f9960160ca0 2024-09-10T11:30:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7fce577fe640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 0x7fce5c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce6408b4a0 con 0x7fce58004800 2024-09-10T11:30:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7fce6e59c640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 0x7fce5c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8421bb9640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a1c0 tx=0x7f840c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c041030 con 0x7f841c06e120 2024-09-10T11:30:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840c030980 con 0x7f841c06e120 2024-09-10T11:30:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c037070 con 0x7f841c06e120 2024-09-10T11:30:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7950007a90 con 0x7f79500e28f0 2024-09-10T11:30:38.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fce6e59c640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 0x7fce5c06e3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce60034000 tx=0x7fce6002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c151850 con 0x7f841c06e120 2024-09-10T11:30:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.160+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c0d07a0 con 0x7f841c06e120 2024-09-10T11:30:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7f9957fff640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 0x7f994006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f995008b6d0 con 0x7f9960160ca0 2024-09-10T11:30:38.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124017070 con 0x7fd13406d7a0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd124005ce0 con 0x7fd13406d7a0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124007310 con 0x7fd13406d7a0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13415ca80 con 0x7fd13406d7a0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1340b5c60 con 0x7fd13406d7a0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.161+0000 7f995f7fe640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 0x7f994006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.162+0000 7f995f7fe640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 0x7f994006e510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f996015df30 tx=0x7f9948008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0d1a00 con 0x7f841c06e120 2024-09-10T11:30:38.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd114005290 con 0x7fd13406d7a0 2024-09-10T11:30:38.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f840c044070 con 0x7f841c06e120 2024-09-10T11:30:38.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40f401a020 con 0x7f40f80cf990 2024-09-10T11:30:38.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f40ee7fc640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 0x7f40dc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40f408ad50 con 0x7f40f80cf990 2024-09-10T11:30:38.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.163+0000 7f8412ffd640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 0x7f840006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.164+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f840c0ab6e0 con 0x7f841c06e120 2024-09-10T11:30:38.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.164+0000 7f84223ba640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 0x7f840006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.164+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f690c058570 con 0x7f6910053080 2024-09-10T11:30:38.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.164+0000 7f40fd4d5640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 0x7f40dc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.164+0000 7f84223ba640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 0x7f840006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f841c152c20 tx=0x7f8418007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.165+0000 7f40fd4d5640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 0x7f40dc06e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40e8034000 tx=0x7f40e802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.166+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40f806d9d0 con 0x7f40f80cf990 2024-09-10T11:30:38.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.169+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9950022020 con 0x7f9960160ca0 2024-09-10T11:30:38.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.169+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd12401a070 con 0x7fd13406d7a0 2024-09-10T11:30:38.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.169+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f840c032e20 con 0x7f841c06e120 2024-09-10T11:30:38.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.169+0000 7fd11b7fe640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 0x7fd12006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.170+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd12408a540 con 0x7fd13406d7a0 2024-09-10T11:30:38.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.170+0000 7fd13259c640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 0x7fd12006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.170+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd124021020 con 0x7fd13406d7a0 2024-09-10T11:30:38.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.171+0000 7fd13259c640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 0x7fd12006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd128009db0 tx=0x7fd128009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.173+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce64019030 con 0x7fce58004800 2024-09-10T11:30:38.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.478+0000 7fcf127c4640 1 -- 172.21.15.99:0/2469510423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcf0c15ac40 con 0x7fcf0c072cd0 2024-09-10T11:30:38.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.479+0000 7fcf08ff9640 1 -- 172.21.15.99:0/2469510423 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf0405d420 con 0x7fcf0c072cd0 2024-09-10T11:30:38.481 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.482+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 msgr2=0x7fcef806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.482+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 0x7fcef806e3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf00034000 tx=0x7fcf0002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 msgr2=0x7fcf0c14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b9f0 tx=0x7fcf0400beb0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 shutdown_connections 2024-09-10T11:30:38.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcef806bf30 0x7fcef806e3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf0c0d0740 0x7fcf0c0b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0c072cd0 0x7fcf0c14fc10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 --2- 172.21.15.99:0/2469510423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf0c06c320 0x7fcf0c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.483+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 >> 172.21.15.99:0/2469510423 conn(0x7fcf0c075870 msgr2=0x7fcf0c15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.484+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 shutdown_connections 2024-09-10T11:30:38.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.484+0000 7fcef27fc640 1 -- 172.21.15.99:0/2469510423 wait complete. 2024-09-10T11:30:38.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-09-10T11:30:38.502 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.522+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f69100ea800 con 0x7f6910053080 2024-09-10T11:30:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.523+0000 7f6914ff9640 1 -- 172.21.15.99:0/290371380 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f690c05c870 con 0x7f6910053080 2024-09-10T11:30:38.524 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 msgr2=0x7f68fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 0x7f68fc06e320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6904034000 tx=0x7f690402c000 comp rx=0 tx=0).stop 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 msgr2=0x7f69100e5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f690c00c750 tx=0x7f690c00cc10 comp rx=0 tx=0).stop 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 shutdown_connections 2024-09-10T11:30:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f68fc06be60 0x7f68fc06e320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69100ea390 0x7f69100e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6910057010 0x7f69100e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 --2- 172.21.15.99:0/290371380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6910053080 0x7f69100e5f60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 >> 172.21.15.99:0/290371380 conn(0x7f691003a520 msgr2=0x7f6910038d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 shutdown_connections 2024-09-10T11:30:38.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.525+0000 7f691d7ea640 1 -- 172.21.15.99:0/290371380 wait complete. 2024-09-10T11:30:38.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-09-10T11:30:38.538 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.561+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7950053080 con 0x7f79500e28f0 2024-09-10T11:30:38.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.561+0000 7f794effd640 1 -- 172.21.15.99:0/80173066 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7954079bc0 con 0x7f79500e28f0 2024-09-10T11:30:38.563 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:38.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.563+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f40f8153ec0 con 0x7f40f80cf990 2024-09-10T11:30:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.563+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9944005850 con 0x7f9960160ca0 2024-09-10T11:30:38.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.563+0000 7f40ee7fc640 1 -- 172.21.15.99:0/1489139598 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40f4021070 con 0x7f40f80cf990 2024-09-10T11:30:38.566 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:38.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f9957fff640 1 -- 172.21.15.99:0/659703434 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f995005a0f0 con 0x7f9960160ca0 2024-09-10T11:30:38.566 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 msgr2=0x7f793c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 0x7f793c06e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f795802dd50 tx=0x7f7958069b40 comp rx=0 tx=0).stop 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 msgr2=0x7f79500d98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f79540097d0 tx=0x7f79540046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 shutdown_connections 2024-09-10T11:30:38.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793c06be30 0x7f793c06e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f79500e28f0 0x7f79500d98f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7950057830 0x7f79500d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 --2- 172.21.15.99:0/80173066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f795003bc60 0x7f79500e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 >> 172.21.15.99:0/80173066 conn(0x7f795003a520 msgr2=0x7f7950053290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 shutdown_connections 2024-09-10T11:30:38.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.564+0000 7f794cff9640 1 -- 172.21.15.99:0/80173066 wait complete. 2024-09-10T11:30:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.565+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd114005850 con 0x7fd13406d7a0 2024-09-10T11:30:38.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7fd11b7fe640 1 -- 172.21.15.99:0/1908229098 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd124058f60 con 0x7fd13406d7a0 2024-09-10T11:30:38.570 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 msgr2=0x7f40dc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 0x7f40dc06e3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40e8034000 tx=0x7f40e802e000 comp rx=0 tx=0).stop 2024-09-10T11:30:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 msgr2=0x7f40f81ed9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f40f400c3c0 tx=0x7f40f400c880 comp rx=0 tx=0).stop 2024-09-10T11:30:38.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fce70161a10 con 0x7fce58004800 2024-09-10T11:30:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 shutdown_connections 2024-09-10T11:30:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f40dc06bf00 0x7f40dc06e3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40f8153ab0 0x7f40f80ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f81630c0 0x7f40f8153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 --2- 172.21.15.99:0/1489139598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f80cf990 0x7f40f81ed9b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 >> 172.21.15.99:0/1489139598 conn(0x7f40f806cd20 msgr2=0x7f40f806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.566+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 shutdown_connections 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f40dbfff640 1 -- 172.21.15.99:0/1489139598 wait complete. 2024-09-10T11:30:38.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7fce577fe640 1 -- 172.21.15.99:0/2270774282 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce640597e0 con 0x7fce58004800 2024-09-10T11:30:38.574 INFO:teuthology.orchestra.run.smithi099.stdout:55834574854 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 msgr2=0x7f994006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 0x7f994006e510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f996015df30 tx=0x7f9948008040 comp rx=0 tx=0).stop 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 msgr2=0x7f996015cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 0x7f996015cd70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f995000f7f0 tx=0x7f995000fcb0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 shutdown_connections 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994006c050 0x7f994006e510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996015d2b0 0x7f99600ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9960160ca0 0x7f996015cd70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 --2- 172.21.15.99:0/659703434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c0051c0 0x7f996015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.567+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 >> 172.21.15.99:0/659703434 conn(0x7f99600b4660 msgr2=0x7f996015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.568+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 shutdown_connections 2024-09-10T11:30:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.568+0000 7f9965c65640 1 -- 172.21.15.99:0/659703434 wait complete. 2024-09-10T11:30:38.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 msgr2=0x7fd12006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 0x7fd12006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd128009db0 tx=0x7fd128009110 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 msgr2=0x7fd13414cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c400 tx=0x7fd12400c800 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 shutdown_connections 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd12006bf00 0x7fd12006e3c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd134072680 0x7fd134146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd134071760 0x7fd1340b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 --2- 172.21.15.99:0/1908229098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd13406d7a0 0x7fd13414cc30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 >> 172.21.15.99:0/1908229098 conn(0x7fd1340b2790 msgr2=0x7fd1340b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 shutdown_connections 2024-09-10T11:30:38.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fd139227640 1 -- 172.21.15.99:0/1908229098 wait complete. 2024-09-10T11:30:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 msgr2=0x7fce5c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 0x7fce5c06e3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fce60034000 tx=0x7fce6002f000 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 msgr2=0x7fce701530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce64011a50 tx=0x7fce64011f10 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 shutdown_connections 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce5c06bf30 0x7fce5c06e3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce700ce180 0x7fce700cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.569+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce700cdd00 0x7fce700cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.570+0000 7fce7505e640 1 --2- 172.21.15.99:0/2270774282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce58004800 0x7fce701530d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.570+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 >> 172.21.15.99:0/2270774282 conn(0x7fce7008e680 msgr2=0x7fce700d1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.570+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 shutdown_connections 2024-09-10T11:30:38.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.570+0000 7fce7505e640 1 -- 172.21.15.99:0/2270774282 wait complete. 2024-09-10T11:30:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.572+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f841c152570 con 0x7f841c06e120 2024-09-10T11:30:38.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.572+0000 7f8412ffd640 1 -- 172.21.15.99:0/3863369932 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f840c079ff0 con 0x7f841c06e120 2024-09-10T11:30:38.580 INFO:teuthology.orchestra.run.smithi099.stdout:55834574853 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 msgr2=0x7f840006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 0x7f840006e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f841c152c20 tx=0x7f8418007a40 comp rx=0 tx=0).stop 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 msgr2=0x7f841c152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f840c02a1c0 tx=0x7f840c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 shutdown_connections 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f840006be30 0x7f840006e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f841c072750 0x7f841c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f841c06e120 0x7f841c152030 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 --2- 172.21.15.99:0/3863369932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f841c068dc0 0x7f841c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 >> 172.21.15.99:0/3863369932 conn(0x7f841c0d0b30 msgr2=0x7f841c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 shutdown_connections 2024-09-10T11:30:38.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.576+0000 7f84233bc640 1 -- 172.21.15.99:0/3863369932 wait complete. 2024-09-10T11:30:38.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-09-10T11:30:38.587 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-09-10T11:30:38.588 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-09-10T11:30:38.589 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-09-10T11:30:38.590 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-09-10T11:30:38.592 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-09-10T11:30:38.595 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:38.595 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:30:38.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.730+0000 7f30f1159640 1 Processor -- start 2024-09-10T11:30:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.730+0000 7f30f1159640 1 -- start start 2024-09-10T11:30:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30f1159640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 0x7f30ec072620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30f1159640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ec15ccc0 con 0x7f30ec1590c0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30f1159640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30ec15c470 con 0x7f30ec1594a0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30f1159640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec15c700 con 0x7f30ec0733c0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30ead9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f30ec1590c0 0x7f30ec15bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45002/0 (socket says 172.21.15.99:45002) 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30ead9d640 1 -- 172.21.15.99:0/2408901211 learned_addr learned my addr 172.21.15.99:0/2408901211 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.731+0000 7f30ea59c640 1 --2- 172.21.15.99:0/2408901211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 0x7f30ec072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220825174 0 0) 0x7f30ec15ccc0 con 0x7f30ec1590c0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30d8003520 con 0x7f30ec1590c0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981157723 0 0) 0x7f30ec15c470 con 0x7f30ec1594a0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f30ec15ccc0 con 0x7f30ec1594a0 2024-09-10T11:30:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4230224429 0 0) 0x7f30d8003520 con 0x7f30ec1590c0 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30ea59c640 1 -- 172.21.15.99:0/2408901211 >> v1:172.21.15.196:6789/0 conn(0x7f30ec1594a0 legacy=0x7f30ec0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30ea59c640 1 -- 172.21.15.99:0/2408901211 >> v1:172.21.15.99:6789/0 conn(0x7f30ec1590c0 legacy=0x7f30ec15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30ea59c640 1 -- 172.21.15.99:0/2408901211 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec15c9e0 con 0x7f30ec0733c0 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.732+0000 7f30ea59c640 1 --2- 172.21.15.99:0/2408901211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 0x7f30ec072620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30d40097d0 tx=0x7f30d402a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d406ea7ad7cb87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4037070 con 0x7f30ec0733c0 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d402ab10 con 0x7f30ec0733c0 2024-09-10T11:30:38.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30e9d9b640 1 -- 172.21.15.99:0/2408901211 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0019f0 con 0x7f30ec0733c0 2024-09-10T11:30:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 -- 172.21.15.99:0/2408901211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 msgr2=0x7f30ec072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 --2- 172.21.15.99:0/2408901211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 0x7f30ec072620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30d40097d0 tx=0x7f30d402a080 comp rx=0 tx=0).stop 2024-09-10T11:30:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 -- 172.21.15.99:0/2408901211 shutdown_connections 2024-09-10T11:30:38.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 --2- 172.21.15.99:0/2408901211 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec0733c0 0x7f30ec072620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 -- 172.21.15.99:0/2408901211 >> 172.21.15.99:0/2408901211 conn(0x7f30ec0d1900 msgr2=0x7f30ec0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 -- 172.21.15.99:0/2408901211 shutdown_connections 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.733+0000 7f30f1159640 1 -- 172.21.15.99:0/2408901211 wait complete. 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 Processor -- start 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 -- start start 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 0x7f30ec155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec1594a0 0x7f30ec1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0d0df0 con 0x7f30ec0733c0 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec0cf8e0 con 0x7f30ec1590c0 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30f1159640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30ec0d0f70 con 0x7f30ec1594a0 2024-09-10T11:30:38.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.735+0000 7f30ead9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54614/0 (socket says 172.21.15.99:54614) 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 -- 172.21.15.99:0/729244414 learned_addr learned my addr 172.21.15.99:0/729244414 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30eb59e640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec1594a0 0x7f30ec1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ea59c640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 0x7f30ec155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 -- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec1594a0 msgr2=0x7f30ec1577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec1594a0 0x7f30ec1577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 -- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 msgr2=0x7f30ec155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 0x7f30ec155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 -- 172.21.15.99:0/729244414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec159cc0 con 0x7f30ec0733c0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ea59c640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 0x7f30ec155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30ead9d640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30dc001f90 tx=0x7f30dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc017070 con 0x7f30ec0733c0 2024-09-10T11:30:38.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30dc012400 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0127f0 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec159fa0 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.736+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec15a560 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.737+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec0d16a0 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.738+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30dc01a070 con 0x7f30ec0733c0 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.738+0000 7f30e37fe640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 0x7f30cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:38.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.738+0000 7f30ea59c640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 0x7f30cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.738+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30dc08ad70 con 0x7f30ec0733c0 2024-09-10T11:30:38.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.738+0000 7f30ea59c640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 0x7f30cc06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30d4034000 tx=0x7f30d402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.742+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f30dc059220 con 0x7f30ec0733c0 2024-09-10T11:30:38.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.906+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30ec072620 con 0x7f30cc06be30 2024-09-10T11:30:38.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.912+0000 7f30e37fe640 1 -- 172.21.15.99:0/729244414 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247485 (secure 0 0 0) 0x7f30ec072620 con 0x7f30cc06be30 2024-09-10T11:30:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:38.919 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 msgr2=0x7f30cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 0x7f30cc06e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f30d4034000 tx=0x7f30d402e000 comp rx=0 tx=0).stop 2024-09-10T11:30:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 msgr2=0x7f30ec15ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f30dc001f90 tx=0x7f30dc00cf90 comp rx=0 tx=0).stop 2024-09-10T11:30:38.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 shutdown_connections 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f30cc06be30 0x7f30cc06e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30ec1594a0 0x7f30ec1577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30ec1590c0 0x7f30ec155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 --2- 172.21.15.99:0/729244414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30ec0733c0 0x7f30ec15ad60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 >> 172.21.15.99:0/729244414 conn(0x7f30ec0d1900 msgr2=0x7f30ec158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 shutdown_connections 2024-09-10T11:30:38.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:38.915+0000 7f30f1159640 1 -- 172.21.15.99:0/729244414 wait complete. 2024-09-10T11:30:38.928 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-10T11:30:36.908656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.554307"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370941+0000","last_change":"2024-09-10T11:30:27.366179+0000","last_active":"2024-09-10T11:30:28.370941+0000","last_peered":"2024-09-10T11:30:28.370941+0000","last_clean":"2024-09-10T11:30:28.370941+0000","last_became_active":"2024-09-10T11:30:27.365953+0000","last_became_peered":"2024-09-10T11:30:27.365953+0000","last_unstale":"2024-09-10T11:30:28.370941+0000","last_undegraded":"2024-09-10T11:30:28.370941+0000","last_fullsized":"2024-09-10T11:30:28.370941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:09.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359715+0000","last_change":"2024-09-10T11:30:27.368422+0000","last_active":"2024-09-10T11:30:28.359715+0000","last_peered":"2024-09-10T11:30:28.359715+0000","last_clean":"2024-09-10T11:30:28.359715+0000","last_became_active":"2024-09-10T11:30:27.366676+0000","last_became_peered":"2024-09-10T11:30:27.366676+0000","last_unstale":"2024-09-10T11:30:28.359715+0000","last_undegraded":"2024-09-10T11:30:28.359715+0000","last_fullsized":"2024-09-10T11:30:28.359715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.234005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359802+0000","last_change":"2024-09-10T11:30:27.370589+0000","last_active":"2024-09-10T11:30:28.359802+0000","last_peered":"2024-09-10T11:30:28.359802+0000","last_clean":"2024-09-10T11:30:28.359802+0000","last_became_active":"2024-09-10T11:30:27.369527+0000","last_became_peered":"2024-09-10T11:30:27.369527+0000","last_unstale":"2024-09-10T11:30:28.359802+0000","last_undegraded":"2024-09-10T11:30:28.359802+0000","last_fullsized":"2024-09-10T11:30:28.359802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:28.644939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659543+0000","last_change":"2024-09-10T11:30:27.368869+0000","last_active":"2024-09-10T11:30:28.659543+0000","last_peered":"2024-09-10T11:30:28.659543+0000","last_clean":"2024-09-10T11:30:28.659543+0000","last_became_active":"2024-09-10T11:30:27.367497+0000","last_became_peered":"2024-09-10T11:30:27.367497+0000","last_unstale":"2024-09-10T11:30:28.659543+0000","last_undegraded":"2024-09-10T11:30:28.659543+0000","last_fullsized":"2024-09-10T11:30:28.659543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:11.740953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371428+0000","last_change":"2024-09-10T11:30:27.372181+0000","last_active":"2024-09-10T11:30:28.371428+0000","last_peered":"2024-09-10T11:30:28.371428+0000","last_clean":"2024-09-10T11:30:28.371428+0000","last_became_active":"2024-09-10T11:30:27.371972+0000","last_became_peered":"2024-09-10T11:30:27.371972+0000","last_unstale":"2024-09-10T11:30:28.371428+0000","last_undegraded":"2024-09-10T11:30:28.371428+0000","last_fullsized":"2024-09-10T11:30:28.371428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:52.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659909+0000","last_change":"2024-09-10T11:30:27.368458+0000","last_active":"2024-09-10T11:30:28.659909+0000","last_peered":"2024-09-10T11:30:28.659909+0000","last_clean":"2024-09-10T11:30:28.659909+0000","last_became_active":"2024-09-10T11:30:27.368163+0000","last_became_peered":"2024-09-10T11:30:27.368163+0000","last_unstale":"2024-09-10T11:30:28.659909+0000","last_undegraded":"2024-09-10T11:30:28.659909+0000","last_fullsized":"2024-09-10T11:30:28.659909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:17.388777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371439+0000","last_change":"2024-09-10T11:30:27.370692+0000","last_active":"2024-09-10T11:30:28.371439+0000","last_peered":"2024-09-10T11:30:28.371439+0000","last_clean":"2024-09-10T11:30:28.371439+0000","last_became_active":"2024-09-10T11:30:27.370323+0000","last_became_peered":"2024-09-10T11:30:27.370323+0000","last_unstale":"2024-09-10T11:30:28.371439+0000","last_undegraded":"2024-09-10T11:30:28.371439+0000","last_fullsized":"2024-09-10T11:30:28.371439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:30.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359963+0000","last_change":"2024-09-10T11:30:27.371135+0000","last_active":"2024-09-10T11:30:28.359963+0000","last_peered":"2024-09-10T11:30:28.359963+0000","last_clean":"2024-09-10T11:30:28.359963+0000","last_became_active":"2024-09-10T11:30:27.370855+0000","last_became_peered":"2024-09-10T11:30:27.370855+0000","last_unstale":"2024-09-10T11:30:28.359963+0000","last_undegraded":"2024-09-10T11:30:28.359963+0000","last_fullsized":"2024-09-10T11:30:28.359963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:08.475065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356981+0000","last_change":"2024-09-10T11:30:27.368622+0000","last_active":"2024-09-10T11:30:28.356981+0000","last_peered":"2024-09-10T11:30:28.356981+0000","last_clean":"2024-09-10T11:30:28.356981+0000","last_became_active":"2024-09-10T11:30:27.368043+0000","last_became_peered":"2024-09-10T11:30:27.368043+0000","last_unstale":"2024-09-10T11:30:28.356981+0000","last_undegraded":"2024-09-10T11:30:28.356981+0000","last_fullsized":"2024-09-10T11:30:28.356981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:11.979652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371169+0000","last_change":"2024-09-10T11:30:27.370344+0000","last_active":"2024-09-10T11:30:28.371169+0000","last_peered":"2024-09-10T11:30:28.371169+0000","last_clean":"2024-09-10T11:30:28.371169+0000","last_became_active":"2024-09-10T11:30:27.370093+0000","last_became_peered":"2024-09-10T11:30:27.370093+0000","last_unstale":"2024-09-10T11:30:28.371169+0000","last_undegraded":"2024-09-10T11:30:28.371169+0000","last_fullsized":"2024-09-10T11:30:28.371169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:55.994388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371076+0000","last_change":"2024-09-10T11:30:27.364571+0000","last_active":"2024-09-10T11:30:28.371076+0000","last_peered":"2024-09-10T11:30:28.371076+0000","last_clean":"2024-09-10T11:30:28.371076+0000","last_became_active":"2024-09-10T11:30:27.364329+0000","last_became_peered":"2024-09-10T11:30:27.364329+0000","last_unstale":"2024-09-10T11:30:28.371076+0000","last_undegraded":"2024-09-10T11:30:28.371076+0000","last_fullsized":"2024-09-10T11:30:28.371076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:56.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371023+0000","last_change":"2024-09-10T11:30:27.362816+0000","last_active":"2024-09-10T11:30:28.371023+0000","last_peered":"2024-09-10T11:30:28.371023+0000","last_clean":"2024-09-10T11:30:28.371023+0000","last_became_active":"2024-09-10T11:30:27.362165+0000","last_became_peered":"2024-09-10T11:30:27.362165+0000","last_unstale":"2024-09-10T11:30:28.371023+0000","last_undegraded":"2024-09-10T11:30:28.371023+0000","last_fullsized":"2024-09-10T11:30:28.371023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:13.704596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356572+0000","last_change":"2024-09-10T11:30:27.361112+0000","last_active":"2024-09-10T11:30:28.356572+0000","last_peered":"2024-09-10T11:30:28.356572+0000","last_clean":"2024-09-10T11:30:28.356572+0000","last_became_active":"2024-09-10T11:30:27.360081+0000","last_became_peered":"2024-09-10T11:30:27.360081+0000","last_unstale":"2024-09-10T11:30:28.356572+0000","last_undegraded":"2024-09-10T11:30:28.356572+0000","last_fullsized":"2024-09-10T11:30:28.356572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:25.075435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356459+0000","last_change":"2024-09-10T11:30:27.360154+0000","last_active":"2024-09-10T11:30:28.356459+0000","last_peered":"2024-09-10T11:30:28.356459+0000","last_clean":"2024-09-10T11:30:28.356459+0000","last_became_active":"2024-09-10T11:30:27.359341+0000","last_became_peered":"2024-09-10T11:30:27.359341+0000","last_unstale":"2024-09-10T11:30:28.356459+0000","last_undegraded":"2024-09-10T11:30:28.356459+0000","last_fullsized":"2024-09-10T11:30:28.356459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.266260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359988+0000","last_change":"2024-09-10T11:30:27.371329+0000","last_active":"2024-09-10T11:30:28.359988+0000","last_peered":"2024-09-10T11:30:28.359988+0000","last_clean":"2024-09-10T11:30:28.359988+0000","last_became_active":"2024-09-10T11:30:27.370948+0000","last_became_peered":"2024-09-10T11:30:27.370948+0000","last_unstale":"2024-09-10T11:30:28.359988+0000","last_undegraded":"2024-09-10T11:30:28.359988+0000","last_fullsized":"2024-09-10T11:30:28.359988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.619700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659901+0000","last_change":"2024-09-10T11:30:27.368565+0000","last_active":"2024-09-10T11:30:28.659901+0000","last_peered":"2024-09-10T11:30:28.659901+0000","last_clean":"2024-09-10T11:30:28.659901+0000","last_became_active":"2024-09-10T11:30:27.368309+0000","last_became_peered":"2024-09-10T11:30:27.368309+0000","last_unstale":"2024-09-10T11:30:28.659901+0000","last_undegraded":"2024-09-10T11:30:28.659901+0000","last_fullsized":"2024-09-10T11:30:28.659901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:33.008256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659831+0000","last_change":"2024-09-10T11:30:27.371513+0000","last_active":"2024-09-10T11:30:28.659831+0000","last_peered":"2024-09-10T11:30:28.659831+0000","last_clean":"2024-09-10T11:30:28.659831+0000","last_became_active":"2024-09-10T11:30:27.371229+0000","last_became_peered":"2024-09-10T11:30:27.371229+0000","last_unstale":"2024-09-10T11:30:28.659831+0000","last_undegraded":"2024-09-10T11:30:28.659831+0000","last_fullsized":"2024-09-10T11:30:28.659831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:28.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359314+0000","last_change":"2024-09-10T11:30:27.366960+0000","last_active":"2024-09-10T11:30:28.359314+0000","last_peered":"2024-09-10T11:30:28.359314+0000","last_clean":"2024-09-10T11:30:28.359314+0000","last_became_active":"2024-09-10T11:30:27.366744+0000","last_became_peered":"2024-09-10T11:30:27.366744+0000","last_unstale":"2024-09-10T11:30:28.359314+0000","last_undegraded":"2024-09-10T11:30:28.359314+0000","last_fullsized":"2024-09-10T11:30:28.359314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:30.599189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660402+0000","last_change":"2024-09-10T11:30:27.369325+0000","last_active":"2024-09-10T11:30:28.660402+0000","last_peered":"2024-09-10T11:30:28.660402+0000","last_clean":"2024-09-10T11:30:28.660402+0000","last_became_active":"2024-09-10T11:30:27.368956+0000","last_became_peered":"2024-09-10T11:30:27.368956+0000","last_unstale":"2024-09-10T11:30:28.660402+0000","last_undegraded":"2024-09-10T11:30:28.660402+0000","last_fullsized":"2024-09-10T11:30:28.660402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.633447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371252+0000","last_change":"2024-09-10T11:30:27.360270+0000","last_active":"2024-09-10T11:30:28.371252+0000","last_peered":"2024-09-10T11:30:28.371252+0000","last_clean":"2024-09-10T11:30:28.371252+0000","last_became_active":"2024-09-10T11:30:27.360000+0000","last_became_peered":"2024-09-10T11:30:27.360000+0000","last_unstale":"2024-09-10T11:30:28.371252+0000","last_undegraded":"2024-09-10T11:30:28.371252+0000","last_fullsized":"2024-09-10T11:30:28.371252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:00.874011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357411+0000","last_change":"2024-09-10T11:30:27.369424+0000","last_active":"2024-09-10T11:30:28.357411+0000","last_peered":"2024-09-10T11:30:28.357411+0000","last_clean":"2024-09-10T11:30:28.357411+0000","last_became_active":"2024-09-10T11:30:27.367954+0000","last_became_peered":"2024-09-10T11:30:27.367954+0000","last_unstale":"2024-09-10T11:30:28.357411+0000","last_undegraded":"2024-09-10T11:30:28.357411+0000","last_fullsized":"2024-09-10T11:30:28.357411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:15.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356963+0000","last_change":"2024-09-10T11:30:27.364058+0000","last_active":"2024-09-10T11:30:28.356963+0000","last_peered":"2024-09-10T11:30:28.356963+0000","last_clean":"2024-09-10T11:30:28.356963+0000","last_became_active":"2024-09-10T11:30:27.363699+0000","last_became_peered":"2024-09-10T11:30:27.363699+0000","last_unstale":"2024-09-10T11:30:28.356963+0000","last_undegraded":"2024-09-10T11:30:28.356963+0000","last_fullsized":"2024-09-10T11:30:28.356963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:36.749159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360157+0000","last_change":"2024-09-10T11:30:27.372337+0000","last_active":"2024-09-10T11:30:28.360157+0000","last_peered":"2024-09-10T11:30:28.360157+0000","last_clean":"2024-09-10T11:30:28.360157+0000","last_became_active":"2024-09-10T11:30:27.372074+0000","last_became_peered":"2024-09-10T11:30:27.372074+0000","last_unstale":"2024-09-10T11:30:28.360157+0000","last_undegraded":"2024-09-10T11:30:28.360157+0000","last_fullsized":"2024-09-10T11:30:28.360157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:39.707264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659804+0000","last_change":"2024-09-10T11:30:27.372831+0000","last_active":"2024-09-10T11:30:28.659804+0000","last_peered":"2024-09-10T11:30:28.659804+0000","last_clean":"2024-09-10T11:30:28.659804+0000","last_became_active":"2024-09-10T11:30:27.371298+0000","last_became_peered":"2024-09-10T11:30:27.371298+0000","last_unstale":"2024-09-10T11:30:28.659804+0000","last_undegraded":"2024-09-10T11:30:28.659804+0000","last_fullsized":"2024-09-10T11:30:28.659804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:04.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360104+0000","last_change":"2024-09-10T11:30:27.370679+0000","last_active":"2024-09-10T11:30:28.360104+0000","last_peered":"2024-09-10T11:30:28.360104+0000","last_clean":"2024-09-10T11:30:28.360104+0000","last_became_active":"2024-09-10T11:30:27.370082+0000","last_became_peered":"2024-09-10T11:30:27.370082+0000","last_unstale":"2024-09-10T11:30:28.360104+0000","last_undegraded":"2024-09-10T11:30:28.360104+0000","last_fullsized":"2024-09-10T11:30:28.360104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:29.103851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660089+0000","last_change":"2024-09-10T11:30:27.366962+0000","last_active":"2024-09-10T11:30:28.660089+0000","last_peered":"2024-09-10T11:30:28.660089+0000","last_clean":"2024-09-10T11:30:28.660089+0000","last_became_active":"2024-09-10T11:30:27.366732+0000","last_became_peered":"2024-09-10T11:30:27.366732+0000","last_unstale":"2024-09-10T11:30:28.660089+0000","last_undegraded":"2024-09-10T11:30:28.660089+0000","last_fullsized":"2024-09-10T11:30:28.660089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.894297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357208+0000","last_change":"2024-09-10T11:30:27.359455+0000","last_active":"2024-09-10T11:30:28.357208+0000","last_peered":"2024-09-10T11:30:28.357208+0000","last_clean":"2024-09-10T11:30:28.357208+0000","last_became_active":"2024-09-10T11:30:27.359192+0000","last_became_peered":"2024-09-10T11:30:27.359192+0000","last_unstale":"2024-09-10T11:30:28.357208+0000","last_undegraded":"2024-09-10T11:30:28.357208+0000","last_fullsized":"2024-09-10T11:30:28.357208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:32.342908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356734+0000","last_change":"2024-09-10T11:30:27.361285+0000","last_active":"2024-09-10T11:30:28.356734+0000","last_peered":"2024-09-10T11:30:28.356734+0000","last_clean":"2024-09-10T11:30:28.356734+0000","last_became_active":"2024-09-10T11:30:27.360381+0000","last_became_peered":"2024-09-10T11:30:27.360381+0000","last_unstale":"2024-09-10T11:30:28.356734+0000","last_undegraded":"2024-09-10T11:30:28.356734+0000","last_fullsized":"2024-09-10T11:30:28.356734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:44.857099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359897+0000","last_change":"2024-09-10T11:30:27.365604+0000","last_active":"2024-09-10T11:30:28.359897+0000","last_peered":"2024-09-10T11:30:28.359897+0000","last_clean":"2024-09-10T11:30:28.359897+0000","last_became_active":"2024-09-10T11:30:27.365092+0000","last_became_peered":"2024-09-10T11:30:27.365092+0000","last_unstale":"2024-09-10T11:30:28.359897+0000","last_undegraded":"2024-09-10T11:30:28.359897+0000","last_fullsized":"2024-09-10T11:30:28.359897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.304449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659583+0000","last_change":"2024-09-10T11:30:27.372482+0000","last_active":"2024-09-10T11:30:28.659583+0000","last_peered":"2024-09-10T11:30:28.659583+0000","last_clean":"2024-09-10T11:30:28.659583+0000","last_became_active":"2024-09-10T11:30:27.371823+0000","last_became_peered":"2024-09-10T11:30:27.371823+0000","last_unstale":"2024-09-10T11:30:28.659583+0000","last_undegraded":"2024-09-10T11:30:28.659583+0000","last_fullsized":"2024-09-10T11:30:28.659583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:00.379890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359182+0000","last_change":"2024-09-10T11:30:27.361947+0000","last_active":"2024-09-10T11:30:28.359182+0000","last_peered":"2024-09-10T11:30:28.359182+0000","last_clean":"2024-09-10T11:30:28.359182+0000","last_became_active":"2024-09-10T11:30:27.361705+0000","last_became_peered":"2024-09-10T11:30:27.361705+0000","last_unstale":"2024-09-10T11:30:28.359182+0000","last_undegraded":"2024-09-10T11:30:28.359182+0000","last_fullsized":"2024-09-10T11:30:28.359182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:49.977081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660062+0000","last_change":"2024-09-10T11:30:27.374439+0000","last_active":"2024-09-10T11:30:28.660062+0000","last_peered":"2024-09-10T11:30:28.660062+0000","last_clean":"2024-09-10T11:30:28.660062+0000","last_became_active":"2024-09-10T11:30:27.373730+0000","last_became_peered":"2024-09-10T11:30:27.373730+0000","last_unstale":"2024-09-10T11:30:28.660062+0000","last_undegraded":"2024-09-10T11:30:28.660062+0000","last_fullsized":"2024-09-10T11:30:28.660062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:55.938451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356881+0000","last_change":"2024-09-10T11:30:27.371303+0000","last_active":"2024-09-10T11:30:28.356881+0000","last_peered":"2024-09-10T11:30:28.356881+0000","last_clean":"2024-09-10T11:30:28.356881+0000","last_became_active":"2024-09-10T11:30:27.370830+0000","last_became_peered":"2024-09-10T11:30:27.370830+0000","last_unstale":"2024-09-10T11:30:28.356881+0000","last_undegraded":"2024-09-10T11:30:28.356881+0000","last_fullsized":"2024-09-10T11:30:28.356881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:11.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359871+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:28.359871+0000","last_peered":"2024-09-10T11:30:28.359871+0000","last_clean":"2024-09-10T11:30:28.359871+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:28.359871+0000","last_undegraded":"2024-09-10T11:30:28.359871+0000","last_fullsized":"2024-09-10T11:30:28.359871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:21.094608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357704+0000","last_change":"2024-09-10T11:30:27.368638+0000","last_active":"2024-09-10T11:30:28.357704+0000","last_peered":"2024-09-10T11:30:28.357704+0000","last_clean":"2024-09-10T11:30:28.357704+0000","last_became_active":"2024-09-10T11:30:27.368113+0000","last_became_peered":"2024-09-10T11:30:27.368113+0000","last_unstale":"2024-09-10T11:30:28.357704+0000","last_undegraded":"2024-09-10T11:30:28.357704+0000","last_fullsized":"2024-09-10T11:30:28.357704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:09.131376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659494+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:28.659494+0000","last_peered":"2024-09-10T11:30:28.659494+0000","last_clean":"2024-09-10T11:30:28.659494+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:28.659494+0000","last_undegraded":"2024-09-10T11:30:28.659494+0000","last_fullsized":"2024-09-10T11:30:28.659494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:51.779467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356585+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:28.356585+0000","last_peered":"2024-09-10T11:30:28.356585+0000","last_clean":"2024-09-10T11:30:28.356585+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:28.356585+0000","last_undegraded":"2024-09-10T11:30:28.356585+0000","last_fullsized":"2024-09-10T11:30:28.356585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:34.623802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360215+0000","last_change":"2024-09-10T11:30:27.370405+0000","last_active":"2024-09-10T11:30:28.360215+0000","last_peered":"2024-09-10T11:30:28.360215+0000","last_clean":"2024-09-10T11:30:28.360215+0000","last_became_active":"2024-09-10T11:30:27.369467+0000","last_became_peered":"2024-09-10T11:30:27.369467+0000","last_unstale":"2024-09-10T11:30:28.360215+0000","last_undegraded":"2024-09-10T11:30:28.360215+0000","last_fullsized":"2024-09-10T11:30:28.360215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:09.083864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660711+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:28.660711+0000","last_peered":"2024-09-10T11:30:28.660711+0000","last_clean":"2024-09-10T11:30:28.660711+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:28.660711+0000","last_undegraded":"2024-09-10T11:30:28.660711+0000","last_fullsized":"2024-09-10T11:30:28.660711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:34.426397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357445+0000","last_change":"2024-09-10T11:30:27.367728+0000","last_active":"2024-09-10T11:30:28.357445+0000","last_peered":"2024-09-10T11:30:28.357445+0000","last_clean":"2024-09-10T11:30:28.357445+0000","last_became_active":"2024-09-10T11:30:27.367459+0000","last_became_peered":"2024-09-10T11:30:27.367459+0000","last_unstale":"2024-09-10T11:30:28.357445+0000","last_undegraded":"2024-09-10T11:30:28.357445+0000","last_fullsized":"2024-09-10T11:30:28.357445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:06.747060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357160+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:28.357160+0000","last_peered":"2024-09-10T11:30:28.357160+0000","last_clean":"2024-09-10T11:30:28.357160+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:28.357160+0000","last_undegraded":"2024-09-10T11:30:28.357160+0000","last_fullsized":"2024-09-10T11:30:28.357160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:44.240242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660192+0000","last_change":"2024-09-10T11:30:27.368579+0000","last_active":"2024-09-10T11:30:28.660192+0000","last_peered":"2024-09-10T11:30:28.660192+0000","last_clean":"2024-09-10T11:30:28.660192+0000","last_became_active":"2024-09-10T11:30:27.368354+0000","last_became_peered":"2024-09-10T11:30:27.368354+0000","last_unstale":"2024-09-10T11:30:28.660192+0000","last_undegraded":"2024-09-10T11:30:28.660192+0000","last_fullsized":"2024-09-10T11:30:28.660192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:38.076870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371291+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:28.371291+0000","last_peered":"2024-09-10T11:30:28.371291+0000","last_clean":"2024-09-10T11:30:28.371291+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:28.371291+0000","last_undegraded":"2024-09-10T11:30:28.371291+0000","last_fullsized":"2024-09-10T11:30:28.371291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:03.475850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359716+0000","last_change":"2024-09-10T11:30:27.366765+0000","last_active":"2024-09-10T11:30:28.359716+0000","last_peered":"2024-09-10T11:30:28.359716+0000","last_clean":"2024-09-10T11:30:28.359716+0000","last_became_active":"2024-09-10T11:30:27.366456+0000","last_became_peered":"2024-09-10T11:30:27.366456+0000","last_unstale":"2024-09-10T11:30:28.359716+0000","last_undegraded":"2024-09-10T11:30:28.359716+0000","last_fullsized":"2024-09-10T11:30:28.359716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:07.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359589+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:28.359589+0000","last_peered":"2024-09-10T11:30:28.359589+0000","last_clean":"2024-09-10T11:30:28.359589+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:28.359589+0000","last_undegraded":"2024-09-10T11:30:28.359589+0000","last_fullsized":"2024-09-10T11:30:28.359589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:49.274560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660264+0000","last_change":"2024-09-10T11:30:27.373963+0000","last_active":"2024-09-10T11:30:28.660264+0000","last_peered":"2024-09-10T11:30:28.660264+0000","last_clean":"2024-09-10T11:30:28.660264+0000","last_became_active":"2024-09-10T11:30:27.373716+0000","last_became_peered":"2024-09-10T11:30:27.373716+0000","last_unstale":"2024-09-10T11:30:28.660264+0000","last_undegraded":"2024-09-10T11:30:28.660264+0000","last_fullsized":"2024-09-10T11:30:28.660264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:42.183989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659777+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:28.659777+0000","last_peered":"2024-09-10T11:30:28.659777+0000","last_clean":"2024-09-10T11:30:28.659777+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:28.659777+0000","last_undegraded":"2024-09-10T11:30:28.659777+0000","last_fullsized":"2024-09-10T11:30:28.659777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:16.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371116+0000","last_change":"2024-09-10T11:30:27.362545+0000","last_active":"2024-09-10T11:30:28.371116+0000","last_peered":"2024-09-10T11:30:28.371116+0000","last_clean":"2024-09-10T11:30:28.371116+0000","last_became_active":"2024-09-10T11:30:27.362303+0000","last_became_peered":"2024-09-10T11:30:27.362303+0000","last_unstale":"2024-09-10T11:30:28.371116+0000","last_undegraded":"2024-09-10T11:30:28.371116+0000","last_fullsized":"2024-09-10T11:30:28.371116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359239+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:28.359239+0000","last_peered":"2024-09-10T11:30:28.359239+0000","last_clean":"2024-09-10T11:30:28.359239+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:28.359239+0000","last_undegraded":"2024-09-10T11:30:28.359239+0000","last_fullsized":"2024-09-10T11:30:28.359239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:17.526780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356863+0000","last_change":"2024-09-10T11:30:27.363967+0000","last_active":"2024-09-10T11:30:28.356863+0000","last_peered":"2024-09-10T11:30:28.356863+0000","last_clean":"2024-09-10T11:30:28.356863+0000","last_became_active":"2024-09-10T11:30:27.363715+0000","last_became_peered":"2024-09-10T11:30:27.363715+0000","last_unstale":"2024-09-10T11:30:28.356863+0000","last_undegraded":"2024-09-10T11:30:28.356863+0000","last_fullsized":"2024-09-10T11:30:28.356863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:30.062610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659978+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:28.659978+0000","last_peered":"2024-09-10T11:30:28.659978+0000","last_clean":"2024-09-10T11:30:28.659978+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:28.659978+0000","last_undegraded":"2024-09-10T11:30:28.659978+0000","last_fullsized":"2024-09-10T11:30:28.659978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:17.448052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660486+0000","last_change":"2024-09-10T11:30:27.371396+0000","last_active":"2024-09-10T11:30:28.660486+0000","last_peered":"2024-09-10T11:30:28.660486+0000","last_clean":"2024-09-10T11:30:28.660486+0000","last_became_active":"2024-09-10T11:30:27.370265+0000","last_became_peered":"2024-09-10T11:30:27.370265+0000","last_unstale":"2024-09-10T11:30:28.660486+0000","last_undegraded":"2024-09-10T11:30:28.660486+0000","last_fullsized":"2024-09-10T11:30:28.660486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:27.284946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371504+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:28.371504+0000","last_peered":"2024-09-10T11:30:28.371504+0000","last_clean":"2024-09-10T11:30:28.371504+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:28.371504+0000","last_undegraded":"2024-09-10T11:30:28.371504+0000","last_fullsized":"2024-09-10T11:30:28.371504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:59.588789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357034+0000","last_change":"2024-09-10T11:30:27.371097+0000","last_active":"2024-09-10T11:30:28.357034+0000","last_peered":"2024-09-10T11:30:28.357034+0000","last_clean":"2024-09-10T11:30:28.357034+0000","last_became_active":"2024-09-10T11:30:27.370403+0000","last_became_peered":"2024-09-10T11:30:27.370403+0000","last_unstale":"2024-09-10T11:30:28.357034+0000","last_undegraded":"2024-09-10T11:30:28.357034+0000","last_fullsized":"2024-09-10T11:30:28.357034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:39.981595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357926+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:28.357926+0000","last_peered":"2024-09-10T11:30:28.357926+0000","last_clean":"2024-09-10T11:30:28.357926+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:28.357926+0000","last_undegraded":"2024-09-10T11:30:28.357926+0000","last_fullsized":"2024-09-10T11:30:28.357926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.466472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659957+0000","last_change":"2024-09-10T11:30:27.370723+0000","last_active":"2024-09-10T11:30:28.659957+0000","last_peered":"2024-09-10T11:30:28.659957+0000","last_clean":"2024-09-10T11:30:28.659957+0000","last_became_active":"2024-09-10T11:30:27.369619+0000","last_became_peered":"2024-09-10T11:30:27.369619+0000","last_unstale":"2024-09-10T11:30:28.659957+0000","last_undegraded":"2024-09-10T11:30:28.659957+0000","last_fullsized":"2024-09-10T11:30:28.659957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:12.483235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370845+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:28.370845+0000","last_peered":"2024-09-10T11:30:28.370845+0000","last_clean":"2024-09-10T11:30:28.370845+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:28.370845+0000","last_undegraded":"2024-09-10T11:30:28.370845+0000","last_fullsized":"2024-09-10T11:30:28.370845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:06.667422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660299+0000","last_change":"2024-09-10T11:30:27.366991+0000","last_active":"2024-09-10T11:30:28.660299+0000","last_peered":"2024-09-10T11:30:28.660299+0000","last_clean":"2024-09-10T11:30:28.660299+0000","last_became_active":"2024-09-10T11:30:27.366752+0000","last_became_peered":"2024-09-10T11:30:27.366752+0000","last_unstale":"2024-09-10T11:30:28.660299+0000","last_undegraded":"2024-09-10T11:30:28.660299+0000","last_fullsized":"2024-09-10T11:30:28.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:47.959643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357478+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:28.357478+0000","last_peered":"2024-09-10T11:30:28.357478+0000","last_clean":"2024-09-10T11:30:28.357478+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:28.357478+0000","last_undegraded":"2024-09-10T11:30:28.357478+0000","last_fullsized":"2024-09-10T11:30:28.357478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:28.210484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370778+0000","last_change":"2024-09-10T11:30:27.363710+0000","last_active":"2024-09-10T11:30:28.370778+0000","last_peered":"2024-09-10T11:30:28.370778+0000","last_clean":"2024-09-10T11:30:28.370778+0000","last_became_active":"2024-09-10T11:30:27.363475+0000","last_became_peered":"2024-09-10T11:30:27.363475+0000","last_unstale":"2024-09-10T11:30:28.370778+0000","last_undegraded":"2024-09-10T11:30:28.370778+0000","last_fullsized":"2024-09-10T11:30:28.370778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:48.768550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359866+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:28.359866+0000","last_peered":"2024-09-10T11:30:28.359866+0000","last_clean":"2024-09-10T11:30:28.359866+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:28.359866+0000","last_undegraded":"2024-09-10T11:30:28.359866+0000","last_fullsized":"2024-09-10T11:30:28.359866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:14.325407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371216+0000","last_change":"2024-09-10T11:30:27.367212+0000","last_active":"2024-09-10T11:30:28.371216+0000","last_peered":"2024-09-10T11:30:28.371216+0000","last_clean":"2024-09-10T11:30:28.371216+0000","last_became_active":"2024-09-10T11:30:27.366516+0000","last_became_peered":"2024-09-10T11:30:27.366516+0000","last_unstale":"2024-09-10T11:30:28.371216+0000","last_undegraded":"2024-09-10T11:30:28.371216+0000","last_fullsized":"2024-09-10T11:30:28.371216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:43.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357377+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:28.357377+0000","last_peered":"2024-09-10T11:30:28.357377+0000","last_clean":"2024-09-10T11:30:28.357377+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:28.357377+0000","last_undegraded":"2024-09-10T11:30:28.357377+0000","last_fullsized":"2024-09-10T11:30:28.357377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:16.594543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660098+0000","last_change":"2024-09-10T11:30:27.363291+0000","last_active":"2024-09-10T11:30:28.660098+0000","last_peered":"2024-09-10T11:30:28.660098+0000","last_clean":"2024-09-10T11:30:28.660098+0000","last_became_active":"2024-09-10T11:30:27.362845+0000","last_became_peered":"2024-09-10T11:30:27.362845+0000","last_unstale":"2024-09-10T11:30:28.660098+0000","last_undegraded":"2024-09-10T11:30:28.660098+0000","last_fullsized":"2024-09-10T11:30:28.660098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.348250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:28.357044+0000","last_peered":"2024-09-10T11:30:28.357044+0000","last_clean":"2024-09-10T11:30:28.357044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:28.357044+0000","last_undegraded":"2024-09-10T11:30:28.357044+0000","last_fullsized":"2024-09-10T11:30:28.357044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660171+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:28.660171+0000","last_peered":"2024-09-10T11:30:28.660171+0000","last_clean":"2024-09-10T11:30:28.660171+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:28.660171+0000","last_undegraded":"2024-09-10T11:30:28.660171+0000","last_fullsized":"2024-09-10T11:30:28.660171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:07.847876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358022+0000","last_change":"2024-09-10T11:30:27.368957+0000","last_active":"2024-09-10T11:30:28.358022+0000","last_peered":"2024-09-10T11:30:28.358022+0000","last_clean":"2024-09-10T11:30:28.358022+0000","last_became_active":"2024-09-10T11:30:27.368412+0000","last_became_peered":"2024-09-10T11:30:27.368412+0000","last_unstale":"2024-09-10T11:30:28.358022+0000","last_undegraded":"2024-09-10T11:30:28.358022+0000","last_fullsized":"2024-09-10T11:30:28.358022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:15.612861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660500+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:28.660500+0000","last_peered":"2024-09-10T11:30:28.660500+0000","last_clean":"2024-09-10T11:30:28.660500+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:28.660500+0000","last_undegraded":"2024-09-10T11:30:28.660500+0000","last_fullsized":"2024-09-10T11:30:28.660500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:18.084482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356890+0000","last_change":"2024-09-10T11:30:27.370947+0000","last_active":"2024-09-10T11:30:28.356890+0000","last_peered":"2024-09-10T11:30:28.356890+0000","last_clean":"2024-09-10T11:30:28.356890+0000","last_became_active":"2024-09-10T11:30:27.370630+0000","last_became_peered":"2024-09-10T11:30:27.370630+0000","last_unstale":"2024-09-10T11:30:28.356890+0000","last_undegraded":"2024-09-10T11:30:28.356890+0000","last_fullsized":"2024-09-10T11:30:28.356890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:42.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359693+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:28.359693+0000","last_peered":"2024-09-10T11:30:28.359693+0000","last_clean":"2024-09-10T11:30:28.359693+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:28.359693+0000","last_undegraded":"2024-09-10T11:30:28.359693+0000","last_fullsized":"2024-09-10T11:30:28.359693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:39.031878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360140+0000","last_change":"2024-09-10T11:30:27.365334+0000","last_active":"2024-09-10T11:30:28.360140+0000","last_peered":"2024-09-10T11:30:28.360140+0000","last_clean":"2024-09-10T11:30:28.360140+0000","last_became_active":"2024-09-10T11:30:27.364781+0000","last_became_peered":"2024-09-10T11:30:27.364781+0000","last_unstale":"2024-09-10T11:30:28.360140+0000","last_undegraded":"2024-09-10T11:30:28.360140+0000","last_fullsized":"2024-09-10T11:30:28.360140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:22.952484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660118+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:28.660118+0000","last_peered":"2024-09-10T11:30:28.660118+0000","last_clean":"2024-09-10T11:30:28.660118+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:28.660118+0000","last_undegraded":"2024-09-10T11:30:28.660118+0000","last_fullsized":"2024-09-10T11:30:28.660118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:52.177357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371203+0000","last_change":"2024-09-10T11:30:27.369498+0000","last_active":"2024-09-10T11:30:28.371203+0000","last_peered":"2024-09-10T11:30:28.371203+0000","last_clean":"2024-09-10T11:30:28.371203+0000","last_became_active":"2024-09-10T11:30:27.369231+0000","last_became_peered":"2024-09-10T11:30:27.369231+0000","last_unstale":"2024-09-10T11:30:28.371203+0000","last_undegraded":"2024-09-10T11:30:28.371203+0000","last_fullsized":"2024-09-10T11:30:28.371203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:05.406568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360323+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:28.360323+0000","last_peered":"2024-09-10T11:30:28.360323+0000","last_clean":"2024-09-10T11:30:28.360323+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:28.360323+0000","last_undegraded":"2024-09-10T11:30:28.360323+0000","last_fullsized":"2024-09-10T11:30:28.360323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:27.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659688+0000","last_change":"2024-09-10T11:30:27.369830+0000","last_active":"2024-09-10T11:30:28.659688+0000","last_peered":"2024-09-10T11:30:28.659688+0000","last_clean":"2024-09-10T11:30:28.659688+0000","last_became_active":"2024-09-10T11:30:27.369147+0000","last_became_peered":"2024-09-10T11:30:27.369147+0000","last_unstale":"2024-09-10T11:30:28.659688+0000","last_undegraded":"2024-09-10T11:30:28.659688+0000","last_fullsized":"2024-09-10T11:30:28.659688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:00.145985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660445+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:28.660445+0000","last_peered":"2024-09-10T11:30:28.660445+0000","last_clean":"2024-09-10T11:30:28.660445+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:28.660445+0000","last_undegraded":"2024-09-10T11:30:28.660445+0000","last_fullsized":"2024-09-10T11:30:28.660445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:02.778492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357067+0000","last_change":"2024-09-10T11:30:27.370822+0000","last_active":"2024-09-10T11:30:28.357067+0000","last_peered":"2024-09-10T11:30:28.357067+0000","last_clean":"2024-09-10T11:30:28.357067+0000","last_became_active":"2024-09-10T11:30:27.370501+0000","last_became_peered":"2024-09-10T11:30:27.370501+0000","last_unstale":"2024-09-10T11:30:28.357067+0000","last_undegraded":"2024-09-10T11:30:28.357067+0000","last_fullsized":"2024-09-10T11:30:28.357067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:05.773002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358086+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:28.358086+0000","last_peered":"2024-09-10T11:30:28.358086+0000","last_clean":"2024-09-10T11:30:28.358086+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:28.358086+0000","last_undegraded":"2024-09-10T11:30:28.358086+0000","last_fullsized":"2024-09-10T11:30:28.358086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:39.722801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357861+0000","last_change":"2024-09-10T11:30:27.362871+0000","last_active":"2024-09-10T11:30:28.357861+0000","last_peered":"2024-09-10T11:30:28.357861+0000","last_clean":"2024-09-10T11:30:28.357861+0000","last_became_active":"2024-09-10T11:30:27.362533+0000","last_became_peered":"2024-09-10T11:30:27.362533+0000","last_unstale":"2024-09-10T11:30:28.357861+0000","last_undegraded":"2024-09-10T11:30:28.357861+0000","last_fullsized":"2024-09-10T11:30:28.357861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:34.967835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660031+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:28.660031+0000","last_peered":"2024-09-10T11:30:28.660031+0000","last_clean":"2024-09-10T11:30:28.660031+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:28.660031+0000","last_undegraded":"2024-09-10T11:30:28.660031+0000","last_fullsized":"2024-09-10T11:30:28.660031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:00.971732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357833+0000","last_change":"2024-09-10T11:30:27.365497+0000","last_active":"2024-09-10T11:30:28.357833+0000","last_peered":"2024-09-10T11:30:28.357833+0000","last_clean":"2024-09-10T11:30:28.357833+0000","last_became_active":"2024-09-10T11:30:27.363947+0000","last_became_peered":"2024-09-10T11:30:27.363947+0000","last_unstale":"2024-09-10T11:30:28.357833+0000","last_undegraded":"2024-09-10T11:30:28.357833+0000","last_fullsized":"2024-09-10T11:30:28.357833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:24.206828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371397+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:28.371397+0000","last_peered":"2024-09-10T11:30:28.371397+0000","last_clean":"2024-09-10T11:30:28.371397+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:28.371397+0000","last_undegraded":"2024-09-10T11:30:28.371397+0000","last_fullsized":"2024-09-10T11:30:28.371397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:26.569057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660619+0000","last_change":"2024-09-10T11:30:27.372245+0000","last_active":"2024-09-10T11:30:28.660619+0000","last_peered":"2024-09-10T11:30:28.660619+0000","last_clean":"2024-09-10T11:30:28.660619+0000","last_became_active":"2024-09-10T11:30:27.371099+0000","last_became_peered":"2024-09-10T11:30:27.371099+0000","last_unstale":"2024-09-10T11:30:28.660619+0000","last_undegraded":"2024-09-10T11:30:28.660619+0000","last_fullsized":"2024-09-10T11:30:28.660619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:39.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357638+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:28.357638+0000","last_peered":"2024-09-10T11:30:28.357638+0000","last_clean":"2024-09-10T11:30:28.357638+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:28.357638+0000","last_undegraded":"2024-09-10T11:30:28.357638+0000","last_fullsized":"2024-09-10T11:30:28.357638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:59.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357610+0000","last_change":"2024-09-10T11:30:27.361895+0000","last_active":"2024-09-10T11:30:28.357610+0000","last_peered":"2024-09-10T11:30:28.357610+0000","last_clean":"2024-09-10T11:30:28.357610+0000","last_became_active":"2024-09-10T11:30:27.361662+0000","last_became_peered":"2024-09-10T11:30:27.361662+0000","last_unstale":"2024-09-10T11:30:28.357610+0000","last_undegraded":"2024-09-10T11:30:28.357610+0000","last_fullsized":"2024-09-10T11:30:28.357610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:08.687699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356658+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:28.356658+0000","last_peered":"2024-09-10T11:30:28.356658+0000","last_clean":"2024-09-10T11:30:28.356658+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:28.356658+0000","last_undegraded":"2024-09-10T11:30:28.356658+0000","last_fullsized":"2024-09-10T11:30:28.356658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:49.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359485+0000","last_change":"2024-09-10T11:30:27.362943+0000","last_active":"2024-09-10T11:30:28.359485+0000","last_peered":"2024-09-10T11:30:28.359485+0000","last_clean":"2024-09-10T11:30:28.359485+0000","last_became_active":"2024-09-10T11:30:27.362709+0000","last_became_peered":"2024-09-10T11:30:27.362709+0000","last_unstale":"2024-09-10T11:30:28.359485+0000","last_undegraded":"2024-09-10T11:30:28.359485+0000","last_fullsized":"2024-09-10T11:30:28.359485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:05.763931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659605+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:28.659605+0000","last_peered":"2024-09-10T11:30:28.659605+0000","last_clean":"2024-09-10T11:30:28.659605+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:28.659605+0000","last_undegraded":"2024-09-10T11:30:28.659605+0000","last_fullsized":"2024-09-10T11:30:28.659605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:01.450636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660335+0000","last_change":"2024-09-10T11:30:27.372613+0000","last_active":"2024-09-10T11:30:28.660335+0000","last_peered":"2024-09-10T11:30:28.660335+0000","last_clean":"2024-09-10T11:30:28.660335+0000","last_became_active":"2024-09-10T11:30:27.370970+0000","last_became_peered":"2024-09-10T11:30:27.370970+0000","last_unstale":"2024-09-10T11:30:28.660335+0000","last_undegraded":"2024-09-10T11:30:28.660335+0000","last_fullsized":"2024-09-10T11:30:28.660335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:51.164359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660329+0000","last_change":"2024-09-10T11:30:27.372047+0000","last_active":"2024-09-10T11:30:28.660329+0000","last_peered":"2024-09-10T11:30:28.660329+0000","last_clean":"2024-09-10T11:30:28.660329+0000","last_became_active":"2024-09-10T11:30:27.370618+0000","last_became_peered":"2024-09-10T11:30:27.370618+0000","last_unstale":"2024-09-10T11:30:28.660329+0000","last_undegraded":"2024-09-10T11:30:28.660329+0000","last_fullsized":"2024-09-10T11:30:28.660329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:47.350659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371285+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:28.371285+0000","last_peered":"2024-09-10T11:30:28.371285+0000","last_clean":"2024-09-10T11:30:28.371285+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:28.371285+0000","last_undegraded":"2024-09-10T11:30:28.371285+0000","last_fullsized":"2024-09-10T11:30:28.371285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:31.153745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357070+0000","last_change":"2024-09-10T11:30:27.360961+0000","last_active":"2024-09-10T11:30:28.357070+0000","last_peered":"2024-09-10T11:30:28.357070+0000","last_clean":"2024-09-10T11:30:28.357070+0000","last_became_active":"2024-09-10T11:30:27.360702+0000","last_became_peered":"2024-09-10T11:30:27.360702+0000","last_unstale":"2024-09-10T11:30:28.357070+0000","last_undegraded":"2024-09-10T11:30:28.357070+0000","last_fullsized":"2024-09-10T11:30:28.357070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:54.000251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371486+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:28.371486+0000","last_peered":"2024-09-10T11:30:28.371486+0000","last_clean":"2024-09-10T11:30:28.371486+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:28.371486+0000","last_undegraded":"2024-09-10T11:30:28.371486+0000","last_fullsized":"2024-09-10T11:30:28.371486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:49.786219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360066+0000","last_change":"2024-09-10T11:30:27.368686+0000","last_active":"2024-09-10T11:30:28.360066+0000","last_peered":"2024-09-10T11:30:28.360066+0000","last_clean":"2024-09-10T11:30:28.360066+0000","last_became_active":"2024-09-10T11:30:27.368329+0000","last_became_peered":"2024-09-10T11:30:27.368329+0000","last_unstale":"2024-09-10T11:30:28.360066+0000","last_undegraded":"2024-09-10T11:30:28.360066+0000","last_fullsized":"2024-09-10T11:30:28.360066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:03.672443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360276+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:28.360276+0000","last_peered":"2024-09-10T11:30:28.360276+0000","last_clean":"2024-09-10T11:30:28.360276+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:28.360276+0000","last_undegraded":"2024-09-10T11:30:28.360276+0000","last_fullsized":"2024-09-10T11:30:28.360276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:06.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360033+0000","last_change":"2024-09-10T11:30:27.372498+0000","last_active":"2024-09-10T11:30:28.360033+0000","last_peered":"2024-09-10T11:30:28.360033+0000","last_clean":"2024-09-10T11:30:28.360033+0000","last_became_active":"2024-09-10T11:30:27.372251+0000","last_became_peered":"2024-09-10T11:30:27.372251+0000","last_unstale":"2024-09-10T11:30:28.360033+0000","last_undegraded":"2024-09-10T11:30:28.360033+0000","last_fullsized":"2024-09-10T11:30:28.360033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:44.924775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357462+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:28.357462+0000","last_peered":"2024-09-10T11:30:28.357462+0000","last_clean":"2024-09-10T11:30:28.357462+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:28.357462+0000","last_undegraded":"2024-09-10T11:30:28.357462+0000","last_fullsized":"2024-09-10T11:30:28.357462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:16.783289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338766,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199926942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338788,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:38.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:30:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.064+0000 7f8f48800640 1 Processor -- start 2024-09-10T11:30:39.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.064+0000 7f8f48800640 1 -- start start 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f48800640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8f440d1290 0x7f8f440d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f48800640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f440d34a0 con 0x7f8f440b5d60 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f48800640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f440d3780 con 0x7f8f440d1bb0 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f48800640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8f440b4550 con 0x7f8f440d1290 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f4359e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8f440b5d60 0x7f8f440cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45004/0 (socket says 172.21.15.99:45004) 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f4359e640 1 -- 172.21.15.99:0/2072362291 learned_addr learned my addr 172.21.15.99:0/2072362291 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586644803 0 0) 0x7f8f440d34a0 con 0x7f8f440b5d60 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f30003520 con 0x7f8f440b5d60 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.065+0000 7f8f42d9d640 1 --2- 172.21.15.99:0/2072362291 >> v2:172.21.15.196:3300/0 conn(0x7f8f440d1290 0x7f8f440d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 829390474 0 0) 0x7f8f30003520 con 0x7f8f440b5d60 2024-09-10T11:30:39.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 >> v2:172.21.15.196:3300/0 conn(0x7f8f440d1290 msgr2=0x7f8f440d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 --2- 172.21.15.99:0/2072362291 >> v2:172.21.15.196:3300/0 conn(0x7f8f440d1290 0x7f8f440d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 >> v1:172.21.15.196:6789/0 conn(0x7f8f440d1bb0 legacy=0x7f8f440cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f440b4830 con 0x7f8f440b5d60 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8f3c0019f0 con 0x7f8f440b5d60 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 >> v1:172.21.15.99:6789/0 conn(0x7f8f440b5d60 legacy=0x7f8f440cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f300039b0 0x7f8f30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f440b4550 con 0x7f8f300051c0 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3000c060 con 0x7f8f300039b0 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3000c260 con 0x7f8f30004800 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f42d9d640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f300039b0 0x7f8f30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f3a59c640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 -- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f30004800 msgr2=0x7f8f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f30004800 0x7f8f30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 -- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f300039b0 msgr2=0x7f8f30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f300039b0 0x7f8f30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.066+0000 7f8f4359e640 1 -- 172.21.15.99:0/2072362291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f30003520 con 0x7f8f300051c0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.067+0000 7f8f4359e640 1 -- 172.21.15.99:0/2072362291 reap_dead start 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.067+0000 7f8f4359e640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b61f0 tx=0x7f8f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea42b539da35306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.067+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00a4a0 con 0x7f8f300051c0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.067+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3c02f070 con 0x7f8f300051c0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.067+0000 7f8f4259c640 1 -- 172.21.15.99:0/2072362291 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c02bcb0 con 0x7f8f300051c0 2024-09-10T11:30:39.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 -- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 msgr2=0x7f8f3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b61f0 tx=0x7f8f30005830 comp rx=0 tx=0).stop 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 -- 172.21.15.99:0/2072362291 shutdown_connections 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 --2- 172.21.15.99:0/2072362291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f3000b9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 -- 172.21.15.99:0/2072362291 >> 172.21.15.99:0/2072362291 conn(0x7f8f440d4b50 msgr2=0x7f8f440d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 -- 172.21.15.99:0/2072362291 shutdown_connections 2024-09-10T11:30:39.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.068+0000 7f8f48800640 1 -- 172.21.15.99:0/2072362291 wait complete. 2024-09-10T11:30:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.069+0000 7f8f48800640 1 Processor -- start 2024-09-10T11:30:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.069+0000 7f8f48800640 1 -- start start 2024-09-10T11:30:39.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f44158270 0x7f8f44158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b6990 0x7f8f440b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f440d3130 con 0x7f8f300051c0 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f440d3a60 con 0x7f8f440b6990 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4415f000 con 0x7f8f44158270 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54620/0 (socket says 172.21.15.99:54620) 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 -- 172.21.15.99:0/85553580 learned_addr learned my addr 172.21.15.99:0/85553580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f3a59c640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f44158270 0x7f8f44158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f4359e640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b6990 0x7f8f440b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 -- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f44158270 msgr2=0x7f8f44158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f44158270 0x7f8f44158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 -- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b6990 msgr2=0x7f8f440b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b6990 0x7f8f440b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 -- 172.21.15.99:0/85553580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4415abd0 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f42d9d640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f340118e0 tx=0x7f8f34011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400faf0 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3400fc90 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.070+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4415aeb0 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.071+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34014440 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.071+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4415b470 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.071+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f440d1670 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.072+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f34012040 con 0x7f8f300051c0 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.072+0000 7f8f3bfff640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 0x7f8f2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.072+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3408b590 con 0x7f8f300051c0 2024-09-10T11:30:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.072+0000 7f8f3a59c640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 0x7f8f2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.073+0000 7f8f3a59c640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 0x7f8f2406e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b7630 tx=0x7f8f2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.077+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f34019030 con 0x7f8f300051c0 2024-09-10T11:30:39.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.239+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f440b4290 con 0x7f8f2406be60 2024-09-10T11:30:39.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.245+0000 7f8f3bfff640 1 -- 172.21.15.99:0/85553580 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247486 (secure 0 0 0) 0x7f8f440b4290 con 0x7f8f2406be60 2024-09-10T11:30:39.250 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:39.251 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:30:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 msgr2=0x7f8f2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 0x7f8f2406e320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b7630 tx=0x7f8f2c009040 comp rx=0 tx=0).stop 2024-09-10T11:30:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 msgr2=0x7f8f441f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f340118e0 tx=0x7f8f34011da0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 shutdown_connections 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f2406be60 0x7f8f2406e320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f440b6990 0x7f8f440b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f44158270 0x7f8f44158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 --2- 172.21.15.99:0/85553580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f300051c0 0x7f8f441f4360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 >> 172.21.15.99:0/85553580 conn(0x7f8f440d4b50 msgr2=0x7f8f440cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 shutdown_connections 2024-09-10T11:30:39.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.248+0000 7f8f48800640 1 -- 172.21.15.99:0/85553580 wait complete. 2024-09-10T11:30:39.261 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T11:30:38.913537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.557799"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370941+0000","last_change":"2024-09-10T11:30:27.366179+0000","last_active":"2024-09-10T11:30:28.370941+0000","last_peered":"2024-09-10T11:30:28.370941+0000","last_clean":"2024-09-10T11:30:28.370941+0000","last_became_active":"2024-09-10T11:30:27.365953+0000","last_became_peered":"2024-09-10T11:30:27.365953+0000","last_unstale":"2024-09-10T11:30:28.370941+0000","last_undegraded":"2024-09-10T11:30:28.370941+0000","last_fullsized":"2024-09-10T11:30:28.370941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:09.985092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359715+0000","last_change":"2024-09-10T11:30:27.368422+0000","last_active":"2024-09-10T11:30:28.359715+0000","last_peered":"2024-09-10T11:30:28.359715+0000","last_clean":"2024-09-10T11:30:28.359715+0000","last_became_active":"2024-09-10T11:30:27.366676+0000","last_became_peered":"2024-09-10T11:30:27.366676+0000","last_unstale":"2024-09-10T11:30:28.359715+0000","last_undegraded":"2024-09-10T11:30:28.359715+0000","last_fullsized":"2024-09-10T11:30:28.359715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:37:11.234005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359802+0000","last_change":"2024-09-10T11:30:27.370589+0000","last_active":"2024-09-10T11:30:28.359802+0000","last_peered":"2024-09-10T11:30:28.359802+0000","last_clean":"2024-09-10T11:30:28.359802+0000","last_became_active":"2024-09-10T11:30:27.369527+0000","last_became_peered":"2024-09-10T11:30:27.369527+0000","last_unstale":"2024-09-10T11:30:28.359802+0000","last_undegraded":"2024-09-10T11:30:28.359802+0000","last_fullsized":"2024-09-10T11:30:28.359802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:28.644939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659543+0000","last_change":"2024-09-10T11:30:27.368869+0000","last_active":"2024-09-10T11:30:28.659543+0000","last_peered":"2024-09-10T11:30:28.659543+0000","last_clean":"2024-09-10T11:30:28.659543+0000","last_became_active":"2024-09-10T11:30:27.367497+0000","last_became_peered":"2024-09-10T11:30:27.367497+0000","last_unstale":"2024-09-10T11:30:28.659543+0000","last_undegraded":"2024-09-10T11:30:28.659543+0000","last_fullsized":"2024-09-10T11:30:28.659543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:11.740953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371428+0000","last_change":"2024-09-10T11:30:27.372181+0000","last_active":"2024-09-10T11:30:28.371428+0000","last_peered":"2024-09-10T11:30:28.371428+0000","last_clean":"2024-09-10T11:30:28.371428+0000","last_became_active":"2024-09-10T11:30:27.371972+0000","last_became_peered":"2024-09-10T11:30:27.371972+0000","last_unstale":"2024-09-10T11:30:28.371428+0000","last_undegraded":"2024-09-10T11:30:28.371428+0000","last_fullsized":"2024-09-10T11:30:28.371428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:52.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659909+0000","last_change":"2024-09-10T11:30:27.368458+0000","last_active":"2024-09-10T11:30:28.659909+0000","last_peered":"2024-09-10T11:30:28.659909+0000","last_clean":"2024-09-10T11:30:28.659909+0000","last_became_active":"2024-09-10T11:30:27.368163+0000","last_became_peered":"2024-09-10T11:30:27.368163+0000","last_unstale":"2024-09-10T11:30:28.659909+0000","last_undegraded":"2024-09-10T11:30:28.659909+0000","last_fullsized":"2024-09-10T11:30:28.659909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:17.388777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371439+0000","last_change":"2024-09-10T11:30:27.370692+0000","last_active":"2024-09-10T11:30:28.371439+0000","last_peered":"2024-09-10T11:30:28.371439+0000","last_clean":"2024-09-10T11:30:28.371439+0000","last_became_active":"2024-09-10T11:30:27.370323+0000","last_became_peered":"2024-09-10T11:30:27.370323+0000","last_unstale":"2024-09-10T11:30:28.371439+0000","last_undegraded":"2024-09-10T11:30:28.371439+0000","last_fullsized":"2024-09-10T11:30:28.371439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:30.005194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359963+0000","last_change":"2024-09-10T11:30:27.371135+0000","last_active":"2024-09-10T11:30:28.359963+0000","last_peered":"2024-09-10T11:30:28.359963+0000","last_clean":"2024-09-10T11:30:28.359963+0000","last_became_active":"2024-09-10T11:30:27.370855+0000","last_became_peered":"2024-09-10T11:30:27.370855+0000","last_unstale":"2024-09-10T11:30:28.359963+0000","last_undegraded":"2024-09-10T11:30:28.359963+0000","last_fullsized":"2024-09-10T11:30:28.359963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:08.475065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356981+0000","last_change":"2024-09-10T11:30:27.368622+0000","last_active":"2024-09-10T11:30:28.356981+0000","last_peered":"2024-09-10T11:30:28.356981+0000","last_clean":"2024-09-10T11:30:28.356981+0000","last_became_active":"2024-09-10T11:30:27.368043+0000","last_became_peered":"2024-09-10T11:30:27.368043+0000","last_unstale":"2024-09-10T11:30:28.356981+0000","last_undegraded":"2024-09-10T11:30:28.356981+0000","last_fullsized":"2024-09-10T11:30:28.356981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:11.979652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371169+0000","last_change":"2024-09-10T11:30:27.370344+0000","last_active":"2024-09-10T11:30:28.371169+0000","last_peered":"2024-09-10T11:30:28.371169+0000","last_clean":"2024-09-10T11:30:28.371169+0000","last_became_active":"2024-09-10T11:30:27.370093+0000","last_became_peered":"2024-09-10T11:30:27.370093+0000","last_unstale":"2024-09-10T11:30:28.371169+0000","last_undegraded":"2024-09-10T11:30:28.371169+0000","last_fullsized":"2024-09-10T11:30:28.371169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:55.994388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371076+0000","last_change":"2024-09-10T11:30:27.364571+0000","last_active":"2024-09-10T11:30:28.371076+0000","last_peered":"2024-09-10T11:30:28.371076+0000","last_clean":"2024-09-10T11:30:28.371076+0000","last_became_active":"2024-09-10T11:30:27.364329+0000","last_became_peered":"2024-09-10T11:30:27.364329+0000","last_unstale":"2024-09-10T11:30:28.371076+0000","last_undegraded":"2024-09-10T11:30:28.371076+0000","last_fullsized":"2024-09-10T11:30:28.371076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:56.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371023+0000","last_change":"2024-09-10T11:30:27.362816+0000","last_active":"2024-09-10T11:30:28.371023+0000","last_peered":"2024-09-10T11:30:28.371023+0000","last_clean":"2024-09-10T11:30:28.371023+0000","last_became_active":"2024-09-10T11:30:27.362165+0000","last_became_peered":"2024-09-10T11:30:27.362165+0000","last_unstale":"2024-09-10T11:30:28.371023+0000","last_undegraded":"2024-09-10T11:30:28.371023+0000","last_fullsized":"2024-09-10T11:30:28.371023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:13.704596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356572+0000","last_change":"2024-09-10T11:30:27.361112+0000","last_active":"2024-09-10T11:30:28.356572+0000","last_peered":"2024-09-10T11:30:28.356572+0000","last_clean":"2024-09-10T11:30:28.356572+0000","last_became_active":"2024-09-10T11:30:27.360081+0000","last_became_peered":"2024-09-10T11:30:27.360081+0000","last_unstale":"2024-09-10T11:30:28.356572+0000","last_undegraded":"2024-09-10T11:30:28.356572+0000","last_fullsized":"2024-09-10T11:30:28.356572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:25.075435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356459+0000","last_change":"2024-09-10T11:30:27.360154+0000","last_active":"2024-09-10T11:30:28.356459+0000","last_peered":"2024-09-10T11:30:28.356459+0000","last_clean":"2024-09-10T11:30:28.356459+0000","last_became_active":"2024-09-10T11:30:27.359341+0000","last_became_peered":"2024-09-10T11:30:27.359341+0000","last_unstale":"2024-09-10T11:30:28.356459+0000","last_undegraded":"2024-09-10T11:30:28.356459+0000","last_fullsized":"2024-09-10T11:30:28.356459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.266260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359988+0000","last_change":"2024-09-10T11:30:27.371329+0000","last_active":"2024-09-10T11:30:28.359988+0000","last_peered":"2024-09-10T11:30:28.359988+0000","last_clean":"2024-09-10T11:30:28.359988+0000","last_became_active":"2024-09-10T11:30:27.370948+0000","last_became_peered":"2024-09-10T11:30:27.370948+0000","last_unstale":"2024-09-10T11:30:28.359988+0000","last_undegraded":"2024-09-10T11:30:28.359988+0000","last_fullsized":"2024-09-10T11:30:28.359988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:40.619700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659901+0000","last_change":"2024-09-10T11:30:27.368565+0000","last_active":"2024-09-10T11:30:28.659901+0000","last_peered":"2024-09-10T11:30:28.659901+0000","last_clean":"2024-09-10T11:30:28.659901+0000","last_became_active":"2024-09-10T11:30:27.368309+0000","last_became_peered":"2024-09-10T11:30:27.368309+0000","last_unstale":"2024-09-10T11:30:28.659901+0000","last_undegraded":"2024-09-10T11:30:28.659901+0000","last_fullsized":"2024-09-10T11:30:28.659901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:33.008256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659831+0000","last_change":"2024-09-10T11:30:27.371513+0000","last_active":"2024-09-10T11:30:28.659831+0000","last_peered":"2024-09-10T11:30:28.659831+0000","last_clean":"2024-09-10T11:30:28.659831+0000","last_became_active":"2024-09-10T11:30:27.371229+0000","last_became_peered":"2024-09-10T11:30:27.371229+0000","last_unstale":"2024-09-10T11:30:28.659831+0000","last_undegraded":"2024-09-10T11:30:28.659831+0000","last_fullsized":"2024-09-10T11:30:28.659831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:28.410078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359314+0000","last_change":"2024-09-10T11:30:27.366960+0000","last_active":"2024-09-10T11:30:28.359314+0000","last_peered":"2024-09-10T11:30:28.359314+0000","last_clean":"2024-09-10T11:30:28.359314+0000","last_became_active":"2024-09-10T11:30:27.366744+0000","last_became_peered":"2024-09-10T11:30:27.366744+0000","last_unstale":"2024-09-10T11:30:28.359314+0000","last_undegraded":"2024-09-10T11:30:28.359314+0000","last_fullsized":"2024-09-10T11:30:28.359314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:30.599189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660402+0000","last_change":"2024-09-10T11:30:27.369325+0000","last_active":"2024-09-10T11:30:28.660402+0000","last_peered":"2024-09-10T11:30:28.660402+0000","last_clean":"2024-09-10T11:30:28.660402+0000","last_became_active":"2024-09-10T11:30:27.368956+0000","last_became_peered":"2024-09-10T11:30:27.368956+0000","last_unstale":"2024-09-10T11:30:28.660402+0000","last_undegraded":"2024-09-10T11:30:28.660402+0000","last_fullsized":"2024-09-10T11:30:28.660402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:08.633447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371252+0000","last_change":"2024-09-10T11:30:27.360270+0000","last_active":"2024-09-10T11:30:28.371252+0000","last_peered":"2024-09-10T11:30:28.371252+0000","last_clean":"2024-09-10T11:30:28.371252+0000","last_became_active":"2024-09-10T11:30:27.360000+0000","last_became_peered":"2024-09-10T11:30:27.360000+0000","last_unstale":"2024-09-10T11:30:28.371252+0000","last_undegraded":"2024-09-10T11:30:28.371252+0000","last_fullsized":"2024-09-10T11:30:28.371252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:00.874011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357411+0000","last_change":"2024-09-10T11:30:27.369424+0000","last_active":"2024-09-10T11:30:28.357411+0000","last_peered":"2024-09-10T11:30:28.357411+0000","last_clean":"2024-09-10T11:30:28.357411+0000","last_became_active":"2024-09-10T11:30:27.367954+0000","last_became_peered":"2024-09-10T11:30:27.367954+0000","last_unstale":"2024-09-10T11:30:28.357411+0000","last_undegraded":"2024-09-10T11:30:28.357411+0000","last_fullsized":"2024-09-10T11:30:28.357411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:15.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356963+0000","last_change":"2024-09-10T11:30:27.364058+0000","last_active":"2024-09-10T11:30:28.356963+0000","last_peered":"2024-09-10T11:30:28.356963+0000","last_clean":"2024-09-10T11:30:28.356963+0000","last_became_active":"2024-09-10T11:30:27.363699+0000","last_became_peered":"2024-09-10T11:30:27.363699+0000","last_unstale":"2024-09-10T11:30:28.356963+0000","last_undegraded":"2024-09-10T11:30:28.356963+0000","last_fullsized":"2024-09-10T11:30:28.356963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:48:36.749159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360157+0000","last_change":"2024-09-10T11:30:27.372337+0000","last_active":"2024-09-10T11:30:28.360157+0000","last_peered":"2024-09-10T11:30:28.360157+0000","last_clean":"2024-09-10T11:30:28.360157+0000","last_became_active":"2024-09-10T11:30:27.372074+0000","last_became_peered":"2024-09-10T11:30:27.372074+0000","last_unstale":"2024-09-10T11:30:28.360157+0000","last_undegraded":"2024-09-10T11:30:28.360157+0000","last_fullsized":"2024-09-10T11:30:28.360157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:39.707264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659804+0000","last_change":"2024-09-10T11:30:27.372831+0000","last_active":"2024-09-10T11:30:28.659804+0000","last_peered":"2024-09-10T11:30:28.659804+0000","last_clean":"2024-09-10T11:30:28.659804+0000","last_became_active":"2024-09-10T11:30:27.371298+0000","last_became_peered":"2024-09-10T11:30:27.371298+0000","last_unstale":"2024-09-10T11:30:28.659804+0000","last_undegraded":"2024-09-10T11:30:28.659804+0000","last_fullsized":"2024-09-10T11:30:28.659804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:04.248905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360104+0000","last_change":"2024-09-10T11:30:27.370679+0000","last_active":"2024-09-10T11:30:28.360104+0000","last_peered":"2024-09-10T11:30:28.360104+0000","last_clean":"2024-09-10T11:30:28.360104+0000","last_became_active":"2024-09-10T11:30:27.370082+0000","last_became_peered":"2024-09-10T11:30:27.370082+0000","last_unstale":"2024-09-10T11:30:28.360104+0000","last_undegraded":"2024-09-10T11:30:28.360104+0000","last_fullsized":"2024-09-10T11:30:28.360104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:29.103851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660089+0000","last_change":"2024-09-10T11:30:27.366962+0000","last_active":"2024-09-10T11:30:28.660089+0000","last_peered":"2024-09-10T11:30:28.660089+0000","last_clean":"2024-09-10T11:30:28.660089+0000","last_became_active":"2024-09-10T11:30:27.366732+0000","last_became_peered":"2024-09-10T11:30:27.366732+0000","last_unstale":"2024-09-10T11:30:28.660089+0000","last_undegraded":"2024-09-10T11:30:28.660089+0000","last_fullsized":"2024-09-10T11:30:28.660089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:10.894297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357208+0000","last_change":"2024-09-10T11:30:27.359455+0000","last_active":"2024-09-10T11:30:28.357208+0000","last_peered":"2024-09-10T11:30:28.357208+0000","last_clean":"2024-09-10T11:30:28.357208+0000","last_became_active":"2024-09-10T11:30:27.359192+0000","last_became_peered":"2024-09-10T11:30:27.359192+0000","last_unstale":"2024-09-10T11:30:28.357208+0000","last_undegraded":"2024-09-10T11:30:28.357208+0000","last_fullsized":"2024-09-10T11:30:28.357208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:32.342908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356734+0000","last_change":"2024-09-10T11:30:27.361285+0000","last_active":"2024-09-10T11:30:28.356734+0000","last_peered":"2024-09-10T11:30:28.356734+0000","last_clean":"2024-09-10T11:30:28.356734+0000","last_became_active":"2024-09-10T11:30:27.360381+0000","last_became_peered":"2024-09-10T11:30:27.360381+0000","last_unstale":"2024-09-10T11:30:28.356734+0000","last_undegraded":"2024-09-10T11:30:28.356734+0000","last_fullsized":"2024-09-10T11:30:28.356734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:44.857099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359897+0000","last_change":"2024-09-10T11:30:27.365604+0000","last_active":"2024-09-10T11:30:28.359897+0000","last_peered":"2024-09-10T11:30:28.359897+0000","last_clean":"2024-09-10T11:30:28.359897+0000","last_became_active":"2024-09-10T11:30:27.365092+0000","last_became_peered":"2024-09-10T11:30:27.365092+0000","last_unstale":"2024-09-10T11:30:28.359897+0000","last_undegraded":"2024-09-10T11:30:28.359897+0000","last_fullsized":"2024-09-10T11:30:28.359897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:56.304449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659583+0000","last_change":"2024-09-10T11:30:27.372482+0000","last_active":"2024-09-10T11:30:28.659583+0000","last_peered":"2024-09-10T11:30:28.659583+0000","last_clean":"2024-09-10T11:30:28.659583+0000","last_became_active":"2024-09-10T11:30:27.371823+0000","last_became_peered":"2024-09-10T11:30:27.371823+0000","last_unstale":"2024-09-10T11:30:28.659583+0000","last_undegraded":"2024-09-10T11:30:28.659583+0000","last_fullsized":"2024-09-10T11:30:28.659583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:00.379890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359182+0000","last_change":"2024-09-10T11:30:27.361947+0000","last_active":"2024-09-10T11:30:28.359182+0000","last_peered":"2024-09-10T11:30:28.359182+0000","last_clean":"2024-09-10T11:30:28.359182+0000","last_became_active":"2024-09-10T11:30:27.361705+0000","last_became_peered":"2024-09-10T11:30:27.361705+0000","last_unstale":"2024-09-10T11:30:28.359182+0000","last_undegraded":"2024-09-10T11:30:28.359182+0000","last_fullsized":"2024-09-10T11:30:28.359182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:49.977081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660062+0000","last_change":"2024-09-10T11:30:27.374439+0000","last_active":"2024-09-10T11:30:28.660062+0000","last_peered":"2024-09-10T11:30:28.660062+0000","last_clean":"2024-09-10T11:30:28.660062+0000","last_became_active":"2024-09-10T11:30:27.373730+0000","last_became_peered":"2024-09-10T11:30:27.373730+0000","last_unstale":"2024-09-10T11:30:28.660062+0000","last_undegraded":"2024-09-10T11:30:28.660062+0000","last_fullsized":"2024-09-10T11:30:28.660062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:55.938451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356881+0000","last_change":"2024-09-10T11:30:27.371303+0000","last_active":"2024-09-10T11:30:28.356881+0000","last_peered":"2024-09-10T11:30:28.356881+0000","last_clean":"2024-09-10T11:30:28.356881+0000","last_became_active":"2024-09-10T11:30:27.370830+0000","last_became_peered":"2024-09-10T11:30:27.370830+0000","last_unstale":"2024-09-10T11:30:28.356881+0000","last_undegraded":"2024-09-10T11:30:28.356881+0000","last_fullsized":"2024-09-10T11:30:28.356881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:11.895301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359871+0000","last_change":"2024-09-10T11:30:26.355518+0000","last_active":"2024-09-10T11:30:28.359871+0000","last_peered":"2024-09-10T11:30:28.359871+0000","last_clean":"2024-09-10T11:30:28.359871+0000","last_became_active":"2024-09-10T11:30:26.355270+0000","last_became_peered":"2024-09-10T11:30:26.355270+0000","last_unstale":"2024-09-10T11:30:28.359871+0000","last_undegraded":"2024-09-10T11:30:28.359871+0000","last_fullsized":"2024-09-10T11:30:28.359871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:21.094608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357704+0000","last_change":"2024-09-10T11:30:27.368638+0000","last_active":"2024-09-10T11:30:28.357704+0000","last_peered":"2024-09-10T11:30:28.357704+0000","last_clean":"2024-09-10T11:30:28.357704+0000","last_became_active":"2024-09-10T11:30:27.368113+0000","last_became_peered":"2024-09-10T11:30:27.368113+0000","last_unstale":"2024-09-10T11:30:28.357704+0000","last_undegraded":"2024-09-10T11:30:28.357704+0000","last_fullsized":"2024-09-10T11:30:28.357704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:09.131376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659494+0000","last_change":"2024-09-10T11:30:26.352434+0000","last_active":"2024-09-10T11:30:28.659494+0000","last_peered":"2024-09-10T11:30:28.659494+0000","last_clean":"2024-09-10T11:30:28.659494+0000","last_became_active":"2024-09-10T11:30:26.351941+0000","last_became_peered":"2024-09-10T11:30:26.351941+0000","last_unstale":"2024-09-10T11:30:28.659494+0000","last_undegraded":"2024-09-10T11:30:28.659494+0000","last_fullsized":"2024-09-10T11:30:28.659494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:51.779467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356585+0000","last_change":"2024-09-10T11:30:26.350846+0000","last_active":"2024-09-10T11:30:28.356585+0000","last_peered":"2024-09-10T11:30:28.356585+0000","last_clean":"2024-09-10T11:30:28.356585+0000","last_became_active":"2024-09-10T11:30:26.350558+0000","last_became_peered":"2024-09-10T11:30:26.350558+0000","last_unstale":"2024-09-10T11:30:28.356585+0000","last_undegraded":"2024-09-10T11:30:28.356585+0000","last_fullsized":"2024-09-10T11:30:28.356585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:34.623802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360215+0000","last_change":"2024-09-10T11:30:27.370405+0000","last_active":"2024-09-10T11:30:28.360215+0000","last_peered":"2024-09-10T11:30:28.360215+0000","last_clean":"2024-09-10T11:30:28.360215+0000","last_became_active":"2024-09-10T11:30:27.369467+0000","last_became_peered":"2024-09-10T11:30:27.369467+0000","last_unstale":"2024-09-10T11:30:28.360215+0000","last_undegraded":"2024-09-10T11:30:28.360215+0000","last_fullsized":"2024-09-10T11:30:28.360215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:09.083864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660711+0000","last_change":"2024-09-10T11:30:26.353656+0000","last_active":"2024-09-10T11:30:28.660711+0000","last_peered":"2024-09-10T11:30:28.660711+0000","last_clean":"2024-09-10T11:30:28.660711+0000","last_became_active":"2024-09-10T11:30:26.353437+0000","last_became_peered":"2024-09-10T11:30:26.353437+0000","last_unstale":"2024-09-10T11:30:28.660711+0000","last_undegraded":"2024-09-10T11:30:28.660711+0000","last_fullsized":"2024-09-10T11:30:28.660711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:34.426397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357445+0000","last_change":"2024-09-10T11:30:27.367728+0000","last_active":"2024-09-10T11:30:28.357445+0000","last_peered":"2024-09-10T11:30:28.357445+0000","last_clean":"2024-09-10T11:30:28.357445+0000","last_became_active":"2024-09-10T11:30:27.367459+0000","last_became_peered":"2024-09-10T11:30:27.367459+0000","last_unstale":"2024-09-10T11:30:28.357445+0000","last_undegraded":"2024-09-10T11:30:28.357445+0000","last_fullsized":"2024-09-10T11:30:28.357445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:06.747060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357160+0000","last_change":"2024-09-10T11:30:26.352082+0000","last_active":"2024-09-10T11:30:28.357160+0000","last_peered":"2024-09-10T11:30:28.357160+0000","last_clean":"2024-09-10T11:30:28.357160+0000","last_became_active":"2024-09-10T11:30:26.351660+0000","last_became_peered":"2024-09-10T11:30:26.351660+0000","last_unstale":"2024-09-10T11:30:28.357160+0000","last_undegraded":"2024-09-10T11:30:28.357160+0000","last_fullsized":"2024-09-10T11:30:28.357160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:44.240242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660192+0000","last_change":"2024-09-10T11:30:27.368579+0000","last_active":"2024-09-10T11:30:28.660192+0000","last_peered":"2024-09-10T11:30:28.660192+0000","last_clean":"2024-09-10T11:30:28.660192+0000","last_became_active":"2024-09-10T11:30:27.368354+0000","last_became_peered":"2024-09-10T11:30:27.368354+0000","last_unstale":"2024-09-10T11:30:28.660192+0000","last_undegraded":"2024-09-10T11:30:28.660192+0000","last_fullsized":"2024-09-10T11:30:28.660192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:38.076870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371291+0000","last_change":"2024-09-10T11:30:26.356148+0000","last_active":"2024-09-10T11:30:28.371291+0000","last_peered":"2024-09-10T11:30:28.371291+0000","last_clean":"2024-09-10T11:30:28.371291+0000","last_became_active":"2024-09-10T11:30:26.355373+0000","last_became_peered":"2024-09-10T11:30:26.355373+0000","last_unstale":"2024-09-10T11:30:28.371291+0000","last_undegraded":"2024-09-10T11:30:28.371291+0000","last_fullsized":"2024-09-10T11:30:28.371291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:03.475850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359716+0000","last_change":"2024-09-10T11:30:27.366765+0000","last_active":"2024-09-10T11:30:28.359716+0000","last_peered":"2024-09-10T11:30:28.359716+0000","last_clean":"2024-09-10T11:30:28.359716+0000","last_became_active":"2024-09-10T11:30:27.366456+0000","last_became_peered":"2024-09-10T11:30:27.366456+0000","last_unstale":"2024-09-10T11:30:28.359716+0000","last_undegraded":"2024-09-10T11:30:28.359716+0000","last_fullsized":"2024-09-10T11:30:28.359716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:07.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359589+0000","last_change":"2024-09-10T11:30:26.347476+0000","last_active":"2024-09-10T11:30:28.359589+0000","last_peered":"2024-09-10T11:30:28.359589+0000","last_clean":"2024-09-10T11:30:28.359589+0000","last_became_active":"2024-09-10T11:30:26.347086+0000","last_became_peered":"2024-09-10T11:30:26.347086+0000","last_unstale":"2024-09-10T11:30:28.359589+0000","last_undegraded":"2024-09-10T11:30:28.359589+0000","last_fullsized":"2024-09-10T11:30:28.359589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:49.274560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660264+0000","last_change":"2024-09-10T11:30:27.373963+0000","last_active":"2024-09-10T11:30:28.660264+0000","last_peered":"2024-09-10T11:30:28.660264+0000","last_clean":"2024-09-10T11:30:28.660264+0000","last_became_active":"2024-09-10T11:30:27.373716+0000","last_became_peered":"2024-09-10T11:30:27.373716+0000","last_unstale":"2024-09-10T11:30:28.660264+0000","last_undegraded":"2024-09-10T11:30:28.660264+0000","last_fullsized":"2024-09-10T11:30:28.660264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:42.183989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659777+0000","last_change":"2024-09-10T11:30:26.347857+0000","last_active":"2024-09-10T11:30:28.659777+0000","last_peered":"2024-09-10T11:30:28.659777+0000","last_clean":"2024-09-10T11:30:28.659777+0000","last_became_active":"2024-09-10T11:30:26.347333+0000","last_became_peered":"2024-09-10T11:30:26.347333+0000","last_unstale":"2024-09-10T11:30:28.659777+0000","last_undegraded":"2024-09-10T11:30:28.659777+0000","last_fullsized":"2024-09-10T11:30:28.659777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:16.946029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371116+0000","last_change":"2024-09-10T11:30:27.362545+0000","last_active":"2024-09-10T11:30:28.371116+0000","last_peered":"2024-09-10T11:30:28.371116+0000","last_clean":"2024-09-10T11:30:28.371116+0000","last_became_active":"2024-09-10T11:30:27.362303+0000","last_became_peered":"2024-09-10T11:30:27.362303+0000","last_unstale":"2024-09-10T11:30:28.371116+0000","last_undegraded":"2024-09-10T11:30:28.371116+0000","last_fullsized":"2024-09-10T11:30:28.371116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:12.866317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359239+0000","last_change":"2024-09-10T11:30:26.351756+0000","last_active":"2024-09-10T11:30:28.359239+0000","last_peered":"2024-09-10T11:30:28.359239+0000","last_clean":"2024-09-10T11:30:28.359239+0000","last_became_active":"2024-09-10T11:30:26.351436+0000","last_became_peered":"2024-09-10T11:30:26.351436+0000","last_unstale":"2024-09-10T11:30:28.359239+0000","last_undegraded":"2024-09-10T11:30:28.359239+0000","last_fullsized":"2024-09-10T11:30:28.359239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:17.526780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356863+0000","last_change":"2024-09-10T11:30:27.363967+0000","last_active":"2024-09-10T11:30:28.356863+0000","last_peered":"2024-09-10T11:30:28.356863+0000","last_clean":"2024-09-10T11:30:28.356863+0000","last_became_active":"2024-09-10T11:30:27.363715+0000","last_became_peered":"2024-09-10T11:30:27.363715+0000","last_unstale":"2024-09-10T11:30:28.356863+0000","last_undegraded":"2024-09-10T11:30:28.356863+0000","last_fullsized":"2024-09-10T11:30:28.356863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:30.062610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659978+0000","last_change":"2024-09-10T11:30:26.354870+0000","last_active":"2024-09-10T11:30:28.659978+0000","last_peered":"2024-09-10T11:30:28.659978+0000","last_clean":"2024-09-10T11:30:28.659978+0000","last_became_active":"2024-09-10T11:30:26.354645+0000","last_became_peered":"2024-09-10T11:30:26.354645+0000","last_unstale":"2024-09-10T11:30:28.659978+0000","last_undegraded":"2024-09-10T11:30:28.659978+0000","last_fullsized":"2024-09-10T11:30:28.659978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:47:17.448052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660486+0000","last_change":"2024-09-10T11:30:27.371396+0000","last_active":"2024-09-10T11:30:28.660486+0000","last_peered":"2024-09-10T11:30:28.660486+0000","last_clean":"2024-09-10T11:30:28.660486+0000","last_became_active":"2024-09-10T11:30:27.370265+0000","last_became_peered":"2024-09-10T11:30:27.370265+0000","last_unstale":"2024-09-10T11:30:28.660486+0000","last_undegraded":"2024-09-10T11:30:28.660486+0000","last_fullsized":"2024-09-10T11:30:28.660486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:27.284946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371504+0000","last_change":"2024-09-10T11:30:26.356377+0000","last_active":"2024-09-10T11:30:28.371504+0000","last_peered":"2024-09-10T11:30:28.371504+0000","last_clean":"2024-09-10T11:30:28.371504+0000","last_became_active":"2024-09-10T11:30:26.355924+0000","last_became_peered":"2024-09-10T11:30:26.355924+0000","last_unstale":"2024-09-10T11:30:28.371504+0000","last_undegraded":"2024-09-10T11:30:28.371504+0000","last_fullsized":"2024-09-10T11:30:28.371504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:59.588789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357034+0000","last_change":"2024-09-10T11:30:27.371097+0000","last_active":"2024-09-10T11:30:28.357034+0000","last_peered":"2024-09-10T11:30:28.357034+0000","last_clean":"2024-09-10T11:30:28.357034+0000","last_became_active":"2024-09-10T11:30:27.370403+0000","last_became_peered":"2024-09-10T11:30:27.370403+0000","last_unstale":"2024-09-10T11:30:28.357034+0000","last_undegraded":"2024-09-10T11:30:28.357034+0000","last_fullsized":"2024-09-10T11:30:28.357034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:39.981595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357926+0000","last_change":"2024-09-10T11:30:26.346879+0000","last_active":"2024-09-10T11:30:28.357926+0000","last_peered":"2024-09-10T11:30:28.357926+0000","last_clean":"2024-09-10T11:30:28.357926+0000","last_became_active":"2024-09-10T11:30:26.346537+0000","last_became_peered":"2024-09-10T11:30:26.346537+0000","last_unstale":"2024-09-10T11:30:28.357926+0000","last_undegraded":"2024-09-10T11:30:28.357926+0000","last_fullsized":"2024-09-10T11:30:28.357926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.466472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659957+0000","last_change":"2024-09-10T11:30:27.370723+0000","last_active":"2024-09-10T11:30:28.659957+0000","last_peered":"2024-09-10T11:30:28.659957+0000","last_clean":"2024-09-10T11:30:28.659957+0000","last_became_active":"2024-09-10T11:30:27.369619+0000","last_became_peered":"2024-09-10T11:30:27.369619+0000","last_unstale":"2024-09-10T11:30:28.659957+0000","last_undegraded":"2024-09-10T11:30:28.659957+0000","last_fullsized":"2024-09-10T11:30:28.659957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:12.483235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370845+0000","last_change":"2024-09-10T11:30:26.358204+0000","last_active":"2024-09-10T11:30:28.370845+0000","last_peered":"2024-09-10T11:30:28.370845+0000","last_clean":"2024-09-10T11:30:28.370845+0000","last_became_active":"2024-09-10T11:30:26.357901+0000","last_became_peered":"2024-09-10T11:30:26.357901+0000","last_unstale":"2024-09-10T11:30:28.370845+0000","last_undegraded":"2024-09-10T11:30:28.370845+0000","last_fullsized":"2024-09-10T11:30:28.370845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:06.667422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660299+0000","last_change":"2024-09-10T11:30:27.366991+0000","last_active":"2024-09-10T11:30:28.660299+0000","last_peered":"2024-09-10T11:30:28.660299+0000","last_clean":"2024-09-10T11:30:28.660299+0000","last_became_active":"2024-09-10T11:30:27.366752+0000","last_became_peered":"2024-09-10T11:30:27.366752+0000","last_unstale":"2024-09-10T11:30:28.660299+0000","last_undegraded":"2024-09-10T11:30:28.660299+0000","last_fullsized":"2024-09-10T11:30:28.660299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:47.959643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357478+0000","last_change":"2024-09-10T11:30:26.355757+0000","last_active":"2024-09-10T11:30:28.357478+0000","last_peered":"2024-09-10T11:30:28.357478+0000","last_clean":"2024-09-10T11:30:28.357478+0000","last_became_active":"2024-09-10T11:30:26.355505+0000","last_became_peered":"2024-09-10T11:30:26.355505+0000","last_unstale":"2024-09-10T11:30:28.357478+0000","last_undegraded":"2024-09-10T11:30:28.357478+0000","last_fullsized":"2024-09-10T11:30:28.357478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:28.210484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.370778+0000","last_change":"2024-09-10T11:30:27.363710+0000","last_active":"2024-09-10T11:30:28.370778+0000","last_peered":"2024-09-10T11:30:28.370778+0000","last_clean":"2024-09-10T11:30:28.370778+0000","last_became_active":"2024-09-10T11:30:27.363475+0000","last_became_peered":"2024-09-10T11:30:27.363475+0000","last_unstale":"2024-09-10T11:30:28.370778+0000","last_undegraded":"2024-09-10T11:30:28.370778+0000","last_fullsized":"2024-09-10T11:30:28.370778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:48.768550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359866+0000","last_change":"2024-09-10T11:30:26.357437+0000","last_active":"2024-09-10T11:30:28.359866+0000","last_peered":"2024-09-10T11:30:28.359866+0000","last_clean":"2024-09-10T11:30:28.359866+0000","last_became_active":"2024-09-10T11:30:26.357195+0000","last_became_peered":"2024-09-10T11:30:26.357195+0000","last_unstale":"2024-09-10T11:30:28.359866+0000","last_undegraded":"2024-09-10T11:30:28.359866+0000","last_fullsized":"2024-09-10T11:30:28.359866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:14.325407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371216+0000","last_change":"2024-09-10T11:30:27.367212+0000","last_active":"2024-09-10T11:30:28.371216+0000","last_peered":"2024-09-10T11:30:28.371216+0000","last_clean":"2024-09-10T11:30:28.371216+0000","last_became_active":"2024-09-10T11:30:27.366516+0000","last_became_peered":"2024-09-10T11:30:27.366516+0000","last_unstale":"2024-09-10T11:30:28.371216+0000","last_undegraded":"2024-09-10T11:30:28.371216+0000","last_fullsized":"2024-09-10T11:30:28.371216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:43.559533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357377+0000","last_change":"2024-09-10T11:30:26.356910+0000","last_active":"2024-09-10T11:30:28.357377+0000","last_peered":"2024-09-10T11:30:28.357377+0000","last_clean":"2024-09-10T11:30:28.357377+0000","last_became_active":"2024-09-10T11:30:26.356667+0000","last_became_peered":"2024-09-10T11:30:26.356667+0000","last_unstale":"2024-09-10T11:30:28.357377+0000","last_undegraded":"2024-09-10T11:30:28.357377+0000","last_fullsized":"2024-09-10T11:30:28.357377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:16.594543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660098+0000","last_change":"2024-09-10T11:30:27.363291+0000","last_active":"2024-09-10T11:30:28.660098+0000","last_peered":"2024-09-10T11:30:28.660098+0000","last_clean":"2024-09-10T11:30:28.660098+0000","last_became_active":"2024-09-10T11:30:27.362845+0000","last_became_peered":"2024-09-10T11:30:27.362845+0000","last_unstale":"2024-09-10T11:30:28.660098+0000","last_undegraded":"2024-09-10T11:30:28.660098+0000","last_fullsized":"2024-09-10T11:30:28.660098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.348250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:30:28.357044+0000","last_peered":"2024-09-10T11:30:28.357044+0000","last_clean":"2024-09-10T11:30:28.357044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:30:28.357044+0000","last_undegraded":"2024-09-10T11:30:28.357044+0000","last_fullsized":"2024-09-10T11:30:28.357044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660171+0000","last_change":"2024-09-10T11:30:26.351942+0000","last_active":"2024-09-10T11:30:28.660171+0000","last_peered":"2024-09-10T11:30:28.660171+0000","last_clean":"2024-09-10T11:30:28.660171+0000","last_became_active":"2024-09-10T11:30:26.350843+0000","last_became_peered":"2024-09-10T11:30:26.350843+0000","last_unstale":"2024-09-10T11:30:28.660171+0000","last_undegraded":"2024-09-10T11:30:28.660171+0000","last_fullsized":"2024-09-10T11:30:28.660171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:07.847876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358022+0000","last_change":"2024-09-10T11:30:27.368957+0000","last_active":"2024-09-10T11:30:28.358022+0000","last_peered":"2024-09-10T11:30:28.358022+0000","last_clean":"2024-09-10T11:30:28.358022+0000","last_became_active":"2024-09-10T11:30:27.368412+0000","last_became_peered":"2024-09-10T11:30:27.368412+0000","last_unstale":"2024-09-10T11:30:28.358022+0000","last_undegraded":"2024-09-10T11:30:28.358022+0000","last_fullsized":"2024-09-10T11:30:28.358022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:15.612861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660500+0000","last_change":"2024-09-10T11:30:26.357737+0000","last_active":"2024-09-10T11:30:28.660500+0000","last_peered":"2024-09-10T11:30:28.660500+0000","last_clean":"2024-09-10T11:30:28.660500+0000","last_became_active":"2024-09-10T11:30:26.357512+0000","last_became_peered":"2024-09-10T11:30:26.357512+0000","last_unstale":"2024-09-10T11:30:28.660500+0000","last_undegraded":"2024-09-10T11:30:28.660500+0000","last_fullsized":"2024-09-10T11:30:28.660500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:46:18.084482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356890+0000","last_change":"2024-09-10T11:30:27.370947+0000","last_active":"2024-09-10T11:30:28.356890+0000","last_peered":"2024-09-10T11:30:28.356890+0000","last_clean":"2024-09-10T11:30:28.356890+0000","last_became_active":"2024-09-10T11:30:27.370630+0000","last_became_peered":"2024-09-10T11:30:27.370630+0000","last_unstale":"2024-09-10T11:30:28.356890+0000","last_undegraded":"2024-09-10T11:30:28.356890+0000","last_fullsized":"2024-09-10T11:30:28.356890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:42.872782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359693+0000","last_change":"2024-09-10T11:30:26.355466+0000","last_active":"2024-09-10T11:30:28.359693+0000","last_peered":"2024-09-10T11:30:28.359693+0000","last_clean":"2024-09-10T11:30:28.359693+0000","last_became_active":"2024-09-10T11:30:26.355134+0000","last_became_peered":"2024-09-10T11:30:26.355134+0000","last_unstale":"2024-09-10T11:30:28.359693+0000","last_undegraded":"2024-09-10T11:30:28.359693+0000","last_fullsized":"2024-09-10T11:30:28.359693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:39.031878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360140+0000","last_change":"2024-09-10T11:30:27.365334+0000","last_active":"2024-09-10T11:30:28.360140+0000","last_peered":"2024-09-10T11:30:28.360140+0000","last_clean":"2024-09-10T11:30:28.360140+0000","last_became_active":"2024-09-10T11:30:27.364781+0000","last_became_peered":"2024-09-10T11:30:27.364781+0000","last_unstale":"2024-09-10T11:30:28.360140+0000","last_undegraded":"2024-09-10T11:30:28.360140+0000","last_fullsized":"2024-09-10T11:30:28.360140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:22.952484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660118+0000","last_change":"2024-09-10T11:30:26.351508+0000","last_active":"2024-09-10T11:30:28.660118+0000","last_peered":"2024-09-10T11:30:28.660118+0000","last_clean":"2024-09-10T11:30:28.660118+0000","last_became_active":"2024-09-10T11:30:26.351227+0000","last_became_peered":"2024-09-10T11:30:26.351227+0000","last_unstale":"2024-09-10T11:30:28.660118+0000","last_undegraded":"2024-09-10T11:30:28.660118+0000","last_fullsized":"2024-09-10T11:30:28.660118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:52.177357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371203+0000","last_change":"2024-09-10T11:30:27.369498+0000","last_active":"2024-09-10T11:30:28.371203+0000","last_peered":"2024-09-10T11:30:28.371203+0000","last_clean":"2024-09-10T11:30:28.371203+0000","last_became_active":"2024-09-10T11:30:27.369231+0000","last_became_peered":"2024-09-10T11:30:27.369231+0000","last_unstale":"2024-09-10T11:30:28.371203+0000","last_undegraded":"2024-09-10T11:30:28.371203+0000","last_fullsized":"2024-09-10T11:30:28.371203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:05.406568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360323+0000","last_change":"2024-09-10T11:30:26.356544+0000","last_active":"2024-09-10T11:30:28.360323+0000","last_peered":"2024-09-10T11:30:28.360323+0000","last_clean":"2024-09-10T11:30:28.360323+0000","last_became_active":"2024-09-10T11:30:26.356109+0000","last_became_peered":"2024-09-10T11:30:26.356109+0000","last_unstale":"2024-09-10T11:30:28.360323+0000","last_undegraded":"2024-09-10T11:30:28.360323+0000","last_fullsized":"2024-09-10T11:30:28.360323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:27.919918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659688+0000","last_change":"2024-09-10T11:30:27.369830+0000","last_active":"2024-09-10T11:30:28.659688+0000","last_peered":"2024-09-10T11:30:28.659688+0000","last_clean":"2024-09-10T11:30:28.659688+0000","last_became_active":"2024-09-10T11:30:27.369147+0000","last_became_peered":"2024-09-10T11:30:27.369147+0000","last_unstale":"2024-09-10T11:30:28.659688+0000","last_undegraded":"2024-09-10T11:30:28.659688+0000","last_fullsized":"2024-09-10T11:30:28.659688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:00.145985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660445+0000","last_change":"2024-09-10T11:30:26.353408+0000","last_active":"2024-09-10T11:30:28.660445+0000","last_peered":"2024-09-10T11:30:28.660445+0000","last_clean":"2024-09-10T11:30:28.660445+0000","last_became_active":"2024-09-10T11:30:26.352932+0000","last_became_peered":"2024-09-10T11:30:26.352932+0000","last_unstale":"2024-09-10T11:30:28.660445+0000","last_undegraded":"2024-09-10T11:30:28.660445+0000","last_fullsized":"2024-09-10T11:30:28.660445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:02.778492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357067+0000","last_change":"2024-09-10T11:30:27.370822+0000","last_active":"2024-09-10T11:30:28.357067+0000","last_peered":"2024-09-10T11:30:28.357067+0000","last_clean":"2024-09-10T11:30:28.357067+0000","last_became_active":"2024-09-10T11:30:27.370501+0000","last_became_peered":"2024-09-10T11:30:27.370501+0000","last_unstale":"2024-09-10T11:30:28.357067+0000","last_undegraded":"2024-09-10T11:30:28.357067+0000","last_fullsized":"2024-09-10T11:30:28.357067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:05.773002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.358086+0000","last_change":"2024-09-10T11:30:26.351464+0000","last_active":"2024-09-10T11:30:28.358086+0000","last_peered":"2024-09-10T11:30:28.358086+0000","last_clean":"2024-09-10T11:30:28.358086+0000","last_became_active":"2024-09-10T11:30:26.350802+0000","last_became_peered":"2024-09-10T11:30:26.350802+0000","last_unstale":"2024-09-10T11:30:28.358086+0000","last_undegraded":"2024-09-10T11:30:28.358086+0000","last_fullsized":"2024-09-10T11:30:28.358086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:39.722801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357861+0000","last_change":"2024-09-10T11:30:27.362871+0000","last_active":"2024-09-10T11:30:28.357861+0000","last_peered":"2024-09-10T11:30:28.357861+0000","last_clean":"2024-09-10T11:30:28.357861+0000","last_became_active":"2024-09-10T11:30:27.362533+0000","last_became_peered":"2024-09-10T11:30:27.362533+0000","last_unstale":"2024-09-10T11:30:28.357861+0000","last_undegraded":"2024-09-10T11:30:28.357861+0000","last_fullsized":"2024-09-10T11:30:28.357861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:03:34.967835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660031+0000","last_change":"2024-09-10T11:30:26.355781+0000","last_active":"2024-09-10T11:30:28.660031+0000","last_peered":"2024-09-10T11:30:28.660031+0000","last_clean":"2024-09-10T11:30:28.660031+0000","last_became_active":"2024-09-10T11:30:26.355532+0000","last_became_peered":"2024-09-10T11:30:26.355532+0000","last_unstale":"2024-09-10T11:30:28.660031+0000","last_undegraded":"2024-09-10T11:30:28.660031+0000","last_fullsized":"2024-09-10T11:30:28.660031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:34:00.971732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357833+0000","last_change":"2024-09-10T11:30:27.365497+0000","last_active":"2024-09-10T11:30:28.357833+0000","last_peered":"2024-09-10T11:30:28.357833+0000","last_clean":"2024-09-10T11:30:28.357833+0000","last_became_active":"2024-09-10T11:30:27.363947+0000","last_became_peered":"2024-09-10T11:30:27.363947+0000","last_unstale":"2024-09-10T11:30:28.357833+0000","last_undegraded":"2024-09-10T11:30:28.357833+0000","last_fullsized":"2024-09-10T11:30:28.357833+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:45:24.206828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371397+0000","last_change":"2024-09-10T11:30:26.352709+0000","last_active":"2024-09-10T11:30:28.371397+0000","last_peered":"2024-09-10T11:30:28.371397+0000","last_clean":"2024-09-10T11:30:28.371397+0000","last_became_active":"2024-09-10T11:30:26.352479+0000","last_became_peered":"2024-09-10T11:30:26.352479+0000","last_unstale":"2024-09-10T11:30:28.371397+0000","last_undegraded":"2024-09-10T11:30:28.371397+0000","last_fullsized":"2024-09-10T11:30:28.371397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:26.569057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660619+0000","last_change":"2024-09-10T11:30:27.372245+0000","last_active":"2024-09-10T11:30:28.660619+0000","last_peered":"2024-09-10T11:30:28.660619+0000","last_clean":"2024-09-10T11:30:28.660619+0000","last_became_active":"2024-09-10T11:30:27.371099+0000","last_became_peered":"2024-09-10T11:30:27.371099+0000","last_unstale":"2024-09-10T11:30:28.660619+0000","last_undegraded":"2024-09-10T11:30:28.660619+0000","last_fullsized":"2024-09-10T11:30:28.660619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:39.468681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357638+0000","last_change":"2024-09-10T11:30:26.350391+0000","last_active":"2024-09-10T11:30:28.357638+0000","last_peered":"2024-09-10T11:30:28.357638+0000","last_clean":"2024-09-10T11:30:28.357638+0000","last_became_active":"2024-09-10T11:30:26.350146+0000","last_became_peered":"2024-09-10T11:30:26.350146+0000","last_unstale":"2024-09-10T11:30:28.357638+0000","last_undegraded":"2024-09-10T11:30:28.357638+0000","last_fullsized":"2024-09-10T11:30:28.357638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:59.504068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357610+0000","last_change":"2024-09-10T11:30:27.361895+0000","last_active":"2024-09-10T11:30:28.357610+0000","last_peered":"2024-09-10T11:30:28.357610+0000","last_clean":"2024-09-10T11:30:28.357610+0000","last_became_active":"2024-09-10T11:30:27.361662+0000","last_became_peered":"2024-09-10T11:30:27.361662+0000","last_unstale":"2024-09-10T11:30:28.357610+0000","last_undegraded":"2024-09-10T11:30:28.357610+0000","last_fullsized":"2024-09-10T11:30:28.357610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:08.687699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.356658+0000","last_change":"2024-09-10T11:30:26.358473+0000","last_active":"2024-09-10T11:30:28.356658+0000","last_peered":"2024-09-10T11:30:28.356658+0000","last_clean":"2024-09-10T11:30:28.356658+0000","last_became_active":"2024-09-10T11:30:26.358231+0000","last_became_peered":"2024-09-10T11:30:26.358231+0000","last_unstale":"2024-09-10T11:30:28.356658+0000","last_undegraded":"2024-09-10T11:30:28.356658+0000","last_fullsized":"2024-09-10T11:30:28.356658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:49.799834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.359485+0000","last_change":"2024-09-10T11:30:27.362943+0000","last_active":"2024-09-10T11:30:28.359485+0000","last_peered":"2024-09-10T11:30:28.359485+0000","last_clean":"2024-09-10T11:30:28.359485+0000","last_became_active":"2024-09-10T11:30:27.362709+0000","last_became_peered":"2024-09-10T11:30:27.362709+0000","last_unstale":"2024-09-10T11:30:28.359485+0000","last_undegraded":"2024-09-10T11:30:28.359485+0000","last_fullsized":"2024-09-10T11:30:28.359485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:05.763931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.659605+0000","last_change":"2024-09-10T11:30:26.352641+0000","last_active":"2024-09-10T11:30:28.659605+0000","last_peered":"2024-09-10T11:30:28.659605+0000","last_clean":"2024-09-10T11:30:28.659605+0000","last_became_active":"2024-09-10T11:30:26.352219+0000","last_became_peered":"2024-09-10T11:30:26.352219+0000","last_unstale":"2024-09-10T11:30:28.659605+0000","last_undegraded":"2024-09-10T11:30:28.659605+0000","last_fullsized":"2024-09-10T11:30:28.659605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:01.450636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660335+0000","last_change":"2024-09-10T11:30:27.372613+0000","last_active":"2024-09-10T11:30:28.660335+0000","last_peered":"2024-09-10T11:30:28.660335+0000","last_clean":"2024-09-10T11:30:28.660335+0000","last_became_active":"2024-09-10T11:30:27.370970+0000","last_became_peered":"2024-09-10T11:30:27.370970+0000","last_unstale":"2024-09-10T11:30:28.660335+0000","last_undegraded":"2024-09-10T11:30:28.660335+0000","last_fullsized":"2024-09-10T11:30:28.660335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:51.164359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.660329+0000","last_change":"2024-09-10T11:30:27.372047+0000","last_active":"2024-09-10T11:30:28.660329+0000","last_peered":"2024-09-10T11:30:28.660329+0000","last_clean":"2024-09-10T11:30:28.660329+0000","last_became_active":"2024-09-10T11:30:27.370618+0000","last_became_peered":"2024-09-10T11:30:27.370618+0000","last_unstale":"2024-09-10T11:30:28.660329+0000","last_undegraded":"2024-09-10T11:30:28.660329+0000","last_fullsized":"2024-09-10T11:30:28.660329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:47.350659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371285+0000","last_change":"2024-09-10T11:30:26.354738+0000","last_active":"2024-09-10T11:30:28.371285+0000","last_peered":"2024-09-10T11:30:28.371285+0000","last_clean":"2024-09-10T11:30:28.371285+0000","last_became_active":"2024-09-10T11:30:26.353535+0000","last_became_peered":"2024-09-10T11:30:26.353535+0000","last_unstale":"2024-09-10T11:30:28.371285+0000","last_undegraded":"2024-09-10T11:30:28.371285+0000","last_fullsized":"2024-09-10T11:30:28.371285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:31.153745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357070+0000","last_change":"2024-09-10T11:30:27.360961+0000","last_active":"2024-09-10T11:30:28.357070+0000","last_peered":"2024-09-10T11:30:28.357070+0000","last_clean":"2024-09-10T11:30:28.357070+0000","last_became_active":"2024-09-10T11:30:27.360702+0000","last_became_peered":"2024-09-10T11:30:27.360702+0000","last_unstale":"2024-09-10T11:30:28.357070+0000","last_undegraded":"2024-09-10T11:30:28.357070+0000","last_fullsized":"2024-09-10T11:30:28.357070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:54.000251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.371486+0000","last_change":"2024-09-10T11:30:26.351609+0000","last_active":"2024-09-10T11:30:28.371486+0000","last_peered":"2024-09-10T11:30:28.371486+0000","last_clean":"2024-09-10T11:30:28.371486+0000","last_became_active":"2024-09-10T11:30:26.351211+0000","last_became_peered":"2024-09-10T11:30:26.351211+0000","last_unstale":"2024-09-10T11:30:28.371486+0000","last_undegraded":"2024-09-10T11:30:28.371486+0000","last_fullsized":"2024-09-10T11:30:28.371486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:49.786219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360066+0000","last_change":"2024-09-10T11:30:27.368686+0000","last_active":"2024-09-10T11:30:28.360066+0000","last_peered":"2024-09-10T11:30:28.360066+0000","last_clean":"2024-09-10T11:30:28.360066+0000","last_became_active":"2024-09-10T11:30:27.368329+0000","last_became_peered":"2024-09-10T11:30:27.368329+0000","last_unstale":"2024-09-10T11:30:28.360066+0000","last_undegraded":"2024-09-10T11:30:28.360066+0000","last_fullsized":"2024-09-10T11:30:28.360066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:03.672443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360276+0000","last_change":"2024-09-10T11:30:26.356867+0000","last_active":"2024-09-10T11:30:28.360276+0000","last_peered":"2024-09-10T11:30:28.360276+0000","last_clean":"2024-09-10T11:30:28.360276+0000","last_became_active":"2024-09-10T11:30:26.356480+0000","last_became_peered":"2024-09-10T11:30:26.356480+0000","last_unstale":"2024-09-10T11:30:28.360276+0000","last_undegraded":"2024-09-10T11:30:28.360276+0000","last_fullsized":"2024-09-10T11:30:28.360276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:06.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.360033+0000","last_change":"2024-09-10T11:30:27.372498+0000","last_active":"2024-09-10T11:30:28.360033+0000","last_peered":"2024-09-10T11:30:28.360033+0000","last_clean":"2024-09-10T11:30:28.360033+0000","last_became_active":"2024-09-10T11:30:27.372251+0000","last_became_peered":"2024-09-10T11:30:27.372251+0000","last_unstale":"2024-09-10T11:30:28.360033+0000","last_undegraded":"2024-09-10T11:30:28.360033+0000","last_fullsized":"2024-09-10T11:30:28.360033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:26.336629+0000","last_clean_scrub_stamp":"2024-09-10T11:30:26.336629+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:44.924775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T11:30:28.357462+0000","last_change":"2024-09-10T11:30:26.357620+0000","last_active":"2024-09-10T11:30:28.357462+0000","last_peered":"2024-09-10T11:30:28.357462+0000","last_clean":"2024-09-10T11:30:28.357462+0000","last_became_active":"2024-09-10T11:30:26.357317+0000","last_became_peered":"2024-09-10T11:30:26.357317+0000","last_unstale":"2024-09-10T11:30:28.357462+0000","last_undegraded":"2024-09-10T11:30:28.357462+0000","last_fullsized":"2024-09-10T11:30:28.357462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:25.330030+0000","last_clean_scrub_stamp":"2024-09-10T11:30:25.330030+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:16.783289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929102,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338766,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199926942,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338788,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:30:39.265 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:30:39.265 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T11:30:39.265 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T11:30:39.265 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T11:30:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.397+0000 7f1e06eac640 1 Processor -- start 2024-09-10T11:30:39.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.397+0000 7f1e06eac640 1 -- start start 2024-09-10T11:30:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e06eac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 0x7f1e000d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e06eac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e000d4e40 0x7f1e000d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e06eac640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0015ff70 con 0x7f1e0015eb80 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e06eac640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1e000d30c0 con 0x7f1e000d4e40 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e06eac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e000d1620 con 0x7f1e001626c0 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e000d4e40 0x7f1e000d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e04c48640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1e0015eb80 0x7f1e000d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45010/0 (socket says 172.21.15.99:45010) 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e000d4e40 0x7f1e000d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54626/0 (socket says 172.21.15.99:54626) 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e04c48640 1 -- 172.21.15.99:0/1150147285 learned_addr learned my addr 172.21.15.99:0/1150147285 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1df7fff640 1 --2- 172.21.15.99:0/1150147285 >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 0x7f1e000d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 -- 172.21.15.99:0/1150147285 >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 msgr2=0x7f1e000d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 --2- 172.21.15.99:0/1150147285 >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 0x7f1e000d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 -- 172.21.15.99:0/1150147285 >> v1:172.21.15.99:6789/0 conn(0x7f1e0015eb80 legacy=0x7f1e000d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1e05449640 1 -- 172.21.15.99:0/1150147285 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e00162bc0 con 0x7f1e000d4e40 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1df77fe640 1 -- 172.21.15.99:0/1150147285 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137487159 0 0) 0x7f1e0015ff70 con 0x7f1e0015eb80 2024-09-10T11:30:39.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.398+0000 7f1df7fff640 1 --2- 172.21.15.99:0/1150147285 >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 0x7f1e000d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1e05449640 1 --2- 172.21.15.99:0/1150147285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e000d4e40 0x7f1e000d1b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009d90 tx=0x7f1df802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b9e0d8e7cc19b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1df77fe640 1 -- 172.21.15.99:0/1150147285 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8031600 con 0x7f1e000d4e40 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1df77fe640 1 -- 172.21.15.99:0/1150147285 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df80317a0 con 0x7f1e000d4e40 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1df77fe640 1 -- 172.21.15.99:0/1150147285 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8031b70 con 0x7f1e000d4e40 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1e06eac640 1 -- 172.21.15.99:0/1150147285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e000d4e40 msgr2=0x7f1e000d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.399+0000 7f1e06eac640 1 --2- 172.21.15.99:0/1150147285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e000d4e40 0x7f1e000d1b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009d90 tx=0x7f1df802a610 comp rx=0 tx=0).stop 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 -- 172.21.15.99:0/1150147285 shutdown_connections 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 --2- 172.21.15.99:0/1150147285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e000d4e40 0x7f1e000d1b20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 --2- 172.21.15.99:0/1150147285 >> v2:172.21.15.196:3300/0 conn(0x7f1e001626c0 0x7f1e000d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 -- 172.21.15.99:0/1150147285 >> 172.21.15.99:0/1150147285 conn(0x7f1e0008e680 msgr2=0x7f1e0009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 -- 172.21.15.99:0/1150147285 shutdown_connections 2024-09-10T11:30:39.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.400+0000 7f1e06eac640 1 -- 172.21.15.99:0/1150147285 wait complete. 2024-09-10T11:30:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.401+0000 7f1e06eac640 1 Processor -- start 2024-09-10T11:30:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.401+0000 7f1e06eac640 1 -- start start 2024-09-10T11:30:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e000d4e40 0x7f1e00152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e0015eb80 0x7f1e000b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00163460 con 0x7f1e001626c0 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e00163180 con 0x7f1e000d4e40 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e000d5290 con 0x7f1e0015eb80 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54634/0 (socket says 172.21.15.99:54634) 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 -- 172.21.15.99:0/2550814435 learned_addr learned my addr 172.21.15.99:0/2550814435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e04c48640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e000d4e40 0x7f1e00152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1df7fff640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e0015eb80 0x7f1e000b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 -- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e0015eb80 msgr2=0x7f1e000b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e0015eb80 0x7f1e000b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 -- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e000d4e40 msgr2=0x7f1e00152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e000d4e40 0x7f1e00152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 -- 172.21.15.99:0/2550814435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e000cac40 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e05449640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009eb0 tx=0x7f1df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df803f070 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8030b60 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e000caf20 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8009450 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.402+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e000cb4e0 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.403+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e000d4900 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.404+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df803b410 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.404+0000 7f1df57fa640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 0x7f1dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.404+0000 7f1e04c48640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 0x7f1dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.404+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1df80ac370 con 0x7f1e001626c0 2024-09-10T11:30:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.405+0000 7f1e04c48640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 0x7f1dec06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.408+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1df807a6d0 con 0x7f1e001626c0 2024-09-10T11:30:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.608+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f1e0006c610 con 0x7f1e001626c0 2024-09-10T11:30:39.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.609+0000 7f1df57fa640 1 -- 172.21.15.99:0/2550814435 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1df807e9d0 con 0x7f1e001626c0 2024-09-10T11:30:39.611 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:39.611 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 msgr2=0x7f1dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 0x7f1dec06e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fd0 tx=0x7f1dfc007de0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 msgr2=0x7f1e000b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009eb0 tx=0x7f1df8004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 shutdown_connections 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1dec06be30 0x7f1dec06e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e001626c0 0x7f1e000b6260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e0015eb80 0x7f1e000b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 --2- 172.21.15.99:0/2550814435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e000d4e40 0x7f1e00152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 >> 172.21.15.99:0/2550814435 conn(0x7f1e0008e680 msgr2=0x7f1e000cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 shutdown_connections 2024-09-10T11:30:39.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.611+0000 7f1e06eac640 1 -- 172.21.15.99:0/2550814435 wait complete. 2024-09-10T11:30:39.623 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T11:30:39.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.720+0000 7fd97d253640 1 Processor -- start 2024-09-10T11:30:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.720+0000 7fd97d253640 1 -- start start 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97d253640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd97815b2d0 0x7fd9780b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97d253640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9780afbb0 0x7fd9780714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97d253640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9780b0020 con 0x7fd97815aef0 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97d253640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd97806ee10 con 0x7fd97815b2d0 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97d253640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9780cd9a0 con 0x7fd9780afbb0 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd97815b2d0 0x7fd9780b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd97815b2d0 0x7fd9780b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54636/0 (socket says 172.21.15.99:54636) 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 -- 172.21.15.99:0/3501132187 learned_addr learned my addr 172.21.15.99:0/3501132187 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd97759e640 1 --2- 172.21.15.99:0/3501132187 >> v2:172.21.15.196:3300/0 conn(0x7fd9780afbb0 0x7fd9780714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 -- 172.21.15.99:0/3501132187 >> v2:172.21.15.196:3300/0 conn(0x7fd9780afbb0 msgr2=0x7fd9780714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 --2- 172.21.15.99:0/3501132187 >> v2:172.21.15.196:3300/0 conn(0x7fd9780afbb0 0x7fd9780714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 -- 172.21.15.99:0/3501132187 >> v1:172.21.15.196:6789/0 conn(0x7fd97815aef0 legacy=0x7fd9780cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.721+0000 7fd96e59c640 1 -- 172.21.15.99:0/3501132187 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9780b0290 con 0x7fd97815b2d0 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.722+0000 7fd96e59c640 1 --2- 172.21.15.99:0/3501132187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd9780b4510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd960009870 tx=0x7fd96002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2830b9f00d3c3be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.722+0000 7fd97659c640 1 -- 172.21.15.99:0/3501132187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960037070 con 0x7fd97815b2d0 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.722+0000 7fd97659c640 1 -- 172.21.15.99:0/3501132187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd960032440 con 0x7fd97815b2d0 2024-09-10T11:30:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.722+0000 7fd97659c640 1 -- 172.21.15.99:0/3501132187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960032810 con 0x7fd97815b2d0 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 -- 172.21.15.99:0/3501132187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 msgr2=0x7fd9780b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 --2- 172.21.15.99:0/3501132187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd9780b4510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd960009870 tx=0x7fd96002a270 comp rx=0 tx=0).stop 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 -- 172.21.15.99:0/3501132187 shutdown_connections 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 --2- 172.21.15.99:0/3501132187 >> v2:172.21.15.196:3300/0 conn(0x7fd9780afbb0 0x7fd9780714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 --2- 172.21.15.99:0/3501132187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd9780b4510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 -- 172.21.15.99:0/3501132187 >> 172.21.15.99:0/3501132187 conn(0x7fd97806bc10 msgr2=0x7fd97806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 -- 172.21.15.99:0/3501132187 shutdown_connections 2024-09-10T11:30:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.723+0000 7fd97d253640 1 -- 172.21.15.99:0/3501132187 wait complete. 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.724+0000 7fd97d253640 1 Processor -- start 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.724+0000 7fd97d253640 1 -- start start 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9780afbb0 0x7fd97814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd97815aef0 0x7fd978155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd97806eb30 con 0x7fd97815b2d0 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd97806e850 con 0x7fd9780afbb0 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97d253640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9780719e0 con 0x7fd97815aef0 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54638/0 (socket says 172.21.15.99:54638) 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 -- 172.21.15.99:0/4018922224 learned_addr learned my addr 172.21.15.99:0/4018922224 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd976d9d640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9780afbb0 0x7fd97814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd96e59c640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd97815aef0 0x7fd978155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 -- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd97815aef0 msgr2=0x7fd978155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd97815aef0 0x7fd978155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 -- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9780afbb0 msgr2=0x7fd97814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9780afbb0 0x7fd97814d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 -- 172.21.15.99:0/4018922224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97814b0e0 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.725+0000 7fd97759e640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd97000cd70 tx=0x7fd97000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.726+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970003b90 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.726+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd970003d30 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.726+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970004240 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.726+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd978150380 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.726+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97815e500 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.727+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd978071da0 con 0x7fd97815b2d0 2024-09-10T11:30:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.727+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9700043e0 con 0x7fd97815b2d0 2024-09-10T11:30:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.727+0000 7fd96ffff640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 0x7fd95806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.727+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd97008b780 con 0x7fd97815b2d0 2024-09-10T11:30:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.727+0000 7fd976d9d640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 0x7fd95806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.728+0000 7fd976d9d640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 0x7fd95806e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd97814f940 tx=0x7fd968009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.732+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd970059b70 con 0x7fd97815b2d0 2024-09-10T11:30:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.928+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd97814aea0 con 0x7fd97815b2d0 2024-09-10T11:30:39.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.929+0000 7fd96ffff640 1 -- 172.21.15.99:0/4018922224 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7fd97005de70 con 0x7fd97815b2d0 2024-09-10T11:30:39.931 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:39.931 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:39.931 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 msgr2=0x7fd95806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 0x7fd95806e2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd97814f940 tx=0x7fd968009040 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 msgr2=0x7fd978157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd97000cd70 tx=0x7fd97000a660 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 shutdown_connections 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd95806be30 0x7fd95806e2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd97815b2d0 0x7fd978157bd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd97815aef0 0x7fd978155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 --2- 172.21.15.99:0/4018922224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9780afbb0 0x7fd97814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 >> 172.21.15.99:0/4018922224 conn(0x7fd97806bc10 msgr2=0x7fd978159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 shutdown_connections 2024-09-10T11:30:39.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:39.931+0000 7fd97d253640 1 -- 172.21.15.99:0/4018922224 wait complete. 2024-09-10T11:30:39.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.039+0000 7fae9b568640 1 Processor -- start 2024-09-10T11:30:40.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.039+0000 7fae9b568640 1 -- start start 2024-09-10T11:30:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae9b568640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fae9406bc10 0x7fae94163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae9b568640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae940b5140 con 0x7fae94160030 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae9b568640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae94162b40 con 0x7fae940b4930 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae9b568640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fae940b5420 con 0x7fae9406bc10 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae98b03640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fae94160030 0x7fae9406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45014/0 (socket says 172.21.15.99:45014) 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae98b03640 1 -- 172.21.15.99:0/464953638 learned_addr learned my addr 172.21.15.99:0/464953638 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821174345 0 0) 0x7fae940b5140 con 0x7fae94160030 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae80003520 con 0x7fae94160030 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae99304640 1 --2- 172.21.15.99:0/464953638 >> v2:172.21.15.196:3300/0 conn(0x7fae9406bc10 0x7fae94163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1758710093 0 0) 0x7fae80003520 con 0x7fae94160030 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 >> v2:172.21.15.196:3300/0 conn(0x7fae9406bc10 msgr2=0x7fae94163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 --2- 172.21.15.99:0/464953638 >> v2:172.21.15.196:3300/0 conn(0x7fae9406bc10 0x7fae94163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 >> v1:172.21.15.196:6789/0 conn(0x7fae940b4930 legacy=0x7fae9406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae94162e20 con 0x7fae94160030 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fae84001670 con 0x7fae94160030 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 >> v1:172.21.15.99:6789/0 conn(0x7fae94160030 legacy=0x7fae9406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 0x7fae80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae80004800 0x7fae80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800051c0 0x7fae8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae940b5420 con 0x7fae800039b0 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae8000c160 con 0x7fae80004800 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.040+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae8000c360 con 0x7fae800051c0 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 0x7fae80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae98b03640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae80004800 0x7fae80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99b05640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800051c0 0x7fae8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 -- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800051c0 msgr2=0x7fae8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800051c0 0x7fae8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 -- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae80004800 msgr2=0x7fae80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae80004800 0x7fae80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 -- 172.21.15.99:0/464953638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae80003520 con 0x7fae800039b0 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99b05640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800051c0 0x7fae8000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99b05640 1 -- 172.21.15.99:0/464953638 reap_dead start 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.041+0000 7fae99304640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 0x7fae80003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ac0 tx=0x7fae80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2818bdde40502899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90036070 con 0x7fae800039b0 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae90031400 con 0x7fae800039b0 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae8bfff640 1 -- 172.21.15.99:0/464953638 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae90009690 con 0x7fae800039b0 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 -- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 msgr2=0x7fae80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 0x7fae80003dd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ac0 tx=0x7fae80005830 comp rx=0 tx=0).stop 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 -- 172.21.15.99:0/464953638 shutdown_connections 2024-09-10T11:30:40.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 --2- 172.21.15.99:0/464953638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae800039b0 0x7fae80003dd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 -- 172.21.15.99:0/464953638 >> 172.21.15.99:0/464953638 conn(0x7fae940b3630 msgr2=0x7fae940b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 -- 172.21.15.99:0/464953638 shutdown_connections 2024-09-10T11:30:40.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.042+0000 7fae9b568640 1 -- 172.21.15.99:0/464953638 wait complete. 2024-09-10T11:30:40.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.043+0000 7fae9b568640 1 Processor -- start 2024-09-10T11:30:40.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.043+0000 7fae9b568640 1 -- start start 2024-09-10T11:30:40.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800039b0 0x7fae9415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 0x7fae941f9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae940d1560 con 0x7fae941432c0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae940b4dc0 con 0x7fae941f8ce0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae9b568640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fae940d16e0 con 0x7fae800039b0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54660/0 (socket says 172.21.15.99:54660) 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 -- 172.21.15.99:0/1960890752 learned_addr learned my addr 172.21.15.99:0/1960890752 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae99304640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800039b0 0x7fae9415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae99b05640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 0x7fae941f9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 -- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800039b0 msgr2=0x7fae9415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800039b0 0x7fae9415e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 -- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 msgr2=0x7fae941f9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 0x7fae941f9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae98b03640 1 -- 172.21.15.99:0/1960890752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9415ee30 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.044+0000 7fae99b05640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 0x7fae941f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae98b03640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae84007340 tx=0x7fae84007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400f940 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae940afd70 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400fd40 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.045+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae941e84b0 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.046+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae941e82a0 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.046+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae8401d460 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.046+0000 7fae89ffb640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 0x7fae7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.047+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae8408ae60 con 0x7fae941432c0 2024-09-10T11:30:40.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.047+0000 7fae99304640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 0x7fae7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.047+0000 7fae99304640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 0x7fae7806e320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae90033000 tx=0x7fae90008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.051+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae84059810 con 0x7fae941432c0 2024-09-10T11:30:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.253+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae940b5140 con 0x7fae941432c0 2024-09-10T11:30:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.254+0000 7fae89ffb640 1 -- 172.21.15.99:0/1960890752 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7fae84059110 con 0x7fae941432c0 2024-09-10T11:30:40.256 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:40.256 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 msgr2=0x7fae7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 0x7fae7806e320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fae90033000 tx=0x7fae90008030 comp rx=0 tx=0).stop 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 msgr2=0x7fae941f87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fae84007340 tx=0x7fae84007800 comp rx=0 tx=0).stop 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 shutdown_connections 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fae7806be60 0x7fae7806e320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae941f8ce0 0x7fae941f9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae941432c0 0x7fae941f87a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 --2- 172.21.15.99:0/1960890752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae800039b0 0x7fae9415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 >> 172.21.15.99:0/1960890752 conn(0x7fae940b3630 msgr2=0x7fae94140ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 shutdown_connections 2024-09-10T11:30:40.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.256+0000 7fae9b568640 1 -- 172.21.15.99:0/1960890752 wait complete. 2024-09-10T11:30:40.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.364+0000 7ff68bd66640 1 Processor -- start 2024-09-10T11:30:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 -- start start 2024-09-10T11:30:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6841626c0 0x7ff6840d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6840d4e40 0x7ff6840d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6840d52f0 con 0x7ff68415eb80 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff6840d30c0 con 0x7ff6840d4e40 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68bd66640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff68415ff70 con 0x7ff6841626c0 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff68a303640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff6840d4e40 0x7ff6840d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff689b02640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff68415eb80 0x7ff6840d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45018/0 (socket says 172.21.15.99:45018) 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.365+0000 7ff689b02640 1 -- 172.21.15.99:0/3331921747 learned_addr learned my addr 172.21.15.99:0/3331921747 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff689301640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6841626c0 0x7ff6840d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff68a303640 1 -- 172.21.15.99:0/3331921747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6841626c0 msgr2=0x7ff6840d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff68a303640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6841626c0 0x7ff6840d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff68a303640 1 -- 172.21.15.99:0/3331921747 >> v1:172.21.15.99:6789/0 conn(0x7ff68415eb80 legacy=0x7ff6840d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff68a303640 1 -- 172.21.15.99:0/3331921747 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6840d1620 con 0x7ff6840d4e40 2024-09-10T11:30:40.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff688b00640 1 -- 172.21.15.99:0/3331921747 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806872783 0 0) 0x7ff674001110 con 0x7ff68415eb80 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.366+0000 7ff68a303640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6840d4e40 0x7ff6840d1b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009d90 tx=0x7ff67c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c19a5f4ffa657 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff688b00640 1 -- 172.21.15.99:0/3331921747 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c031600 con 0x7ff6840d4e40 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff688b00640 1 -- 172.21.15.99:0/3331921747 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff67c0317a0 con 0x7ff6840d4e40 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff688b00640 1 -- 172.21.15.99:0/3331921747 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c031b70 con 0x7ff6840d4e40 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 -- 172.21.15.99:0/3331921747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6840d4e40 msgr2=0x7ff6840d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6840d4e40 0x7ff6840d1b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff67c009d90 tx=0x7ff67c02a5d0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 -- 172.21.15.99:0/3331921747 shutdown_connections 2024-09-10T11:30:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6840d4e40 0x7ff6840d1b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 --2- 172.21.15.99:0/3331921747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6841626c0 0x7ff6840d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 -- 172.21.15.99:0/3331921747 >> 172.21.15.99:0/3331921747 conn(0x7ff68408e680 msgr2=0x7ff68409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 -- 172.21.15.99:0/3331921747 shutdown_connections 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.367+0000 7ff68bd66640 1 -- 172.21.15.99:0/3331921747 wait complete. 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.368+0000 7ff68bd66640 1 Processor -- start 2024-09-10T11:30:40.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.368+0000 7ff68bd66640 1 -- start start 2024-09-10T11:30:40.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6840d4e40 0x7ff6841572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6841626c0 0x7ff6840d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff684163180 con 0x7ff68415eb80 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff684162ea0 con 0x7ff6841626c0 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68bd66640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff684163460 con 0x7ff6840d4e40 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff689301640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff689301640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54678/0 (socket says 172.21.15.99:54678) 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff689301640 1 -- 172.21.15.99:0/1770421856 learned_addr learned my addr 172.21.15.99:0/1770421856 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff68a303640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6841626c0 0x7ff6840d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.369+0000 7ff689b02640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6840d4e40 0x7ff6841572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 -- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6840d4e40 msgr2=0x7ff6841572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6840d4e40 0x7ff6841572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 -- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6841626c0 msgr2=0x7ff6840d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6841626c0 0x7ff6840d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 -- 172.21.15.99:0/1770421856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68415bbc0 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff689301640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff67800d600 tx=0x7ff67800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678004240 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6780043e0 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678003830 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68415bea0 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.370+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68415c460 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.371+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6840d4900 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.371+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6780039d0 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.372+0000 7ff6827fc640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 0x7ff67006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.372+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff67808b170 con 0x7ff68415eb80 2024-09-10T11:30:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.372+0000 7ff689b02640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 0x7ff67006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.372+0000 7ff689b02640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 0x7ff67006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff674000f60 tx=0x7ff674007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.376+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff678059b20 con 0x7ff68415eb80 2024-09-10T11:30:40.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.542+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6840c8570 con 0x7ff68415eb80 2024-09-10T11:30:40.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.543+0000 7ff6827fc640 1 -- 172.21.15.99:0/1770421856 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7ff678059420 con 0x7ff68415eb80 2024-09-10T11:30:40.545 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:40.545 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:40.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.545+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 msgr2=0x7ff67006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.545+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 0x7ff67006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff674000f60 tx=0x7ff674007c00 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.545+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 msgr2=0x7ff6841608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.545+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff67800d600 tx=0x7ff67800dac0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 shutdown_connections 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67006be30 0x7ff67006e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6841626c0 0x7ff6840d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff68415eb80 0x7ff6841608d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 --2- 172.21.15.99:0/1770421856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6840d4e40 0x7ff6841572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 >> 172.21.15.99:0/1770421856 conn(0x7ff68408e680 msgr2=0x7ff6840d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 shutdown_connections 2024-09-10T11:30:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.546+0000 7ff68bd66640 1 -- 172.21.15.99:0/1770421856 wait complete. 2024-09-10T11:30:40.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:40.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.654+0000 7f6e7b703640 1 Processor -- start 2024-09-10T11:30:40.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 -- start start 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7415eb80 0x7f6e740d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e741626c0 0x7f6e740d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6e740d4e40 0x7f6e740d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6e740d52d0 con 0x7f6e741626c0 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6e740d1620 con 0x7f6e740d4e40 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.655+0000 7f6e7b703640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e74162bc0 con 0x7f6e7415eb80 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e741626c0 0x7f6e740d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e741626c0 0x7f6e740d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54688/0 (socket says 172.21.15.99:54688) 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 -- 172.21.15.99:0/742638835 learned_addr learned my addr 172.21.15.99:0/742638835 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e7949f640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7415eb80 0x7f6e740d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e79ca0640 1 --2- 172.21.15.99:0/742638835 >> v2:172.21.15.196:3300/0 conn(0x7f6e740d4e40 0x7f6e740d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 -- 172.21.15.99:0/742638835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7415eb80 msgr2=0x7f6e740d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7415eb80 0x7f6e740d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 -- 172.21.15.99:0/742638835 >> v2:172.21.15.196:3300/0 conn(0x7f6e740d4e40 msgr2=0x7f6e740d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:40.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/742638835 >> v2:172.21.15.196:3300/0 conn(0x7f6e740d4e40 0x7f6e740d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 -- 172.21.15.99:0/742638835 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e74162ea0 con 0x7f6e741626c0 2024-09-10T11:30:40.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.656+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e741626c0 0x7f6e740d4900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e74162600 tx=0x7f6e6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f182bb70b5b889e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e6bfff640 1 -- 172.21.15.99:0/742638835 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64037070 con 0x7f6e741626c0 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e6bfff640 1 -- 172.21.15.99:0/742638835 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e64032440 con 0x7f6e741626c0 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e6bfff640 1 -- 172.21.15.99:0/742638835 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64032810 con 0x7f6e741626c0 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 -- 172.21.15.99:0/742638835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e741626c0 msgr2=0x7f6e740d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e741626c0 0x7f6e740d4900 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e74162600 tx=0x7f6e6402a140 comp rx=0 tx=0).stop 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 -- 172.21.15.99:0/742638835 shutdown_connections 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 --2- 172.21.15.99:0/742638835 >> v2:172.21.15.196:3300/0 conn(0x7f6e740d4e40 0x7f6e740d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e741626c0 0x7f6e740d4900 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 --2- 172.21.15.99:0/742638835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7415eb80 0x7f6e740d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 -- 172.21.15.99:0/742638835 >> 172.21.15.99:0/742638835 conn(0x7f6e7408e680 msgr2=0x7f6e7409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 -- 172.21.15.99:0/742638835 shutdown_connections 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.657+0000 7f6e7b703640 1 -- 172.21.15.99:0/742638835 wait complete. 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.658+0000 7f6e7b703640 1 Processor -- start 2024-09-10T11:30:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.658+0000 7f6e7b703640 1 -- start start 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 0x7f6e740b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 0x7f6e740b9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e740d2ef0 con 0x7f6e7415eb80 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e74163460 con 0x7f6e741626c0 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7b703640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e740d3070 con 0x7f6e740d4e40 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54690/0 (socket says 172.21.15.99:54690) 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 -- 172.21.15.99:0/1271240912 learned_addr learned my addr 172.21.15.99:0/1271240912 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e79ca0640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 0x7f6e740b9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e7949f640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 0x7f6e740b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 -- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 msgr2=0x7f6e740b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 0x7f6e740b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 -- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 msgr2=0x7f6e740b9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 0x7f6e740b9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.659+0000 7f6e78c9e640 1 -- 172.21.15.99:0/1271240912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e740babc0 con 0x7f6e7415eb80 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e79ca0640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 0x7f6e740b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e7949f640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 0x7f6e740b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e78c9e640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e6402a180 tx=0x7f6e640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64041070 con 0x7f6e7415eb80 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6403c890 con 0x7f6e7415eb80 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e740ba600 con 0x7f6e7415eb80 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64037040 con 0x7f6e7415eb80 2024-09-10T11:30:40.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.660+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e740b8630 con 0x7f6e7415eb80 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.661+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e740d29b0 con 0x7f6e7415eb80 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.662+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e64044070 con 0x7f6e7415eb80 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.662+0000 7f6e69ffb640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 0x7f6e5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.662+0000 7f6e7949f640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 0x7f6e5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.662+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e640abb00 con 0x7f6e7415eb80 2024-09-10T11:30:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.663+0000 7f6e7949f640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 0x7f6e5806e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00b970 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:40.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.667+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e740babc0 con 0x7f6e7415eb80 2024-09-10T11:30:40.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.863+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e7406de80 con 0x7f6e7415eb80 2024-09-10T11:30:40.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.864+0000 7f6e69ffb640 1 -- 172.21.15.99:0/1271240912 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7f6e6407a660 con 0x7f6e7415eb80 2024-09-10T11:30:40.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:40.866 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:40.866 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 msgr2=0x7f6e5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 0x7f6e5806e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00b970 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 msgr2=0x7f6e74157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e6402a180 tx=0x7f6e640046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 shutdown_connections 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e5806be30 0x7f6e5806e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e741626c0 0x7f6e740b9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e7415eb80 0x7f6e74157d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 --2- 172.21.15.99:0/1271240912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e740d4e40 0x7f6e740b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:40.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 >> 172.21.15.99:0/1271240912 conn(0x7f6e7408e680 msgr2=0x7f6e740b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:40.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 shutdown_connections 2024-09-10T11:30:40.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:40.866+0000 7f6e7b703640 1 -- 172.21.15.99:0/1271240912 wait complete. 2024-09-10T11:30:41.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:30:26.684199+0000', 'modified': '2024-09-10T11:30:28.690762+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4303}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4303': {'gid': 4303, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/2032905820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 2032905820}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 2032905820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4303, 'qdb_cluster': [4303]} 2024-09-10T11:30:41.281 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:30:41.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T11:30:41.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.379+0000 7f62fe237640 1 Processor -- start 2024-09-10T11:30:41.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.379+0000 7f62fe237640 1 -- start start 2024-09-10T11:30:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62fe237640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62fe237640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62f80d3a30 0x7f62f80d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62fe237640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f80d2990 con 0x7f62f80d15a0 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62fe237640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f62f80d2c70 con 0x7f62f80d3a30 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62fe237640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62f80d2f50 con 0x7f62f815efc0 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62f80d3a30 0x7f62f80d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62f80d3a30 0x7f62f80d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54706/0 (socket says 172.21.15.99:54706) 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 -- 172.21.15.99:0/2342511684 learned_addr learned my addr 172.21.15.99:0/2342511684 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f77fe640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 -- 172.21.15.99:0/2342511684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 msgr2=0x7f62f80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 -- 172.21.15.99:0/2342511684 >> v1:172.21.15.196:6789/0 conn(0x7f62f80d15a0 legacy=0x7f62f8162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.380+0000 7f62f6ffd640 1 -- 172.21.15.99:0/2342511684 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f80d3230 con 0x7f62f80d3a30 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.381+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d3a30 0x7f62f80d0930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098a0 tx=0x7f62ec02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5425daab31d854fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.381+0000 7f62f67fc640 1 -- 172.21.15.99:0/2342511684 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec037070 con 0x7f62f80d3a30 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.381+0000 7f62f67fc640 1 -- 172.21.15.99:0/2342511684 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec032440 con 0x7f62f80d3a30 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.381+0000 7f62f67fc640 1 -- 172.21.15.99:0/2342511684 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec032810 con 0x7f62f80d3a30 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 -- 172.21.15.99:0/2342511684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d3a30 msgr2=0x7f62f80d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d3a30 0x7f62f80d0930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098a0 tx=0x7f62ec02a250 comp rx=0 tx=0).stop 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 -- 172.21.15.99:0/2342511684 shutdown_connections 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d3a30 0x7f62f80d0930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 --2- 172.21.15.99:0/2342511684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 -- 172.21.15.99:0/2342511684 >> 172.21.15.99:0/2342511684 conn(0x7f62f80d4c20 msgr2=0x7f62f80d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 -- 172.21.15.99:0/2342511684 shutdown_connections 2024-09-10T11:30:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.382+0000 7f62fe237640 1 -- 172.21.15.99:0/2342511684 wait complete. 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.383+0000 7f62fe237640 1 Processor -- start 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.383+0000 7f62fe237640 1 -- start start 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62f80d3a30 0x7f62f80cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62f80d1de0 con 0x7f62f80d15a0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62f80d1c60 con 0x7f62f80d3a30 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62f8163110 con 0x7f62f815efc0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54722/0 (socket says 172.21.15.99:54722) 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 -- 172.21.15.99:0/1634315942 learned_addr learned my addr 172.21.15.99:0/1634315942 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62f80d3a30 0x7f62f80cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f7fff640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 -- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 msgr2=0x7f62f80b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 -- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62f80d3a30 msgr2=0x7f62f80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62f80d3a30 0x7f62f80cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 -- 172.21.15.99:0/1634315942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f815f5b0 con 0x7f62f80d15a0 2024-09-10T11:30:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f7fff640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62f77fe640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c820 tx=0x7f62e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62dffff640 1 -- 172.21.15.99:0/1634315942 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8015070 con 0x7f62f80d15a0 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62dffff640 1 -- 172.21.15.99:0/1634315942 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8004020 con 0x7f62f80d15a0 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f815f890 con 0x7f62f80d15a0 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62dffff640 1 -- 172.21.15.99:0/1634315942 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80043f0 con 0x7f62f80d15a0 2024-09-10T11:30:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.384+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f80ce060 con 0x7f62f80d15a0 2024-09-10T11:30:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.386+0000 7f62dffff640 1 -- 172.21.15.99:0/1634315942 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62e8007830 con 0x7f62f80d15a0 2024-09-10T11:30:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.386+0000 7f62dffff640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 0x7f62e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.386+0000 7f62dffff640 1 -- 172.21.15.99:0/1634315942 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62e808abe0 con 0x7f62f80d15a0 2024-09-10T11:30:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.386+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 0x7f62e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.387+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 0x7f62e406e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f62ec034000 tx=0x7f62ec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.387+0000 7f6300fce740 1 Processor -- start 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.387+0000 7f6300fce740 1 -- start start 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fefa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fa48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d156f0c520 con 0x55d156fa7a80 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d156eb43e0 con 0x55d156faa2c0 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d156f084e0 con 0x55d156faba50 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54724/0 (socket says 172.21.15.99:54724) 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3153087067 learned_addr learned my addr 172.21.15.99:0/3153087067 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fefa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 msgr2=0x55d156fa48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f7fff640 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fa48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fa48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 msgr2=0x55d156fefa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fefa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.388+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3153087067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d156ecf2f0 con 0x55d156fa7a80 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0099c0 tx=0x7f62ec02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ff338681261f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f62dd7fa640 1 -- 172.21.15.99:0/3153087067 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec037040 con 0x55d156fa7a80 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f62dd7fa640 1 -- 172.21.15.99:0/3153087067 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec003ba0 con 0x55d156fa7a80 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f62dd7fa640 1 -- 172.21.15.99:0/3153087067 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec003f70 con 0x55d156fa7a80 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f6300fce740 1 -- 172.21.15.99:0/3153087067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 msgr2=0x55d156faa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f6300fce740 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0099c0 tx=0x7f62ec02c000 comp rx=0 tx=0).stop 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f6300fce740 1 -- 172.21.15.99:0/3153087067 shutdown_connections 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f6300fce740 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fa48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.389+0000 7f6300fce740 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156fa7a80 0x55d156faa6a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 --2- 172.21.15.99:0/3153087067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fefa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 -- 172.21.15.99:0/3153087067 >> 172.21.15.99:0/3153087067 conn(0x55d156fa6a80 msgr2=0x55d156feef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 -- 172.21.15.99:0/3153087067 shutdown_connections 2024-09-10T11:30:41.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 -- 172.21.15.99:0/3153087067 wait complete. 2024-09-10T11:30:41.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 -- start start 2024-09-10T11:30:41.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.390+0000 7f6300fce740 1 Processor -- start 2024-09-10T11:30:41.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fcaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156fcd460 0x55d156fe88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d156fa5f20 con 0x55d156faba50 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d156e95450 con 0x55d156faa2c0 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d156ff0290 con 0x55d156fcd460 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f6300fce740 10 client.0 _ll_register_callbacks cb 0x55d156c1eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54726/0 (socket says 172.21.15.99:54726) 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3941593471 learned_addr learned my addr 172.21.15.99:0/3941593471 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fcaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f7fff640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156fcd460 0x55d156fe88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156fcd460 msgr2=0x55d156fe88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156fcd460 0x55d156fe88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 msgr2=0x55d156fcaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fcaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3941593471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d156c1f2d0 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0099c0 tx=0x7f62ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac7bedcb46b0faa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec037070 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.391+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62ec03ea20 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ec003ba0 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f62ec058050 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 10 client.4593 fetch_fsmap learned FSMap version 6 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55d156ce96b0 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec003ba0 con 0x55d156faba50 2024-09-10T11:30:41.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f62ec05b020 con 0x55d156faba50 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 10 client.4593 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 10 client.4593 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 20 client.4593 populate_metadata read hostname 'smithi099' 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fd2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 4 client.4593 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 --> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d156f19c50 con 0x55d156fd2a00 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fd2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62d67fc640 10 client.4593 ms_handle_connect on v2:172.21.15.196:6834/2032905820 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.392+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fd2de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253e7efdbf0d1747 server_cookie=b633e949bb65157a in_seq=0 out_seq=0 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.393+0000 7f62d67fc640 1 -- 172.21.15.99:0/3941593471 <== mds.0 v2:172.21.15.196:6834/2032905820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d156f19c50 con 0x55d156fd2a00 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.393+0000 7f62d67fc640 10 client.4593 handle_command_reply: tid=0 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.393+0000 7f6300fce740 1 client.4593 shutdown 2024-09-10T11:30:41.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 msgr2=0x55d156fccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0099c0 tx=0x7f62ec004730 comp rx=0 tx=0).stop 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 shutdown_connections 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fd2de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f62d67fc640 0 client.4593 ms_handle_reset on v2:172.21.15.196:6834/2032905820 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156fcd460 0x55d156fe88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d156fccf20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 --2- 172.21.15.99:0/3941593471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156faa2c0 0x55d156fcaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 >> 172.21.15.99:0/3941593471 conn(0x55d156fa7a80 msgr2=0x55d156feddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 shutdown_connections 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 1 -- 172.21.15.99:0/3941593471 wait complete. 2024-09-10T11:30:41.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.394+0000 7f6300fce740 20 client.4593 trim_cache size 0 max 16384 2024-09-10T11:30:41.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.411+0000 7f6300fce740 1 Processor -- start 2024-09-10T11:30:41.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.411+0000 7f6300fce740 1 -- start start 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fd34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fecee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62f82002f0 con 0x55d156faa2c0 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81ff7d0 con 0x55d156fcd460 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62f81ffcd0 con 0x55d156faba50 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54730/0 (socket says 172.21.15.99:54730) 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 -- 172.21.15.99:0/2850907168 learned_addr learned my addr 172.21.15.99:0/2850907168 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 -- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 msgr2=0x55d156fd34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fd34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 -- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 msgr2=0x55d156fecee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f7fff640 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fecee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fecee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f77fe640 1 -- 172.21.15.99:0/2850907168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d156f9f470 con 0x55d156faa2c0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.412+0000 7f62f7fff640 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f62f77fe640 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62e80553a0 tx=0x7f62e8078060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69169a2ef17052a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f62dcff9640 1 -- 172.21.15.99:0/2850907168 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e801e040 con 0x55d156faa2c0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f62dcff9640 1 -- 172.21.15.99:0/2850907168 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8057240 con 0x55d156faa2c0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f62dcff9640 1 -- 172.21.15.99:0/2850907168 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8058390 con 0x55d156faa2c0 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 -- 172.21.15.99:0/2850907168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 msgr2=0x55d156fd2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62e80553a0 tx=0x7f62e8078060 comp rx=0 tx=0).stop 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 -- 172.21.15.99:0/2850907168 shutdown_connections 2024-09-10T11:30:41.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fecee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faba50 0x55d156fd34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 --2- 172.21.15.99:0/2850907168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faa2c0 0x55d156fd2f90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.413+0000 7f6300fce740 1 -- 172.21.15.99:0/2850907168 >> 172.21.15.99:0/2850907168 conn(0x55d156fa7a80 msgr2=0x55d156fcbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.414+0000 7f6300fce740 1 -- 172.21.15.99:0/2850907168 shutdown_connections 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.414+0000 7f6300fce740 1 -- 172.21.15.99:0/2850907168 wait complete. 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.414+0000 7f6300fce740 1 -- start start 2024-09-10T11:30:41.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.414+0000 7f6300fce740 1 Processor -- start 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faa2c0 0x55d156fd3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fa39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d156fe90e0 con 0x55d156faba50 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d156fa5660 con 0x55d156fcd460 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55d157002800 con 0x55d156faa2c0 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f6300fce740 10 client.0 _ll_register_callbacks cb 0x7f62ec004470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54732/0 (socket says 172.21.15.99:54732) 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3679840103 learned_addr learned my addr 172.21.15.99:0/3679840103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faa2c0 0x55d156fd3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f7fff640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fa39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faa2c0 msgr2=0x55d156fd3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faa2c0 0x55d156fd3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 msgr2=0x55d156fa39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fa39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 -- 172.21.15.99:0/3679840103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d156fcc880 con 0x55d156faba50 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f7fff640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fa39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62f6ffd640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0090b0 tx=0x7f62ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c46c24501ef97d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.415+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec003ba0 con 0x55d156faba50 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62ec03e460 con 0x55d156faba50 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e8057240 con 0x55d156faba50 2024-09-10T11:30:41.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f62ec03ed80 con 0x55d156faba50 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 10 client.4599 fetch_fsmap learned FSMap version 6 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55d156fec1d0 con 0x55d156faba50 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e8057240 con 0x55d156faba50 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f62ec05b030 con 0x55d156faba50 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 10 client.4599 fetch_fsmap finished waiting for FSMap version 6 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 10 client.4599 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 20 client.4599 populate_metadata read hostname 'smithi099' 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fa3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 4 client.4599 mds_command: new command op to 4303 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 --> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] -- command(tid 0: {"prefix": "status"}) -- 0x55d156f13fa0 con 0x55d156fd2a00 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.416+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fa3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62f77fe640 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fa3f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f722300d6e7c9c09 server_cookie=232f8e5a81981d0d in_seq=0 out_seq=0 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62d67fc640 10 client.4599 ms_handle_connect on v2:172.21.15.196:6834/2032905820 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62d67fc640 1 -- 172.21.15.99:0/3679840103 <== mds.0 v2:172.21.15.196:6834/2032905820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55d156f13fa0 con 0x55d156fd2a00 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62d67fc640 10 client.4599 handle_command_reply: tid=0 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 client.4599 shutdown 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 msgr2=0x55d15706ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0090b0 tx=0x7f62ec0046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 shutdown_connections 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:6834/2032905820,v1:172.21.15.196:6835/2032905820] conn(0x55d156fd2a00 0x55d156fa3f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d156fcd460 0x55d156fa39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62d67fc640 0 client.4599 ms_handle_reset on v2:172.21.15.196:6834/2032905820 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d156faba50 0x55d15706ed70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 --2- 172.21.15.99:0/3679840103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d156faa2c0 0x55d156fd3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 >> 172.21.15.99:0/3679840103 conn(0x55d156fa7a80 msgr2=0x55d156feedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 shutdown_connections 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 1 -- 172.21.15.99:0/3679840103 wait complete. 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f6300fce740 20 client.4599 trim_cache size 0 max 16384 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4303, 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:30:41.426 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 14.052004066, 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 6, 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 20, 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 20, 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 20.083120605000001 2024-09-10T11:30:41.427 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 msgr2=0x7f62e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 0x7f62e406e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f62ec034000 tx=0x7f62ec02e000 comp rx=0 tx=0).stop 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 msgr2=0x7f62f81f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.417+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62e800c820 tx=0x7f62e800cce0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 shutdown_connections 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62e406be30 0x7f62e406e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62f815efc0 0x7f62f80b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62f80d3a30 0x7f62f80cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 --2- 172.21.15.99:0/1634315942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62f80d15a0 0x7f62f81f4bb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 >> 172.21.15.99:0/1634315942 conn(0x7f62f80d4c20 msgr2=0x7f62f80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 shutdown_connections 2024-09-10T11:30:41.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.418+0000 7f62fe237640 1 -- 172.21.15.99:0/1634315942 wait complete. 2024-09-10T11:30:41.430 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T11:30:41.440 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T11:30:41.440 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T11:30:41.441 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T11:30:41.441 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T11:30:41.481 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T11:30:41.481 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T11:30:41.481 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:41.482 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:41.482 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:41.482 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:30:41.482 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:41.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:41.483 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:30:41.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:41.499 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-10T11:30:41.565 INFO:teuthology.orchestra.run.smithi099.stderr:Cannot find device "ceph-brx" 2024-09-10T11:30:41.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:30:41.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:41.567 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:30:41.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:41.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-09-10T11:30:41.662 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-09-10T11:30:41.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:30:41.664 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T11:30:41.664 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:30:41.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:30:41.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.763+0000 7f87adce4640 1 Processor -- start 2024-09-10T11:30:41.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.763+0000 7f87adce4640 1 -- start start 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a8163580 0x7f87a80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 0x7f87a80d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87a806cd20 0x7f87a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f87a80b3620 con 0x7f87a806cd20 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f87a806d690 con 0x7f87a80b3960 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87adce4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87a806d960 con 0x7f87a8163580 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87a7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87a806cd20 0x7f87a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87a7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87a806cd20 0x7f87a806d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54740/0 (socket says 172.21.15.99:54740) 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.764+0000 7f87a7fff640 1 -- 172.21.15.99:0/4123374056 learned_addr learned my addr 172.21.15.99:0/4123374056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 -- 172.21.15.99:0/4123374056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a8163580 msgr2=0x7f87a80b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a77fe640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a8163580 0x7f87a80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a6ffd640 1 --2- 172.21.15.99:0/4123374056 >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 0x7f87a80d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a8163580 0x7f87a80b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 -- 172.21.15.99:0/4123374056 >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 msgr2=0x7f87a80d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 --2- 172.21.15.99:0/4123374056 >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 0x7f87a80d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 -- 172.21.15.99:0/4123374056 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a806dc40 con 0x7f87a806cd20 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a6ffd640 1 --2- 172.21.15.99:0/4123374056 >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 0x7f87a80d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a7fff640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a806d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d3a80 tx=0x7f87a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f0186293e4cf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.765+0000 7f87a67fc640 1 -- 172.21.15.99:0/4123374056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0036070 con 0x7f87a806cd20 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87a67fc640 1 -- 172.21.15.99:0/4123374056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a0031600 con 0x7f87a806cd20 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87a67fc640 1 -- 172.21.15.99:0/4123374056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a00319d0 con 0x7f87a806cd20 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 -- 172.21.15.99:0/4123374056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 msgr2=0x7f87a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a806d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d3a80 tx=0x7f87a002a610 comp rx=0 tx=0).stop 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 -- 172.21.15.99:0/4123374056 shutdown_connections 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a806d150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 --2- 172.21.15.99:0/4123374056 >> v2:172.21.15.196:3300/0 conn(0x7f87a80b3960 0x7f87a80d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 --2- 172.21.15.99:0/4123374056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a8163580 0x7f87a80b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 -- 172.21.15.99:0/4123374056 >> 172.21.15.99:0/4123374056 conn(0x7f87a80b4570 msgr2=0x7f87a80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 -- 172.21.15.99:0/4123374056 shutdown_connections 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.766+0000 7f87adce4640 1 -- 172.21.15.99:0/4123374056 wait complete. 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.767+0000 7f87adce4640 1 Processor -- start 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.767+0000 7f87adce4640 1 -- start start 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a80b3960 0x7f87a80ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8163580 0x7f87a80cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80d2120 con 0x7f87a806cd20 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87a80d16c0 con 0x7f87a8163580 2024-09-10T11:30:41.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87adce4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87a80af850 con 0x7f87a80b3960 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:54756/0 (socket says 172.21.15.99:54756) 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 -- 172.21.15.99:0/1772283312 learned_addr learned my addr 172.21.15.99:0/1772283312 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 -- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a80b3960 msgr2=0x7f87a80ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a80b3960 0x7f87a80ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 -- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8163580 msgr2=0x7f87a80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.768+0000 7f87a77fe640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8163580 0x7f87a80cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f87a77fe640 1 -- 172.21.15.99:0/1772283312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a81f5ea0 con 0x7f87a806cd20 2024-09-10T11:30:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f87a77fe640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f879800c820 tx=0x7f879800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8798015070 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8798004020 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87980043f0 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a80cb390 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.769+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a80cb910 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.770+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87a806d150 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.770+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8798007830 con 0x7f87a806cd20 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.771+0000 7f878ffff640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 0x7f879406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.771+0000 7f87a6ffd640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 0x7f879406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:41.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.771+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f879808ac00 con 0x7f87a806cd20 2024-09-10T11:30:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.771+0000 7f87a6ffd640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 0x7f879406e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b09f0 tx=0x7f879c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.775+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8798091080 con 0x7f87a806cd20 2024-09-10T11:30:41.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.973+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f87a80cbb10 con 0x7f87a806cd20 2024-09-10T11:30:41.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.974+0000 7f878ffff640 1 -- 172.21.15.99:0/1772283312 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f87980595b0 con 0x7f87a806cd20 2024-09-10T11:30:41.976 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 msgr2=0x7f879406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 0x7f879406e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b09f0 tx=0x7f879c008040 comp rx=0 tx=0).stop 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 msgr2=0x7f87a815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f879800c820 tx=0x7f879800cce0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 shutdown_connections 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879406be60 0x7f879406e320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87a8163580 0x7f87a80cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87a80b3960 0x7f87a80ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 --2- 172.21.15.99:0/1772283312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a806cd20 0x7f87a815dcb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:41.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 >> 172.21.15.99:0/1772283312 conn(0x7f87a80b4570 msgr2=0x7f87a806f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:41.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 shutdown_connections 2024-09-10T11:30:41.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:41.976+0000 7f87adce4640 1 -- 172.21.15.99:0/1772283312 wait complete. 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:30:41.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:30:41.989 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:30:41.989 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:30:41.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:41.989 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:30:42.004 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:30:42.005 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42807sec preferred_lft 42807sec 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:30:42.006 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:42.006 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T11:30:42.006 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T11:30:42.007 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:42.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:42.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:42.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:42.207 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:30:42.233 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:30:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:42.235 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:30:42.314 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:30:42.314 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:30:42.314 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:30:42.314 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:30:42.314 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:30:42.315 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:42.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:42.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:42.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:42.608 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:30:42.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:42.624 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-09-10T11:30:42.680 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:42.680 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:42.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:42.681 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T11:30:42.681 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:42.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:42.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:42.799 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T11:30:42.799 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:42.799 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T11:30:42.800 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:42.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:42.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:42.922 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:42.922 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:42.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:30:42.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T11:30:42.922 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:42.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:42.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:42.999 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:30:42.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:42.999 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:43.015 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:30:43.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:43.015 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:43.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:30:43.135 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:30:43.191 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:43.192 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:43.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:43.194 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:30:43.246 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:30:43.247 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:30:43.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:30:43.313 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:30:43.370 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:43.371 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:43.372 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:43.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:43.373 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:30:43.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:30:43.379+0000 7fac52492480 -1 init, newargv = 0x557498e10460 newargc=15 2024-09-10T11:30:43.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[35082]: starting ceph client 2024-09-10T11:30:43.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[35082]: starting fuse 2024-09-10T11:30:43.396 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:30:43.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:30:43.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:30:43.397 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:30:43.398 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:30:43.513 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.35082.asok 2024-09-10T11:30:43.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:43.523 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35082.asok 2024-09-10T11:30:43.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:43.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35082.asok status 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "35082", 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4611, 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4611 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2460579183 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2460579183 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4611 192.168.0.1:0/2460579183", 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2460579183", 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 6, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 20, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:30:43.659 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:30:43.665 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:30:43.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:30:43.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.762+0000 7fe3a3665640 1 Processor -- start 2024-09-10T11:30:43.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.762+0000 7fe3a3665640 1 -- start start 2024-09-10T11:30:43.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a3665640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe39c0b3960 0x7fe39c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a3665640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe39c0b3160 0x7fe39c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a3665640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe39c0b3620 con 0x7fe39c163580 2024-09-10T11:30:43.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a3665640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe39c06d960 con 0x7fe39c0b3960 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a3665640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe39c06dc40 con 0x7fe39c0b3160 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe39c0b3960 0x7fe39c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe39c0b3960 0x7fe39c0b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46546/0 (socket says 172.21.15.99:46546) 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 -- 172.21.15.99:0/3287089056 learned_addr learned my addr 172.21.15.99:0/3287089056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a1c02640 1 --2- 172.21.15.99:0/3287089056 >> v2:172.21.15.196:3300/0 conn(0x7fe39c0b3160 0x7fe39c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 -- 172.21.15.99:0/3287089056 >> v2:172.21.15.196:3300/0 conn(0x7fe39c0b3160 msgr2=0x7fe39c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 --2- 172.21.15.99:0/3287089056 >> v2:172.21.15.196:3300/0 conn(0x7fe39c0b3160 0x7fe39c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 -- 172.21.15.99:0/3287089056 >> v1:172.21.15.99:6789/0 conn(0x7fe39c163580 legacy=0x7fe39c0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.763+0000 7fe3a0c00640 1 -- 172.21.15.99:0/3287089056 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c06df20 con 0x7fe39c0b3960 2024-09-10T11:30:43.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.764+0000 7fe3a0c00640 1 --2- 172.21.15.99:0/3287089056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c0b3960 0x7fe39c0b5090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe39c073250 tx=0x7fe38c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c429e56e2486f580 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:43.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.764+0000 7fe393fff640 1 -- 172.21.15.99:0/3287089056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c037070 con 0x7fe39c0b3960 2024-09-10T11:30:43.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.764+0000 7fe393fff640 1 -- 172.21.15.99:0/3287089056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe38c032440 con 0x7fe39c0b3960 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.764+0000 7fe393fff640 1 -- 172.21.15.99:0/3287089056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38c032810 con 0x7fe39c0b3960 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 -- 172.21.15.99:0/3287089056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c0b3960 msgr2=0x7fe39c0b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 --2- 172.21.15.99:0/3287089056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c0b3960 0x7fe39c0b5090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe39c073250 tx=0x7fe38c02a150 comp rx=0 tx=0).stop 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 -- 172.21.15.99:0/3287089056 shutdown_connections 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 --2- 172.21.15.99:0/3287089056 >> v2:172.21.15.196:3300/0 conn(0x7fe39c0b3160 0x7fe39c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 --2- 172.21.15.99:0/3287089056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c0b3960 0x7fe39c0b5090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 -- 172.21.15.99:0/3287089056 >> 172.21.15.99:0/3287089056 conn(0x7fe39c0b4570 msgr2=0x7fe39c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 -- 172.21.15.99:0/3287089056 shutdown_connections 2024-09-10T11:30:43.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.765+0000 7fe3a3665640 1 -- 172.21.15.99:0/3287089056 wait complete. 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.766+0000 7fe3a3665640 1 Processor -- start 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.766+0000 7fe3a3665640 1 -- start start 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 0x7fe39c1f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 0x7fe39c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c06d3e0 con 0x7fe39c163580 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c06d260 con 0x7fe39c0b3160 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a3665640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c06d560 con 0x7fe39c0b3960 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46556/0 (socket says 172.21.15.99:46556) 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 -- 172.21.15.99:0/4243066173 learned_addr learned my addr 172.21.15.99:0/4243066173 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1401640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 0x7fe39c1f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a0c00640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 0x7fe39c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 -- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 msgr2=0x7fe39c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 0x7fe39c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 -- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 msgr2=0x7fe39c1f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 0x7fe39c1f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1c02640 1 -- 172.21.15.99:0/4243066173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c0b77a0 con 0x7fe39c163580 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a1401640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 0x7fe39c1f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.767+0000 7fe3a0c00640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 0x7fe39c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe3a1c02640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe39400cc30 tx=0x7fe39400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39400ce40 con 0x7fe39c163580 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe394003990 con 0x7fe39c163580 2024-09-10T11:30:43.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c0b7a80 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394003d40 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.768+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c0b8040 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.769+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c0d1280 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.769+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe394003ee0 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.770+0000 7fe391ffb640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 0x7fe38006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.770+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe39408b2a0 con 0x7fe39c163580 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.770+0000 7fe3a1401640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 0x7fe38006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:43.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.770+0000 7fe3a1401640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 0x7fe38006e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0a0ba0 tx=0x7fe398009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:43.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.774+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe39c0d1280 con 0x7fe39c163580 2024-09-10T11:30:43.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.973+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe39c06cd20 con 0x7fe39c163580 2024-09-10T11:30:43.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.973+0000 7fe391ffb640 1 -- 172.21.15.99:0/4243066173 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fe394014030 con 0x7fe39c163580 2024-09-10T11:30:43.975 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:30:43.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 msgr2=0x7fe38006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 0x7fe38006e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0a0ba0 tx=0x7fe398009040 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 msgr2=0x7fe39c0c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe39400cc30 tx=0x7fe39400a5c0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 shutdown_connections 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe38006be60 0x7fe38006e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe39c163580 0x7fe39c0c89d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe39c0b3960 0x7fe39c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 --2- 172.21.15.99:0/4243066173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe39c0b3160 0x7fe39c1f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 >> 172.21.15.99:0/4243066173 conn(0x7fe39c0b4570 msgr2=0x7fe39c0d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 shutdown_connections 2024-09-10T11:30:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:43.975+0000 7fe3a3665640 1 -- 172.21.15.99:0/4243066173 wait complete. 2024-09-10T11:30:43.988 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:30:43.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:30:43.989 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T11:30:43.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:43.989 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:44.005 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42805sec preferred_lft 42805sec 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:44.006 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link tentative 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.007 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:30:44.067 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:30:44.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.069 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:30:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:30:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:30:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:30:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:30:44.126 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:30:44.126 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:44.126 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:44.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:30:44.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:30:44.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:30:44.127 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:44.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:44.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.250 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:30:44.266 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:30:44.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.267 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-09-10T11:30:44.322 INFO:teuthology.orchestra.run.smithi099.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T11:30:44.322 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:44.322 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:44.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:30:44.323 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T11:30:44.323 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:44.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:44.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:44.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether c6:88:91:ef:88:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.464 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::c488:91ff:feef:880a/64 scope link tentative 2024-09-10T11:30:44.465 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.530 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether c6:88:91:ef:88:0a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T11:30:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T11:30:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::c488:91ff:feef:880a/64 scope link tentative 2024-09-10T11:30:44.531 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:44.532 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T11:30:44.532 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T11:30:44.533 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T11:30:44.533 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:44.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:44.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:44.685 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:44.685 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:30:44.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.1 up 2024-09-10T11:30:44.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T11:30:44.685 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:30:44.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:44.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:44.761 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-10T11:30:44.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:44.761 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:44.775 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T11:30:44.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:44.776 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:44.829 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:30:44.894 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:30:44.951 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:30:44.952 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:30:44.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:44.953 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:30:45.006 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:30:45.006 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-10T11:30:45.006 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-10T11:30:45.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:30:45.072 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:30:45.128 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:30:45.129 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:30:45.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:45.130 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:30:45.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:2024-09-10T11:30:45.128+0000 7f465b11f480 -1 init, newargv = 0x55fcbd57e160 newargc=15 2024-09-10T11:30:45.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:ceph-fuse[35577]: starting ceph client 2024-09-10T11:30:45.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:ceph-fuse[35577]: starting fuse 2024-09-10T11:30:45.146 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:30:45.146 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-09-10T11:30:45.146 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:30:45.147 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.2")) 2024-09-10T11:30:45.148 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:30:45.264 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.2.35577.asok 2024-09-10T11:30:45.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:45.274 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35577.asok 2024-09-10T11:30:45.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:45.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35577.asok status 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "2", 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:30:45.408 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "35577", 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4626, 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4626 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:30:45.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3282483160 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3282483160 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4626 192.168.0.2:0/3282483160", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.2:0/3282483160", 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:30:45.410 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 6, 2024-09-10T11:30:45.411 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 20, 2024-09-10T11:30:45.411 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:30:45.411 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:30:45.411 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:30:45.411 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:30:45.417 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:30:45.417 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:30:45.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.512+0000 7fafd709b640 1 Processor -- start 2024-09-10T11:30:45.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.512+0000 7fafd709b640 1 -- start start 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd709b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafd00b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd709b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fafd0070610 0x7fafd0070a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd709b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd00b5d70 con 0x7fafd0146490 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd709b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fafd0070fa0 con 0x7fafd0070610 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd709b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00b6050 con 0x7fafd0073300 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd4e37640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fafd0146490 0x7fafd00d1020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34756/0 (socket says 172.21.15.99:34756) 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd4e37640 1 -- 172.21.15.99:0/1875821191 learned_addr learned my addr 172.21.15.99:0/1875821191 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafcffff640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafd00b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.513+0000 7fafd5638640 1 --2- 172.21.15.99:0/1875821191 >> v2:172.21.15.196:3300/0 conn(0x7fafd0070610 0x7fafd0070a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3226649126 0 0) 0x7fafd00b5d70 con 0x7fafd0146490 2024-09-10T11:30:45.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc0003520 con 0x7fafd0146490 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 494423759 0 0) 0x7fafc0003520 con 0x7fafd0146490 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 msgr2=0x7fafd00b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafd00b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 >> v2:172.21.15.196:3300/0 conn(0x7fafd0070610 msgr2=0x7fafd0070a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 --2- 172.21.15.99:0/1875821191 >> v2:172.21.15.196:3300/0 conn(0x7fafd0070610 0x7fafd0070a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd00b6330 con 0x7fafd0146490 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fafc4001670 con 0x7fafd0146490 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 >> v1:172.21.15.99:6789/0 conn(0x7fafd0146490 legacy=0x7fafd00d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafc0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafc0004d40 0x7fafc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafd0070fa0 con 0x7fafc00039b0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00b6050 con 0x7fafc0004d40 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafc000b9c0 con 0x7fafd0073300 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafcffff640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafc0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd5638640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafc0004d40 0x7fafc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 -- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 msgr2=0x7fafc0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafd0073300 0x7fafc0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 -- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafc0004d40 msgr2=0x7fafc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafc0004d40 0x7fafc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd4e37640 1 -- 172.21.15.99:0/1875821191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc0003520 con 0x7fafc00039b0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.514+0000 7fafd5638640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafc0004d40 0x7fafc00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.515+0000 7fafd4e37640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c210 tx=0x7fafc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299b6a0b8e3137cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:45.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.515+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007d00 con 0x7fafc00039b0 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.515+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc4005ad0 con 0x7fafc00039b0 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.515+0000 7fafcf7fe640 1 -- 172.21.15.99:0/1875821191 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007070 con 0x7fafc00039b0 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 -- 172.21.15.99:0/1875821191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc00039b0 msgr2=0x7fafc0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 --2- 172.21.15.99:0/1875821191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafc00039b0 0x7fafc0003dd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c210 tx=0x7fafc0005830 comp rx=0 tx=0).stop 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd5638640 1 -- 172.21.15.99:0/1875821191 reap_dead start 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 -- 172.21.15.99:0/1875821191 shutdown_connections 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 -- 172.21.15.99:0/1875821191 >> 172.21.15.99:0/1875821191 conn(0x7fafd0072680 msgr2=0x7fafd0072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 -- 172.21.15.99:0/1875821191 shutdown_connections 2024-09-10T11:30:45.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.516+0000 7fafd709b640 1 -- 172.21.15.99:0/1875821191 wait complete. 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.517+0000 7fafd709b640 1 Processor -- start 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.517+0000 7fafd709b640 1 -- start start 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafc00039b0 0x7fafd015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd00ad850 0x7fafd00adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00b6cb0 con 0x7fafd01425c0 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00b6b30 con 0x7fafd00ad850 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00b6e30 con 0x7fafc00039b0 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46586/0 (socket says 172.21.15.99:46586) 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 -- 172.21.15.99:0/3924207376 learned_addr learned my addr 172.21.15.99:0/3924207376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:45.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd5638640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd00ad850 0x7fafd00adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd4e37640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafc00039b0 0x7fafd015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 -- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafc00039b0 msgr2=0x7fafd015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafc00039b0 0x7fafd015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 -- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd00ad850 msgr2=0x7fafd00adcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd00ad850 0x7fafd00adcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 -- 172.21.15.99:0/3924207376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd013f500 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcffff640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0119d0 tx=0x7fafbc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00fb50 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc00fcf0 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd0141520 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.518+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00fea0 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.519+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd0141ae0 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.519+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd015fb00 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.520+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafbc012040 con 0x7fafd01425c0 2024-09-10T11:30:45.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.520+0000 7fafcd7fa640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 0x7fafb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:45.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.520+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fafbc08b340 con 0x7fafd01425c0 2024-09-10T11:30:45.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.520+0000 7fafd4e37640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 0x7fafb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:45.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.521+0000 7fafd4e37640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 0x7fafb406e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:45.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.525+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fafd015fb00 con 0x7fafd01425c0 2024-09-10T11:30:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.727+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fafd0070a60 con 0x7fafd01425c0 2024-09-10T11:30:45.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.727+0000 7fafcd7fa640 1 -- 172.21.15.99:0/3924207376 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fafbc019030 con 0x7fafd01425c0 2024-09-10T11:30:45.729 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 msgr2=0x7fafb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 0x7fafb406e320 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4007000 comp rx=0 tx=0).stop 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 msgr2=0x7fafd00b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0119d0 tx=0x7fafbc011e90 comp rx=0 tx=0).stop 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 shutdown_connections 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fafb406be60 0x7fafb406e320 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fafd00ad850 0x7fafd00adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafd01425c0 0x7fafd00b0650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 --2- 172.21.15.99:0/3924207376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fafc00039b0 0x7fafd015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 >> 172.21.15.99:0/3924207376 conn(0x7fafd0072680 msgr2=0x7fafd015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 shutdown_connections 2024-09-10T11:30:45.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:45.729+0000 7fafd709b640 1 -- 172.21.15.99:0/3924207376 wait complete. 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:30:45.742 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:30:45.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:45.743 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:30:45.762 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:45.762 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:45.762 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:45.762 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:45.763 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:30:45.764 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42819sec preferred_lft 42819sec 2024-09-10T11:30:45.764 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:30:45.764 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:45.764 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T11:30:45.764 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:45.860 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:46.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:46.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:46.068 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:30:46.094 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:30:46.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:46.096 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:30:46.168 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:30:46.168 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:30:46.168 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:30:46.168 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:30:46.168 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:30:46.168 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:30:46.169 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:46.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:46.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:46.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:46.429 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:30:46.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:46.444 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-09-10T11:30:46.500 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:46.500 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:46.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:46.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T11:30:46.500 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:46.587 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:46.610 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:46.616 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T11:30:46.617 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:46.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:46.766 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:46.773 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:46.773 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:46.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:30:46.773 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T11:30:46.773 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:46.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:46.843 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:46.854 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:30:46.854 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:46.854 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:46.870 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:30:46.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:46.870 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:46.919 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:30:46.985 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:47.041 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:30:47.041 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:30:47.041 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:30:47.042 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:47.043 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:47.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.044 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:30:47.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:30:47.100 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:30:47.142 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:30:47.166 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:30:47.223 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:47.224 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:47.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.225 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:30:47.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:30:47.227+0000 7ff57d217480 -1 init, newargv = 0x555afdc2dc80 newargc=15 2024-09-10T11:30:47.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33004]: starting ceph client 2024-09-10T11:30:47.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33004]: starting fuse 2024-09-10T11:30:47.243 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:30:47.244 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:30:47.244 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:30:47.245 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:47.246 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:30:47.246 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:30:47.360 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.33004.asok 2024-09-10T11:30:47.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:47.371 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33004.asok 2024-09-10T11:30:47.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.371 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33004.asok status 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33004", 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:30:47.505 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4651, 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4651 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4123570850 2024-09-10T11:30:47.506 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4123570850 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4651 192.168.0.1:0/4123570850", 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/4123570850", 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 6, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 20, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:30:47.507 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:30:47.508 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:30:47.512 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:30:47.513 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:30:47.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.621+0000 7fb4e7e23640 1 Processor -- start 2024-09-10T11:30:47.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 -- start start 2024-09-10T11:30:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4e0162760 con 0x7fb4e0070ab0 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb4e0162a40 con 0x7fb4e0148400 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.622+0000 7fb4e7e23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e0162d20 con 0x7fb4e006c150 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e5bbf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e63c0640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb4e0070ab0 0x7fb4e0072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:49166/0 (socket says 172.21.15.99:49166) 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e5bbf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:49766/0 (socket says 172.21.15.99:49766) 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:59456/0 (socket says 172.21.15.99:59456) 2024-09-10T11:30:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e63c0640 1 -- 172.21.15.99:0/2564103438 learned_addr learned my addr 172.21.15.99:0/2564103438 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 -- 172.21.15.99:0/2564103438 >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 msgr2=0x7fb4e006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 --2- 172.21.15.99:0/2564103438 >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 -- 172.21.15.99:0/2564103438 >> v1:172.21.15.196:6789/0 conn(0x7fb4e0070ab0 legacy=0x7fb4e0072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e4bbd640 1 -- 172.21.15.99:0/2564103438 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1146061208 0 0) 0x7fb4e0162760 con 0x7fb4e0070ab0 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e53be640 1 -- 172.21.15.99:0/2564103438 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e0163000 con 0x7fb4e006c150 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.623+0000 7fb4e5bbf640 1 --2- 172.21.15.99:0/2564103438 >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.624+0000 7fb4e53be640 1 --2- 172.21.15.99:0/2564103438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00097a0 tx=0x7fb4d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999498156ec95fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.624+0000 7fb4e4bbd640 1 -- 172.21.15.99:0/2564103438 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0037070 con 0x7fb4e006c150 2024-09-10T11:30:47.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.624+0000 7fb4e4bbd640 1 -- 172.21.15.99:0/2564103438 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0032440 con 0x7fb4e006c150 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.624+0000 7fb4e4bbd640 1 -- 172.21.15.99:0/2564103438 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0032810 con 0x7fb4e006c150 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 -- 172.21.15.99:0/2564103438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 msgr2=0x7fb4e0070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/2564103438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00097a0 tx=0x7fb4d002a180 comp rx=0 tx=0).stop 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 -- 172.21.15.99:0/2564103438 shutdown_connections 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/2564103438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e006c150 0x7fb4e0070570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/2564103438 >> v2:172.21.15.196:3300/0 conn(0x7fb4e0148400 0x7fb4e006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 -- 172.21.15.99:0/2564103438 >> 172.21.15.99:0/2564103438 conn(0x7fb4e00d1900 msgr2=0x7fb4e00d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 -- 172.21.15.99:0/2564103438 shutdown_connections 2024-09-10T11:30:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.625+0000 7fb4e7e23640 1 -- 172.21.15.99:0/2564103438 wait complete. 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.626+0000 7fb4e7e23640 1 Processor -- start 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.626+0000 7fb4e7e23640 1 -- start start 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e0070ab0 0x7fb4e015e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e0148400 0x7fb4e01443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e00d0700 con 0x7fb4e006c150 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e006c570 con 0x7fb4e0070ab0 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e7e23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4e00d0880 con 0x7fb4e0148400 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46598/0 (socket says 172.21.15.99:46598) 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 -- 172.21.15.99:0/3388792602 learned_addr learned my addr 172.21.15.99:0/3388792602 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e53be640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e0070ab0 0x7fb4e015e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e63c0640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e0148400 0x7fb4e01443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 -- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e0148400 msgr2=0x7fb4e01443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e0148400 0x7fb4e01443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 -- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e0070ab0 msgr2=0x7fb4e015e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e0070ab0 0x7fb4e015e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 -- 172.21.15.99:0/3388792602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e0142ba0 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.627+0000 7fb4e5bbf640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800e960 tx=0x7fb4d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.628+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8017070 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.628+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d800ca00 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.628+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e00b66a0 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.628+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800cdd0 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.628+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e00b6c60 con 0x7fb4e006c150 2024-09-10T11:30:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.629+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e015e830 con 0x7fb4e006c150 2024-09-10T11:30:47.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.629+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4d801a070 con 0x7fb4e006c150 2024-09-10T11:30:47.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.630+0000 7fb4d67fc640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:47.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.630+0000 7fb4e53be640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:47.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.630+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d808af90 con 0x7fb4e006c150 2024-09-10T11:30:47.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.630+0000 7fb4e53be640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 0x7fb4c406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0034000 tx=0x7fb4d002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:47.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.635+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4d8059380 con 0x7fb4e006c150 2024-09-10T11:30:47.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.833+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb4e0070570 con 0x7fb4e006c150 2024-09-10T11:30:47.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.833+0000 7fb4d67fc640 1 -- 172.21.15.99:0/3388792602 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7fb4d805d680 con 0x7fb4e006c150 2024-09-10T11:30:47.835 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:30:47.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 msgr2=0x7fb4c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 0x7fb4c406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0034000 tx=0x7fb4d002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 msgr2=0x7fb4e00b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800e960 tx=0x7fb4d800ee20 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 shutdown_connections 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4c406be30 0x7fb4c406e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4e0148400 0x7fb4e01443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4e0070ab0 0x7fb4e015e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 --2- 172.21.15.99:0/3388792602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4e006c150 0x7fb4e00b7180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 >> 172.21.15.99:0/3388792602 conn(0x7fb4e00d1900 msgr2=0x7fb4e00af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 shutdown_connections 2024-09-10T11:30:47.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:47.835+0000 7fb4e7e23640 1 -- 172.21.15.99:0/3388792602 wait complete. 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:30:47.848 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T11:30:47.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.848 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:47.865 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42816sec preferred_lft 42816sec 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.866 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link tentative 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:47.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.867 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:30:47.929 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:30:47.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:47.931 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:30:47.987 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:30:47.987 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:30:47.987 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:30:47.988 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:30:47.988 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:30:47.988 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:48.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:48.109 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:48.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:48.115 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:30:48.130 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:30:48.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:48.130 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-09-10T11:30:48.186 INFO:teuthology.orchestra.run.smithi196.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T11:30:48.187 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:48.187 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:48.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:30:48.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T11:30:48.187 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:48.274 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:48.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:48.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:48.303 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T11:30:48.327 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:48.327 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether b2:af:7b:f7:50:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::b0af:7bff:fef7:50f2/64 scope link 2024-09-10T11:30:48.328 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:48.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:30:48.395 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether b2:af:7b:f7:50:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::b0af:7bff:fef7:50f2/64 scope link 2024-09-10T11:30:48.396 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:30:48.397 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T11:30:48.398 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:48.487 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:48.545 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:48.551 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:30:48.551 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:30:48.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-09-10T11:30:48.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T11:30:48.552 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:30:48.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:30:48.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:48.628 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-10T11:30:48.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:48.628 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:48.643 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T11:30:48.643 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:30:48.643 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:48.699 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:30:48.765 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:30:48.822 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:30:48.823 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:48.824 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:48.825 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:48.825 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:30:48.825 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:30:48.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:48.825 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:30:48.878 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:30:48.879 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-09-10T11:30:48.879 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-10T11:30:48.921 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:30:48.948 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:30:49.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:2024-09-10T11:30:49.000+0000 7fe3e72f9480 -1 init, newargv = 0x55d52601c080 newargc=15 2024-09-10T11:30:49.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33460]: starting ceph client 2024-09-10T11:30:49.005 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:30:49.005 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:30:49.005 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:30:49.005 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:30:49.005 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:30:49.006 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:30:49.007 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:49.008 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:30:49.008 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:49.008 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:30:49.008 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:30:49.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.008 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:30:49.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33460]: starting fuse 2024-09-10T11:30:49.023 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:30:49.023 INFO:teuthology.orchestra.run.smithi196.stdout:44 2024-09-10T11:30:49.023 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-09-10T11:30:49.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:30:49.024 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:30:49.025 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.3")) 2024-09-10T11:30:49.026 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:30:49.142 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.3.33460.asok 2024-09-10T11:30:49.145 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:30:49.153 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33460.asok 2024-09-10T11:30:49.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33460.asok status 2024-09-10T11:30:49.284 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:30:49.284 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:30:49.284 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "3", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33460", 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4666, 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:30:49.285 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4666 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 748530453 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-09-10T11:30:49.286 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 748530453 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4666 192.168.0.2:0/748530453", 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.2:0/748530453", 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 6, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 20, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:30:49.287 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:30:49.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.293 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:30:49.293 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:49.312 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:30:49.312 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:49.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.383 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:30:49.383 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:49.442 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:30:49.442 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:49.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.443 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:49.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.513 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:30:49.514 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:49.529 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:30:49.530 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:49.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:49.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.595 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:30:49.595 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:49.650 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:30:49.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:49.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:49.651 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:49.715 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T11:30:49.830 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-09-10T11:30:49.831 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-09-10T11:30:49.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:49.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.941+0000 7f746d68b640 1 Processor -- start 2024-09-10T11:30:49.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.941+0000 7f746d68b640 1 -- start start 2024-09-10T11:30:49.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f746d68b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f746815eb80 0x7f74680d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f746d68b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74681626c0 0x7f74680d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f746d68b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74680d52d0 con 0x7f74680d4e40 2024-09-10T11:30:49.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f746d68b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f746815ff70 con 0x7f74681626c0 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f746d68b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f74680d1620 con 0x7f746815eb80 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f745ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74681626c0 0x7f74680d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f74677fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f74680d4e40 0x7f74680d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34764/0 (socket says 172.21.15.99:34764) 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f74677fe640 1 -- 172.21.15.99:0/457936321 learned_addr learned my addr 172.21.15.99:0/457936321 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f7466ffd640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f746815eb80 0x7f74680d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f745ffff640 1 -- 172.21.15.99:0/457936321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f746815eb80 msgr2=0x7f74680d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f745ffff640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f746815eb80 0x7f74680d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f745ffff640 1 -- 172.21.15.99:0/457936321 >> v1:172.21.15.99:6789/0 conn(0x7f74680d4e40 legacy=0x7f74680d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f74667fc640 1 -- 172.21.15.99:0/457936321 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3833261638 0 0) 0x7f74680d52d0 con 0x7f74680d4e40 2024-09-10T11:30:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.942+0000 7f745ffff640 1 -- 172.21.15.99:0/457936321 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7468162bc0 con 0x7f74681626c0 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.943+0000 7f745ffff640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74681626c0 0x7f74680d4900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74500097a0 tx=0x7f745002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d5ad8b9392e2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.943+0000 7f74667fc640 1 -- 172.21.15.99:0/457936321 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450037070 con 0x7f74681626c0 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.943+0000 7f74667fc640 1 -- 172.21.15.99:0/457936321 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745002ae50 con 0x7f74681626c0 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.943+0000 7f74667fc640 1 -- 172.21.15.99:0/457936321 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450032660 con 0x7f74681626c0 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 -- 172.21.15.99:0/457936321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74681626c0 msgr2=0x7f74680d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74681626c0 0x7f74680d4900 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74500097a0 tx=0x7f745002a100 comp rx=0 tx=0).stop 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 -- 172.21.15.99:0/457936321 shutdown_connections 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74681626c0 0x7f74680d4900 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 --2- 172.21.15.99:0/457936321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f746815eb80 0x7f74680d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 -- 172.21.15.99:0/457936321 >> 172.21.15.99:0/457936321 conn(0x7f746808e680 msgr2=0x7f746809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:49.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 -- 172.21.15.99:0/457936321 shutdown_connections 2024-09-10T11:30:49.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.944+0000 7f746d68b640 1 -- 172.21.15.99:0/457936321 wait complete. 2024-09-10T11:30:49.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.945+0000 7f746d68b640 1 Processor -- start 2024-09-10T11:30:49.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.945+0000 7f746d68b640 1 -- start start 2024-09-10T11:30:49.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74680d4e40 0x7f74680b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74681626c0 0x7f7468152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7468163460 con 0x7f746815eb80 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7468163180 con 0x7f74681626c0 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f746d68b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f74680d2ef0 con 0x7f74680d4e40 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46614/0 (socket says 172.21.15.99:46614) 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 -- 172.21.15.99:0/2329538520 learned_addr learned my addr 172.21.15.99:0/2329538520 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f7466ffd640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74680d4e40 0x7f74680b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f74677fe640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74681626c0 0x7f7468152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 -- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74680d4e40 msgr2=0x7f74680b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74680d4e40 0x7f74680b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 -- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74681626c0 msgr2=0x7f7468152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74681626c0 0x7f7468152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 -- 172.21.15.99:0/2329538520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74680b7b90 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745ffff640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7450000c00 tx=0x7f74500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.946+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450041070 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.947+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f745003c670 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.947+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74680b7e00 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.947+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450037040 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.947+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74681f9090 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.948+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74680d29b0 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.948+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7450044070 con 0x7f746815eb80 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.949+0000 7f745f7fe640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 0x7f744806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.949+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74500ab4d0 con 0x7f746815eb80 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.949+0000 7f7466ffd640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 0x7f744806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.949+0000 7f7466ffd640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 0x7f744806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74580079a0 tx=0x7f7458008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:49.953+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7450079830 con 0x7f746815eb80 2024-09-10T11:30:50.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.119+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74680b6330 con 0x7f746815eb80 2024-09-10T11:30:50.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.120+0000 7f745f7fe640 1 -- 172.21.15.99:0/2329538520 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7f74500a9e20 con 0x7f746815eb80 2024-09-10T11:30:50.121 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:50.122 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 msgr2=0x7f744806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 0x7f744806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74580079a0 tx=0x7f7458008040 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 msgr2=0x7f74680b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7450000c00 tx=0x7f74500046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 shutdown_connections 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.122+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f744806be30 0x7f744806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74681626c0 0x7f7468152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f746815eb80 0x7f74680b6da0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 --2- 172.21.15.99:0/2329538520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74680d4e40 0x7f74680b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 >> 172.21.15.99:0/2329538520 conn(0x7f746808e680 msgr2=0x7f746809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 shutdown_connections 2024-09-10T11:30:50.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.123+0000 7f746d68b640 1 -- 172.21.15.99:0/2329538520 wait complete. 2024-09-10T11:30:50.134 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T11:30:24.498187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T11:30:25.666653+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:30:50.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:30:50.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 Processor -- start 2024-09-10T11:30:50.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 -- start start 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f779015ee00 0x7f77900b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77900b40e0 con 0x7f77900b0090 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77900b5ab0 con 0x7f779015f1e0 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.235+0000 7f7795db9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7790073c50 con 0x7f779015ee00 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f77900b0090 0x7f77900b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34776/0 (socket says 172.21.15.99:34776) 2024-09-10T11:30:50.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778ffff640 1 -- 172.21.15.99:0/638965219 learned_addr learned my addr 172.21.15.99:0/638965219 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778f7fe640 1 --2- 172.21.15.99:0/638965219 >> v2:172.21.15.196:3300/0 conn(0x7f779015ee00 0x7f77900b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778e7fc640 1 -- 172.21.15.99:0/638965219 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584643038 0 0) 0x7f77900b40e0 con 0x7f77900b0090 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778e7fc640 1 -- 172.21.15.99:0/638965219 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7778003520 con 0x7f77900b0090 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778f7fe640 1 -- 172.21.15.99:0/638965219 >> v1:172.21.15.196:6789/0 conn(0x7f779015f1e0 legacy=0x7f77900b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778f7fe640 1 -- 172.21.15.99:0/638965219 >> v1:172.21.15.99:6789/0 conn(0x7f77900b0090 legacy=0x7f77900b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.236+0000 7f778f7fe640 1 -- 172.21.15.99:0/638965219 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77900b4380 con 0x7f779015ee00 2024-09-10T11:30:50.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f778f7fe640 1 --2- 172.21.15.99:0/638965219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f779015ee00 0x7f77900b5f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77800097a0 tx=0x7f778002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a4016d9623843f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f778e7fc640 1 -- 172.21.15.99:0/638965219 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780037070 con 0x7f779015ee00 2024-09-10T11:30:50.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f778e7fc640 1 -- 172.21.15.99:0/638965219 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7780032440 con 0x7f779015ee00 2024-09-10T11:30:50.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f778e7fc640 1 -- 172.21.15.99:0/638965219 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77800327f0 con 0x7f779015ee00 2024-09-10T11:30:50.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f7795db9640 1 -- 172.21.15.99:0/638965219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f779015ee00 msgr2=0x7f77900b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f7795db9640 1 --2- 172.21.15.99:0/638965219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f779015ee00 0x7f77900b5f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77800097a0 tx=0x7f778002a230 comp rx=0 tx=0).stop 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.237+0000 7f7795db9640 1 -- 172.21.15.99:0/638965219 shutdown_connections 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.238+0000 7f7795db9640 1 --2- 172.21.15.99:0/638965219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f779015ee00 0x7f77900b5f90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.238+0000 7f7795db9640 1 -- 172.21.15.99:0/638965219 >> 172.21.15.99:0/638965219 conn(0x7f77900730c0 msgr2=0x7f77900734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.238+0000 7f7795db9640 1 -- 172.21.15.99:0/638965219 shutdown_connections 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.238+0000 7f7795db9640 1 -- 172.21.15.99:0/638965219 wait complete. 2024-09-10T11:30:50.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.239+0000 7f7795db9640 1 Processor -- start 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.239+0000 7f7795db9640 1 -- start start 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77900b0090 0x7f7790161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 0x7f779014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77900b0600 con 0x7f779015ee00 2024-09-10T11:30:50.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77900afd60 con 0x7f77900b0090 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77900b0780 con 0x7f779015f1e0 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46630/0 (socket says 172.21.15.99:46630) 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 -- 172.21.15.99:0/2949498077 learned_addr learned my addr 172.21.15.99:0/2949498077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778f7fe640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77900b0090 0x7f7790161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778ffff640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 0x7f779014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 -- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 msgr2=0x7f779014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 0x7f779014a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 -- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77900b0090 msgr2=0x7f7790161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77900b0090 0x7f7790161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 -- 172.21.15.99:0/2949498077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779014ce30 con 0x7f779015ee00 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778ffff640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 0x7f779014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:50.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f778effd640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f778400c3c0 tx=0x7f778400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840052d0 con 0x7f779015ee00 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7784005490 con 0x7f779015ee00 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779014d110 con 0x7f779015ee00 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784007260 con 0x7f779015ee00 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.240+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779014d6d0 con 0x7f779015ee00 2024-09-10T11:30:50.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.241+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77900b5f90 con 0x7f779015ee00 2024-09-10T11:30:50.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.242+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7784007400 con 0x7f779015ee00 2024-09-10T11:30:50.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.242+0000 7f7777fff640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 0x7f777c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.242+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f778408a200 con 0x7f779015ee00 2024-09-10T11:30:50.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.242+0000 7f778f7fe640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 0x7f777c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.243+0000 7f778f7fe640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 0x7f777c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7780034000 tx=0x7f778002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.246+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77840585f0 con 0x7f779015ee00 2024-09-10T11:30:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.413+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f779014d8a0 con 0x7f779015ee00 2024-09-10T11:30:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.413+0000 7f7777fff640 1 -- 172.21.15.99:0/2949498077 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f778405c8f0 con 0x7f779015ee00 2024-09-10T11:30:50.414 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.415+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 msgr2=0x7f777c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.415+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 0x7f777c06e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7780034000 tx=0x7f778002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.415+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 msgr2=0x7f7790149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.415+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f778400c3c0 tx=0x7f778400c880 comp rx=0 tx=0).stop 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 shutdown_connections 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f777c06be30 0x7f777c06e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f779015f1e0 0x7f779014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f779015ee00 0x7f7790149f20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 --2- 172.21.15.99:0/2949498077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77900b0090 0x7f7790161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 >> 172.21.15.99:0/2949498077 conn(0x7f77900730c0 msgr2=0x7f779006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 shutdown_connections 2024-09-10T11:30:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.416+0000 7f7795db9640 1 -- 172.21.15.99:0/2949498077 wait complete. 2024-09-10T11:30:50.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T11:30:50.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.527+0000 7ff47bf6d640 1 Processor -- start 2024-09-10T11:30:50.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.527+0000 7ff47bf6d640 1 -- start start 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47bf6d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff47415c7b0 0x7ff4740d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47bf6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47406f000 0x7ff47406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47bf6d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4740ceb10 con 0x7ff4740d0fb0 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47bf6d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff4740d1420 con 0x7ff47415c7b0 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47bf6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4740b3840 con 0x7ff47406f000 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479d09640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff47415c7b0 0x7ff4740d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479508640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff4740d0fb0 0x7ff47406e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34788/0 (socket says 172.21.15.99:34788) 2024-09-10T11:30:50.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479508640 1 -- 172.21.15.99:0/1597658149 learned_addr learned my addr 172.21.15.99:0/1597658149 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479d09640 1 --2- 172.21.15.99:0/1597658149 >> v2:172.21.15.99:3300/0 conn(0x7ff47415c7b0 0x7ff4740d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46640/0 (socket says 172.21.15.99:46640) 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff47a50a640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47406f000 0x7ff47406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479d09640 1 -- 172.21.15.99:0/1597658149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47406f000 msgr2=0x7ff47406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479d09640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47406f000 0x7ff47406d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.528+0000 7ff479d09640 1 -- 172.21.15.99:0/1597658149 >> v1:172.21.15.99:6789/0 conn(0x7ff4740d0fb0 legacy=0x7ff47406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff479d09640 1 -- 172.21.15.99:0/1597658149 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4740b3a90 con 0x7ff47415c7b0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff478d07640 1 -- 172.21.15.99:0/1597658149 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3420123896 0 0) 0x7ff464001110 con 0x7ff4740d0fb0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff479d09640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47415c7b0 0x7ff4740d0a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff4700097a0 tx=0x7ff47002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf6b109fbeaad9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff478d07640 1 -- 172.21.15.99:0/1597658149 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470037070 con 0x7ff47415c7b0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff478d07640 1 -- 172.21.15.99:0/1597658149 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff470032440 con 0x7ff47415c7b0 2024-09-10T11:30:50.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.529+0000 7ff478d07640 1 -- 172.21.15.99:0/1597658149 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff470032810 con 0x7ff47415c7b0 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 -- 172.21.15.99:0/1597658149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47415c7b0 msgr2=0x7ff4740d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47415c7b0 0x7ff4740d0a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff4700097a0 tx=0x7ff47002a180 comp rx=0 tx=0).stop 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 -- 172.21.15.99:0/1597658149 shutdown_connections 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47406f000 0x7ff47406d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/1597658149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47415c7b0 0x7ff4740d0a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 -- 172.21.15.99:0/1597658149 >> 172.21.15.99:0/1597658149 conn(0x7ff47406bc10 msgr2=0x7ff47406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 -- 172.21.15.99:0/1597658149 shutdown_connections 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.530+0000 7ff47bf6d640 1 -- 172.21.15.99:0/1597658149 wait complete. 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.531+0000 7ff47bf6d640 1 Processor -- start 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.531+0000 7ff47bf6d640 1 -- start start 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 0x7ff47413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 0x7ff47413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740b4330 con 0x7ff4740d0fb0 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4740b4050 con 0x7ff47406f000 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47bf6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4740b4610 con 0x7ff47415c7b0 2024-09-10T11:30:50.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46654/0 (socket says 172.21.15.99:46654) 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 -- 172.21.15.99:0/2242182208 learned_addr learned my addr 172.21.15.99:0/2242182208 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47a50a640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 0x7ff47413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479d09640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 0x7ff47413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 -- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 msgr2=0x7ff47413d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 0x7ff47413d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 -- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 msgr2=0x7ff47413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 0x7ff47413ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 -- 172.21.15.99:0/2242182208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff47414fa70 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff47a50a640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 0x7ff47413d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479d09640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 0x7ff47413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.532+0000 7ff479508640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c7b0 tx=0x7ff46400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.533+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464018070 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.533+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4640072d0 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.533+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff47414fd50 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.533+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640047e0 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.533+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff47413def0 con 0x7ff4740d0fb0 2024-09-10T11:30:50.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.534+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff47415cc20 con 0x7ff4740d0fb0 2024-09-10T11:30:50.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.534+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff46401b070 con 0x7ff4740d0fb0 2024-09-10T11:30:50.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.535+0000 7ff46a7fc640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 0x7ff45806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.535+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff46408a1e0 con 0x7ff4740d0fb0 2024-09-10T11:30:50.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.535+0000 7ff479d09640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 0x7ff45806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.535+0000 7ff479d09640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 0x7ff45806e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff470034000 tx=0x7ff47002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.539+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4740ceb10 con 0x7ff4740d0fb0 2024-09-10T11:30:50.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.705+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff4740b09c0 con 0x7ff4740d0fb0 2024-09-10T11:30:50.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.706+0000 7ff46a7fc640 1 -- 172.21.15.99:0/2242182208 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff464058b90 con 0x7ff4740d0fb0 2024-09-10T11:30:50.707 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 msgr2=0x7ff45806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 0x7ff45806e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff470034000 tx=0x7ff47002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 msgr2=0x7ff47414f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff46400c7b0 tx=0x7ff46400cc70 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 shutdown_connections 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45806be30 0x7ff45806e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47415c7b0 0x7ff47413d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4740d0fb0 0x7ff47414f020 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 --2- 172.21.15.99:0/2242182208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff47406f000 0x7ff47413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 >> 172.21.15.99:0/2242182208 conn(0x7ff47406bc10 msgr2=0x7ff4740b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 shutdown_connections 2024-09-10T11:30:50.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.708+0000 7ff47bf6d640 1 -- 172.21.15.99:0/2242182208 wait complete. 2024-09-10T11:30:50.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T11:30:50.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.820+0000 7f7476986640 1 Processor -- start 2024-09-10T11:30:50.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.820+0000 7f7476986640 1 -- start start 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7476986640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 0x7f7470073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7476986640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f747015c060 0x7f747015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7476986640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7470152e60 con 0x7f74700d1200 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7476986640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f74700d15e0 con 0x7f747015c060 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7476986640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7470072680 con 0x7f747006ece0 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7474f23640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f747015c060 0x7f747015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7474f23640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f747015c060 0x7f747015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46662/0 (socket says 172.21.15.99:46662) 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.821+0000 7f7474f23640 1 -- 172.21.15.99:0/2452715926 learned_addr learned my addr 172.21.15.99:0/2452715926 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f746f7fe640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 0x7f7470073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f7474f23640 1 -- 172.21.15.99:0/2452715926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 msgr2=0x7f7470073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:50.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f7474f23640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 0x7f7470073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f7474f23640 1 -- 172.21.15.99:0/2452715926 >> v1:172.21.15.99:6789/0 conn(0x7f74700d1200 legacy=0x7f747006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f746effd640 1 -- 172.21.15.99:0/2452715926 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64126642 0 0) 0x7f7470152e60 con 0x7f74700d1200 2024-09-10T11:30:50.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f7474f23640 1 -- 172.21.15.99:0/2452715926 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7470072960 con 0x7f747015c060 2024-09-10T11:30:50.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f746f7fe640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 0x7f7470073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.822+0000 7f7474f23640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f747015c060 0x7f747015c4a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f746800d150 tx=0x7f746802c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa33115474ae4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f746effd640 1 -- 172.21.15.99:0/2452715926 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468039070 con 0x7f747015c060 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f746effd640 1 -- 172.21.15.99:0/2452715926 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7468009600 con 0x7f747015c060 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f746effd640 1 -- 172.21.15.99:0/2452715926 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74680099d0 con 0x7f747015c060 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 -- 172.21.15.99:0/2452715926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f747015c060 msgr2=0x7f747015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f747015c060 0x7f747015c4a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f746800d150 tx=0x7f746802c890 comp rx=0 tx=0).stop 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 -- 172.21.15.99:0/2452715926 shutdown_connections 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f747015c060 0x7f747015c4a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 --2- 172.21.15.99:0/2452715926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747006ece0 0x7f7470073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 -- 172.21.15.99:0/2452715926 >> 172.21.15.99:0/2452715926 conn(0x7f747006bc10 msgr2=0x7f747006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 -- 172.21.15.99:0/2452715926 shutdown_connections 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.823+0000 7f7476986640 1 -- 172.21.15.99:0/2452715926 wait complete. 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.824+0000 7f7476986640 1 Processor -- start 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 -- start start 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747006ece0 0x7f7470159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 0x7f7470147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f747015cd90 con 0x7f74700d1200 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7470072f20 con 0x7f747006ece0 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f7476986640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f747015c9e0 con 0x7f747015c060 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f746f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f746f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46676/0 (socket says 172.21.15.99:46676) 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.825+0000 7f746f7fe640 1 -- 172.21.15.99:0/443371056 learned_addr learned my addr 172.21.15.99:0/443371056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:50.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f7474f23640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 0x7f7470147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746ffff640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747006ece0 0x7f7470159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 -- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 msgr2=0x7f7470147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 0x7f7470147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 -- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747006ece0 msgr2=0x7f7470159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747006ece0 0x7f7470159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 -- 172.21.15.99:0/443371056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74701582b0 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f7474f23640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 0x7f7470147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746f7fe640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f746400d7d0 tx=0x7f746400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464004030 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74640041d0 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7470158590 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400b6c0 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.826+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7470158b50 con 0x7f74700d1200 2024-09-10T11:30:50.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.827+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7470073440 con 0x7f74700d1200 2024-09-10T11:30:50.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.828+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f746400b860 con 0x7f74700d1200 2024-09-10T11:30:50.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.828+0000 7f746cff9640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 0x7f745c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.828+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f746408b170 con 0x7f74700d1200 2024-09-10T11:30:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.828+0000 7f746ffff640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 0x7f745c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.828+0000 7f746ffff640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 0x7f745c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7460000f60 tx=0x7f7460009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.832+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7464059b20 con 0x7f74700d1200 2024-09-10T11:30:50.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.998+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f74700ccff0 con 0x7f74700d1200 2024-09-10T11:30:50.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:50.999+0000 7f746cff9640 1 -- 172.21.15.99:0/443371056 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7464059420 con 0x7f74700d1200 2024-09-10T11:30:51.000 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 msgr2=0x7f745c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 0x7f745c06e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7460000f60 tx=0x7f7460009040 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 msgr2=0x7f7470157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f746400d7d0 tx=0x7f746400dc90 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 shutdown_connections 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f745c06be30 0x7f745c06e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f747015c060 0x7f7470147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74700d1200 0x7f7470157a80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 --2- 172.21.15.99:0/443371056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f747006ece0 0x7f7470159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 >> 172.21.15.99:0/443371056 conn(0x7f747006bc10 msgr2=0x7f74700b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 shutdown_connections 2024-09-10T11:30:51.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.001+0000 7f7476986640 1 -- 172.21.15.99:0/443371056 wait complete. 2024-09-10T11:30:51.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-09-10T11:30:51.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.110+0000 7fb6f7791640 1 Processor -- start 2024-09-10T11:30:51.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 -- start start 2024-09-10T11:30:51.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6f00b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6f00b3660 0x7fb6f00d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f006df80 con 0x7fb6f00d1240 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb6f00d2120 con 0x7fb6f00b3660 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f7791640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f006e260 con 0x7fb6f00cfb90 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f5d2e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb6f00d1240 0x7fb6f006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34818/0 (socket says 172.21.15.99:34818) 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.111+0000 7fb6f5d2e640 1 -- 172.21.15.99:0/3653294321 learned_addr learned my addr 172.21.15.99:0/3653294321 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f4d2c640 1 --2- 172.21.15.99:0/3653294321 >> v2:172.21.15.196:3300/0 conn(0x7fb6f00b3660 0x7fb6f00d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f552d640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6f00b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551730412 0 0) 0x7fb6f006df80 con 0x7fb6f00d1240 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d8003520 con 0x7fb6f00d1240 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1435008652 0 0) 0x7fb6d8003520 con 0x7fb6f00d1240 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 msgr2=0x7fb6f00b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:51.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6f00b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 >> v2:172.21.15.196:3300/0 conn(0x7fb6f00b3660 msgr2=0x7fb6f00d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 --2- 172.21.15.99:0/3653294321 >> v2:172.21.15.196:3300/0 conn(0x7fb6f00b3660 0x7fb6f00d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00d1ad0 con 0x7fb6f00d1240 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fb6e80019f0 con 0x7fb6f00d1240 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 >> v1:172.21.15.99:6789/0 conn(0x7fb6f00d1240 legacy=0x7fb6f006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d80039b0 0x7fb6d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00d2120 con 0x7fb6d8004d40 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f006e260 con 0x7fb6d80039b0 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d800b9c0 con 0x7fb6f00cfb90 2024-09-10T11:30:51.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f4d2c640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f552d640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d80039b0 0x7fb6d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 -- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 msgr2=0x7fb6d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00cfb90 0x7fb6d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 -- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d80039b0 msgr2=0x7fb6d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6d80039b0 0x7fb6d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.112+0000 7fb6f5d2e640 1 -- 172.21.15.99:0/3653294321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d8003520 con 0x7fb6d8004d40 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.113+0000 7fb6f5d2e640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6d80051c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80013d0 tx=0x7fb6d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df220ce0ec725168 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.113+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800a4e0 con 0x7fb6d8004d40 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.113+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e802f070 con 0x7fb6d8004d40 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.113+0000 7fb6effff640 1 -- 172.21.15.99:0/3653294321 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e802bce0 con 0x7fb6d8004d40 2024-09-10T11:30:51.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 -- 172.21.15.99:0/3653294321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 msgr2=0x7fb6d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 --2- 172.21.15.99:0/3653294321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6d80051c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80013d0 tx=0x7fb6d8005830 comp rx=0 tx=0).stop 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f5d2e640 1 -- 172.21.15.99:0/3653294321 reap_dead start 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 -- 172.21.15.99:0/3653294321 shutdown_connections 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 -- 172.21.15.99:0/3653294321 >> 172.21.15.99:0/3653294321 conn(0x7fb6f00b4650 msgr2=0x7fb6f00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 -- 172.21.15.99:0/3653294321 shutdown_connections 2024-09-10T11:30:51.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.114+0000 7fb6f7791640 1 -- 172.21.15.99:0/3653294321 wait complete. 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.115+0000 7fb6f7791640 1 Processor -- start 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.115+0000 7fb6f7791640 1 -- start start 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00b2820 0x7fb6f00b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6f00b0680 0x7fb6f00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00d00f0 con 0x7fb6d8004d40 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f00cff70 con 0x7fb6f00b0680 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f7791640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f00d16f0 con 0x7fb6f00b2820 2024-09-10T11:30:51.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46680/0 (socket says 172.21.15.99:46680) 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 -- 172.21.15.99:0/2107281687 learned_addr learned my addr 172.21.15.99:0/2107281687 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f5d2e640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6f00b0680 0x7fb6f00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f4d2c640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00b2820 0x7fb6f00b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 -- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00b2820 msgr2=0x7fb6f00b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00b2820 0x7fb6f00b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 -- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6f00b0680 msgr2=0x7fb6f00c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6f00b0680 0x7fb6f00c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 -- 172.21.15.99:0/2107281687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f00b2cb0 con 0x7fb6d8004d40 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.116+0000 7fb6f552d640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00118e0 tx=0x7fb6e0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.117+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000fa30 con 0x7fb6d8004d40 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.117+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e000fbd0 con 0x7fb6d8004d40 2024-09-10T11:30:51.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.117+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f00b2f90 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.117+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0014440 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.117+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f0154dd0 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.118+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f006df80 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.118+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6e0012040 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.119+0000 7fb6edffb640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 0x7fb6dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.119+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb6e008b320 con 0x7fb6d8004d40 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.119+0000 7fb6f4d2c640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 0x7fb6dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.119+0000 7fb6f4d2c640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 0x7fb6dc06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00cd0c0 tx=0x7fb6e4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.123+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6e0019030 con 0x7fb6d8004d40 2024-09-10T11:30:51.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.349+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7fb6f0162550 con 0x7fb6d8004d40 2024-09-10T11:30:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.414+0000 7fb6edffb640 1 -- 172.21.15.99:0/2107281687 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v35) ==== 133+0+0 (secure 0 0 0) 0x7fb6e0059d60 con 0x7fb6d8004d40 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 msgr2=0x7fb6dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 0x7fb6dc06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00cd0c0 tx=0x7fb6e4009190 comp rx=0 tx=0).stop 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 msgr2=0x7fb6f0153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00118e0 tx=0x7fb6e0011ce0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 shutdown_connections 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb6dc06be30 0x7fb6dc06e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6f00b0680 0x7fb6f00c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6f00b2820 0x7fb6f00b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 --2- 172.21.15.99:0/2107281687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6d8004d40 0x7fb6f0153f00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 >> 172.21.15.99:0/2107281687 conn(0x7fb6f00b4650 msgr2=0x7fb6f006f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 shutdown_connections 2024-09-10T11:30:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.416+0000 7fb6f7791640 1 -- 172.21.15.99:0/2107281687 wait complete. 2024-09-10T11:30:51.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-09-10T11:30:51.536 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2024-09-10T11:30:51.536 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-09-10T11:30:51.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:30:51.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.636+0000 7f60d411f640 1 Processor -- start 2024-09-10T11:30:51.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.636+0000 7f60d411f640 1 -- start start 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d411f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 0x7f60cc073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d411f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60cc15c770 0x7f60cc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d411f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60cc152e60 con 0x7f60cc06e5d0 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d411f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f60cc06ef50 con 0x7f60cc15c770 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d411f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60cc0d15e0 con 0x7f60cc0d1200 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60cc15c770 0x7f60cc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d16ba640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f60cc06e5d0 0x7f60cc15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34820/0 (socket says 172.21.15.99:34820) 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60cc15c770 0x7f60cc06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46696/0 (socket says 172.21.15.99:46696) 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d16ba640 1 -- 172.21.15.99:0/937029700 learned_addr learned my addr 172.21.15.99:0/937029700 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:51.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d1ebb640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 0x7f60cc073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 -- 172.21.15.99:0/937029700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 msgr2=0x7f60cc073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 0x7f60cc073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 -- 172.21.15.99:0/937029700 >> v1:172.21.15.99:6789/0 conn(0x7f60cc06e5d0 legacy=0x7f60cc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d0eb9640 1 -- 172.21.15.99:0/937029700 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 148697199 0 0) 0x7f60cc152e60 con 0x7f60cc06e5d0 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d26bc640 1 -- 172.21.15.99:0/937029700 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc072680 con 0x7f60cc15c770 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.637+0000 7f60d1ebb640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 0x7f60cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.638+0000 7f60d26bc640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc15c770 0x7f60cc06ea10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60c40097a0 tx=0x7f60c402c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e0bb767ce97f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.638+0000 7f60d0eb9640 1 -- 172.21.15.99:0/937029700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4039070 con 0x7f60cc15c770 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.638+0000 7f60d0eb9640 1 -- 172.21.15.99:0/937029700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4034640 con 0x7f60cc15c770 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.638+0000 7f60d0eb9640 1 -- 172.21.15.99:0/937029700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c402ce60 con 0x7f60cc15c770 2024-09-10T11:30:51.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.638+0000 7f60d411f640 1 -- 172.21.15.99:0/937029700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc15c770 msgr2=0x7f60cc06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc15c770 0x7f60cc06ea10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60c40097a0 tx=0x7f60c402c310 comp rx=0 tx=0).stop 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 -- 172.21.15.99:0/937029700 shutdown_connections 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc15c770 0x7f60cc06ea10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 --2- 172.21.15.99:0/937029700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc0d1200 0x7f60cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 -- 172.21.15.99:0/937029700 >> 172.21.15.99:0/937029700 conn(0x7f60cc06bc10 msgr2=0x7f60cc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 -- 172.21.15.99:0/937029700 shutdown_connections 2024-09-10T11:30:51.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.639+0000 7f60d411f640 1 -- 172.21.15.99:0/937029700 wait complete. 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.640+0000 7f60d411f640 1 Processor -- start 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.640+0000 7f60d411f640 1 -- start start 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc06e5d0 0x7f60cc154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc15c770 0x7f60cc15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc072f20 con 0x7f60cc0d1200 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60cc072c40 con 0x7f60cc06e5d0 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60cc15cd90 con 0x7f60cc15c770 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46704/0 (socket says 172.21.15.99:46704) 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 -- 172.21.15.99:0/2379410520 learned_addr learned my addr 172.21.15.99:0/2379410520 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d1ebb640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc06e5d0 0x7f60cc154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d26bc640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc15c770 0x7f60cc15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 -- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc15c770 msgr2=0x7f60cc15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc15c770 0x7f60cc15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 -- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc06e5d0 msgr2=0x7f60cc154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc06e5d0 0x7f60cc154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 -- 172.21.15.99:0/2379410520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60cc1f41c0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d16ba640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c440 tx=0x7f60bc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc018070 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc0074e0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60cc1f44a0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc004a20 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.641+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60cc1443b0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.642+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60cc06e3f0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.643+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60bc01b070 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.643+0000 7f60c27fc640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 0x7f60b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.643+0000 7f60d1ebb640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 0x7f60b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.643+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f60bc08a0b0 con 0x7f60cc0d1200 2024-09-10T11:30:51.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.644+0000 7f60d1ebb640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 0x7f60b006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60c8008ee0 tx=0x7f60c8008430 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:51.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.648+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f60cc06e3f0 con 0x7f60cc0d1200 2024-09-10T11:30:51.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.812+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f60cc0b4bf0 con 0x7f60cc0d1200 2024-09-10T11:30:51.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.815+0000 7f60c27fc640 1 -- 172.21.15.99:0/2379410520 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) ==== 122+0+0 (secure 0 0 0) 0x7f60bc058af0 con 0x7f60cc0d1200 2024-09-10T11:30:51.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 msgr2=0x7f60b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 0x7f60b006e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f60c8008ee0 tx=0x7f60c8008430 comp rx=0 tx=0).stop 2024-09-10T11:30:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 msgr2=0x7f60cc1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c440 tx=0x7f60bc00c900 comp rx=0 tx=0).stop 2024-09-10T11:30:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 shutdown_connections 2024-09-10T11:30:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60b006be30 0x7f60b006e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60cc15c770 0x7f60cc15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60cc0d1200 0x7f60cc1f3850 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 --2- 172.21.15.99:0/2379410520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60cc06e5d0 0x7f60cc154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 >> 172.21.15.99:0/2379410520 conn(0x7f60cc06bc10 msgr2=0x7f60cc06dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 shutdown_connections 2024-09-10T11:30:51.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:51.817+0000 7f60d411f640 1 -- 172.21.15.99:0/2379410520 wait complete. 2024-09-10T11:30:51.829 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:30:51.830 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:30:51.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:30:51.939 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:30:51.940 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:51.940 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:51.956 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:51.956 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:52.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:52.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.151+0000 7f83182b7640 1 Processor -- start 2024-09-10T11:30:52.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.151+0000 7f83182b7640 1 -- start start 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 0x7f83100cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f831015cc30 0x7f83100d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83100cf7f0 0x7f83100ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f83100d1540 con 0x7f831015cc30 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f83100d1770 con 0x7f83100cf410 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f83182b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f83100cd810 con 0x7f83100cf7f0 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f831015cc30 0x7f83100d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f831015cc30 0x7f83100d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46706/0 (socket says 172.21.15.99:46706) 2024-09-10T11:30:52.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 -- 172.21.15.99:0/3277546769 learned_addr learned my addr 172.21.15.99:0/3277546769 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8316053640 1 --2- 172.21.15.99:0/3277546769 >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 0x7f83100cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8316854640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83100cf7f0 0x7f83100ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 -- 172.21.15.99:0/3277546769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83100cf7f0 msgr2=0x7f83100ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83100cf7f0 0x7f83100ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 -- 172.21.15.99:0/3277546769 >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 msgr2=0x7f83100cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 --2- 172.21.15.99:0/3277546769 >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 0x7f83100cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8315852640 1 -- 172.21.15.99:0/3277546769 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100cdaf0 con 0x7f831015cc30 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.152+0000 7f8316053640 1 --2- 172.21.15.99:0/3277546769 >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 0x7f83100cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.153+0000 7f8315852640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f831015cc30 0x7f83100d1000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8310072600 tx=0x7f830002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe41ecb73b578e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.153+0000 7f8315051640 1 -- 172.21.15.99:0/3277546769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300037070 con 0x7f831015cc30 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.153+0000 7f8315051640 1 -- 172.21.15.99:0/3277546769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830002ae00 con 0x7f831015cc30 2024-09-10T11:30:52.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.153+0000 7f8315051640 1 -- 172.21.15.99:0/3277546769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300032660 con 0x7f831015cc30 2024-09-10T11:30:52.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 -- 172.21.15.99:0/3277546769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f831015cc30 msgr2=0x7f83100d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f831015cc30 0x7f83100d1000 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8310072600 tx=0x7f830002a0b0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 -- 172.21.15.99:0/3277546769 shutdown_connections 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83100cf7f0 0x7f83100ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 --2- 172.21.15.99:0/3277546769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f831015cc30 0x7f83100d1000 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 --2- 172.21.15.99:0/3277546769 >> v2:172.21.15.196:3300/0 conn(0x7f83100cf410 0x7f83100cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 -- 172.21.15.99:0/3277546769 >> 172.21.15.99:0/3277546769 conn(0x7f83100b4570 msgr2=0x7f83100b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 -- 172.21.15.99:0/3277546769 shutdown_connections 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.154+0000 7f83182b7640 1 -- 172.21.15.99:0/3277546769 wait complete. 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.155+0000 7f83182b7640 1 Processor -- start 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.155+0000 7f83182b7640 1 -- start start 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83100cf7f0 0x7f831006ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f831015cc30 0x7f83100704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83100cfdc0 con 0x7f83100cf410 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83100cfc40 con 0x7f83100cf7f0 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f83182b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f83100ce830 con 0x7f831015cc30 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46718/0 (socket says 172.21.15.99:46718) 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 -- 172.21.15.99:0/3958847284 learned_addr learned my addr 172.21.15.99:0/3958847284 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8315852640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83100cf7f0 0x7f831006ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316854640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f831015cc30 0x7f83100704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 -- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f831015cc30 msgr2=0x7f83100704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f831015cc30 0x7f83100704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 -- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83100cf7f0 msgr2=0x7f831006ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83100cf7f0 0x7f831006ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 -- 172.21.15.99:0/3958847284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83100c8d60 con 0x7f83100cf410 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8316053640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c7c0 tx=0x7f830c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.156+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c015070 con 0x7f83100cf410 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.157+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f830c007a30 con 0x7f83100cf410 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.157+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8310158a10 con 0x7f83100cf410 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.157+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c007e00 con 0x7f83100cf410 2024-09-10T11:30:52.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.157+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8310158fd0 con 0x7f83100cf410 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.158+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83101595e0 con 0x7f83100cf410 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.158+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f830c018070 con 0x7f83100cf410 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.158+0000 7f8306ffd640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.159+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f830c08ad60 con 0x7f83100cf410 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.159+0000 7f8315852640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.159+0000 7f8315852640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 0x7f82f406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8300034000 tx=0x7f830002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.163+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83100c8d60 con 0x7f83100cf410 2024-09-10T11:30:52.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.344+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83100b89a0 con 0x7f83100cf410 2024-09-10T11:30:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.344+0000 7f8306ffd640 1 -- 172.21.15.99:0/3958847284 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f830c059710 con 0x7f83100cf410 2024-09-10T11:30:52.345 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 msgr2=0x7f82f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 0x7f82f406e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8300034000 tx=0x7f830002e000 comp rx=0 tx=0).stop 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 msgr2=0x7f83100c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f830c00c7c0 tx=0x7f830c00cc80 comp rx=0 tx=0).stop 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 shutdown_connections 2024-09-10T11:30:52.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f82f406be30 0x7f82f406e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f831015cc30 0x7f83100704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83100cf7f0 0x7f831006ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 --2- 172.21.15.99:0/3958847284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83100cf410 0x7f83100c8820 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 >> 172.21.15.99:0/3958847284 conn(0x7f83100b4570 msgr2=0x7f83100b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 shutdown_connections 2024-09-10T11:30:52.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.346+0000 7f83182b7640 1 -- 172.21.15.99:0/3958847284 wait complete. 2024-09-10T11:30:52.358 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:52.358 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:52.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:52.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.512+0000 7f1a557a6640 1 Processor -- start 2024-09-10T11:30:52.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.512+0000 7f1a557a6640 1 -- start start 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a557a6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a500d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a557a6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a500d2990 con 0x7f1a500d3a30 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a557a6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a500d2c70 con 0x7f1a500d1cb0 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a557a6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a500d2f50 con 0x7f1a5015efc0 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a4e7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1a500d3a30 0x7f1a500d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34828/0 (socket says 172.21.15.99:34828) 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a4e7fc640 1 -- 172.21.15.99:0/1045494005 learned_addr learned my addr 172.21.15.99:0/1045494005 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a4effd640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a500d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714008115 0 0) 0x7f1a500d2990 con 0x7f1a500d3a30 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.513+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a38003520 con 0x7f1a500d3a30 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270582456 0 0) 0x7f1a500d2c70 con 0x7f1a500d1cb0 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a500d2990 con 0x7f1a500d1cb0 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3300285427 0 0) 0x7f1a38003520 con 0x7f1a500d3a30 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 msgr2=0x7f1a500d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a500d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 >> v1:172.21.15.196:6789/0 conn(0x7f1a500d1cb0 legacy=0x7f1a500b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a500d3230 con 0x7f1a500d3a30 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1a44001670 con 0x7f1a500d3a30 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 >> v1:172.21.15.99:6789/0 conn(0x7f1a500d3a30 legacy=0x7f1a500d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a38003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a38004800 0x7f1a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a500d2f50 con 0x7f1a38003b70 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3800ba70 con 0x7f1a38004800 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3800bc40 con 0x7f1a5015efc0 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a38003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4e7fc640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a38004800 0x7f1a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4f7fe640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 -- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 msgr2=0x7f1a380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5015efc0 0x7f1a380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 -- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a38004800 msgr2=0x7f1a38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a38004800 0x7f1a38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4effd640 1 -- 172.21.15.99:0/1045494005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a38003520 con 0x7f1a38003b70 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.514+0000 7f1a4e7fc640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a38004800 0x7f1a38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a4effd640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a38003f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ac0 tx=0x7f1a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d068cd99df84a2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40036070 con 0x7f1a38003b70 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a40031400 con 0x7f1a38003b70 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a4dffb640 1 -- 172.21.15.99:0/1045494005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40009690 con 0x7f1a38003b70 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a557a6640 1 -- 172.21.15.99:0/1045494005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 msgr2=0x7f1a38003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a557a6640 1 --2- 172.21.15.99:0/1045494005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a38003f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ac0 tx=0x7f1a38005830 comp rx=0 tx=0).stop 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a4f7fe640 1 -- 172.21.15.99:0/1045494005 reap_dead start 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a557a6640 1 -- 172.21.15.99:0/1045494005 shutdown_connections 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.515+0000 7f1a557a6640 1 -- 172.21.15.99:0/1045494005 >> 172.21.15.99:0/1045494005 conn(0x7f1a500d4c20 msgr2=0x7f1a500d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.516+0000 7f1a557a6640 1 -- 172.21.15.99:0/1045494005 shutdown_connections 2024-09-10T11:30:52.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.516+0000 7f1a557a6640 1 -- 172.21.15.99:0/1045494005 wait complete. 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.517+0000 7f1a557a6640 1 Processor -- start 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.517+0000 7f1a557a6640 1 -- start start 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a50152cf0 0x7f1a501f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 0x7f1a501f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a500b5080 con 0x7f1a38003b70 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1a500b4f00 con 0x7f1a50157720 2024-09-10T11:30:52.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1a500b5200 con 0x7f1a50152cf0 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46736/0 (socket says 172.21.15.99:46736) 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 -- 172.21.15.99:0/3748207995 learned_addr learned my addr 172.21.15.99:0/3748207995 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4f7fe640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 0x7f1a501f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4e7fc640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a50152cf0 0x7f1a501f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 -- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a50152cf0 msgr2=0x7f1a501f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a50152cf0 0x7f1a501f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 -- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 msgr2=0x7f1a501f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 0x7f1a501f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 -- 172.21.15.99:0/3748207995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a500c9fa0 con 0x7f1a38003b70 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4f7fe640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 0x7f1a501f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:52.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a4effd640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ac0 tx=0x7f1a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40041020 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a4003c470 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a500b7e50 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40036040 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.518+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a500ba500 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.519+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a500b4560 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.520+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a40044020 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.520+0000 7f1a377fe640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 0x7f1a3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.520+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1a400ab620 con 0x7f1a38003b70 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.520+0000 7f1a4e7fc640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 0x7f1a3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.521+0000 7f1a4e7fc640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 0x7f1a3c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1a501530d0 tx=0x7f1a44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.525+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a4004b030 con 0x7f1a38003b70 2024-09-10T11:30:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.698+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a500ba8a0 con 0x7f1a38003b70 2024-09-10T11:30:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.699+0000 7f1a377fe640 1 -- 172.21.15.99:0/3748207995 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f1a40079fd0 con 0x7f1a38003b70 2024-09-10T11:30:52.699 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.700+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 msgr2=0x7f1a3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.700+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 0x7f1a3c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1a501530d0 tx=0x7f1a44009040 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.700+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 msgr2=0x7f1a501527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.700+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ac0 tx=0x7f1a40004730 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 shutdown_connections 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1a3c06be60 0x7f1a3c06e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a50157720 0x7f1a501f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a50152cf0 0x7f1a501f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 --2- 172.21.15.99:0/3748207995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a38003b70 0x7f1a501527b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 >> 172.21.15.99:0/3748207995 conn(0x7f1a500d4c20 msgr2=0x7f1a500cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 shutdown_connections 2024-09-10T11:30:52.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.701+0000 7f1a557a6640 1 -- 172.21.15.99:0/3748207995 wait complete. 2024-09-10T11:30:52.712 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:30:52.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:52.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:30:52.737 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:30:52.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:52.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:52.738 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:30:52.799 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:52.799 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:52.817 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:52.817 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:52.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.974+0000 7f0401398640 1 Processor -- start 2024-09-10T11:30:52.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.974+0000 7f0401398640 1 -- start start 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f0401398640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03fc15efc0 0x7f03fc0d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f0401398640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f0401398640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d2990 con 0x7f03fc0d3a30 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f0401398640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc0d2c70 con 0x7f03fc15efc0 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f0401398640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0d2f50 con 0x7f03fc0d1cb0 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03faffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03fc15efc0 0x7f03fc0d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03faffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03fc15efc0 0x7f03fc0d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57022/0 (socket says 172.21.15.99:57022) 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03faffd640 1 -- 172.21.15.99:0/1476251459 learned_addr learned my addr 172.21.15.99:0/1476251459 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03fb7fe640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03faffd640 1 -- 172.21.15.99:0/1476251459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 msgr2=0x7f03fc0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.975+0000 7f03faffd640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc0d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03faffd640 1 -- 172.21.15.99:0/1476251459 >> v1:172.21.15.196:6789/0 conn(0x7f03fc0d3a30 legacy=0x7f03fc0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03faffd640 1 -- 172.21.15.99:0/1476251459 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0d3230 con 0x7f03fc15efc0 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03faffd640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc15efc0 0x7f03fc0d03f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009e50 tx=0x7f03ec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcde16833dbba207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03f9ffb640 1 -- 172.21.15.99:0/1476251459 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec036070 con 0x7f03fc15efc0 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03f9ffb640 1 -- 172.21.15.99:0/1476251459 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec031600 con 0x7f03fc15efc0 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.976+0000 7f03f9ffb640 1 -- 172.21.15.99:0/1476251459 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0319d0 con 0x7f03fc15efc0 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 -- 172.21.15.99:0/1476251459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc15efc0 msgr2=0x7f03fc0d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc15efc0 0x7f03fc0d03f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009e50 tx=0x7f03ec02a710 comp rx=0 tx=0).stop 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 -- 172.21.15.99:0/1476251459 shutdown_connections 2024-09-10T11:30:52.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 --2- 172.21.15.99:0/1476251459 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc15efc0 0x7f03fc0d03f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 -- 172.21.15.99:0/1476251459 >> 172.21.15.99:0/1476251459 conn(0x7f03fc0d4c20 msgr2=0x7f03fc0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 -- 172.21.15.99:0/1476251459 shutdown_connections 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.977+0000 7f0401398640 1 -- 172.21.15.99:0/1476251459 wait complete. 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.978+0000 7f0401398640 1 Processor -- start 2024-09-10T11:30:52.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.978+0000 7f0401398640 1 -- start start 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03fc15efc0 0x7f03fc0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0b4970 con 0x7f03fc0d3a30 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0b47f0 con 0x7f03fc15efc0 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f0401398640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0b4af0 con 0x7f03fc0d1cb0 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57024/0 (socket says 172.21.15.99:57024) 2024-09-10T11:30:52.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 -- 172.21.15.99:0/1208985118 learned_addr learned my addr 172.21.15.99:0/1208985118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03faffd640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fb7fe640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03fc15efc0 0x7f03fc0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 -- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 msgr2=0x7f03fc153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 -- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03fc15efc0 msgr2=0x7f03fc0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.979+0000 7f03fa7fc640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03fc15efc0 0x7f03fc0cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f03fa7fc640 1 -- 172.21.15.99:0/1208985118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc159580 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f03fa7fc640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c3c0 tx=0x7f03f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0017070 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f00052f0 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc1f5a40 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0003850 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.980+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1f6000 con 0x7f03fc0d3a30 2024-09-10T11:30:52.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.981+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc0b4e80 con 0x7f03fc0d3a30 2024-09-10T11:30:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.981+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03f001a070 con 0x7f03fc0d3a30 2024-09-10T11:30:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.982+0000 7f03e37fe640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 0x7f03e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.982+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f03f008a0a0 con 0x7f03fc0d3a30 2024-09-10T11:30:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.982+0000 7f03faffd640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 0x7f03e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:52.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.982+0000 7f03faffd640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 0x7f03e806e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03ec033000 tx=0x7f03ec009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:52.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:52.986+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03f0058400 con 0x7f03fc0d3a30 2024-09-10T11:30:53.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.165+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03fc08ec70 con 0x7f03fc0d3a30 2024-09-10T11:30:53.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.165+0000 7f03e37fe640 1 -- 172.21.15.99:0/1208985118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f03f005c700 con 0x7f03fc0d3a30 2024-09-10T11:30:53.166 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 msgr2=0x7f03e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 0x7f03e806e320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f03ec033000 tx=0x7f03ec009000 comp rx=0 tx=0).stop 2024-09-10T11:30:53.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 msgr2=0x7f03fc159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c3c0 tx=0x7f03f000c880 comp rx=0 tx=0).stop 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 shutdown_connections 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03e806be60 0x7f03e806e320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03fc15efc0 0x7f03fc0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03fc0d3a30 0x7f03fc159040 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 --2- 172.21.15.99:0/1208985118 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03fc0d1cb0 0x7f03fc153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 >> 172.21.15.99:0/1208985118 conn(0x7f03fc0d4c20 msgr2=0x7f03fc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 shutdown_connections 2024-09-10T11:30:53.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.167+0000 7f0401398640 1 -- 172.21.15.99:0/1208985118 wait complete. 2024-09-10T11:30:53.179 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:53.179 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:53.198 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:53.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.297+0000 7fb82619f640 1 Processor -- start 2024-09-10T11:30:53.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.297+0000 7fb82619f640 1 -- start start 2024-09-10T11:30:53.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb82619f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 0x7fb8200b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb82619f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb8200d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb82619f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8200b3620 con 0x7fb82006cd20 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb82619f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb82006d960 con 0x7fb820163580 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb82619f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb82006dc40 con 0x7fb8200b3960 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb82006cd20 0x7fb82006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59798/0 (socket says 172.21.15.99:59798) 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81ffff640 1 -- 172.21.15.99:0/3266060309 learned_addr learned my addr 172.21.15.99:0/3266060309 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81f7fe640 1 --2- 172.21.15.99:0/3266060309 >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 0x7fb8200b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81effd640 1 --2- 172.21.15.99:0/3266060309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb8200d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81e7fc640 1 -- 172.21.15.99:0/3266060309 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4107838751 0 0) 0x7fb8200b3620 con 0x7fb82006cd20 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81e7fc640 1 -- 172.21.15.99:0/3266060309 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb808003520 con 0x7fb82006cd20 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81effd640 1 -- 172.21.15.99:0/3266060309 >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 msgr2=0x7fb8200b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81effd640 1 --2- 172.21.15.99:0/3266060309 >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 0x7fb8200b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81effd640 1 -- 172.21.15.99:0/3266060309 >> v1:172.21.15.99:6789/0 conn(0x7fb82006cd20 legacy=0x7fb82006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81effd640 1 -- 172.21.15.99:0/3266060309 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82006df20 con 0x7fb8200b3960 2024-09-10T11:30:53.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.298+0000 7fb81f7fe640 1 --2- 172.21.15.99:0/3266060309 >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 0x7fb8200b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.299+0000 7fb81effd640 1 --2- 172.21.15.99:0/3266060309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb8200d1be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=664a50b598c9e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.299+0000 7fb81e7fc640 1 -- 172.21.15.99:0/3266060309 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814037070 con 0x7fb8200b3960 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.299+0000 7fb81e7fc640 1 -- 172.21.15.99:0/3266060309 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81402ae50 con 0x7fb8200b3960 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.299+0000 7fb81e7fc640 1 -- 172.21.15.99:0/3266060309 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814032660 con 0x7fb8200b3960 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 -- 172.21.15.99:0/3266060309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 msgr2=0x7fb8200d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 --2- 172.21.15.99:0/3266060309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb8200d1be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb8140097a0 tx=0x7fb81402a100 comp rx=0 tx=0).stop 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 -- 172.21.15.99:0/3266060309 shutdown_connections 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 --2- 172.21.15.99:0/3266060309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb8200d1be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 --2- 172.21.15.99:0/3266060309 >> v2:172.21.15.196:3300/0 conn(0x7fb820163580 0x7fb8200b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 -- 172.21.15.99:0/3266060309 >> 172.21.15.99:0/3266060309 conn(0x7fb8200b4570 msgr2=0x7fb8200b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 -- 172.21.15.99:0/3266060309 shutdown_connections 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.300+0000 7fb82619f640 1 -- 172.21.15.99:0/3266060309 wait complete. 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.301+0000 7fb82619f640 1 Processor -- start 2024-09-10T11:30:53.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.301+0000 7fb82619f640 1 -- start start 2024-09-10T11:30:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82006cd20 0x7fb8201591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb820161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200b3160 con 0x7fb820163580 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8200d2120 con 0x7fb82006cd20 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb82619f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb8200b32e0 con 0x7fb8200b3960 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57040/0 (socket says 172.21.15.99:57040) 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 -- 172.21.15.99:0/2830587690 learned_addr learned my addr 172.21.15.99:0/2830587690 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81effd640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb820161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81f7fe640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82006cd20 0x7fb8201591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 -- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 msgr2=0x7fb820161e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb820161e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 -- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82006cd20 msgr2=0x7fb8201591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82006cd20 0x7fb8201591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.302+0000 7fb81ffff640 1 -- 172.21.15.99:0/2830587690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8201f4c30 con 0x7fb820163580 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb81ffff640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb81800caa0 tx=0x7fb81800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818017070 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb81800ac00 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb818012400 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8201f4ef0 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.303+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8201f54b0 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.304+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8200d1be0 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.304+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb81801a070 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.305+0000 7fb807fff640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 0x7fb80c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.305+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb81808abc0 con 0x7fb820163580 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.305+0000 7fb81f7fe640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 0x7fb80c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.305+0000 7fb81f7fe640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 0x7fb80c06e320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8200ccaf0 tx=0x7fb810009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.309+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb818058ea0 con 0x7fb820163580 2024-09-10T11:30:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.486+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb82006f7f0 con 0x7fb820163580 2024-09-10T11:30:53.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.486+0000 7fb807fff640 1 -- 172.21.15.99:0/2830587690 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fb81805d1a0 con 0x7fb820163580 2024-09-10T11:30:53.488 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:53.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 msgr2=0x7fb80c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 0x7fb80c06e320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb8200ccaf0 tx=0x7fb810009190 comp rx=0 tx=0).stop 2024-09-10T11:30:53.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 msgr2=0x7fb8201623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb81800caa0 tx=0x7fb81800cf60 comp rx=0 tx=0).stop 2024-09-10T11:30:53.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 shutdown_connections 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb80c06be60 0x7fb80c06e320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb820163580 0x7fb8201623a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8200b3960 0x7fb820161e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 --2- 172.21.15.99:0/2830587690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb82006cd20 0x7fb8201591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 >> 172.21.15.99:0/2830587690 conn(0x7fb8200b4570 msgr2=0x7fb82015f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 shutdown_connections 2024-09-10T11:30:53.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.489+0000 7fb82619f640 1 -- 172.21.15.99:0/2830587690 wait complete. 2024-09-10T11:30:53.500 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:30:53.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:53.501 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:30:53.526 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:30:53.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:53.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:53.527 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:30:53.588 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:53.588 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:53.604 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:53.604 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:53.660 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.798+0000 7f90c4fcf640 1 Processor -- start 2024-09-10T11:30:53.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.798+0000 7f90c4fcf640 1 -- start start 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90c4fcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b4150 0x7f90c00b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90c4fcf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f90c00b0df0 0x7f90c015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90c4fcf640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90c0072650 con 0x7f90c015cb70 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90c4fcf640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f90c00b1240 con 0x7f90c00b0df0 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90c4fcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f90c00734f0 con 0x7f90c00b4150 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f90c00b0df0 0x7f90c015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f90c00b0df0 0x7f90c015cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57056/0 (socket says 172.21.15.99:57056) 2024-09-10T11:30:53.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 -- 172.21.15.99:0/924065845 learned_addr learned my addr 172.21.15.99:0/924065845 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bdd9b640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b4150 0x7f90c00b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 -- 172.21.15.99:0/924065845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b4150 msgr2=0x7f90c00b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b4150 0x7f90c00b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 -- 172.21.15.99:0/924065845 >> v1:172.21.15.196:6789/0 conn(0x7f90c015cb70 legacy=0x7f90c0072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.799+0000 7f90bed9d640 1 -- 172.21.15.99:0/924065845 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c015db70 con 0x7f90c00b0df0 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90bed9d640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b0df0 0x7f90c015cf50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009870 tx=0x7f90b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edd0d7b09ce86b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90bd59a640 1 -- 172.21.15.99:0/924065845 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0037070 con 0x7f90c00b0df0 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90bd59a640 1 -- 172.21.15.99:0/924065845 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b0032440 con 0x7f90c00b0df0 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90bd59a640 1 -- 172.21.15.99:0/924065845 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0032810 con 0x7f90c00b0df0 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90c4fcf640 1 -- 172.21.15.99:0/924065845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b0df0 msgr2=0x7f90c015cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.800+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b0df0 0x7f90c015cf50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009870 tx=0x7f90b002a270 comp rx=0 tx=0).stop 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 -- 172.21.15.99:0/924065845 shutdown_connections 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b0df0 0x7f90c015cf50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/924065845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b4150 0x7f90c00b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 -- 172.21.15.99:0/924065845 >> 172.21.15.99:0/924065845 conn(0x7f90c00b3630 msgr2=0x7f90c00b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:53.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 -- 172.21.15.99:0/924065845 shutdown_connections 2024-09-10T11:30:53.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.801+0000 7f90c4fcf640 1 -- 172.21.15.99:0/924065845 wait complete. 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.802+0000 7f90c4fcf640 1 Processor -- start 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.802+0000 7f90c4fcf640 1 -- start start 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b0df0 0x7f90c0142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90c015cb70 0x7f90c0160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90c015e130 con 0x7f90c00b4150 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90c00728c0 con 0x7f90c015cb70 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f90c015d490 con 0x7f90c00b0df0 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57072/0 (socket says 172.21.15.99:57072) 2024-09-10T11:30:53.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 -- 172.21.15.99:0/588055481 learned_addr learned my addr 172.21.15.99:0/588055481 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90be59c640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b0df0 0x7f90c0142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bed9d640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90c015cb70 0x7f90c0160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 -- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b0df0 msgr2=0x7f90c0142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b0df0 0x7f90c0142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 -- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90c015cb70 msgr2=0x7f90c0160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90c015cb70 0x7f90c0160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 -- 172.21.15.99:0/588055481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c01e7fd0 con 0x7f90c00b4150 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90bdd9b640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90b4007ef0 tx=0x7f90b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4014620 con 0x7f90c00b4150 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b4019070 con 0x7f90c00b4150 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c01e82b0 con 0x7f90c00b4150 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4014a00 con 0x7f90c00b4150 2024-09-10T11:30:53.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.803+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c01430b0 con 0x7f90c00b4150 2024-09-10T11:30:53.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.804+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c0158bf0 con 0x7f90c00b4150 2024-09-10T11:30:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.805+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90b4014ba0 con 0x7f90c00b4150 2024-09-10T11:30:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.805+0000 7f90aeffd640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 0x7f90a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.805+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90b408b5f0 con 0x7f90c00b4150 2024-09-10T11:30:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.805+0000 7f90be59c640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 0x7f90a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:53.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.806+0000 7f90be59c640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 0x7f90a006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90c015c350 tx=0x7f90b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:53.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.810+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90c0158bf0 con 0x7f90c00b4150 2024-09-10T11:30:53.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.987+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90c015d8c0 con 0x7f90c00b4150 2024-09-10T11:30:53.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.987+0000 7f90aeffd640 1 -- 172.21.15.99:0/588055481 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f90b40599e0 con 0x7f90c00b4150 2024-09-10T11:30:53.989 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:53.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 msgr2=0x7f90a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 0x7f90a006e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f90c015c350 tx=0x7f90b8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 msgr2=0x7f90c01e7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f90b4007ef0 tx=0x7f90b400e4b0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 shutdown_connections 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f90a006be30 0x7f90a006e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90c015cb70 0x7f90c0160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90c00b4150 0x7f90c01e7510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 --2- 172.21.15.99:0/588055481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90c00b0df0 0x7f90c0142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 >> 172.21.15.99:0/588055481 conn(0x7f90c00b3630 msgr2=0x7f90c00b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 shutdown_connections 2024-09-10T11:30:53.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:53.989+0000 7f90c4fcf640 1 -- 172.21.15.99:0/588055481 wait complete. 2024-09-10T11:30:54.002 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:30:54.002 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:54.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.156+0000 7f1d08931640 1 Processor -- start 2024-09-10T11:30:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.156+0000 7f1d08931640 1 -- start start 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d040d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 0x7f1d040d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d040d4e40 0x7f1d040d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1d040d52d0 con 0x7f1d040d4e40 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1d040d1620 con 0x7f1d041626c0 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.157+0000 7f1d08931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d04162bc0 con 0x7f1d0415eb80 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d040d4e40 0x7f1d040d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d040d4e40 0x7f1d040d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57078/0 (socket says 172.21.15.99:57078) 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 -- 172.21.15.99:0/3027294332 learned_addr learned my addr 172.21.15.99:0/3027294332 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d0259c640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d040d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d01d9b640 1 --2- 172.21.15.99:0/3027294332 >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 0x7f1d040d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 -- 172.21.15.99:0/3027294332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 msgr2=0x7f1d040d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d040d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 -- 172.21.15.99:0/3027294332 >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 msgr2=0x7f1d040d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 --2- 172.21.15.99:0/3027294332 >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 0x7f1d040d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d02d9d640 1 -- 172.21.15.99:0/3027294332 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d04162ea0 con 0x7f1d040d4e40 2024-09-10T11:30:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.158+0000 7f1d01d9b640 1 --2- 172.21.15.99:0/3027294332 >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 0x7f1d040d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d02d9d640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040d1b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1d0408db70 tx=0x7f1cfc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99d2deef5f34f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d0159a640 1 -- 172.21.15.99:0/3027294332 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc036070 con 0x7f1d040d4e40 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d0159a640 1 -- 172.21.15.99:0/3027294332 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cfc031600 con 0x7f1d040d4e40 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d0159a640 1 -- 172.21.15.99:0/3027294332 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0319d0 con 0x7f1d040d4e40 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 -- 172.21.15.99:0/3027294332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 msgr2=0x7f1d040d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040d1b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1d0408db70 tx=0x7f1cfc02a610 comp rx=0 tx=0).stop 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 -- 172.21.15.99:0/3027294332 shutdown_connections 2024-09-10T11:30:54.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040d1b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 --2- 172.21.15.99:0/3027294332 >> v2:172.21.15.196:3300/0 conn(0x7f1d041626c0 0x7f1d040d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 --2- 172.21.15.99:0/3027294332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d040d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 -- 172.21.15.99:0/3027294332 >> 172.21.15.99:0/3027294332 conn(0x7f1d0408e680 msgr2=0x7f1d0409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:54.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 -- 172.21.15.99:0/3027294332 shutdown_connections 2024-09-10T11:30:54.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.159+0000 7f1d08931640 1 -- 172.21.15.99:0/3027294332 wait complete. 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 Processor -- start 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 -- start start 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d0414dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d041626c0 0x7f1d040b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d040d2ef0 con 0x7f1d040d4e40 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04163460 con 0x7f1d041626c0 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d08931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d040d3070 con 0x7f1d0415eb80 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.161+0000 7f1d0259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57090/0 (socket says 172.21.15.99:57090) 2024-09-10T11:30:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 -- 172.21.15.99:0/1666400185 learned_addr learned my addr 172.21.15.99:0/1666400185 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 -- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 msgr2=0x7f1d0414dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d0414dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 -- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d041626c0 msgr2=0x7f1d040b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d041626c0 0x7f1d040b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 -- 172.21.15.99:0/1666400185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d040b8340 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d0259c640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e8a0 tx=0x7f1cf400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4017070 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cf400c940 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d040b8620 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400cd30 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.162+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d040b8be0 con 0x7f1d040d4e40 2024-09-10T11:30:54.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.163+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d040d29b0 con 0x7f1d040d4e40 2024-09-10T11:30:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.163+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cf401a070 con 0x7f1d040d4e40 2024-09-10T11:30:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.164+0000 7f1cf2ffd640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 0x7f1ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.164+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1cf408b110 con 0x7f1d040d4e40 2024-09-10T11:30:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.164+0000 7f1d01d9b640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 0x7f1ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.164+0000 7f1d01d9b640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 0x7f1ce406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d040cba20 tx=0x7f1cf8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.168+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1cf4059a40 con 0x7f1d040d4e40 2024-09-10T11:30:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.344+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d040b8030 con 0x7f1d040d4e40 2024-09-10T11:30:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.344+0000 7f1cf2ffd640 1 -- 172.21.15.99:0/1666400185 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f1cf4059340 con 0x7f1d040d4e40 2024-09-10T11:30:54.345 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 msgr2=0x7f1ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 0x7f1ce406e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1d040cba20 tx=0x7f1cf8008040 comp rx=0 tx=0).stop 2024-09-10T11:30:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 msgr2=0x7f1d040b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e8a0 tx=0x7f1cf400ed60 comp rx=0 tx=0).stop 2024-09-10T11:30:54.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 shutdown_connections 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ce406be30 0x7f1ce406e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d041626c0 0x7f1d040b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d0415eb80 0x7f1d0414dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 --2- 172.21.15.99:0/1666400185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d040d4e40 0x7f1d040b7af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 >> 172.21.15.99:0/1666400185 conn(0x7f1d0408e680 msgr2=0x7f1d040d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 shutdown_connections 2024-09-10T11:30:54.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.346+0000 7f1d08931640 1 -- 172.21.15.99:0/1666400185 wait complete. 2024-09-10T11:30:54.358 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:30:54.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:54.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-10T11:30:54.383 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:30:54.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:30:54.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:54.384 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:30:54.440 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:54.440 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:54.459 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:54.459 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:54.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:54.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.616+0000 7fee10711640 1 Processor -- start 2024-09-10T11:30:54.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.616+0000 7fee10711640 1 -- start start 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee10711640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fee080d1200 0x7fee08073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee10711640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee0815c770 0x7fee0806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee10711640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee08152e60 con 0x7fee0806e5d0 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee10711640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fee0806ef80 con 0x7fee0815c770 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee10711640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fee080d15e0 con 0x7fee080d1200 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0ecae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee0815c770 0x7fee0806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0dcac640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fee0806e5d0 0x7fee0815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59806/0 (socket says 172.21.15.99:59806) 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0dcac640 1 -- 172.21.15.99:0/947683426 learned_addr learned my addr 172.21.15.99:0/947683426 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0e4ad640 1 --2- 172.21.15.99:0/947683426 >> v2:172.21.15.196:3300/0 conn(0x7fee080d1200 0x7fee08073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0ecae640 1 -- 172.21.15.99:0/947683426 >> v2:172.21.15.196:3300/0 conn(0x7fee080d1200 msgr2=0x7fee08073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0ecae640 1 --2- 172.21.15.99:0/947683426 >> v2:172.21.15.196:3300/0 conn(0x7fee080d1200 0x7fee08073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0ecae640 1 -- 172.21.15.99:0/947683426 >> v1:172.21.15.99:6789/0 conn(0x7fee0806e5d0 legacy=0x7fee0815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0d4ab640 1 -- 172.21.15.99:0/947683426 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304593051 0 0) 0x7fee08152e60 con 0x7fee0806e5d0 2024-09-10T11:30:54.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.617+0000 7fee0ecae640 1 -- 172.21.15.99:0/947683426 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee08072680 con 0x7fee0815c770 2024-09-10T11:30:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.618+0000 7fee0ecae640 1 --2- 172.21.15.99:0/947683426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee0806ea40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee080d1850 tx=0x7fee0002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0564794ff94a1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.618+0000 7fee0d4ab640 1 -- 172.21.15.99:0/947683426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00007740 con 0x7fee0815c770 2024-09-10T11:30:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.618+0000 7fee0d4ab640 1 -- 172.21.15.99:0/947683426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee000078e0 con 0x7fee0815c770 2024-09-10T11:30:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.618+0000 7fee0d4ab640 1 -- 172.21.15.99:0/947683426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00007cb0 con 0x7fee0815c770 2024-09-10T11:30:54.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 -- 172.21.15.99:0/947683426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 msgr2=0x7fee0806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 --2- 172.21.15.99:0/947683426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee0806ea40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee080d1850 tx=0x7fee0002a5f0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 -- 172.21.15.99:0/947683426 shutdown_connections 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 --2- 172.21.15.99:0/947683426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee0806ea40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 --2- 172.21.15.99:0/947683426 >> v2:172.21.15.196:3300/0 conn(0x7fee080d1200 0x7fee08073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 -- 172.21.15.99:0/947683426 >> 172.21.15.99:0/947683426 conn(0x7fee0806bc10 msgr2=0x7fee0806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 -- 172.21.15.99:0/947683426 shutdown_connections 2024-09-10T11:30:54.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.619+0000 7fee10711640 1 -- 172.21.15.99:0/947683426 wait complete. 2024-09-10T11:30:54.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.620+0000 7fee10711640 1 Processor -- start 2024-09-10T11:30:54.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.620+0000 7fee10711640 1 -- start start 2024-09-10T11:30:54.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee0806e5d0 0x7fee08151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 0x7fee08154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee08072f20 con 0x7fee0815c770 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee08072c40 con 0x7fee0806e5d0 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee0815cd90 con 0x7fee080d1200 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57112/0 (socket says 172.21.15.99:57112) 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 -- 172.21.15.99:0/3524396553 learned_addr learned my addr 172.21.15.99:0/3524396553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0e4ad640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee0806e5d0 0x7fee08151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0dcac640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 0x7fee08154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 -- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 msgr2=0x7fee08154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 0x7fee08154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 -- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee0806e5d0 msgr2=0x7fee08151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee0806e5d0 0x7fee08151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 -- 172.21.15.99:0/3524396553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0814c670 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0dcac640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 0x7fee08154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee0ecae640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee00009e50 tx=0x7fee00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0003f070 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00031bf0 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.621+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee08133f30 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.622+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00044460 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.622+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081344f0 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.623+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee08073440 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.623+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee00007790 con 0x7fee0815c770 2024-09-10T11:30:54.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.623+0000 7fedfeffd640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.624+0000 7fee0e4ad640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.624+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fee000ac720 con 0x7fee0815c770 2024-09-10T11:30:54.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.624+0000 7fee0e4ad640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 0x7fedec06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee040079a0 tx=0x7fee04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.628+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee0007b040 con 0x7fee0815c770 2024-09-10T11:30:54.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.802+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee080b45b0 con 0x7fee0815c770 2024-09-10T11:30:54.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.802+0000 7fedfeffd640 1 -- 172.21.15.99:0/3524396553 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fee0007a940 con 0x7fee0815c770 2024-09-10T11:30:54.803 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:54.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 msgr2=0x7fedec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 0x7fedec06e2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fee040079a0 tx=0x7fee04008040 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 msgr2=0x7fee08155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fee00009e50 tx=0x7fee00004a40 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 shutdown_connections 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fedec06be30 0x7fedec06e2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee0815c770 0x7fee08155180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee080d1200 0x7fee08154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 --2- 172.21.15.99:0/3524396553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee0806e5d0 0x7fee08151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.804+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 >> 172.21.15.99:0/3524396553 conn(0x7fee0806bc10 msgr2=0x7fee0815f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.805+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 shutdown_connections 2024-09-10T11:30:54.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.805+0000 7fee10711640 1 -- 172.21.15.99:0/3524396553 wait complete. 2024-09-10T11:30:54.817 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:30:54.817 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:30:54.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:30:54.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.937+0000 7ff087865640 1 Processor -- start 2024-09-10T11:30:54.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.937+0000 7ff087865640 1 -- start start 2024-09-10T11:30:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff087865640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff08015eb80 0x7ff0800d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff087865640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0800d5010 0x7ff0800d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff087865640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08015ff70 con 0x7ff0801626c0 2024-09-10T11:30:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff087865640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff0800d1620 con 0x7ff08015eb80 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff087865640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff080162bc0 con 0x7ff0800d5010 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff084e00640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff0801626c0 0x7ff0800d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59808/0 (socket says 172.21.15.99:59808) 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff084e00640 1 -- 172.21.15.99:0/1009299611 learned_addr learned my addr 172.21.15.99:0/1009299611 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff085601640 1 --2- 172.21.15.99:0/1009299611 >> v2:172.21.15.196:3300/0 conn(0x7ff08015eb80 0x7ff0800d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff085e02640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0800d5010 0x7ff0800d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff077fff640 1 -- 172.21.15.99:0/1009299611 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978802218 0 0) 0x7ff08015ff70 con 0x7ff0801626c0 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.938+0000 7ff077fff640 1 -- 172.21.15.99:0/1009299611 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff06c003520 con 0x7ff0801626c0 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.939+0000 7ff085601640 1 -- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0800d5010 msgr2=0x7ff0800d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.939+0000 7ff085601640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0800d5010 0x7ff0800d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.939+0000 7ff085601640 1 -- 172.21.15.99:0/1009299611 >> v1:172.21.15.99:6789/0 conn(0x7ff0801626c0 legacy=0x7ff0800d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.939+0000 7ff085601640 1 -- 172.21.15.99:0/1009299611 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff080162ea0 con 0x7ff08015eb80 2024-09-10T11:30:54.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.939+0000 7ff085601640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08015eb80 0x7ff0800d29b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097d0 tx=0x7ff07c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d714bd75cc51ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff077fff640 1 -- 172.21.15.99:0/1009299611 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c037070 con 0x7ff08015eb80 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff077fff640 1 -- 172.21.15.99:0/1009299611 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff07c02ae40 con 0x7ff08015eb80 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff077fff640 1 -- 172.21.15.99:0/1009299611 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c032660 con 0x7ff08015eb80 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 -- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08015eb80 msgr2=0x7ff0800d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08015eb80 0x7ff0800d29b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0097d0 tx=0x7ff07c02a130 comp rx=0 tx=0).stop 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 -- 172.21.15.99:0/1009299611 shutdown_connections 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0800d5010 0x7ff0800d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 --2- 172.21.15.99:0/1009299611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08015eb80 0x7ff0800d29b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 -- 172.21.15.99:0/1009299611 >> 172.21.15.99:0/1009299611 conn(0x7ff08008e680 msgr2=0x7ff08009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 -- 172.21.15.99:0/1009299611 shutdown_connections 2024-09-10T11:30:54.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.940+0000 7ff087865640 1 -- 172.21.15.99:0/1009299611 wait complete. 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 Processor -- start 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 -- start start 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 0x7ff0800ba440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08015eb80 0x7ff0800b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800d2ef0 con 0x7ff0801626c0 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff080163460 con 0x7ff0800d5010 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff087865640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0800d3070 con 0x7ff08015eb80 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff085e02640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff085e02640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57122/0 (socket says 172.21.15.99:57122) 2024-09-10T11:30:54.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.942+0000 7ff085e02640 1 -- 172.21.15.99:0/536368539 learned_addr learned my addr 172.21.15.99:0/536368539 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085601640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 0x7ff0800ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 -- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08015eb80 msgr2=0x7ff0800b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff084e00640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08015eb80 0x7ff0800b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08015eb80 0x7ff0800b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 -- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 msgr2=0x7ff0800ba440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 0x7ff0800ba440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 -- 172.21.15.99:0/536368539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0801f55b0 con 0x7ff0801626c0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085601640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 0x7ff0800ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff085e02640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff078009cb0 tx=0x7ff07800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078018020 con 0x7ff0801626c0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078014480 con 0x7ff0801626c0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0800b2870 con 0x7ff0801626c0 2024-09-10T11:30:54.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078014870 con 0x7ff0801626c0 2024-09-10T11:30:54.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.943+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0800b2e30 con 0x7ff0801626c0 2024-09-10T11:30:54.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.944+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0800d29b0 con 0x7ff0801626c0 2024-09-10T11:30:54.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.945+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff078005d10 con 0x7ff0801626c0 2024-09-10T11:30:54.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.945+0000 7ff075ffb640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 0x7ff06406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:54.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.945+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff07808bbe0 con 0x7ff0801626c0 2024-09-10T11:30:54.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.945+0000 7ff085601640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 0x7ff06406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:54.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.946+0000 7ff085601640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 0x7ff06406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff07c034000 tx=0x7ff07c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:54.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:54.949+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff078022030 con 0x7ff0801626c0 2024-09-10T11:30:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.125+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff08015dbf0 con 0x7ff0801626c0 2024-09-10T11:30:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.125+0000 7ff075ffb640 1 -- 172.21.15.99:0/536368539 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ff078010040 con 0x7ff0801626c0 2024-09-10T11:30:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 msgr2=0x7ff06406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 0x7ff06406e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff07c034000 tx=0x7ff07c02e000 comp rx=0 tx=0).stop 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 msgr2=0x7ff0800b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff078009cb0 tx=0x7ff07800c490 comp rx=0 tx=0).stop 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 shutdown_connections 2024-09-10T11:30:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff06406be30 0x7ff06406e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0801626c0 0x7ff0800b83e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08015eb80 0x7ff0800b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 --2- 172.21.15.99:0/536368539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0800d5010 0x7ff0800ba440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 >> 172.21.15.99:0/536368539 conn(0x7ff08008e680 msgr2=0x7ff08015d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 shutdown_connections 2024-09-10T11:30:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.127+0000 7ff087865640 1 -- 172.21.15.99:0/536368539 wait complete. 2024-09-10T11:30:55.139 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:30:55.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:55.139 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-10T11:30:55.170 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:30:55.170 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:30:55.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:30:55.170 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:30:55.228 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:30:55.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:55.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.325+0000 7eff03ce6640 1 Processor -- start 2024-09-10T11:30:55.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.325+0000 7eff03ce6640 1 -- start start 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff03ce6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 0x7efefc0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff03ce6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efefc0afbb0 0x7efefc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff03ce6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efefc0b0020 con 0x7efefc15b2d0 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff03ce6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7efefc06ee10 con 0x7efefc0afbb0 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff03ce6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efefc0cd9a0 con 0x7efefc15aef0 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efefc0afbb0 0x7efefc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff01281640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efefc15b2d0 0x7efefc0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59822/0 (socket says 172.21.15.99:59822) 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff01281640 1 -- 172.21.15.99:0/2789993884 learned_addr learned my addr 172.21.15.99:0/2789993884 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efefc0afbb0 0x7efefc0714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57136/0 (socket says 172.21.15.99:57136) 2024-09-10T11:30:55.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff01a82640 1 --2- 172.21.15.99:0/2789993884 >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 0x7efefc0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 -- 172.21.15.99:0/2789993884 >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 msgr2=0x7efefc0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:55.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 --2- 172.21.15.99:0/2789993884 >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 0x7efefc0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 -- 172.21.15.99:0/2789993884 >> v1:172.21.15.99:6789/0 conn(0x7efefc15b2d0 legacy=0x7efefc0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff02283640 1 -- 172.21.15.99:0/2789993884 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0b0290 con 0x7efefc0afbb0 2024-09-10T11:30:55.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.326+0000 7eff01a82640 1 --2- 172.21.15.99:0/2789993884 >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 0x7efefc0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:30:55.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff02283640 1 --2- 172.21.15.99:0/2789993884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc0714a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efef400aea0 tx=0x7efef402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2edae14b8cb83bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff00a80640 1 -- 172.21.15.99:0/2789993884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4037070 con 0x7efefc0afbb0 2024-09-10T11:30:55.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff00a80640 1 -- 172.21.15.99:0/2789993884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef4008640 con 0x7efefc0afbb0 2024-09-10T11:30:55.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff00a80640 1 -- 172.21.15.99:0/2789993884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef4008a10 con 0x7efefc0afbb0 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 -- 172.21.15.99:0/2789993884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 msgr2=0x7efefc0714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2789993884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc0714a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efef400aea0 tx=0x7efef402b7f0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 -- 172.21.15.99:0/2789993884 shutdown_connections 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2789993884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc0714a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2789993884 >> v2:172.21.15.196:3300/0 conn(0x7efefc15aef0 0x7efefc0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.327+0000 7eff03ce6640 1 -- 172.21.15.99:0/2789993884 >> 172.21.15.99:0/2789993884 conn(0x7efefc06bc10 msgr2=0x7efefc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.328+0000 7eff03ce6640 1 -- 172.21.15.99:0/2789993884 shutdown_connections 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.328+0000 7eff03ce6640 1 -- 172.21.15.99:0/2789993884 wait complete. 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 Processor -- start 2024-09-10T11:30:55.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 -- start start 2024-09-10T11:30:55.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efefc15aef0 0x7efefc15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efefc15b2d0 0x7efefc158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efefc06eb30 con 0x7efefc0afbb0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.329+0000 7eff03ce6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efefc06e850 con 0x7efefc15b2d0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff03ce6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efefc0719e0 con 0x7efefc15aef0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57146/0 (socket says 172.21.15.99:57146) 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 -- 172.21.15.99:0/2856869698 learned_addr learned my addr 172.21.15.99:0/2856869698 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff02283640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efefc15b2d0 0x7efefc158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01281640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efefc15aef0 0x7efefc15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 -- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efefc15aef0 msgr2=0x7efefc15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efefc15aef0 0x7efefc15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 -- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efefc15b2d0 msgr2=0x7efefc158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efefc15b2d0 0x7efefc158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 -- 172.21.15.99:0/2856869698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efefc0aeac0 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff01a82640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efef800c7f0 tx=0x7efef800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8015070 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef80079d0 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8007da0 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efefc0aeda0 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.330+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efefc0af360 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.331+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efefc071da0 con 0x7efefc0afbb0 2024-09-10T11:30:55.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.332+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efef8018070 con 0x7efefc0afbb0 2024-09-10T11:30:55.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.332+0000 7efef27fc640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 0x7efee006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.332+0000 7eff01281640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 0x7efee006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.332+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efef808ad20 con 0x7efefc0afbb0 2024-09-10T11:30:55.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.333+0000 7eff01281640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 0x7efee006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d0f60 tx=0x7efeec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.336+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efef8059760 con 0x7efefc0afbb0 2024-09-10T11:30:55.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.536+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efefc159000 con 0x7efefc0afbb0 2024-09-10T11:30:55.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.537+0000 7efef27fc640 1 -- 172.21.15.99:0/2856869698 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7efef8059060 con 0x7efefc0afbb0 2024-09-10T11:30:55.538 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:55.538 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:55.538 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 msgr2=0x7efee006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 0x7efee006e2f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d0f60 tx=0x7efeec009040 comp rx=0 tx=0).stop 2024-09-10T11:30:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 msgr2=0x7efefc154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efef800c7f0 tx=0x7efef800cbf0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 shutdown_connections 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efee006be30 0x7efee006e2f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efefc15b2d0 0x7efefc158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efefc15aef0 0x7efefc15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 --2- 172.21.15.99:0/2856869698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efefc0afbb0 0x7efefc154280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 >> 172.21.15.99:0/2856869698 conn(0x7efefc06bc10 msgr2=0x7efefc072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 shutdown_connections 2024-09-10T11:30:55.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.539+0000 7eff03ce6640 1 -- 172.21.15.99:0/2856869698 wait complete. 2024-09-10T11:30:55.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:55.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.649+0000 7f5c7fc47640 1 Processor -- start 2024-09-10T11:30:55.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.649+0000 7f5c7fc47640 1 -- start start 2024-09-10T11:30:55.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7fc47640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c7806bc10 0x7f5c78163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7fc47640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c780b4930 0x7f5c7806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7fc47640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c780b5140 con 0x7f5c78160030 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7fc47640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5c780b5420 con 0x7f5c780b4930 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7fc47640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5c78162b40 con 0x7f5c7806bc10 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7e1e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c780b4930 0x7f5c7806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7d1e2640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5c78160030 0x7f5c7806bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59828/0 (socket says 172.21.15.99:59828) 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7e1e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c780b4930 0x7f5c7806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57150/0 (socket says 172.21.15.99:57150) 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7d1e2640 1 -- 172.21.15.99:0/2713347828 learned_addr learned my addr 172.21.15.99:0/2713347828 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.650+0000 7f5c7d9e3640 1 --2- 172.21.15.99:0/2713347828 >> v2:172.21.15.196:3300/0 conn(0x7f5c7806bc10 0x7f5c78163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7e1e4640 1 -- 172.21.15.99:0/2713347828 >> v2:172.21.15.196:3300/0 conn(0x7f5c7806bc10 msgr2=0x7f5c78163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7e1e4640 1 --2- 172.21.15.99:0/2713347828 >> v2:172.21.15.196:3300/0 conn(0x7f5c7806bc10 0x7f5c78163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7e1e4640 1 -- 172.21.15.99:0/2713347828 >> v1:172.21.15.99:6789/0 conn(0x7f5c78160030 legacy=0x7f5c7806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7e1e4640 1 -- 172.21.15.99:0/2713347828 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c78162e20 con 0x7f5c780b4930 2024-09-10T11:30:55.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7c9e1640 1 -- 172.21.15.99:0/2713347828 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2607997511 0 0) 0x7f5c780b5140 con 0x7f5c78160030 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7e1e4640 1 --2- 172.21.15.99:0/2713347828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c780b4930 0x7f5c7806e5d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5c78073870 tx=0x7f5c7002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df872c99c8e9284a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7c9e1640 1 -- 172.21.15.99:0/2713347828 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007740 con 0x7f5c780b4930 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.651+0000 7f5c7c9e1640 1 -- 172.21.15.99:0/2713347828 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c700078e0 con 0x7f5c780b4930 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7c9e1640 1 -- 172.21.15.99:0/2713347828 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c70007cb0 con 0x7f5c780b4930 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2713347828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c780b4930 msgr2=0x7f5c7806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2713347828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c780b4930 0x7f5c7806e5d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5c78073870 tx=0x7f5c7002a5f0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2713347828 shutdown_connections 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2713347828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c780b4930 0x7f5c7806e5d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2713347828 >> v2:172.21.15.196:3300/0 conn(0x7f5c7806bc10 0x7f5c78163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2713347828 >> 172.21.15.99:0/2713347828 conn(0x7f5c780b3630 msgr2=0x7f5c780b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2713347828 shutdown_connections 2024-09-10T11:30:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.652+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2713347828 wait complete. 2024-09-10T11:30:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.653+0000 7f5c7fc47640 1 Processor -- start 2024-09-10T11:30:55.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.653+0000 7f5c7fc47640 1 -- start start 2024-09-10T11:30:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c780b4930 0x7f5c7815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c78160030 0x7f5c7815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780d1560 con 0x7f5c7806bc10 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780b4dc0 con 0x7f5c78160030 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7fc47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5c780d16e0 con 0x7f5c780b4930 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57162/0 (socket says 172.21.15.99:57162) 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 -- 172.21.15.99:0/2678084731 learned_addr learned my addr 172.21.15.99:0/2678084731 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 -- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c780b4930 msgr2=0x7f5c7815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c780b4930 0x7f5c7815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 -- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c78160030 msgr2=0x7f5c7815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c78160030 0x7f5c7815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.654+0000 7f5c7d9e3640 1 -- 172.21.15.99:0/2678084731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7815d0e0 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c7d9e3640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e870 tx=0x7f5c7400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017070 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7400c940 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7815d3b0 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400cd30 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.655+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c78159e40 con 0x7f5c7806bc10 2024-09-10T11:30:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.656+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c780b5140 con 0x7f5c7806bc10 2024-09-10T11:30:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.656+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c7401a070 con 0x7f5c7806bc10 2024-09-10T11:30:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.657+0000 7f5c6e7fc640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 0x7f5c5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.657+0000 7f5c7d1e2640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 0x7f5c5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.657+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5c7408ad10 con 0x7f5c7806bc10 2024-09-10T11:30:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.657+0000 7f5c7d1e2640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 0x7f5c5c06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007fd0 tx=0x7f5c68007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.661+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c740596c0 con 0x7f5c7806bc10 2024-09-10T11:30:55.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.864+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c7815fc60 con 0x7f5c7806bc10 2024-09-10T11:30:55.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.865+0000 7f5c6e7fc640 1 -- 172.21.15.99:0/2678084731 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7f5c74058fc0 con 0x7f5c7806bc10 2024-09-10T11:30:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:55.866 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:55.866 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 msgr2=0x7f5c5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 0x7f5c5c06e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007fd0 tx=0x7f5c68007960 comp rx=0 tx=0).stop 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 msgr2=0x7f5c781588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e870 tx=0x7f5c7400ed30 comp rx=0 tx=0).stop 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 shutdown_connections 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5c06be30 0x7f5c5c06e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c78160030 0x7f5c7815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c780b4930 0x7f5c7815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 --2- 172.21.15.99:0/2678084731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c7806bc10 0x7f5c781588c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 >> 172.21.15.99:0/2678084731 conn(0x7f5c780b3630 msgr2=0x7f5c780b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 shutdown_connections 2024-09-10T11:30:55.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.867+0000 7f5c7fc47640 1 -- 172.21.15.99:0/2678084731 wait complete. 2024-09-10T11:30:55.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:55.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.978+0000 7fce85731640 1 Processor -- start 2024-09-10T11:30:55.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.978+0000 7fce85731640 1 -- start start 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce85731640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce80148400 0x7fce8006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce85731640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 0x7fce80070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce85731640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce80162760 con 0x7fce80070ab0 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce85731640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fce80162a40 con 0x7fce80148400 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce85731640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce80162d20 con 0x7fce8006c150 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce7e7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 0x7fce80070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.979+0000 7fce7effd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce80148400 0x7fce8006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7f7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fce80070ab0 0x7fce80072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:34350/0 (socket says 172.21.15.99:34350) 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7effd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fce80148400 0x7fce8006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:33154/0 (socket says 172.21.15.99:33154) 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7f7fe640 1 -- 172.21.15.99:0/2354871518 learned_addr learned my addr 172.21.15.99:0/2354871518 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:55.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7e7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 0x7fce80070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:51386/0 (socket says 172.21.15.99:51386) 2024-09-10T11:30:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7effd640 1 -- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 msgr2=0x7fce80070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7effd640 1 --2- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 0x7fce80070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7effd640 1 -- 172.21.15.99:0/2354871518 >> v1:172.21.15.196:6789/0 conn(0x7fce80070ab0 legacy=0x7fce80072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7effd640 1 -- 172.21.15.99:0/2354871518 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce80163000 con 0x7fce80148400 2024-09-10T11:30:55.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.980+0000 7fce7dffb640 1 -- 172.21.15.99:0/2354871518 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1806917440 0 0) 0x7fce80162760 con 0x7fce80070ab0 2024-09-10T11:30:55.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce7effd640 1 --2- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80148400 0x7fce8006bc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce70009d90 tx=0x7fce7002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36db350a8782701a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce7dffb640 1 -- 172.21.15.99:0/2354871518 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70007740 con 0x7fce80148400 2024-09-10T11:30:55.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce7dffb640 1 -- 172.21.15.99:0/2354871518 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce700078e0 con 0x7fce80148400 2024-09-10T11:30:55.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce7dffb640 1 -- 172.21.15.99:0/2354871518 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70007cb0 con 0x7fce80148400 2024-09-10T11:30:55.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 -- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80148400 msgr2=0x7fce8006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 --2- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80148400 0x7fce8006bc10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fce70009d90 tx=0x7fce7002a660 comp rx=0 tx=0).stop 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 -- 172.21.15.99:0/2354871518 shutdown_connections 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 --2- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce8006c150 0x7fce80070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 --2- 172.21.15.99:0/2354871518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80148400 0x7fce8006bc10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 -- 172.21.15.99:0/2354871518 >> 172.21.15.99:0/2354871518 conn(0x7fce800d1900 msgr2=0x7fce800d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 -- 172.21.15.99:0/2354871518 shutdown_connections 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.981+0000 7fce85731640 1 -- 172.21.15.99:0/2354871518 wait complete. 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 Processor -- start 2024-09-10T11:30:55.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 -- start start 2024-09-10T11:30:55.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8006c150 0x7fce8015fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce80070ab0 0x7fce801396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d0700 con 0x7fce80148400 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce8006c570 con 0x7fce8006c150 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.983+0000 7fce85731640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fce800d0880 con 0x7fce80070ab0 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57172/0 (socket says 172.21.15.99:57172) 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 -- 172.21.15.99:0/3382472176 learned_addr learned my addr 172.21.15.99:0/3382472176 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7e7fc640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce80070ab0 0x7fce801396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7effd640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8006c150 0x7fce8015fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 -- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce80070ab0 msgr2=0x7fce801396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce80070ab0 0x7fce801396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 -- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8006c150 msgr2=0x7fce8015fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:55.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8006c150 0x7fce8015fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 -- 172.21.15.99:0/3382472176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce801612d0 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce7f7fe640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c480 tx=0x7fce7800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78018070 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce78007500 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce80160230 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce78004a20 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.984+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce801607f0 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.985+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce80070570 con 0x7fce80148400 2024-09-10T11:30:55.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.986+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce78004bc0 con 0x7fce80148400 2024-09-10T11:30:55.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.986+0000 7fce677fe640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 0x7fce6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:55.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.986+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce7808a0d0 con 0x7fce80148400 2024-09-10T11:30:55.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.986+0000 7fce7effd640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 0x7fce6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:55.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.987+0000 7fce7effd640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 0x7fce6c06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce70034000 tx=0x7fce70033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:55.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:55.990+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce78058b50 con 0x7fce80148400 2024-09-10T11:30:56.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.156+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce800b1b40 con 0x7fce80148400 2024-09-10T11:30:56.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.157+0000 7fce677fe640 1 -- 172.21.15.99:0/3382472176 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7fce78058450 con 0x7fce80148400 2024-09-10T11:30:56.158 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:56.158 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 msgr2=0x7fce6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 0x7fce6c06e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce70034000 tx=0x7fce70033040 comp rx=0 tx=0).stop 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 msgr2=0x7fce800b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fce7800c480 tx=0x7fce7800c940 comp rx=0 tx=0).stop 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 shutdown_connections 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fce6c06be30 0x7fce6c06e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce80148400 0x7fce800b6320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce80070ab0 0x7fce801396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 --2- 172.21.15.99:0/3382472176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8006c150 0x7fce8015fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 >> 172.21.15.99:0/3382472176 conn(0x7fce800d1900 msgr2=0x7fce800ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 shutdown_connections 2024-09-10T11:30:56.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.159+0000 7fce85731640 1 -- 172.21.15.99:0/3382472176 wait complete. 2024-09-10T11:30:56.171 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:30:56.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:56.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.268+0000 7fdea8d0c640 1 Processor -- start 2024-09-10T11:30:56.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.268+0000 7fdea8d0c640 1 -- start start 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea8d0c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdea40b3960 0x7fdea40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea8d0c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea40b3620 con 0x7fdea40b3160 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea8d0c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea406d960 con 0x7fdea4163580 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea8d0c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdea406dc40 con 0x7fdea40b3960 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea2d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdea40b3160 0x7fdea406cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59836/0 (socket says 172.21.15.99:59836) 2024-09-10T11:30:56.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea2d9d640 1 -- 172.21.15.99:0/4062186615 learned_addr learned my addr 172.21.15.99:0/4062186615 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325268000 0 0) 0x7fdea40b3620 con 0x7fdea40b3160 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/4062186615 >> v2:172.21.15.196:3300/0 conn(0x7fdea40b3960 0x7fdea40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.269+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde90003520 con 0x7fdea40b3160 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3842533322 0 0) 0x7fde90003520 con 0x7fdea40b3160 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 >> v2:172.21.15.196:3300/0 conn(0x7fdea40b3960 msgr2=0x7fdea40b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 --2- 172.21.15.99:0/4062186615 >> v2:172.21.15.196:3300/0 conn(0x7fdea40b3960 0x7fdea40b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 >> v1:172.21.15.196:6789/0 conn(0x7fdea4163580 legacy=0x7fdea40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea406df20 con 0x7fdea40b3160 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fde9c001670 con 0x7fdea40b3160 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 >> v1:172.21.15.99:6789/0 conn(0x7fdea40b3160 legacy=0x7fdea406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde900039b0 0x7fde90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 0x7fde90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde900052e0 0x7fde9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdea406dc40 con 0x7fde90004800 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde9000c1b0 con 0x7fde900052e0 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fde9000c3b0 con 0x7fde900039b0 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 0x7fde90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea259c640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde900039b0 0x7fde90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea2d9d640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde900052e0 0x7fde9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 -- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde900039b0 msgr2=0x7fde90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde900039b0 0x7fde90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 -- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde900052e0 msgr2=0x7fde9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde900052e0 0x7fde9000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea1d9b640 1 -- 172.21.15.99:0/4062186615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde90003520 con 0x7fde90004800 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea2d9d640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde900052e0 0x7fde9000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:56.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.270+0000 7fdea2d9d640 1 -- 172.21.15.99:0/4062186615 reap_dead start 2024-09-10T11:30:56.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 0x7fde90004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdea4071df0 tx=0x7fde90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e198582e2dc9ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c036070 con 0x7fde90004800 2024-09-10T11:30:56.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c031400 con 0x7fde90004800 2024-09-10T11:30:56.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea159a640 1 -- 172.21.15.99:0/4062186615 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c009690 con 0x7fde90004800 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea8d0c640 1 -- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 msgr2=0x7fde90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.271+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 0x7fde90004c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdea4071df0 tx=0x7fde90005830 comp rx=0 tx=0).stop 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.272+0000 7fdea8d0c640 1 -- 172.21.15.99:0/4062186615 shutdown_connections 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.272+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/4062186615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fde90004800 0x7fde90004c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.272+0000 7fdea8d0c640 1 -- 172.21.15.99:0/4062186615 >> 172.21.15.99:0/4062186615 conn(0x7fdea40b4570 msgr2=0x7fdea40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.272+0000 7fdea8d0c640 1 -- 172.21.15.99:0/4062186615 shutdown_connections 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.272+0000 7fdea8d0c640 1 -- 172.21.15.99:0/4062186615 wait complete. 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.273+0000 7fdea8d0c640 1 Processor -- start 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.273+0000 7fdea8d0c640 1 -- start start 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde90004800 0x7fdea40cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdea40b7be0 0x7fdea40c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdea40b5210 con 0x7fdea4154db0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdea40b5090 con 0x7fde90004800 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdea40b5390 con 0x7fdea40b7be0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57188/0 (socket says 172.21.15.99:57188) 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 -- 172.21.15.99:0/3718943143 learned_addr learned my addr 172.21.15.99:0/3718943143 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea259c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde90004800 0x7fdea40cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea2d9d640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdea40b7be0 0x7fdea40c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 -- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdea40b7be0 msgr2=0x7fdea40c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdea40b7be0 0x7fdea40c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 -- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde90004800 msgr2=0x7fdea40cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde90004800 0x7fdea40cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 -- 172.21.15.99:0/3718943143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea40cddd0 con 0x7fdea4154db0 2024-09-10T11:30:56.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea1d9b640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fde8c034040 tx=0x7fde8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c03f030 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8c037040 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea40ce0b0 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c036070 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.274+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea40ce670 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.276+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea40d1be0 con 0x7fdea4154db0 2024-09-10T11:30:56.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.276+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde8c042070 con 0x7fdea4154db0 2024-09-10T11:30:56.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.276+0000 7fde9affd640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 0x7fde8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.276+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fde8c0aa320 con 0x7fdea4154db0 2024-09-10T11:30:56.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.276+0000 7fdea259c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 0x7fde8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.277+0000 7fdea259c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 0x7fde8406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fde94000f80 tx=0x7fde94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.281+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fde8c078680 con 0x7fdea4154db0 2024-09-10T11:30:56.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.480+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdea4070aa0 con 0x7fdea4154db0 2024-09-10T11:30:56.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.480+0000 7fde9affd640 1 -- 172.21.15.99:0/3718943143 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7fde8c07c980 con 0x7fdea4154db0 2024-09-10T11:30:56.481 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:56.482 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:56.482 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:56.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 msgr2=0x7fde8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 0x7fde8406e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fde94000f80 tx=0x7fde94009040 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 msgr2=0x7fdea40b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fde8c034040 tx=0x7fde8c004730 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 shutdown_connections 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fde8406be30 0x7fde8406e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdea40b7be0 0x7fdea40c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdea4154db0 0x7fdea40b76a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 --2- 172.21.15.99:0/3718943143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde90004800 0x7fdea40cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 >> 172.21.15.99:0/3718943143 conn(0x7fdea40b4570 msgr2=0x7fdea406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 shutdown_connections 2024-09-10T11:30:56.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.482+0000 7fdea8d0c640 1 -- 172.21.15.99:0/3718943143 wait complete. 2024-09-10T11:30:56.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:30:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.591+0000 7fa3d42af640 1 Processor -- start 2024-09-10T11:30:56.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.591+0000 7fa3d42af640 1 -- start start 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d42af640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0b2460 0x7fa3cc166700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d42af640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3cc073d70 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d42af640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3cc0b5360 con 0x7fa3cc15ba90 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d42af640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa3cc0b55f0 con 0x7fa3cc0b2460 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d184a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa3cc0b02a0 0x7fa3cc0aee90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59846/0 (socket says 172.21.15.99:59846) 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d184a640 1 -- 172.21.15.99:0/2018814594 learned_addr learned my addr 172.21.15.99:0/2018814594 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d284c640 1 --2- 172.21.15.99:0/2018814594 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0b2460 0x7fa3cc166700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916164428 0 0) 0x7fa3cc073d70 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3b8003520 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3949084112 0 0) 0x7fa3cc0b5360 con 0x7fa3cc15ba90 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3cc073d70 con 0x7fa3cc15ba90 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3539731317 0 0) 0x7fa3b8003520 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0b2460 msgr2=0x7fa3cc166700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 --2- 172.21.15.99:0/2018814594 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0b2460 0x7fa3cc166700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 >> v1:172.21.15.196:6789/0 conn(0x7fa3cc15ba90 legacy=0x7fa3cc0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc0b0970 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa3bc001670 con 0x7fa3cc0b02a0 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d284c640 1 --2- 172.21.15.99:0/2018814594 >> v2:172.21.15.196:3300/0 conn(0x7fa3cc0b2460 0x7fa3cc166700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 >> v1:172.21.15.99:6789/0 conn(0x7fa3cc0b02a0 legacy=0x7fa3cc0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.592+0000 7fa3d1049640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8003b40 0x7fa3b8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d1049640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 0x7fa3b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d1049640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3b80051c0 0x7fa3b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0b55f0 con 0x7fa3b8004800 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b800c040 con 0x7fa3b8003b40 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b800c240 con 0x7fa3b80051c0 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 0x7fa3b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d204b640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8003b40 0x7fa3b8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d284c640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3b80051c0 0x7fa3b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 -- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3b80051c0 msgr2=0x7fa3b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3b80051c0 0x7fa3b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 -- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8003b40 msgr2=0x7fa3b8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:56.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8003b40 0x7fa3b8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d184a640 1 -- 172.21.15.99:0/2018814594 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b8003520 con 0x7fa3b8004800 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d284c640 1 -- 172.21.15.99:0/2018814594 reap_dead start 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.593+0000 7fa3d204b640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8003b40 0x7fa3b8003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d184a640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 0x7fa3b8004c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c190 tx=0x7fa3b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a906c577546307 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc007ce0 con 0x7fa3b8004800 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc005ad0 con 0x7fa3b8004800 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d1049640 1 -- 172.21.15.99:0/2018814594 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc007070 con 0x7fa3b8004800 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 -- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 msgr2=0x7fa3b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 0x7fa3b8004c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c190 tx=0x7fa3b8005830 comp rx=0 tx=0).stop 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 -- 172.21.15.99:0/2018814594 shutdown_connections 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 --2- 172.21.15.99:0/2018814594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3b8004800 0x7fa3b8004c80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 -- 172.21.15.99:0/2018814594 >> 172.21.15.99:0/2018814594 conn(0x7fa3cc0730c0 msgr2=0x7fa3cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 -- 172.21.15.99:0/2018814594 shutdown_connections 2024-09-10T11:30:56.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.594+0000 7fa3d42af640 1 -- 172.21.15.99:0/2018814594 wait complete. 2024-09-10T11:30:56.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.595+0000 7fa3d42af640 1 Processor -- start 2024-09-10T11:30:56.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 -- start start 2024-09-10T11:30:56.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8004800 0x7fa3cc141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 0x7fa3cc157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc15be70 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa3cc0b0f30 con 0x7fa3b8004800 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d42af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa3cc15bff0 con 0x7fa3cc1579a0 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d184a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d184a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57220/0 (socket says 172.21.15.99:57220) 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d184a640 1 -- 172.21.15.99:0/3188131090 learned_addr learned my addr 172.21.15.99:0/3188131090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.596+0000 7fa3d204b640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8004800 0x7fa3cc141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d284c640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 0x7fa3cc157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 -- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 msgr2=0x7fa3cc157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 0x7fa3cc157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 -- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8004800 msgr2=0x7fa3cc141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8004800 0x7fa3cc141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 -- 172.21.15.99:0/3188131090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3cc09ed60 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d284c640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 0x7fa3cc157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d184a640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02ad60 tx=0x7fa3bc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc037040 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc001470 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc001c10 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3cc09dcc0 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.597+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3cc09e280 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.598+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3cc0b1450 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.598+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3bc043030 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.599+0000 7fa3c2ffd640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 0x7fa3b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.599+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3bc0aa1f0 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.599+0000 7fa3d204b640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 0x7fa3b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.599+0000 7fa3d204b640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 0x7fa3b006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc051220 tx=0x7fa3c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.603+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3bc04a020 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.799+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3cc0af4a0 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.800+0000 7fa3c2ffd640 1 -- 172.21.15.99:0/3188131090 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5220 (secure 0 0 0) 0x7fa3bc0443c0 con 0x7fa3cc09b1a0 2024-09-10T11:30:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:56.801 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"btime":"2024-09-10T11:30:29:365941+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/906933744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":906933744},{"type":"v1","addr":"172.21.15.99:6836","nonce":906933744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3021743957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":3021743957},{"type":"v1","addr":"172.21.15.99:6837","nonce":3021743957}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1960318420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1960318420},{"type":"v1","addr":"172.21.15.196:6833","nonce":1960318420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:30:26.684199+0000","modified":"2024-09-10T11:30:28.690762+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4303},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4303":{"gid":4303,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/2032905820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2032905820},{"type":"v1","addr":"172.21.15.196:6835","nonce":2032905820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4303,"qdb_cluster":[4303]},"id":1}]} 2024-09-10T11:30:56.801 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 msgr2=0x7fa3b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 0x7fa3b006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc051220 tx=0x7fa3c8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 msgr2=0x7fa3cc09b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc02ad60 tx=0x7fa3bc0019c0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 shutdown_connections 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3b006be30 0x7fa3b006e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3cc1579a0 0x7fa3cc157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3cc09b1a0 0x7fa3cc09b620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 --2- 172.21.15.99:0/3188131090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3b8004800 0x7fa3cc141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 >> 172.21.15.99:0/3188131090 conn(0x7fa3cc0730c0 msgr2=0x7fa3cc06dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 shutdown_connections 2024-09-10T11:30:56.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.802+0000 7fa3d42af640 1 -- 172.21.15.99:0/3188131090 wait complete. 2024-09-10T11:30:56.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:30:56.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.911+0000 7fc3c8578640 1 Processor -- start 2024-09-10T11:30:56.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.911+0000 7fc3c8578640 1 -- start start 2024-09-10T11:30:56.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c8578640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c8578640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c006d8c0 con 0x7fc3c0162e50 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c8578640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c00b3870 con 0x7fc3c01485b0 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c8578640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c006dbc0 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c6b15640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc3c0162e50 0x7fc3c006e050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59862/0 (socket says 172.21.15.99:59862) 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c6b15640 1 -- 172.21.15.99:0/1427590586 learned_addr learned my addr 172.21.15.99:0/1427590586 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.912+0000 7fc3c5b13640 1 --2- 172.21.15.99:0/1427590586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181006359 0 0) 0x7fc3c006d8c0 con 0x7fc3c0162e50 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3a8003520 con 0x7fc3c0162e50 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 391157079 0 0) 0x7fc3c00b3870 con 0x7fc3c01485b0 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c006d8c0 con 0x7fc3c01485b0 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1427590586 >> v1:172.21.15.196:6789/0 conn(0x7fc3c01485b0 legacy=0x7fc3c00b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1427590586 >> v1:172.21.15.99:6789/0 conn(0x7fc3c0162e50 legacy=0x7fc3c006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.913+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1427590586 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0148990 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c5b13640 1 --2- 172.21.15.99:0/1427590586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0162910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097d0 tx=0x7fc3b402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bb9c89c1564225 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4037070 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b4032440 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c5312640 1 -- 172.21.15.99:0/1427590586 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4032810 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c8578640 1 -- 172.21.15.99:0/1427590586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 msgr2=0x7fc3c0162910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1427590586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0162910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40097d0 tx=0x7fc3b402a210 comp rx=0 tx=0).stop 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c8578640 1 -- 172.21.15.99:0/1427590586 shutdown_connections 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1427590586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0162910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.914+0000 7fc3c8578640 1 -- 172.21.15.99:0/1427590586 >> 172.21.15.99:0/1427590586 conn(0x7fc3c00b2a80 msgr2=0x7fc3c00b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.915+0000 7fc3c8578640 1 -- 172.21.15.99:0/1427590586 shutdown_connections 2024-09-10T11:30:56.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.915+0000 7fc3c8578640 1 -- 172.21.15.99:0/1427590586 wait complete. 2024-09-10T11:30:56.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 Processor -- start 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 -- start start 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 0x7fc3c00ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00d06a0 con 0x7fc3c01485b0 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c00d03c0 con 0x7fc3c0162e50 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.916+0000 7fc3c8578640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c006e760 con 0x7fc3c00cfbe0 2024-09-10T11:30:56.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57234/0 (socket says 172.21.15.99:57234) 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1274113446 learned_addr learned my addr 172.21.15.99:0/1274113446 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c6b15640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 0x7fc3c00ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c6314640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 msgr2=0x7fc3c0160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 msgr2=0x7fc3c00ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 0x7fc3c00ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 -- 172.21.15.99:0/1274113446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c01e8120 con 0x7fc3c01485b0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c6b15640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 0x7fc3c00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c5b13640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4035040 tx=0x7fc3b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4041020 con 0x7fc3c01485b0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b403d720 con 0x7fc3c01485b0 2024-09-10T11:30:56.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c01e8400 con 0x7fc3c01485b0 2024-09-10T11:30:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4037040 con 0x7fc3c01485b0 2024-09-10T11:30:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.917+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01e89c0 con 0x7fc3c01485b0 2024-09-10T11:30:56.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.918+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c0162910 con 0x7fc3c01485b0 2024-09-10T11:30:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.919+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3b4044020 con 0x7fc3c01485b0 2024-09-10T11:30:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.919+0000 7fc3beffd640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 0x7fc3ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.919+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3b40ab5a0 con 0x7fc3c01485b0 2024-09-10T11:30:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.919+0000 7fc3c6314640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 0x7fc3ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:56.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.920+0000 7fc3c6314640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 0x7fc3ac06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00b4800 tx=0x7fc3b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:56.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:56.924+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3b404b030 con 0x7fc3c01485b0 2024-09-10T11:30:57.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.091+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3c00d0950 con 0x7fc3c01485b0 2024-09-10T11:30:57.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.092+0000 7fc3beffd640 1 -- 172.21.15.99:0/1274113446 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16669 (secure 0 0 0) 0x7fc3b4079f60 con 0x7fc3c01485b0 2024-09-10T11:30:57.093 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:30:57.094 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":20,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:30:28.353917+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:30:24.498187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T11:30:25.666653+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:30:57.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 msgr2=0x7fc3ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 0x7fc3ac06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00b4800 tx=0x7fc3b8009040 comp rx=0 tx=0).stop 2024-09-10T11:30:57.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 msgr2=0x7fc3c01e91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4035040 tx=0x7fc3b40046c0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 shutdown_connections 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc3ac06be30 0x7fc3ac06e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3c0162e50 0x7fc3c00ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3c01485b0 0x7fc3c01e91c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 --2- 172.21.15.99:0/1274113446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3c00cfbe0 0x7fc3c0160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 >> 172.21.15.99:0/1274113446 conn(0x7fc3c00b2a80 msgr2=0x7fc3c0067c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 shutdown_connections 2024-09-10T11:30:57.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.094+0000 7fc3c8578640 1 -- 172.21.15.99:0/1274113446 wait complete. 2024-09-10T11:30:57.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:30:57.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.205+0000 7fb6938af640 1 Processor -- start 2024-09-10T11:30:57.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.205+0000 7fb6938af640 1 -- start start 2024-09-10T11:30:57.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb6938af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb68c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb6938af640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb68c06ee10 con 0x7fb68c15b2d0 2024-09-10T11:30:57.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb6938af640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb68c0afff0 con 0x7fb68c0afbb0 2024-09-10T11:30:57.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb6938af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c0cd9a0 con 0x7fb68c15aef0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb690e4a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb68c15b2d0 0x7fb68c0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59870/0 (socket says 172.21.15.99:59870) 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb690e4a640 1 -- 172.21.15.99:0/1519024421 learned_addr learned my addr 172.21.15.99:0/1519024421 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb69164b640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb68c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506506655 0 0) 0x7fb68c06ee10 con 0x7fb68c15b2d0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb678003520 con 0x7fb68c15b2d0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884190523 0 0) 0x7fb68c0afff0 con 0x7fb68c0afbb0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb68c06ee10 con 0x7fb68c0afbb0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3447657306 0 0) 0x7fb678003520 con 0x7fb68c15b2d0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 msgr2=0x7fb68c0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb68c0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 >> v1:172.21.15.196:6789/0 conn(0x7fb68c0afbb0 legacy=0x7fb68c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c0b0290 con 0x7fb68c15b2d0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.206+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fb67c001670 con 0x7fb68c15b2d0 2024-09-10T11:30:57.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 >> v1:172.21.15.99:6789/0 conn(0x7fb68c15b2d0 legacy=0x7fb68c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb678003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004140 0x7fb678004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb678004d40 0x7fb6780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c0cd9a0 con 0x7fb678004d40 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb67800bb00 con 0x7fb678004140 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb67800bd00 con 0x7fb68c15aef0 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb678004d40 0x7fb6780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb69164b640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb678003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb690e4a640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004140 0x7fb678004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 -- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 msgr2=0x7fb678003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c15aef0 0x7fb678003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 -- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004140 msgr2=0x7fb678004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:57.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004140 0x7fb678004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb691e4c640 1 -- 172.21.15.99:0/1519024421 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb678003520 con 0x7fb678004d40 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.207+0000 7fb690e4a640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004140 0x7fb678004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb691e4c640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb678004d40 0x7fb6780051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb684001fd0 tx=0x7fb678005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be7c4b80aaa59a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684001bf0 con 0x7fb678004d40 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb684003b70 con 0x7fb678004d40 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb683fff640 1 -- 172.21.15.99:0/1519024421 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6840050d0 con 0x7fb678004d40 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 -- 172.21.15.99:0/1519024421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb678004d40 msgr2=0x7fb6780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 --2- 172.21.15.99:0/1519024421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb678004d40 0x7fb6780051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb684001fd0 tx=0x7fb678005830 comp rx=0 tx=0).stop 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb691e4c640 1 -- 172.21.15.99:0/1519024421 reap_dead start 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 -- 172.21.15.99:0/1519024421 shutdown_connections 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 -- 172.21.15.99:0/1519024421 >> 172.21.15.99:0/1519024421 conn(0x7fb68c06bc10 msgr2=0x7fb68c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 -- 172.21.15.99:0/1519024421 shutdown_connections 2024-09-10T11:30:57.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.208+0000 7fb6938af640 1 -- 172.21.15.99:0/1519024421 wait complete. 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 Processor -- start 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 -- start start 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004d40 0x7fb68c1f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c0abc00 0x7fb68c0ac080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c06eb30 con 0x7fb68c0adda0 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c06e850 con 0x7fb678004d40 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb6938af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb68c071bb0 con 0x7fb68c0abc00 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb690e4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb690e4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57246/0 (socket says 172.21.15.99:57246) 2024-09-10T11:30:57.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.210+0000 7fb690e4a640 1 -- 172.21.15.99:0/1160968649 learned_addr learned my addr 172.21.15.99:0/1160968649 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb69164b640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004d40 0x7fb68c1f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb691e4c640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c0abc00 0x7fb68c0ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 -- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c0abc00 msgr2=0x7fb68c0ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c0abc00 0x7fb68c0ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 -- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004d40 msgr2=0x7fb68c1f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004d40 0x7fb68c1f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 -- 172.21.15.99:0/1160968649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c0ae8f0 con 0x7fb68c0adda0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb690e4a640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e760 tx=0x7fb67c00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00c660 con 0x7fb68c0adda0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb67c018070 con 0x7fb68c0adda0 2024-09-10T11:30:57.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c0aebd0 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00ca20 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.211+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c0af190 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.212+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb68c0b4510 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.213+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb67c00cbc0 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.213+0000 7fb681ffb640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 0x7fb67006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.213+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb67c08a7b0 con 0x7fb68c0adda0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.213+0000 7fb69164b640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 0x7fb67006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.214+0000 7fb69164b640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 0x7fb67006e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb688009bc0 tx=0x7fb688013040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.218+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb67c0591f0 con 0x7fb68c0adda0 2024-09-10T11:30:57.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.417+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb68c158610 con 0x7fb68c0adda0 2024-09-10T11:30:57.418 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:30:57.417+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:30:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.419+0000 7fb681ffb640 1 -- 172.21.15.99:0/1160968649 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7fb67c058af0 con 0x7fb68c0adda0 2024-09-10T11:30:57.419 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 msgr2=0x7fb67006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 0x7fb67006e320 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb688009bc0 tx=0x7fb688013040 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 msgr2=0x7fb68c0ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00e760 tx=0x7fb67c00eb60 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 shutdown_connections 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb67006be60 0x7fb67006e320 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb68c0abc00 0x7fb68c0ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb68c0adda0 0x7fb68c0ae220 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 --2- 172.21.15.99:0/1160968649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb678004d40 0x7fb68c1f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 >> 172.21.15.99:0/1160968649 conn(0x7fb68c06bc10 msgr2=0x7fb68c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:57.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 shutdown_connections 2024-09-10T11:30:57.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.422+0000 7fb6938af640 1 -- 172.21.15.99:0/1160968649 wait complete. 2024-09-10T11:30:57.425 INFO:tasks.ceph.mds.d.smithi196.stderr: -1967> 2024-09-10T11:30:32.330+0000 7f067ea6f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:30:57.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:30:57.455 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:30:57.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.533+0000 7f7efc58d640 1 Processor -- start 2024-09-10T11:30:57.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.533+0000 7f7efc58d640 1 -- start start 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.534+0000 7f7efc58d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ef415eb80 0x7f7ef40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.534+0000 7f7efc58d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 0x7f7ef40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.534+0000 7f7efc58d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef415ff70 con 0x7f7ef41626c0 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.534+0000 7f7efc58d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef40d1620 con 0x7f7ef415eb80 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.534+0000 7f7efc58d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef4162bc0 con 0x7f7ef40d5010 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7ef9b28640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7ef41626c0 0x7f7ef40d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59872/0 (socket says 172.21.15.99:59872) 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7ef9b28640 1 -- 172.21.15.99:0/3506885383 learned_addr learned my addr 172.21.15.99:0/3506885383 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:57.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efab2a640 1 --2- 172.21.15.99:0/3506885383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 0x7f7ef40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efa329640 1 --2- 172.21.15.99:0/3506885383 >> v2:172.21.15.196:3300/0 conn(0x7f7ef415eb80 0x7f7ef40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7ef9327640 1 -- 172.21.15.99:0/3506885383 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1027458851 0 0) 0x7f7ef415ff70 con 0x7f7ef41626c0 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7ef9327640 1 -- 172.21.15.99:0/3506885383 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7edc003520 con 0x7f7ef41626c0 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efab2a640 1 -- 172.21.15.99:0/3506885383 >> v2:172.21.15.196:3300/0 conn(0x7f7ef415eb80 msgr2=0x7f7ef40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efab2a640 1 --2- 172.21.15.99:0/3506885383 >> v2:172.21.15.196:3300/0 conn(0x7f7ef415eb80 0x7f7ef40d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efab2a640 1 -- 172.21.15.99:0/3506885383 >> v1:172.21.15.99:6789/0 conn(0x7f7ef41626c0 legacy=0x7f7ef40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.535+0000 7f7efab2a640 1 -- 172.21.15.99:0/3506885383 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4162ea0 con 0x7f7ef40d5010 2024-09-10T11:30:57.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efab2a640 1 --2- 172.21.15.99:0/3506885383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 0x7f7ef40d1b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009dc0 tx=0x7f7ee402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c5aaa6b3783530 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7ef9327640 1 -- 172.21.15.99:0/3506885383 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4007740 con 0x7f7ef40d5010 2024-09-10T11:30:57.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7ef9327640 1 -- 172.21.15.99:0/3506885383 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee40078e0 con 0x7f7ef40d5010 2024-09-10T11:30:57.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7ef9327640 1 -- 172.21.15.99:0/3506885383 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4007c90 con 0x7f7ef40d5010 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 -- 172.21.15.99:0/3506885383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 msgr2=0x7f7ef40d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 --2- 172.21.15.99:0/3506885383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 0x7f7ef40d1b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009dc0 tx=0x7f7ee402a5f0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 -- 172.21.15.99:0/3506885383 shutdown_connections 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 --2- 172.21.15.99:0/3506885383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef40d5010 0x7f7ef40d1b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 --2- 172.21.15.99:0/3506885383 >> v2:172.21.15.196:3300/0 conn(0x7f7ef415eb80 0x7f7ef40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.536+0000 7f7efc58d640 1 -- 172.21.15.99:0/3506885383 >> 172.21.15.99:0/3506885383 conn(0x7f7ef408e680 msgr2=0x7f7ef409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.537+0000 7f7efc58d640 1 -- 172.21.15.99:0/3506885383 shutdown_connections 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.537+0000 7f7efc58d640 1 -- 172.21.15.99:0/3506885383 wait complete. 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 Processor -- start 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 -- start start 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40d5010 0x7f7ef414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef41626c0 0x7f7ef4161670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40d2ef0 con 0x7f7ef415eb80 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4163460 con 0x7f7ef40d5010 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7efc58d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40d3070 con 0x7f7ef41626c0 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.538+0000 7f7ef9b28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57248/0 (socket says 172.21.15.99:57248) 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 -- 172.21.15.99:0/1927906009 learned_addr learned my addr 172.21.15.99:0/1927906009 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7efab2a640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef41626c0 0x7f7ef4161670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7efa329640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40d5010 0x7f7ef414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 -- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef41626c0 msgr2=0x7f7ef4161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef41626c0 0x7f7ef4161670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 -- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40d5010 msgr2=0x7f7ef414fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40d5010 0x7f7ef414fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:57.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 -- 172.21.15.99:0/1927906009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef40cc690 con 0x7f7ef415eb80 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef9b28640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800c390 tx=0x7f7ee800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8017070 con 0x7f7ef415eb80 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8005d00 con 0x7f7ef415eb80 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8007310 con 0x7f7ef415eb80 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef41f7e10 con 0x7f7ef415eb80 2024-09-10T11:30:57.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.539+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef40cc2d0 con 0x7f7ef415eb80 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.540+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef40d29b0 con 0x7f7ef415eb80 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.541+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ee801a070 con 0x7f7ef415eb80 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.541+0000 7f7ef2ffd640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 0x7f7ee006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.541+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f7ee808a160 con 0x7f7ef415eb80 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.541+0000 7f7efa329640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 0x7f7ee006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:57.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.542+0000 7f7efa329640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 0x7f7ee006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009aa0 tx=0x7f7eec009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:57.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.545+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ee80584f0 con 0x7f7ef415eb80 2024-09-10T11:30:57.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:57.747+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7ef41f7bc0 con 0x7f7ef415eb80 2024-09-10T11:30:57.761 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:30:57.764 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:30:58.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.441+0000 7f7ef2ffd640 1 -- 172.21.15.99:0/1927906009 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f7ee805c7f0 con 0x7f7ef415eb80 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 msgr2=0x7f7ee006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 0x7f7ee006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009aa0 tx=0x7f7eec009190 comp rx=0 tx=0).stop 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 msgr2=0x7f7ef40cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800c390 tx=0x7f7ee800c850 comp rx=0 tx=0).stop 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 shutdown_connections 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ee006be30 0x7f7ee006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ef41626c0 0x7f7ef4161670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ef415eb80 0x7f7ef40cbd90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 --2- 172.21.15.99:0/1927906009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ef40d5010 0x7f7ef414fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 >> 172.21.15.99:0/1927906009 conn(0x7f7ef408e680 msgr2=0x7f7ef40d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:58.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 shutdown_connections 2024-09-10T11:30:58.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.444+0000 7f7efc58d640 1 -- 172.21.15.99:0/1927906009 wait complete. 2024-09-10T11:30:58.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:30:58.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.557+0000 7f93ec4ac640 1 Processor -- start 2024-09-10T11:30:58.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.558+0000 7f93ec4ac640 1 -- start start 2024-09-10T11:30:58.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f93e40d1200 0x7f93e4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e406e5d0 0x7f93e406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93e415c060 0x7f93e415c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f93e406ef50 con 0x7f93e415c060 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f93e4152e60 con 0x7f93e40d1200 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ec4ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93e40d15e0 con 0x7f93e406e5d0 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93e415c060 0x7f93e415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f93e415c060 0x7f93e415c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57250/0 (socket says 172.21.15.99:57250) 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 -- 172.21.15.99:0/3657617554 learned_addr learned my addr 172.21.15.99:0/3657617554 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93e9a47640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e406e5d0 0x7f93e406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 -- 172.21.15.99:0/3657617554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e406e5d0 msgr2=0x7f93e406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93ea248640 1 --2- 172.21.15.99:0/3657617554 >> v2:172.21.15.196:3300/0 conn(0x7f93e40d1200 0x7f93e4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e406e5d0 0x7f93e406ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 -- 172.21.15.99:0/3657617554 >> v2:172.21.15.196:3300/0 conn(0x7f93e40d1200 msgr2=0x7f93e4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:58.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 --2- 172.21.15.99:0/3657617554 >> v2:172.21.15.196:3300/0 conn(0x7f93e40d1200 0x7f93e4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.559+0000 7f93eaa49640 1 -- 172.21.15.99:0/3657617554 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e4072680 con 0x7f93e415c060 2024-09-10T11:30:58.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.560+0000 7f93eaa49640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e415c060 0x7f93e415c480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93e40d1850 tx=0x7f93dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd52095434b61cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.560+0000 7f93e9246640 1 -- 172.21.15.99:0/3657617554 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc036070 con 0x7f93e415c060 2024-09-10T11:30:58.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.560+0000 7f93e9246640 1 -- 172.21.15.99:0/3657617554 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc031600 con 0x7f93e415c060 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.560+0000 7f93e9246640 1 -- 172.21.15.99:0/3657617554 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0319d0 con 0x7f93e415c060 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 -- 172.21.15.99:0/3657617554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e415c060 msgr2=0x7f93e415c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e415c060 0x7f93e415c480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f93e40d1850 tx=0x7f93dc02a610 comp rx=0 tx=0).stop 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 -- 172.21.15.99:0/3657617554 shutdown_connections 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e415c060 0x7f93e415c480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/3657617554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e406e5d0 0x7f93e406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/3657617554 >> v2:172.21.15.196:3300/0 conn(0x7f93e40d1200 0x7f93e4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 -- 172.21.15.99:0/3657617554 >> 172.21.15.99:0/3657617554 conn(0x7f93e406bc10 msgr2=0x7f93e406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 -- 172.21.15.99:0/3657617554 shutdown_connections 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.561+0000 7f93ec4ac640 1 -- 172.21.15.99:0/3657617554 wait complete. 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.562+0000 7f93ec4ac640 1 Processor -- start 2024-09-10T11:30:58.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.562+0000 7f93ec4ac640 1 -- start start 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e40d1200 0x7f93e414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 0x7f93e41f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4072f20 con 0x7f93e406e5d0 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4072c40 con 0x7f93e40d1200 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f93e415cd90 con 0x7f93e415c060 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57256/0 (socket says 172.21.15.99:57256) 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 -- 172.21.15.99:0/1814985975 learned_addr learned my addr 172.21.15.99:0/1814985975 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93e9a47640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e40d1200 0x7f93e414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93eaa49640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 0x7f93e41f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 -- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 msgr2=0x7f93e41f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 0x7f93e41f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 -- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e40d1200 msgr2=0x7f93e414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e40d1200 0x7f93e414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 -- 172.21.15.99:0/1814985975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e40c9bc0 con 0x7f93e406e5d0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93eaa49640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 0x7f93e41f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ea248640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c7f0 tx=0x7f93e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0015070 con 0x7f93e406e5d0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e0004220 con 0x7f93e406e5d0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e41f4e60 con 0x7f93e406e5d0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0007620 con 0x7f93e406e5d0 2024-09-10T11:30:58.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.563+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e41f5420 con 0x7f93e406e5d0 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.564+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e406ea10 con 0x7f93e406e5d0 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.565+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f93e00077c0 con 0x7f93e406e5d0 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.565+0000 7f93d6ffd640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.565+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f93e008aeb0 con 0x7f93e406e5d0 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.565+0000 7f93e9a47640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:58.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.566+0000 7f93e9a47640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 0x7f93c806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd0c0 tx=0x7f93d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:58.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.569+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f93e00591b0 con 0x7f93e406e5d0 2024-09-10T11:30:58.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:58.735+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93e4073550 con 0x7f93e406e5d0 2024-09-10T11:30:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.448+0000 7f93d6ffd640 1 -- 172.21.15.99:0/1814985975 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f93e005d4b0 con 0x7f93e406e5d0 2024-09-10T11:30:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:30:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 msgr2=0x7f93c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 0x7f93c806e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93e40cd0c0 tx=0x7f93d8008040 comp rx=0 tx=0).stop 2024-09-10T11:30:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 msgr2=0x7f93e414d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c7f0 tx=0x7f93e000ccb0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 shutdown_connections 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f93c806be30 0x7f93c806e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e415c060 0x7f93e41f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e40d1200 0x7f93e414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 --2- 172.21.15.99:0/1814985975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93e406e5d0 0x7f93e414d7c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.450+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 >> 172.21.15.99:0/1814985975 conn(0x7f93e406bc10 msgr2=0x7f93e415d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.451+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 shutdown_connections 2024-09-10T11:30:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.451+0000 7f93ec4ac640 1 -- 172.21.15.99:0/1814985975 wait complete. 2024-09-10T11:30:59.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:30:59.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.566+0000 7f8dd3282640 1 Processor -- start 2024-09-10T11:30:59.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.566+0000 7f8dd3282640 1 -- start start 2024-09-10T11:30:59.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd3282640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8dcc073440 0x7f8dcc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:59.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd3282640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dcc152e60 con 0x7f8dcc0d1200 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd3282640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dcc0d15e0 con 0x7f8dcc06ece0 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd3282640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8dcc072680 con 0x7f8dcc073440 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd181f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8dcc073440 0x7f8dcc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd101e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8dcc0d1200 0x7f8dcc06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59878/0 (socket says 172.21.15.99:59878) 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd101e640 1 -- 172.21.15.99:0/1896253746 learned_addr learned my addr 172.21.15.99:0/1896253746 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd181f640 1 -- 172.21.15.99:0/1896253746 >> v1:172.21.15.196:6789/0 conn(0x7f8dcc06ece0 legacy=0x7f8dcc15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dd181f640 1 -- 172.21.15.99:0/1896253746 >> v1:172.21.15.99:6789/0 conn(0x7f8dcc0d1200 legacy=0x7f8dcc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.567+0000 7f8dc3fff640 1 -- 172.21.15.99:0/1896253746 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602664941 0 0) 0x7f8dcc152e60 con 0x7f8dcc0d1200 2024-09-10T11:30:59.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.568+0000 7f8dd181f640 1 -- 172.21.15.99:0/1896253746 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc072960 con 0x7f8dcc073440 2024-09-10T11:30:59.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.568+0000 7f8dd181f640 1 --2- 172.21.15.99:0/1896253746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc073440 0x7f8dcc15c770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009d90 tx=0x7f8dc402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9fad2b71c92550 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:59.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.568+0000 7f8dc3fff640 1 -- 172.21.15.99:0/1896253746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4031600 con 0x7f8dcc073440 2024-09-10T11:30:59.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.568+0000 7f8dc3fff640 1 -- 172.21.15.99:0/1896253746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc40317a0 con 0x7f8dcc073440 2024-09-10T11:30:59.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.568+0000 7f8dc3fff640 1 -- 172.21.15.99:0/1896253746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4031b70 con 0x7f8dcc073440 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 -- 172.21.15.99:0/1896253746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc073440 msgr2=0x7f8dcc15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 --2- 172.21.15.99:0/1896253746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc073440 0x7f8dcc15c770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009d90 tx=0x7f8dc402a610 comp rx=0 tx=0).stop 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 -- 172.21.15.99:0/1896253746 shutdown_connections 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 --2- 172.21.15.99:0/1896253746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc073440 0x7f8dcc15c770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 -- 172.21.15.99:0/1896253746 >> 172.21.15.99:0/1896253746 conn(0x7f8dcc06bc10 msgr2=0x7f8dcc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 -- 172.21.15.99:0/1896253746 shutdown_connections 2024-09-10T11:30:59.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.569+0000 7f8dd3282640 1 -- 172.21.15.99:0/1896253746 wait complete. 2024-09-10T11:30:59.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.570+0000 7f8dd3282640 1 Processor -- start 2024-09-10T11:30:59.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.570+0000 7f8dd3282640 1 -- start start 2024-09-10T11:30:59.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8dcc073440 0x7f8dcc1f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dcc0d1200 0x7f8dcc1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc15cd90 con 0x7f8dcc06ece0 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8dcc072f20 con 0x7f8dcc0d1200 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd3282640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8dcc159b50 con 0x7f8dcc073440 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57284/0 (socket says 172.21.15.99:57284) 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 -- 172.21.15.99:0/330316232 learned_addr learned my addr 172.21.15.99:0/330316232 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd081d640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8dcc073440 0x7f8dcc1f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd181f640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dcc0d1200 0x7f8dcc1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 -- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8dcc073440 msgr2=0x7f8dcc1f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8dcc073440 0x7f8dcc1f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 -- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dcc0d1200 msgr2=0x7f8dcc1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dcc0d1200 0x7f8dcc1f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 -- 172.21.15.99:0/330316232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dcc156700 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.571+0000 7f8dd101e640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c460 tx=0x7f8dc800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.572+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8018070 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.572+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dc8007500 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.572+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dcc157690 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.572+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8004a20 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.572+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dcc157c50 con 0x7f8dcc06ece0 2024-09-10T11:30:59.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.573+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dcc160000 con 0x7f8dcc06ece0 2024-09-10T11:30:59.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.573+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8dc801b070 con 0x7f8dcc06ece0 2024-09-10T11:30:59.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.574+0000 7f8dc1ffb640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:59.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.574+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7f8dc8089f20 con 0x7f8dcc06ece0 2024-09-10T11:30:59.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.574+0000 7f8dd081d640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:59.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.574+0000 7f8dd081d640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 0x7f8db006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc1599d0 tx=0x7f8dbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:59.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.578+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8dcc160000 con 0x7f8dcc06ece0 2024-09-10T11:30:59.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:30:59.743+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8dcc159ed0 con 0x7f8dcc06ece0 2024-09-10T11:31:00.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.449+0000 7f8dc1ffb640 1 -- 172.21.15.99:0/330316232 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f8dc8058ac0 con 0x7f8dcc06ece0 2024-09-10T11:31:00.450 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 msgr2=0x7f8db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 0x7f8db006e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc1599d0 tx=0x7f8dbc009040 comp rx=0 tx=0).stop 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 msgr2=0x7f8dcc0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800c460 tx=0x7f8dc800c920 comp rx=0 tx=0).stop 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 shutdown_connections 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8db006be30 0x7f8db006e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8dcc0d1200 0x7f8dcc1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8dcc073440 0x7f8dcc1f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:00.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 --2- 172.21.15.99:0/330316232 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8dcc06ece0 0x7f8dcc0ceca0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:00.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 >> 172.21.15.99:0/330316232 conn(0x7f8dcc06bc10 msgr2=0x7f8dcc15bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:00.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 shutdown_connections 2024-09-10T11:31:00.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:00.453+0000 7f8dd3282640 1 -- 172.21.15.99:0/330316232 wait complete. 2024-09-10T11:31:00.470 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:31:00.470 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:31:00.470 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:31:00.470 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:00.470 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:31:00.471 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:31:00.471 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:31:00.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:00.471 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:31:00.471 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:31:00.471 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:31:00.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:00.472 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:31:00.472 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:31:00.472 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:31:00.472 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:00.484 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:31:00.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:31:00.486 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:31:00.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:31:00.487 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:31:00.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:31:00.526 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:31:00.526 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:00.528 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:31:00.528 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:00.529 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:31:00.529 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:00.601 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:31:00.608 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:31:00.608 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:31:03.559 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-10T11:31:06.484 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:31:06.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:31:06.487 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:31:06.487 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:06.488 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:31:06.571 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:31:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.598+0000 7f14f892c640 1 Processor -- start 2024-09-10T11:31:06.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.598+0000 7f14f892c640 1 -- start start 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f892c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 0x7f14f406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f892c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f14f406ee30 0x7f14f406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f892c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f40ceb10 con 0x7f14f415c7b0 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f892c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f14f406f2a0 con 0x7f14f406ee30 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f892c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14f40b3840 con 0x7f14f40d1180 2024-09-10T11:31:06.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f2d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f14f406ee30 0x7f14f406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f259c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f14f415c7b0 0x7f14f40d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54092/0 (socket says 172.21.15.99:54092) 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f259c640 1 -- 172.21.15.99:0/2043694754 learned_addr learned my addr 172.21.15.99:0/2043694754 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f2d9d640 1 -- 172.21.15.99:0/2043694754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 msgr2=0x7f14f406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f1d9b640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 0x7f14f406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f2d9d640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 0x7f14f406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f2d9d640 1 -- 172.21.15.99:0/2043694754 >> v1:172.21.15.99:6789/0 conn(0x7f14f415c7b0 legacy=0x7f14f40d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f159a640 1 -- 172.21.15.99:0/2043694754 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560399066 0 0) 0x7f14e4001110 con 0x7f14f415c7b0 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f2d9d640 1 -- 172.21.15.99:0/2043694754 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f40b3a90 con 0x7f14f406ee30 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.599+0000 7f14f1d9b640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 0x7f14f406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.600+0000 7f14f2d9d640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f406ee30 0x7f14f406d4c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009d90 tx=0x7f14ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559483993c46f541 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.600+0000 7f14f159a640 1 -- 172.21.15.99:0/2043694754 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec007740 con 0x7f14f406ee30 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.600+0000 7f14f159a640 1 -- 172.21.15.99:0/2043694754 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ec0078e0 con 0x7f14f406ee30 2024-09-10T11:31:06.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.600+0000 7f14f159a640 1 -- 172.21.15.99:0/2043694754 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec007cb0 con 0x7f14f406ee30 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 -- 172.21.15.99:0/2043694754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f406ee30 msgr2=0x7f14f406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f406ee30 0x7f14f406d4c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009d90 tx=0x7f14ec02a660 comp rx=0 tx=0).stop 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 -- 172.21.15.99:0/2043694754 shutdown_connections 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f406ee30 0x7f14f406d4c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 --2- 172.21.15.99:0/2043694754 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f40d1180 0x7f14f406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 -- 172.21.15.99:0/2043694754 >> 172.21.15.99:0/2043694754 conn(0x7f14f406bc10 msgr2=0x7f14f406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 -- 172.21.15.99:0/2043694754 shutdown_connections 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.601+0000 7f14f892c640 1 -- 172.21.15.99:0/2043694754 wait complete. 2024-09-10T11:31:06.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.602+0000 7f14f892c640 1 Processor -- start 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.602+0000 7f14f892c640 1 -- start start 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f406ee30 0x7f14f4160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f415c7b0 0x7f14f41f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14f40b4330 con 0x7f14f40d1180 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14f40b4050 con 0x7f14f415c7b0 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f892c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14f40b4610 con 0x7f14f406ee30 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47532/0 (socket says 172.21.15.99:47532) 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 -- 172.21.15.99:0/1664797865 learned_addr learned my addr 172.21.15.99:0/1664797865 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 -- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f406ee30 msgr2=0x7f14f4160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:06.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f406ee30 0x7f14f4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 -- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f415c7b0 msgr2=0x7f14f41f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f415c7b0 0x7f14f41f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.603+0000 7f14f1d9b640 1 -- 172.21.15.99:0/1664797865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f415eca0 con 0x7f14f40d1180 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14f1d9b640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c7f0 tx=0x7f14e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8015070 con 0x7f14f40d1180 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e8007a30 con 0x7f14f40d1180 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f415ef80 con 0x7f14f40d1180 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8007e00 con 0x7f14f40d1180 2024-09-10T11:31:06.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.604+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f415f540 con 0x7f14f40d1180 2024-09-10T11:31:06.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.605+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f415cc70 con 0x7f14f40d1180 2024-09-10T11:31:06.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.605+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14e8018070 con 0x7f14f40d1180 2024-09-10T11:31:06.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.606+0000 7f14e2ffd640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:06.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.606+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f14e808ab50 con 0x7f14f40d1180 2024-09-10T11:31:06.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.606+0000 7f14f259c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:06.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.606+0000 7f14f259c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14e4007fd0 tx=0x7f14e4007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:06.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.610+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14e8059800 con 0x7f14f40d1180 2024-09-10T11:31:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:06.786+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f14f415b9c0 con 0x7f14f40d1180 2024-09-10T11:31:08.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.483+0000 7f14e2ffd640 1 -- 172.21.15.99:0/1664797865 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f14e8059100 con 0x7f14f40d1180 2024-09-10T11:31:08.485 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:31:08.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 msgr2=0x7f14d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f14e4007fd0 tx=0x7f14e4007960 comp rx=0 tx=0).stop 2024-09-10T11:31:08.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 msgr2=0x7f14f41f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c7f0 tx=0x7f14e800cbf0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 shutdown_connections 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f415c7b0 0x7f14f41f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f40d1180 0x7f14f41f5fd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 --2- 172.21.15.99:0/1664797865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f406ee30 0x7f14f4160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 >> 172.21.15.99:0/1664797865 conn(0x7f14f406bc10 msgr2=0x7f14f40b24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 shutdown_connections 2024-09-10T11:31:08.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.486+0000 7f14f892c640 1 -- 172.21.15.99:0/1664797865 wait complete. 2024-09-10T11:31:08.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:31:08.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.600+0000 7fc793cc9640 1 Processor -- start 2024-09-10T11:31:08.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 -- start start 2024-09-10T11:31:08.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc78c160030 0x7fc78c0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc78c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78c0ce430 con 0x7fc78c0d1760 2024-09-10T11:31:08.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc78c0ce200 con 0x7fc78c160030 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc793cc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c0ce710 con 0x7fc78c0d0130 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc791264640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc78c0d1760 0x7fc78c0cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54102/0 (socket says 172.21.15.99:54102) 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.601+0000 7fc791264640 1 -- 172.21.15.99:0/2602833883 learned_addr learned my addr 172.21.15.99:0/2602833883 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc791a65640 1 --2- 172.21.15.99:0/2602833883 >> v2:172.21.15.196:3300/0 conn(0x7fc78c160030 0x7fc78c0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2887274118 0 0) 0x7fc78c0ce430 con 0x7fc78c0d1760 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc792266640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc78c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc778003520 con 0x7fc78c0d1760 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3207380480 0 0) 0x7fc778003520 con 0x7fc78c0d1760 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 msgr2=0x7fc78c0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc78c0cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 >> v2:172.21.15.196:3300/0 conn(0x7fc78c160030 msgr2=0x7fc78c0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 --2- 172.21.15.99:0/2602833883 >> v2:172.21.15.196:3300/0 conn(0x7fc78c160030 0x7fc78c0b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c0d1240 con 0x7fc78c0d1760 2024-09-10T11:31:08.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fc788001690 con 0x7fc78c0d1760 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 >> v1:172.21.15.99:6789/0 conn(0x7fc78c0d1760 legacy=0x7fc78c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c0ce710 con 0x7fc778004800 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc77800b9c0 con 0x7fc7780039b0 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc77800bb40 con 0x7fc78c0d0130 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc791264640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc778004800 0x7fc778004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc791a65640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.602+0000 7fc792266640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 -- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 msgr2=0x7fc7780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc78c0d0130 0x7fc7780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 -- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7780039b0 msgr2=0x7fc778003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:08.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7780039b0 0x7fc778003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 -- 172.21.15.99:0/2602833883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc778003520 con 0x7fc778004800 2024-09-10T11:31:08.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc791264640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc778004800 0x7fc778004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc778005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ed7785b2f6325 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78802e070 con 0x7fc778004800 2024-09-10T11:31:08.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788005af0 con 0x7fc778004800 2024-09-10T11:31:08.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.603+0000 7fc790a63640 1 -- 172.21.15.99:0/2602833883 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788007d70 con 0x7fc778004800 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 -- 172.21.15.99:0/2602833883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc778004800 msgr2=0x7fc778004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 --2- 172.21.15.99:0/2602833883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc778004800 0x7fc778004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc778005830 comp rx=0 tx=0).stop 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc792266640 1 -- 172.21.15.99:0/2602833883 reap_dead start 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 -- 172.21.15.99:0/2602833883 shutdown_connections 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 -- 172.21.15.99:0/2602833883 >> 172.21.15.99:0/2602833883 conn(0x7fc78c0b4650 msgr2=0x7fc78c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 -- 172.21.15.99:0/2602833883 shutdown_connections 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.604+0000 7fc793cc9640 1 -- 172.21.15.99:0/2602833883 wait complete. 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.605+0000 7fc793cc9640 1 Processor -- start 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.605+0000 7fc793cc9640 1 -- start start 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc778004800 0x7fc78c1f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc78c0cc140 0x7fc78c0cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c160590 con 0x7fc78c0c9b90 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc78c160410 con 0x7fc78c0cc140 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc793cc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc78c160710 con 0x7fc778004800 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc791264640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc791264640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47540/0 (socket says 172.21.15.99:47540) 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc791264640 1 -- 172.21.15.99:0/1111426846 learned_addr learned my addr 172.21.15.99:0/1111426846 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc791a65640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc778004800 0x7fc78c1f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.606+0000 7fc792266640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc78c0cc140 0x7fc78c0cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 -- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc778004800 msgr2=0x7fc78c1f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc778004800 0x7fc78c1f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 -- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc78c0cc140 msgr2=0x7fc78c0cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc78c0cc140 0x7fc78c0cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 -- 172.21.15.99:0/1111426846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78c1f5500 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc791264640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc788034040 tx=0x7fc788001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788042070 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc788007480 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc78c15e5c0 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788007d70 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.607+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78c0ca3a0 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.608+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78c0cbe40 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.608+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc788045070 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.609+0000 7fc77e7fc640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 0x7fc77006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.609+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fc7880ab2a0 con 0x7fc78c0c9b90 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.609+0000 7fc791a65640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 0x7fc77006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.609+0000 7fc791a65640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 0x7fc77006e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc78c1f8bf0 tx=0x7fc780009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.617+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc78c15e5c0 con 0x7fc78c0c9b90 2024-09-10T11:31:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.847+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc78c0b38f0 con 0x7fc78c0c9b90 2024-09-10T11:31:08.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.847+0000 7fc77e7fc640 1 -- 172.21.15.99:0/1111426846 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fc78c1f5500 con 0x7fc78c0c9b90 2024-09-10T11:31:08.850 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:08.850 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:08.850 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:31:08.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.851 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:31:08.853 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:31:08.855 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.856 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.857 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.858 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.859 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:31:08.860 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.861 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:08.862 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mds", 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBmwo7oMhAAPBkhV8/KbAOVpJ79lFeC/Q==", 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.864 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBmb5boMhAAO8km34BTXdcv/YTOJXSmPg==", 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-osd", 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBm/pzoMhAA6+Y9QaDVhaS1bzC66cAj1Q==", 2024-09-10T11:31:08.865 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBmd6PoMhAAwCLGyg8yR7FFqaxEZBqiNQ==", 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBmjavoMhAAdDKef2LpZjFLM8JQDltx0A==", 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDCLeBmWbLoMhAAFgNmCE6sR1p7g4W/nPuC/g==", 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.868 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:08.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:08.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:08.870 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:08.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 msgr2=0x7fc77006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 0x7fc77006e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc78c1f8bf0 tx=0x7fc780009190 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 msgr2=0x7fc78c15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc788034040 tx=0x7fc788001a60 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 shutdown_connections 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc77006be30 0x7fc77006e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc78c0cc140 0x7fc78c0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc78c0c9b90 0x7fc78c15e080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 --2- 172.21.15.99:0/1111426846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc778004800 0x7fc78c1f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 >> 172.21.15.99:0/1111426846 conn(0x7fc78c0b4650 msgr2=0x7fc78c0b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.849+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 shutdown_connections 2024-09-10T11:31:08.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.850+0000 7fc793cc9640 1 -- 172.21.15.99:0/1111426846 wait complete. 2024-09-10T11:31:08.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T11:31:08.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.975+0000 7f66bbaa4640 1 Processor -- start 2024-09-10T11:31:08.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.975+0000 7f66bbaa4640 1 -- start start 2024-09-10T11:31:08.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66bbaa4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66bbaa4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66b41626c0 0x7f66b40d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66bbaa4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b40d52d0 con 0x7f66b40d4e40 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66bbaa4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f66b415ff70 con 0x7f66b41626c0 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66bbaa4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66b40d1620 con 0x7f66b415eb80 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66b9840640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66b903f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66b41626c0 0x7f66b40d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66b9840640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:57360/0 (socket says 172.21.15.99:57360) 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.976+0000 7f66b9840640 1 -- 172.21.15.99:0/1962671052 learned_addr learned my addr 172.21.15.99:0/1962671052 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.977+0000 7f66b9840640 1 -- 172.21.15.99:0/1962671052 >> v2:172.21.15.196:3300/0 conn(0x7f66b41626c0 msgr2=0x7f66b40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:08.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.977+0000 7f66b9840640 1 --2- 172.21.15.99:0/1962671052 >> v2:172.21.15.196:3300/0 conn(0x7f66b41626c0 0x7f66b40d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.977+0000 7f66b9840640 1 -- 172.21.15.99:0/1962671052 >> v1:172.21.15.196:6789/0 conn(0x7f66b40d4e40 legacy=0x7f66b40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.977+0000 7f66b9840640 1 -- 172.21.15.99:0/1962671052 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b4162bc0 con 0x7f66b415eb80 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.977+0000 7f66b9840640 1 --2- 172.21.15.99:0/1962671052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009870 tx=0x7f66ac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90970a19160a24c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66b883e640 1 -- 172.21.15.99:0/1962671052 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac037070 con 0x7f66b415eb80 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66b883e640 1 -- 172.21.15.99:0/1962671052 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac032440 con 0x7f66b415eb80 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66b883e640 1 -- 172.21.15.99:0/1962671052 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac032810 con 0x7f66b415eb80 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 -- 172.21.15.99:0/1962671052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 msgr2=0x7f66b40d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/1962671052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009870 tx=0x7f66ac02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 -- 172.21.15.99:0/1962671052 shutdown_connections 2024-09-10T11:31:08.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/1962671052 >> v2:172.21.15.196:3300/0 conn(0x7f66b41626c0 0x7f66b40d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/1962671052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b415eb80 0x7f66b40d29b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.978+0000 7f66bbaa4640 1 -- 172.21.15.99:0/1962671052 >> 172.21.15.99:0/1962671052 conn(0x7f66b408e680 msgr2=0x7f66b409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:08.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.979+0000 7f66bbaa4640 1 -- 172.21.15.99:0/1962671052 shutdown_connections 2024-09-10T11:31:08.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.979+0000 7f66bbaa4640 1 -- 172.21.15.99:0/1962671052 wait complete. 2024-09-10T11:31:08.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.980+0000 7f66bbaa4640 1 Processor -- start 2024-09-10T11:31:08.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.980+0000 7f66bbaa4640 1 -- start start 2024-09-10T11:31:08.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.980+0000 7f66bbaa4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 0x7f66b4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b41626c0 0x7f66b415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4163460 con 0x7f66b40d4e40 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66b4163180 con 0x7f66b415eb80 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66b40d2ef0 con 0x7f66b41626c0 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47550/0 (socket says 172.21.15.99:47550) 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b903f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 0x7f66b4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66ba041640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b41626c0 0x7f66b415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 -- 172.21.15.99:0/2310032884 learned_addr learned my addr 172.21.15.99:0/2310032884 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 -- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b41626c0 msgr2=0x7f66b415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b41626c0 0x7f66b415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 -- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 msgr2=0x7f66b4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 0x7f66b4159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 -- 172.21.15.99:0/2310032884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b40b2420 con 0x7f66b40d4e40 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b903f640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 0x7f66b4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b9840640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f66ac000c00 tx=0x7f66ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac037040 con 0x7f66b40d4e40 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac042070 con 0x7f66b40d4e40 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.981+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b40b2700 con 0x7f66b40d4e40 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.982+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac030880 con 0x7f66b40d4e40 2024-09-10T11:31:08.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.982+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b41e9f90 con 0x7f66b40d4e40 2024-09-10T11:31:08.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.983+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b40d29b0 con 0x7f66b40d4e40 2024-09-10T11:31:08.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.983+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66ac02ad10 con 0x7f66b40d4e40 2024-09-10T11:31:08.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.983+0000 7f66b27fc640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 0x7f66a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:08.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.984+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f66ac0aae40 con 0x7f66b40d4e40 2024-09-10T11:31:08.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.984+0000 7f66b903f640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 0x7f66a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:08.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.984+0000 7f66b903f640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 0x7f66a006e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66b41f83c0 tx=0x7f66a800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:08.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:08.988+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66ac079580 con 0x7f66b40d4e40 2024-09-10T11:31:09.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.247+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f66b40b5920 con 0x7f66b40d4e40 2024-09-10T11:31:09.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.249+0000 7f66b27fc640 1 -- 172.21.15.99:0/2310032884 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f66ac07d880 con 0x7f66b40d4e40 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 msgr2=0x7f66a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 0x7f66a006e320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f66b41f83c0 tx=0x7f66a800b040 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 msgr2=0x7f66b415b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f66ac000c00 tx=0x7f66ac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 shutdown_connections 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a006be60 0x7f66a006e320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b41626c0 0x7f66b415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b415eb80 0x7f66b4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 --2- 172.21.15.99:0/2310032884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b40d4e40 0x7f66b415b9d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 >> 172.21.15.99:0/2310032884 conn(0x7f66b408e680 msgr2=0x7f66b415d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:09.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 shutdown_connections 2024-09-10T11:31:09.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.251+0000 7f66bbaa4640 1 -- 172.21.15.99:0/2310032884 wait complete. 2024-09-10T11:31:09.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T11:31:09.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.366+0000 7f078e9e4640 1 Processor -- start 2024-09-10T11:31:09.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.366+0000 7f078e9e4640 1 -- start start 2024-09-10T11:31:09.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f078e9e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 0x7f078815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f078e9e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0788070b70 con 0x7f078815d5c0 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f078e9e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0788070e00 con 0x7f078806bc10 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f078e9e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0788160cd0 con 0x7f0788161160 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f0787fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f078815d5c0 0x7f0788070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54116/0 (socket says 172.21.15.99:54116) 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f0787fff640 1 -- 172.21.15.99:0/2636551524 learned_addr learned my addr 172.21.15.99:0/2636551524 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f07877fe640 1 --2- 172.21.15.99:0/2636551524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 0x7f078815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797324165 0 0) 0x7f0788070b70 con 0x7f078815d5c0 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.367+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0774003520 con 0x7f078815d5c0 2024-09-10T11:31:09.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.368+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2430363547 0 0) 0x7f0774003520 con 0x7f078815d5c0 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.368+0000 7f07877fe640 1 -- 172.21.15.99:0/2636551524 >> v1:172.21.15.196:6789/0 conn(0x7f078806bc10 legacy=0x7f078806e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.368+0000 7f07877fe640 1 -- 172.21.15.99:0/2636551524 >> v1:172.21.15.99:6789/0 conn(0x7f078815d5c0 legacy=0x7f0788070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.368+0000 7f07877fe640 1 -- 172.21.15.99:0/2636551524 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0788160680 con 0x7f0788161160 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.368+0000 7f07877fe640 1 --2- 172.21.15.99:0/2636551524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 0x7f078815d9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07700097d0 tx=0x7f077002a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f85a67160d09012 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770037070 con 0x7f0788161160 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077002ab40 con 0x7f0788161160 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f0786ffd640 1 -- 172.21.15.99:0/2636551524 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778001670 con 0x7f0788161160 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 -- 172.21.15.99:0/2636551524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 msgr2=0x7f078815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 --2- 172.21.15.99:0/2636551524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 0x7f078815d9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f07700097d0 tx=0x7f077002a0b0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 -- 172.21.15.99:0/2636551524 shutdown_connections 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 --2- 172.21.15.99:0/2636551524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0788161160 0x7f078815d9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 -- 172.21.15.99:0/2636551524 >> 172.21.15.99:0/2636551524 conn(0x7f07880d1900 msgr2=0x7f07880d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 -- 172.21.15.99:0/2636551524 shutdown_connections 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.369+0000 7f078e9e4640 1 -- 172.21.15.99:0/2636551524 wait complete. 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.370+0000 7f078e9e4640 1 Processor -- start 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 -- start start 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f078806bc10 0x7f078814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f078815d5c0 0x7f0788140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f078806c200 con 0x7f0788161160 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f078806c080 con 0x7f078806bc10 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078e9e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f078806c380 con 0x7f078815d5c0 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078cf81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078cf81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47560/0 (socket says 172.21.15.99:47560) 2024-09-10T11:31:09.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f078cf81640 1 -- 172.21.15.99:0/1473475990 learned_addr learned my addr 172.21.15.99:0/1473475990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.371+0000 7f0787fff640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f078806bc10 0x7f078814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f07877fe640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f078815d5c0 0x7f0788140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 -- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f078815d5c0 msgr2=0x7f0788140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f078815d5c0 0x7f0788140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 -- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f078806bc10 msgr2=0x7f078814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f078806bc10 0x7f078814f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 -- 172.21.15.99:0/1473475990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078813dea0 con 0x7f0788161160 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078cf81640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f078000caf0 tx=0x7f0780002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780017070 con 0x7f0788161160 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0780003da0 con 0x7f0788161160 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078000a940 con 0x7f0788161160 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0788151200 con 0x7f0788161160 2024-09-10T11:31:09.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.372+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881517c0 con 0x7f0788161160 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.373+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078815d9a0 con 0x7f0788161160 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.374+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f078001a070 con 0x7f0788161160 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.374+0000 7f0784ff9640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 0x7f076806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.374+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f078008a280 con 0x7f0788161160 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.374+0000 7f0787fff640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 0x7f076806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.375+0000 7f0787fff640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 0x7f076806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0788144880 tx=0x7f0778001da0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.378+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0780058930 con 0x7f0788161160 2024-09-10T11:31:09.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.619+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f0788152630 con 0x7f0788161160 2024-09-10T11:31:09.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.621+0000 7f0784ff9640 1 -- 172.21.15.99:0/1473475990 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f078005cc30 con 0x7f0788161160 2024-09-10T11:31:09.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 msgr2=0x7f076806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 0x7f076806e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0788144880 tx=0x7f0778001da0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 msgr2=0x7f07881407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f078000caf0 tx=0x7f0780002090 comp rx=0 tx=0).stop 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 shutdown_connections 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076806be30 0x7f076806e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0788161160 0x7f07881407a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f078815d5c0 0x7f0788140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 --2- 172.21.15.99:0/1473475990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f078806bc10 0x7f078814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 >> 172.21.15.99:0/1473475990 conn(0x7f07880d1900 msgr2=0x7f078815dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:09.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 shutdown_connections 2024-09-10T11:31:09.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.623+0000 7f078e9e4640 1 -- 172.21.15.99:0/1473475990 wait complete. 2024-09-10T11:31:09.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T11:31:09.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.734+0000 7f05ea03c640 1 Processor -- start 2024-09-10T11:31:09.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.734+0000 7f05ea03c640 1 -- start start 2024-09-10T11:31:09.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05ea03c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05e40cfb90 0x7f05e40b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05ea03c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f05e40d1410 0x7f05e40cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05ea03c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05e406df80 con 0x7f05e40b3660 2024-09-10T11:31:09.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05ea03c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f05e40d18a0 con 0x7f05e40cfb90 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05ea03c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f05e40d2120 con 0x7f05e40d1410 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05e40cfb90 0x7f05e40b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05dbfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f05e40b3660 0x7f05e40d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54118/0 (socket says 172.21.15.99:54118) 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05dbfff640 1 -- 172.21.15.99:0/127873151 learned_addr learned my addr 172.21.15.99:0/127873151 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e3fff640 1 --2- 172.21.15.99:0/127873151 >> v2:172.21.15.196:3300/0 conn(0x7f05e40d1410 0x7f05e40cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e37fe640 1 -- 172.21.15.99:0/127873151 >> v2:172.21.15.196:3300/0 conn(0x7f05e40d1410 msgr2=0x7f05e40cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e37fe640 1 --2- 172.21.15.99:0/127873151 >> v2:172.21.15.196:3300/0 conn(0x7f05e40d1410 0x7f05e40cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e37fe640 1 -- 172.21.15.99:0/127873151 >> v1:172.21.15.99:6789/0 conn(0x7f05e40b3660 legacy=0x7f05e40d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e2ffd640 1 -- 172.21.15.99:0/127873151 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527736780 0 0) 0x7f05e406df80 con 0x7f05e40b3660 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.735+0000 7f05e37fe640 1 -- 172.21.15.99:0/127873151 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40d1ad0 con 0x7f05e40cfb90 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.736+0000 7f05e37fe640 1 --2- 172.21.15.99:0/127873151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40cfb90 0x7f05e40b5170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05e4073970 tx=0x7f05d402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ce3e50b583deb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.736+0000 7f05e2ffd640 1 -- 172.21.15.99:0/127873151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4037070 con 0x7f05e40cfb90 2024-09-10T11:31:09.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.736+0000 7f05e2ffd640 1 -- 172.21.15.99:0/127873151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d4032440 con 0x7f05e40cfb90 2024-09-10T11:31:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.736+0000 7f05e2ffd640 1 -- 172.21.15.99:0/127873151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d402ae60 con 0x7f05e40cfb90 2024-09-10T11:31:09.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 -- 172.21.15.99:0/127873151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40cfb90 msgr2=0x7f05e40b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 --2- 172.21.15.99:0/127873151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40cfb90 0x7f05e40b5170 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05e4073970 tx=0x7f05d402a110 comp rx=0 tx=0).stop 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 -- 172.21.15.99:0/127873151 shutdown_connections 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 --2- 172.21.15.99:0/127873151 >> v2:172.21.15.196:3300/0 conn(0x7f05e40d1410 0x7f05e40cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 --2- 172.21.15.99:0/127873151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40cfb90 0x7f05e40b5170 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 -- 172.21.15.99:0/127873151 >> 172.21.15.99:0/127873151 conn(0x7f05e40b4650 msgr2=0x7f05e40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 -- 172.21.15.99:0/127873151 shutdown_connections 2024-09-10T11:31:09.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.737+0000 7f05ea03c640 1 -- 172.21.15.99:0/127873151 wait complete. 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.738+0000 7f05ea03c640 1 Processor -- start 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.738+0000 7f05ea03c640 1 -- start start 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e40b3660 0x7f05e41f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 0x7f05e40b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f05e406dd30 con 0x7f05e40d1410 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05e406e260 con 0x7f05e40b3660 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05ea03c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f05e40cff70 con 0x7f05e40cfb90 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47570/0 (socket says 172.21.15.99:47570) 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 -- 172.21.15.99:0/3566972999 learned_addr learned my addr 172.21.15.99:0/3566972999 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e37fe640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e40b3660 0x7f05e41f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05dbfff640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 0x7f05e40b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 -- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 msgr2=0x7f05e40b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 0x7f05e40b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 -- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e40b3660 msgr2=0x7f05e41f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.739+0000 7f05e3fff640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e40b3660 0x7f05e41f7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05e3fff640 1 -- 172.21.15.99:0/3566972999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e40b0660 con 0x7f05e40d1410 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05dbfff640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 0x7f05e40b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05e3fff640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007e40 tx=0x7f05dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc014620 con 0x7f05e40d1410 2024-09-10T11:31:09.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05dc019070 con 0x7f05e40d1410 2024-09-10T11:31:09.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc014a20 con 0x7f05e40d1410 2024-09-10T11:31:09.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40b0940 con 0x7f05e40d1410 2024-09-10T11:31:09.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.740+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e40b0f00 con 0x7f05e40d1410 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.741+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e4070ae0 con 0x7f05e40d1410 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.741+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05dc014bc0 con 0x7f05e40d1410 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.742+0000 7f05e0ff9640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 0x7f05c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.742+0000 7f05e37fe640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 0x7f05c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.742+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f05dc08b380 con 0x7f05e40d1410 2024-09-10T11:31:09.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.742+0000 7f05e37fe640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 0x7f05c406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05d4034000 tx=0x7f05d402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:09.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.746+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f05e4070ae0 con 0x7f05e40d1410 2024-09-10T11:31:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.980+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f05e40b3420 con 0x7f05e40d1410 2024-09-10T11:31:09.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.981+0000 7f05e0ff9640 1 -- 172.21.15.99:0/3566972999 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f05dc05a080 con 0x7f05e40d1410 2024-09-10T11:31:09.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 msgr2=0x7f05c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 0x7f05c406e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f05d4034000 tx=0x7f05d402e000 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 msgr2=0x7f05e40b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f05dc007e40 tx=0x7f05dc00e4b0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 shutdown_connections 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f05c406be30 0x7f05c406e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05e40d1410 0x7f05e40b2560 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e40cfb90 0x7f05e40b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 --2- 172.21.15.99:0/3566972999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e40b3660 0x7f05e41f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 >> 172.21.15.99:0/3566972999 conn(0x7f05e40b4650 msgr2=0x7f05e4070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.983+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 shutdown_connections 2024-09-10T11:31:09.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:09.984+0000 7f05ea03c640 1 -- 172.21.15.99:0/3566972999 wait complete. 2024-09-10T11:31:09.996 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T11:31:10.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.094+0000 7efe69718640 1 Processor -- start 2024-09-10T11:31:10.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.094+0000 7efe69718640 1 -- start start 2024-09-10T11:31:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe69718640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe6415b2d0 0x7efe640b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe69718640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe640afbb0 0x7efe640714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe69718640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640b0020 con 0x7efe6415aef0 2024-09-10T11:31:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe69718640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7efe6406ee10 con 0x7efe640afbb0 2024-09-10T11:31:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe69718640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efe640cd9a0 con 0x7efe6415b2d0 2024-09-10T11:31:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe640afbb0 0x7efe640714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe640afbb0 0x7efe640714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47586/0 (socket says 172.21.15.99:47586) 2024-09-10T11:31:10.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 -- 172.21.15.99:0/678176726 learned_addr learned my addr 172.21.15.99:0/678176726 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 -- 172.21.15.99:0/678176726 >> v2:172.21.15.196:3300/0 conn(0x7efe6415b2d0 msgr2=0x7efe640b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 --2- 172.21.15.99:0/678176726 >> v2:172.21.15.196:3300/0 conn(0x7efe6415b2d0 0x7efe640b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 -- 172.21.15.99:0/678176726 >> v1:172.21.15.196:6789/0 conn(0x7efe6415aef0 legacy=0x7efe640cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.095+0000 7efe637fe640 1 -- 172.21.15.99:0/678176726 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640b0290 con 0x7efe640afbb0 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe637fe640 1 --2- 172.21.15.99:0/678176726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe640afbb0 0x7efe640714a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009e50 tx=0x7efe5c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336bd4998af6b8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe627fc640 1 -- 172.21.15.99:0/678176726 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c037070 con 0x7efe640afbb0 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe627fc640 1 -- 172.21.15.99:0/678176726 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c032600 con 0x7efe640afbb0 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe627fc640 1 -- 172.21.15.99:0/678176726 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0329d0 con 0x7efe640afbb0 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 -- 172.21.15.99:0/678176726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe640afbb0 msgr2=0x7efe640714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 --2- 172.21.15.99:0/678176726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe640afbb0 0x7efe640714a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009e50 tx=0x7efe5c02b820 comp rx=0 tx=0).stop 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 -- 172.21.15.99:0/678176726 shutdown_connections 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 --2- 172.21.15.99:0/678176726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe640afbb0 0x7efe640714a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 --2- 172.21.15.99:0/678176726 >> v2:172.21.15.196:3300/0 conn(0x7efe6415b2d0 0x7efe640b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.096+0000 7efe69718640 1 -- 172.21.15.99:0/678176726 >> 172.21.15.99:0/678176726 conn(0x7efe6406bc10 msgr2=0x7efe6406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.097+0000 7efe69718640 1 -- 172.21.15.99:0/678176726 shutdown_connections 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.097+0000 7efe69718640 1 -- 172.21.15.99:0/678176726 wait complete. 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 Processor -- start 2024-09-10T11:31:10.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 -- start start 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 0x7efe640ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 0x7efe6415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406eb30 con 0x7efe6415aef0 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406e850 con 0x7efe6415b2d0 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe69718640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe640719e0 con 0x7efe640afbb0 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.098+0000 7efe5bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47596/0 (socket says 172.21.15.99:47596) 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 -- 172.21.15.99:0/2143939330 learned_addr learned my addr 172.21.15.99:0/2143939330 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:10.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 -- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 msgr2=0x7efe640ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe637fe640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 0x7efe6415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe62ffd640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 0x7efe640ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 0x7efe640ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 -- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 msgr2=0x7efe6415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 0x7efe6415dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 -- 172.21.15.99:0/2143939330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640cde00 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe62ffd640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 0x7efe640ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe637fe640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 0x7efe6415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5bfff640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00b440 tx=0x7efe4c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c015020 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c011890 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011c80 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe640ce0e0 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.099+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe640ce6a0 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.100+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe64071da0 con 0x7efe6415aef0 2024-09-10T11:31:10.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.101+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe4c018020 con 0x7efe6415aef0 2024-09-10T11:31:10.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.101+0000 7efe5b7fe640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.101+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7efe4c08acb0 con 0x7efe6415aef0 2024-09-10T11:31:10.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.101+0000 7efe62ffd640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.101+0000 7efe62ffd640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 0x7efe4406e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe54007fd0 tx=0x7efe54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.105+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe4c01f020 con 0x7efe6415aef0 2024-09-10T11:31:10.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-10T11:31:10.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.333+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7efe640ce870 con 0x7efe6415aef0 2024-09-10T11:31:10.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.335+0000 7efe5b7fe640 1 -- 172.21.15.99:0/2143939330 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7efe4c00d040 con 0x7efe6415aef0 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 msgr2=0x7efe4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 0x7efe4406e2f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe54007fd0 tx=0x7efe54009040 comp rx=0 tx=0).stop 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 msgr2=0x7efe640aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00b440 tx=0x7efe4c00b900 comp rx=0 tx=0).stop 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 shutdown_connections 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe4406be30 0x7efe4406e2f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe6415b2d0 0x7efe6415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe6415aef0 0x7efe640aa490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 --2- 172.21.15.99:0/2143939330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe640afbb0 0x7efe640ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 >> 172.21.15.99:0/2143939330 conn(0x7efe6406bc10 msgr2=0x7efe64072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:10.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 shutdown_connections 2024-09-10T11:31:10.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.337+0000 7efe69718640 1 -- 172.21.15.99:0/2143939330 wait complete. 2024-09-10T11:31:10.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T11:31:10.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.447+0000 7ff0d0874640 1 Processor -- start 2024-09-10T11:31:10.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.447+0000 7ff0d0874640 1 -- start start 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0d0874640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff0cc0d1180 0x7ff0cc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0d0874640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc0ceb10 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0d0874640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0cc0b3a90 con 0x7ff0cc06ee30 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0d0874640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff0cc06f280 con 0x7ff0cc0d1180 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0ca59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff0cc15c7b0 0x7ff0cc0d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54134/0 (socket says 172.21.15.99:54134) 2024-09-10T11:31:10.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0ca59c640 1 -- 172.21.15.99:0/1756910379 learned_addr learned my addr 172.21.15.99:0/1756910379 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/1756910379 >> v2:172.21.15.196:3300/0 conn(0x7ff0cc0d1180 0x7ff0cc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.448+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1420624255 0 0) 0x7ff0bc001490 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0b4003520 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828609092 0 0) 0x7ff0cc0b3a90 con 0x7ff0cc06ee30 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0bc001490 con 0x7ff0cc06ee30 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398008766 0 0) 0x7ff0b4003520 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 >> v2:172.21.15.196:3300/0 conn(0x7ff0cc0d1180 msgr2=0x7ff0cc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 --2- 172.21.15.99:0/1756910379 >> v2:172.21.15.196:3300/0 conn(0x7ff0cc0d1180 0x7ff0cc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 >> v1:172.21.15.196:6789/0 conn(0x7ff0cc06ee30 legacy=0x7ff0cc06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc0b3d70 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7ff0bc001bc0 con 0x7ff0cc15c7b0 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 >> v1:172.21.15.99:6789/0 conn(0x7ff0cc15c7b0 legacy=0x7ff0cc0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0b4003b90 0x7ff0b4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 0x7ff0b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b40051c0 0x7ff0b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc06f280 con 0x7ff0b4004800 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400c090 con 0x7ff0b40051c0 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b400c290 con 0x7ff0b4003b90 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 0x7ff0b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0ca59c640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0b4003b90 0x7ff0b4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0cad9d640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b40051c0 0x7ff0b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0b4003b90 msgr2=0x7ff0b4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0b4003b90 0x7ff0b4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b40051c0 msgr2=0x7ff0b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b40051c0 0x7ff0b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/1756910379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b4003520 con 0x7ff0b4004800 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0cad9d640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b40051c0 0x7ff0b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:10.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.449+0000 7ff0cad9d640 1 -- 172.21.15.99:0/1756910379 reap_dead start 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.450+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 0x7ff0b4004c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009ac0 tx=0x7ff0b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b062df5d94e7812 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.450+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0036070 con 0x7ff0b4004800 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.450+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c0031400 con 0x7ff0b4004800 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.450+0000 7ff0c959a640 1 -- 172.21.15.99:0/1756910379 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0009690 con 0x7ff0b4004800 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 -- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 msgr2=0x7ff0b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 0x7ff0b4004c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009ac0 tx=0x7ff0b4005830 comp rx=0 tx=0).stop 2024-09-10T11:31:10.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 -- 172.21.15.99:0/1756910379 shutdown_connections 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 --2- 172.21.15.99:0/1756910379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b4004800 0x7ff0b4004c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 -- 172.21.15.99:0/1756910379 >> 172.21.15.99:0/1756910379 conn(0x7ff0cc06bc10 msgr2=0x7ff0cc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 -- 172.21.15.99:0/1756910379 shutdown_connections 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.451+0000 7ff0d0874640 1 -- 172.21.15.99:0/1756910379 wait complete. 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.452+0000 7ff0d0874640 1 Processor -- start 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.452+0000 7ff0d0874640 1 -- start start 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b4004800 0x7ff0cc14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0cc14f490 0x7ff0cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b4330 con 0x7ff0cc073050 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b3840 con 0x7ff0b4004800 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0d0874640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0cc0b4610 con 0x7ff0cc14f490 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47626/0 (socket says 172.21.15.99:47626) 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/936388173 learned_addr learned my addr 172.21.15.99:0/936388173 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0ca59c640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b4004800 0x7ff0cc14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0cad9d640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0cc14f490 0x7ff0cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0cc14f490 msgr2=0x7ff0cc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0cc14f490 0x7ff0cc15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b4004800 msgr2=0x7ff0cc14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b4004800 0x7ff0cc14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 -- 172.21.15.99:0/936388173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0cc14d8c0 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.453+0000 7ff0c9d9b640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009be0 tx=0x7ff0c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.454+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c003f020 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.454+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c0037040 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.454+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0cc071fb0 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.454+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0036070 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.454+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0cc072570 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.455+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0cc15ccb0 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.455+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0c0042020 con 0x7ff0cc073050 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.456+0000 7ff0baffd640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 0x7ff0ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.456+0000 7ff0ca59c640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 0x7ff0ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.456+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ff0c00aaa40 con 0x7ff0cc073050 2024-09-10T11:31:10.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.456+0000 7ff0ca59c640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 0x7ff0ac06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0013d0 tx=0x7ff0bc006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.460+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0c0049030 con 0x7ff0cc073050 2024-09-10T11:31:10.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.691+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7ff0cc158d50 con 0x7ff0cc073050 2024-09-10T11:31:10.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.693+0000 7ff0baffd640 1 -- 172.21.15.99:0/936388173 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7ff0c00aae20 con 0x7ff0cc073050 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 msgr2=0x7ff0ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 0x7ff0ac06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0013d0 tx=0x7ff0bc006040 comp rx=0 tx=0).stop 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 msgr2=0x7ff0cc0716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009be0 tx=0x7ff0c0004730 comp rx=0 tx=0).stop 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 shutdown_connections 2024-09-10T11:31:10.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff0ac06be60 0x7ff0ac06e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff0cc14f490 0x7ff0cc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0cc073050 0x7ff0cc0716b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 --2- 172.21.15.99:0/936388173 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0b4004800 0x7ff0cc14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 >> 172.21.15.99:0/936388173 conn(0x7ff0cc06bc10 msgr2=0x7ff0cc0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 shutdown_connections 2024-09-10T11:31:10.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.696+0000 7ff0d0874640 1 -- 172.21.15.99:0/936388173 wait complete. 2024-09-10T11:31:10.709 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T11:31:10.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.805+0000 7f45050c3640 1 Processor -- start 2024-09-10T11:31:10.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 -- start start 2024-09-10T11:31:10.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 0x7f4500163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4500160030 0x7f450006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 0x7f450006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f45000b5420 con 0x7f4500160030 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f450006c530 con 0x7f450006bc10 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f45050c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45000b5140 con 0x7f45000b4930 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.806+0000 7f44fe59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4500160030 0x7f450006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4500160030 0x7f450006bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47640/0 (socket says 172.21.15.99:47640) 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 -- 172.21.15.99:0/551824781 learned_addr learned my addr 172.21.15.99:0/551824781 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 -- 172.21.15.99:0/551824781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 msgr2=0x7f450006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fed9d640 1 --2- 172.21.15.99:0/551824781 >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 0x7f4500163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44ff59e640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 0x7f450006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 0x7f450006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 -- 172.21.15.99:0/551824781 >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 msgr2=0x7f4500163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 --2- 172.21.15.99:0/551824781 >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 0x7f4500163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 -- 172.21.15.99:0/551824781 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4500162b40 con 0x7f4500160030 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fed9d640 1 --2- 172.21.15.99:0/551824781 >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 0x7f4500163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44ff59e640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 0x7f450006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.807+0000 7f44fe59c640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4500160030 0x7f450006bff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4500073870 tx=0x7f44f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9070580a3d88f097 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f44fdd9b640 1 -- 172.21.15.99:0/551824781 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0037070 con 0x7f4500160030 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f44fdd9b640 1 -- 172.21.15.99:0/551824781 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f0032440 con 0x7f4500160030 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f44fdd9b640 1 -- 172.21.15.99:0/551824781 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0032810 con 0x7f4500160030 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 -- 172.21.15.99:0/551824781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4500160030 msgr2=0x7f450006bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4500160030 0x7f450006bff0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4500073870 tx=0x7f44f002a140 comp rx=0 tx=0).stop 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 -- 172.21.15.99:0/551824781 shutdown_connections 2024-09-10T11:31:10.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45000b4930 0x7f450006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 --2- 172.21.15.99:0/551824781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4500160030 0x7f450006bff0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 --2- 172.21.15.99:0/551824781 >> v2:172.21.15.196:3300/0 conn(0x7f450006bc10 0x7f4500163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 -- 172.21.15.99:0/551824781 >> 172.21.15.99:0/551824781 conn(0x7f45000b3630 msgr2=0x7f45000b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 -- 172.21.15.99:0/551824781 shutdown_connections 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.808+0000 7f45050c3640 1 -- 172.21.15.99:0/551824781 wait complete. 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 Processor -- start 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 -- start start 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f450006bc10 0x7f450015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4500160030 0x7f45001431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45000b4d90 con 0x7f45000b4930 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4500163100 con 0x7f4500160030 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.810+0000 7f45050c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45000af810 con 0x7f450006bc10 2024-09-10T11:31:10.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47646/0 (socket says 172.21.15.99:47646) 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 -- 172.21.15.99:0/3623026814 learned_addr learned my addr 172.21.15.99:0/3623026814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fed9d640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f450006bc10 0x7f450015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44ff59e640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4500160030 0x7f45001431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 -- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f450006bc10 msgr2=0x7f450015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f450006bc10 0x7f450015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 -- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4500160030 msgr2=0x7f45001431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4500160030 0x7f45001431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 -- 172.21.15.99:0/3623026814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450013f5a0 con 0x7f45000b4930 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44fe59c640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f44f002a180 tx=0x7f44f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0037040 con 0x7f45000b4930 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f0042070 con 0x7f45000b4930 2024-09-10T11:31:10.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0030bb0 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450013f840 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.811+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45000b04e0 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.812+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45000d1560 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.813+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44f0047460 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.813+0000 7f44f77fe640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 0x7f44e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.813+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f44f00ab6d0 con 0x7f45000b4930 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.813+0000 7f44fed9d640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 0x7f44e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.814+0000 7f44fed9d640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 0x7f44e006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44e800a950 tx=0x7f44e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:10.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:10.817+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44f0045360 con 0x7f45000b4930 2024-09-10T11:31:11.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.047+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f45000b2c40 con 0x7f45000b4930 2024-09-10T11:31:11.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.049+0000 7f44f77fe640 1 -- 172.21.15.99:0/3623026814 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f44f007a460 con 0x7f45000b4930 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 msgr2=0x7f44e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 0x7f44e006e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f44e800a950 tx=0x7f44e8008040 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 msgr2=0x7f45000affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f44f002a180 tx=0x7f44f00046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 shutdown_connections 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f44e006be30 0x7f44e006e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4500160030 0x7f45001431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45000b4930 0x7f45000affa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 --2- 172.21.15.99:0/3623026814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f450006bc10 0x7f450015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 >> 172.21.15.99:0/3623026814 conn(0x7f45000b3630 msgr2=0x7f45000b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 shutdown_connections 2024-09-10T11:31:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.052+0000 7f45050c3640 1 -- 172.21.15.99:0/3623026814 wait complete. 2024-09-10T11:31:11.065 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:31:11.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.160+0000 7fd0f4f4b640 1 Processor -- start 2024-09-10T11:31:11.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.160+0000 7fd0f4f4b640 1 -- start start 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0f4f4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f0163580 0x7fd0f00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0f4f4b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f00b3960 0x7fd0f00d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0f4f4b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0f00b3620 con 0x7fd0f006cd20 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0f4f4b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd0f006d960 con 0x7fd0f00b3960 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0f4f4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f006dc40 con 0x7fd0f0163580 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f00b3960 0x7fd0f00d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0f00b3960 0x7fd0f00d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47660/0 (socket says 172.21.15.99:47660) 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 -- 172.21.15.99:0/3078396292 learned_addr learned my addr 172.21.15.99:0/3078396292 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 -- 172.21.15.99:0/3078396292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f0163580 msgr2=0x7fd0f00b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0ee59c640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f0163580 0x7fd0f00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f0163580 0x7fd0f00b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 -- 172.21.15.99:0/3078396292 >> v1:172.21.15.196:6789/0 conn(0x7fd0f006cd20 legacy=0x7fd0f006d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:11.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.161+0000 7fd0edd9b640 1 -- 172.21.15.99:0/3078396292 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f006df20 con 0x7fd0f00b3960 2024-09-10T11:31:11.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0edd9b640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f00b3960 0x7fd0f00d1be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009870 tx=0x7fd0e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e8a34e38d1bacf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0ed59a640 1 -- 172.21.15.99:0/3078396292 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0037070 con 0x7fd0f00b3960 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0ed59a640 1 -- 172.21.15.99:0/3078396292 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e0032440 con 0x7fd0f00b3960 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0ed59a640 1 -- 172.21.15.99:0/3078396292 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0032810 con 0x7fd0f00b3960 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/3078396292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f00b3960 msgr2=0x7fd0f00d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.162+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f00b3960 0x7fd0f00d1be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009870 tx=0x7fd0e002a180 comp rx=0 tx=0).stop 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/3078396292 shutdown_connections 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f00b3960 0x7fd0f00d1be0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/3078396292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f0163580 0x7fd0f00b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/3078396292 >> 172.21.15.99:0/3078396292 conn(0x7fd0f00b4570 msgr2=0x7fd0f00b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/3078396292 shutdown_connections 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.163+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/3078396292 wait complete. 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.164+0000 7fd0f4f4b640 1 Processor -- start 2024-09-10T11:31:11.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.164+0000 7fd0f4f4b640 1 -- start start 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f006cd20 0x7fd0f0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f00b3960 0x7fd0f00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f00b3160 con 0x7fd0f0163580 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f00d2120 con 0x7fd0f006cd20 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f00b32e0 con 0x7fd0f00b3960 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47666/0 (socket says 172.21.15.99:47666) 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 -- 172.21.15.99:0/1083625341 learned_addr learned my addr 172.21.15.99:0/1083625341 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0ee59c640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f006cd20 0x7fd0f0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0edd9b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f00b3960 0x7fd0f00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 -- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f00b3960 msgr2=0x7fd0f00cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f00b3960 0x7fd0f00cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 -- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f006cd20 msgr2=0x7fd0f0155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f006cd20 0x7fd0f0155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 -- 172.21.15.99:0/1083625341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f00cb680 con 0x7fd0f0163580 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0eed9d640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800b9a0 tx=0x7fd0e800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8019070 con 0x7fd0f0163580 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e800cba0 con 0x7fd0f0163580 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f00cb960 con 0x7fd0f0163580 2024-09-10T11:31:11.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8014440 con 0x7fd0f0163580 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.165+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f00cbf20 con 0x7fd0f0163580 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.166+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f00d1be0 con 0x7fd0f0163580 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.167+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0e801c070 con 0x7fd0f0163580 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.167+0000 7fd0e6ffd640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 0x7fd0d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.167+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd0e808a630 con 0x7fd0f0163580 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.167+0000 7fd0ee59c640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 0x7fd0d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.168+0000 7fd0ee59c640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 0x7fd0d006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0f01f8a30 tx=0x7fd0d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.172+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd0e8058d70 con 0x7fd0f0163580 2024-09-10T11:31:11.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.338+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd0f00d27a0 con 0x7fd0f0163580 2024-09-10T11:31:11.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.338+0000 7fd0e6ffd640 1 -- 172.21.15.99:0/1083625341 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7fd0e805d070 con 0x7fd0f0163580 2024-09-10T11:31:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:11.341 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:31:11.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:31:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:31:11.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:11.342 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:31:11.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 msgr2=0x7fd0d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 0x7fd0d006e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd0f01f8a30 tx=0x7fd0d8008040 comp rx=0 tx=0).stop 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 msgr2=0x7fd0f015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800b9a0 tx=0x7fd0e800be60 comp rx=0 tx=0).stop 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 shutdown_connections 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd0d006be30 0x7fd0d006e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f0163580 0x7fd0f015da10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0f00b3960 0x7fd0f00cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 --2- 172.21.15.99:0/1083625341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0f006cd20 0x7fd0f0155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 >> 172.21.15.99:0/1083625341 conn(0x7fd0f00b4570 msgr2=0x7fd0f00b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:11.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 shutdown_connections 2024-09-10T11:31:11.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.340+0000 7fd0f4f4b640 1 -- 172.21.15.99:0/1083625341 wait complete. 2024-09-10T11:31:11.353 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:31:11.354 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:31:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.454+0000 7fa4cef98640 1 Processor -- start 2024-09-10T11:31:11.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.454+0000 7fa4cef98640 1 -- start start 2024-09-10T11:31:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cef98640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cef98640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 0x7fa4c8163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cef98640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4c80d5b70 con 0x7fa4c80d16f0 2024-09-10T11:31:11.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cef98640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c80d0d30 con 0x7fa4c80b59b0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cef98640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa4c80b4d30 con 0x7fa4c80d5740 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4c7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4c7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47678/0 (socket says 172.21.15.99:47678) 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4c7fff640 1 -- 172.21.15.99:0/3478121668 learned_addr learned my addr 172.21.15.99:0/3478121668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.455+0000 7fa4cd535640 1 --2- 172.21.15.99:0/3478121668 >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 0x7fa4c8163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c7fff640 1 -- 172.21.15.99:0/3478121668 >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 msgr2=0x7fa4c8163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c7fff640 1 --2- 172.21.15.99:0/3478121668 >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 0x7fa4c8163420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c7fff640 1 -- 172.21.15.99:0/3478121668 >> v1:172.21.15.196:6789/0 conn(0x7fa4c80d16f0 legacy=0x7fa4c8162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c7fff640 1 -- 172.21.15.99:0/3478121668 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c80d2150 con 0x7fa4c80b59b0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4cd535640 1 --2- 172.21.15.99:0/3478121668 >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 0x7fa4c8163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c7fff640 1 --2- 172.21.15.99:0/3478121668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b63c0 tx=0x7fa4bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e25defa580d66f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.456+0000 7fa4c77fe640 1 -- 172.21.15.99:0/3478121668 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037070 con 0x7fa4c80b59b0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4c77fe640 1 -- 172.21.15.99:0/3478121668 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc032440 con 0x7fa4c80b59b0 2024-09-10T11:31:11.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4c77fe640 1 -- 172.21.15.99:0/3478121668 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc032810 con 0x7fa4c80b59b0 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 -- 172.21.15.99:0/3478121668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80b59b0 msgr2=0x7fa4c80d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 --2- 172.21.15.99:0/3478121668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa4c80b63c0 tx=0x7fa4bc02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 -- 172.21.15.99:0/3478121668 shutdown_connections 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 --2- 172.21.15.99:0/3478121668 >> v2:172.21.15.196:3300/0 conn(0x7fa4c80d5740 0x7fa4c8163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 --2- 172.21.15.99:0/3478121668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80b59b0 0x7fa4c80d1ad0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 -- 172.21.15.99:0/3478121668 >> 172.21.15.99:0/3478121668 conn(0x7fa4c80d4c20 msgr2=0x7fa4c80d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 -- 172.21.15.99:0/3478121668 shutdown_connections 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.457+0000 7fa4cef98640 1 -- 172.21.15.99:0/3478121668 wait complete. 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.458+0000 7fa4cef98640 1 Processor -- start 2024-09-10T11:31:11.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.458+0000 7fa4cef98640 1 -- start start 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80b59b0 0x7fa4c8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4c80d5740 0x7fa4c80b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80d23f0 con 0x7fa4c80d16f0 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c80b52f0 con 0x7fa4c80b59b0 2024-09-10T11:31:11.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cef98640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c80b4f80 con 0x7fa4c80d5740 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47690/0 (socket says 172.21.15.99:47690) 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4c7fff640 1 -- 172.21.15.99:0/1154967847 learned_addr learned my addr 172.21.15.99:0/1154967847 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4cd535640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4c80d5740 0x7fa4c80b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.459+0000 7fa4ccd34640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80b59b0 0x7fa4c8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 -- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4c80d5740 msgr2=0x7fa4c80b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4c80d5740 0x7fa4c80b9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 -- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80b59b0 msgr2=0x7fa4c8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80b59b0 0x7fa4c8158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 -- 172.21.15.99:0/1154967847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c8159d00 con 0x7fa4c80d16f0 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c7fff640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02a220 tx=0x7fa4bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc041020 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0306c0 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c8159a20 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037040 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.460+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c815c2a0 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.461+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c815bdf0 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.462+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4bc044020 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.462+0000 7fa4c57fa640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 0x7fa4b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.462+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fa4bc0ab350 con 0x7fa4c80d16f0 2024-09-10T11:31:11.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.462+0000 7fa4ccd34640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 0x7fa4b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:11.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.462+0000 7fa4ccd34640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 0x7fa4b406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8001500 tx=0x7fa4b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:11.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.466+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4bc04b030 con 0x7fa4c80d16f0 2024-09-10T11:31:11.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:11.633+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa4c80d1ad0 con 0x7fa4c80d16f0 2024-09-10T11:31:12.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.494+0000 7fa4c57fa640 1 -- 172.21.15.99:0/1154967847 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7fa4bc07a050 con 0x7fa4c80d16f0 2024-09-10T11:31:12.496 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 msgr2=0x7fa4b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 0x7fa4b406e2f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8001500 tx=0x7fa4b8009040 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 msgr2=0x7fa4c8159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc02a220 tx=0x7fa4bc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 shutdown_connections 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b406be30 0x7fa4b406e2f0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8001500 tx=0x7fa4b8009040 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4c80d5740 0x7fa4c80b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4c80d16f0 0x7fa4c8159f40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 --2- 172.21.15.99:0/1154967847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4c80b59b0 0x7fa4c8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.498+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 >> 172.21.15.99:0/1154967847 conn(0x7fa4c80d4c20 msgr2=0x7fa4c80d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:12.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.499+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 shutdown_connections 2024-09-10T11:31:12.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.499+0000 7fa4cef98640 1 -- 172.21.15.99:0/1154967847 wait complete. 2024-09-10T11:31:12.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:31:12.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 Processor -- start 2024-09-10T11:31:12.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 -- start start 2024-09-10T11:31:12.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd18160030 0x7efd1806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd180b5140 con 0x7efd18163620 2024-09-10T11:31:12.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd18162b40 con 0x7efd1806bc10 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.624+0000 7efd1ec5c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7efd180b5420 con 0x7efd18160030 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd17fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efd18163620 0x7efd180b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50038/0 (socket says 172.21.15.99:50038) 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd17fff640 1 -- 172.21.15.99:0/3732121351 learned_addr learned my addr 172.21.15.99:0/3732121351 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd1d1f9640 1 --2- 172.21.15.99:0/3732121351 >> v2:172.21.15.196:3300/0 conn(0x7efd18160030 0x7efd1806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806756335 0 0) 0x7efd180b5140 con 0x7efd18163620 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd00003520 con 0x7efd18163620 2024-09-10T11:31:12.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 240705864 0 0) 0x7efd18162b40 con 0x7efd1806bc10 2024-09-10T11:31:12.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd180b5140 con 0x7efd1806bc10 2024-09-10T11:31:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3905885937 0 0) 0x7efd00003520 con 0x7efd18163620 2024-09-10T11:31:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 >> v2:172.21.15.196:3300/0 conn(0x7efd18160030 msgr2=0x7efd1806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 --2- 172.21.15.99:0/3732121351 >> v2:172.21.15.196:3300/0 conn(0x7efd18160030 0x7efd1806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 >> v1:172.21.15.196:6789/0 conn(0x7efd1806bc10 legacy=0x7efd1806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd18162e20 con 0x7efd18163620 2024-09-10T11:31:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7efd0c001670 con 0x7efd18163620 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 >> v1:172.21.15.99:6789/0 conn(0x7efd18163620 legacy=0x7efd180b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd00003b40 0x7efd00003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd000051c0 0x7efd0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd180b5420 con 0x7efd00004800 2024-09-10T11:31:12.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd0000c0f0 con 0x7efd000051c0 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd0000c2f0 con 0x7efd00003b40 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd17fff640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd1c9f8640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd00003b40 0x7efd00003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.625+0000 7efd1d1f9640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd000051c0 0x7efd0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 -- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd00003b40 msgr2=0x7efd00003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd00003b40 0x7efd00003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 -- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd000051c0 msgr2=0x7efd0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd000051c0 0x7efd0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 -- 172.21.15.99:0/3732121351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd00003520 con 0x7efd00004800 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd1d1f9640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd000051c0 0x7efd0000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd1d1f9640 1 -- 172.21.15.99:0/3732121351 reap_dead start 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd1c9f8640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd00003b40 0x7efd00003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd17fff640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd00004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c190 tx=0x7efd00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abd4a5737c2b149 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c007ce0 con 0x7efd00004800 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd0c005ad0 con 0x7efd00004800 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.626+0000 7efd177fe640 1 -- 172.21.15.99:0/3732121351 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c007070 con 0x7efd00004800 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 msgr2=0x7efd00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd00004c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00c190 tx=0x7efd00005830 comp rx=0 tx=0).stop 2024-09-10T11:31:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3732121351 shutdown_connections 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3732121351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd00004c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3732121351 >> 172.21.15.99:0/3732121351 conn(0x7efd180b3630 msgr2=0x7efd180b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3732121351 shutdown_connections 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.627+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3732121351 wait complete. 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.628+0000 7efd1ec5c640 1 Processor -- start 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.628+0000 7efd1ec5c640 1 -- start start 2024-09-10T11:31:12.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 0x7efd18146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 0x7efd180afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efd180d16e0 con 0x7efd00004800 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efd180d1560 con 0x7efd18144940 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1ec5c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efd1806bff0 con 0x7efd18146740 2024-09-10T11:31:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1c9f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1c9f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36722/0 (socket says 172.21.15.99:36722) 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1c9f8640 1 -- 172.21.15.99:0/3444736513 learned_addr learned my addr 172.21.15.99:0/3444736513 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd17fff640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 0x7efd18146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1d1f9640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 0x7efd180afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.629+0000 7efd1c9f8640 1 -- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 msgr2=0x7efd180afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1c9f8640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 0x7efd180afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1c9f8640 1 -- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 msgr2=0x7efd18146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1c9f8640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 0x7efd18146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1c9f8640 1 -- 172.21.15.99:0/3444736513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd18143900 con 0x7efd00004800 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd17fff640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 0x7efd18146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1d1f9640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 0x7efd180afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1c9f8640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e870 tx=0x7efd0800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:12.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08003b80 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd08003d20 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd08006b50 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd18143b80 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.630+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd18144140 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.631+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd18162b40 con 0x7efd00004800 2024-09-10T11:31:12.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.631+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd0801a020 con 0x7efd00004800 2024-09-10T11:31:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.632+0000 7efd157fa640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 0x7efd0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.632+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7efd0808a630 con 0x7efd00004800 2024-09-10T11:31:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.632+0000 7efd17fff640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 0x7efd0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.632+0000 7efd17fff640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 0x7efd0406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fd0 tx=0x7efd0c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:12.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.636+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd18143900 con 0x7efd00004800 2024-09-10T11:31:12.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:12.800+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd181627f0 con 0x7efd00004800 2024-09-10T11:31:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.499+0000 7efd157fa640 1 -- 172.21.15.99:0/3444736513 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7efd080591c0 con 0x7efd00004800 2024-09-10T11:31:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:31:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 msgr2=0x7efd0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 0x7efd0406e2f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd0c007fd0 tx=0x7efd0c009030 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 msgr2=0x7efd1815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e870 tx=0x7efd0800ed30 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 shutdown_connections 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efd0406be30 0x7efd0406e2f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd18146740 0x7efd180afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd18144940 0x7efd18146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 --2- 172.21.15.99:0/3444736513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efd00004800 0x7efd1815e510 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.501+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 >> 172.21.15.99:0/3444736513 conn(0x7efd180b3630 msgr2=0x7efd180b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.502+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 shutdown_connections 2024-09-10T11:31:13.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.502+0000 7efd1ec5c640 1 -- 172.21.15.99:0/3444736513 wait complete. 2024-09-10T11:31:13.521 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:31:13.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.622+0000 7fc468c7b640 1 Processor -- start 2024-09-10T11:31:13.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.623+0000 7fc468c7b640 1 -- start start 2024-09-10T11:31:13.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.623+0000 7fc468c7b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc4641626c0 0x7fc4640d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.623+0000 7fc468c7b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4640d4e40 0x7fc4640d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.623+0000 7fc468c7b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc46415ff70 con 0x7fc46415eb80 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc468c7b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc4640d30c0 con 0x7fc4641626c0 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc468c7b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc4640d1620 con 0x7fc4640d4e40 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc461d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc4641626c0 0x7fc4640d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc46259c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc46415eb80 0x7fc4640d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50046/0 (socket says 172.21.15.99:50046) 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc46259c640 1 -- 172.21.15.99:0/1814917553 learned_addr learned my addr 172.21.15.99:0/1814917553 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc462d9d640 1 --2- 172.21.15.99:0/1814917553 >> v2:172.21.15.196:3300/0 conn(0x7fc4640d4e40 0x7fc4640d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc461d9b640 1 -- 172.21.15.99:0/1814917553 >> v2:172.21.15.196:3300/0 conn(0x7fc4640d4e40 msgr2=0x7fc4640d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc461d9b640 1 --2- 172.21.15.99:0/1814917553 >> v2:172.21.15.196:3300/0 conn(0x7fc4640d4e40 0x7fc4640d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc461d9b640 1 -- 172.21.15.99:0/1814917553 >> v1:172.21.15.99:6789/0 conn(0x7fc46415eb80 legacy=0x7fc4640d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc461d9b640 1 -- 172.21.15.99:0/1814917553 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc464162bc0 con 0x7fc4641626c0 2024-09-10T11:31:13.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.624+0000 7fc46159a640 1 -- 172.21.15.99:0/1814917553 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090246585 0 0) 0x7fc46415ff70 con 0x7fc46415eb80 2024-09-10T11:31:13.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc461d9b640 1 --2- 172.21.15.99:0/1814917553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4641626c0 0x7fc4640d4900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0097a0 tx=0x7fc44c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab84ca98881156b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:13.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc46159a640 1 -- 172.21.15.99:0/1814917553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c037070 con 0x7fc4641626c0 2024-09-10T11:31:13.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc46159a640 1 -- 172.21.15.99:0/1814917553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c02ae50 con 0x7fc4641626c0 2024-09-10T11:31:13.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc46159a640 1 -- 172.21.15.99:0/1814917553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c032660 con 0x7fc4641626c0 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 -- 172.21.15.99:0/1814917553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4641626c0 msgr2=0x7fc4640d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 --2- 172.21.15.99:0/1814917553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4641626c0 0x7fc4640d4900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0097a0 tx=0x7fc44c02a140 comp rx=0 tx=0).stop 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 -- 172.21.15.99:0/1814917553 shutdown_connections 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 --2- 172.21.15.99:0/1814917553 >> v2:172.21.15.196:3300/0 conn(0x7fc4640d4e40 0x7fc4640d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 --2- 172.21.15.99:0/1814917553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4641626c0 0x7fc4640d4900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 -- 172.21.15.99:0/1814917553 >> 172.21.15.99:0/1814917553 conn(0x7fc46408e680 msgr2=0x7fc46409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 -- 172.21.15.99:0/1814917553 shutdown_connections 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.625+0000 7fc468c7b640 1 -- 172.21.15.99:0/1814917553 wait complete. 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 Processor -- start 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 -- start start 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4640d4e40 0x7fc46414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4641626c0 0x7fc4640b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc464163460 con 0x7fc46415eb80 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc464163180 con 0x7fc4640d4e40 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.627+0000 7fc468c7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc4640d5290 con 0x7fc4641626c0 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36746/0 (socket says 172.21.15.99:36746) 2024-09-10T11:31:13.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 -- 172.21.15.99:0/3942636263 learned_addr learned my addr 172.21.15.99:0/3942636263 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc46259c640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4640d4e40 0x7fc46414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc462d9d640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4641626c0 0x7fc4640b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 -- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4641626c0 msgr2=0x7fc4640b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4641626c0 0x7fc4640b8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 -- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4640d4e40 msgr2=0x7fc46414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4640d4e40 0x7fc46414ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 -- 172.21.15.99:0/3942636263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4640b3bb0 con 0x7fc46415eb80 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc461d9b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0097a0 tx=0x7fc44c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c037040 con 0x7fc46415eb80 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c004030 con 0x7fc46415eb80 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46415b190 con 0x7fc46415eb80 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c004420 con 0x7fc46415eb80 2024-09-10T11:31:13.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.628+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46415b750 con 0x7fc46415eb80 2024-09-10T11:31:13.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.629+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4640d4900 con 0x7fc46415eb80 2024-09-10T11:31:13.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.630+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc44c003610 con 0x7fc46415eb80 2024-09-10T11:31:13.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.630+0000 7fc45affd640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:13.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.630+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7fc44c0abc00 con 0x7fc46415eb80 2024-09-10T11:31:13.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.630+0000 7fc46259c640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:13.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.631+0000 7fc46259c640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 0x7fc44406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc45c007ed0 tx=0x7fc45c007e60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:13.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.635+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc44c0b2080 con 0x7fc46415eb80 2024-09-10T11:31:13.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:13.832+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc46409e320 con 0x7fc46415eb80 2024-09-10T11:31:14.518 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T11:31:14.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.515+0000 7fc45affd640 1 -- 172.21.15.99:0/3942636263 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fc44c07a6a0 con 0x7fc46415eb80 2024-09-10T11:31:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.524+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 msgr2=0x7fc44406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.524+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 0x7fc44406e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc45c007ed0 tx=0x7fc45c007e60 comp rx=0 tx=0).stop 2024-09-10T11:31:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.524+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 msgr2=0x7fc4640b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.524+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0097a0 tx=0x7fc44c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 shutdown_connections 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc44406be30 0x7fc44406e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4641626c0 0x7fc4640b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc46415eb80 0x7fc4640b31d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 --2- 172.21.15.99:0/3942636263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4640d4e40 0x7fc46414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.525+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 >> 172.21.15.99:0/3942636263 conn(0x7fc46408e680 msgr2=0x7fc4640d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.526+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 shutdown_connections 2024-09-10T11:31:14.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.526+0000 7fc468c7b640 1 -- 172.21.15.99:0/3942636263 wait complete. 2024-09-10T11:31:14.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.639+0000 7fd38ace6640 1 Processor -- start 2024-09-10T11:31:14.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.639+0000 7fd38ace6640 1 -- start start 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd38ace6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd38406ece0 0x7fd384073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd38ace6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd38415c060 0x7fd38415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd38ace6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd384152e60 con 0x7fd3840d1200 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd38ace6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd3840d15e0 con 0x7fd38406ece0 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd38ace6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd384072680 con 0x7fd38415c060 2024-09-10T11:31:14.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd37bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd38406ece0 0x7fd384073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd37bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd38406ece0 0x7fd384073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36748/0 (socket says 172.21.15.99:36748) 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.640+0000 7fd37bfff640 1 -- 172.21.15.99:0/3221797516 learned_addr learned my addr 172.21.15.99:0/3221797516 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd389283640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd38415c060 0x7fd38415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37bfff640 1 -- 172.21.15.99:0/3221797516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd38415c060 msgr2=0x7fd38415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37bfff640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd38415c060 0x7fd38415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37bfff640 1 -- 172.21.15.99:0/3221797516 >> v1:172.21.15.196:6789/0 conn(0x7fd3840d1200 legacy=0x7fd38406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37bfff640 1 -- 172.21.15.99:0/3221797516 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd384072960 con 0x7fd38406ece0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37bfff640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38406ece0 0x7fd384073440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd374009870 tx=0x7fd37402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6f4ade5f7c32bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37b7fe640 1 -- 172.21.15.99:0/3221797516 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374037070 con 0x7fd38406ece0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37b7fe640 1 -- 172.21.15.99:0/3221797516 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd374032440 con 0x7fd38406ece0 2024-09-10T11:31:14.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.641+0000 7fd37b7fe640 1 -- 172.21.15.99:0/3221797516 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374032810 con 0x7fd38406ece0 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 -- 172.21.15.99:0/3221797516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38406ece0 msgr2=0x7fd384073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38406ece0 0x7fd384073440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd374009870 tx=0x7fd37402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 -- 172.21.15.99:0/3221797516 shutdown_connections 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd38415c060 0x7fd38415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3221797516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38406ece0 0x7fd384073440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 -- 172.21.15.99:0/3221797516 >> 172.21.15.99:0/3221797516 conn(0x7fd38406bc10 msgr2=0x7fd38406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 -- 172.21.15.99:0/3221797516 shutdown_connections 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.642+0000 7fd38ace6640 1 -- 172.21.15.99:0/3221797516 wait complete. 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.643+0000 7fd38ace6640 1 Processor -- start 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.643+0000 7fd38ace6640 1 -- start start 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38406ece0 0x7fd38413c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3840d1200 0x7fd384151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd38415cd90 con 0x7fd38415c060 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd384072f20 con 0x7fd38406ece0 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd38ace6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd38415ca00 con 0x7fd3840d1200 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd389283640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd389283640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36756/0 (socket says 172.21.15.99:36756) 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd389283640 1 -- 172.21.15.99:0/3385364664 learned_addr learned my addr 172.21.15.99:0/3385364664 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd37bfff640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3840d1200 0x7fd384151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.644+0000 7fd388a82640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38406ece0 0x7fd38413c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 -- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3840d1200 msgr2=0x7fd384151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3840d1200 0x7fd384151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 -- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38406ece0 msgr2=0x7fd38413c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38406ece0 0x7fd38413c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 -- 172.21.15.99:0/3385364664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd384168860 con 0x7fd38415c060 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd389283640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd37c007e40 tx=0x7fd37c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014620 con 0x7fd38415c060 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37c019070 con 0x7fd38415c060 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd384168b40 con 0x7fd38415c060 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c014a20 con 0x7fd38415c060 2024-09-10T11:31:14.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.645+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38415b370 con 0x7fd38415c060 2024-09-10T11:31:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.646+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd384073440 con 0x7fd38415c060 2024-09-10T11:31:14.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.646+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd37c014bc0 con 0x7fd38415c060 2024-09-10T11:31:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.647+0000 7fd3797fa640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 0x7fd37006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.647+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fd37c08b5e0 con 0x7fd38415c060 2024-09-10T11:31:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.647+0000 7fd388a82640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 0x7fd37006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.647+0000 7fd388a82640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 0x7fd37006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3800016e0 tx=0x7fd380009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.651+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd37c059940 con 0x7fd38415c060 2024-09-10T11:31:14.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.816+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd38406e240 con 0x7fd38415c060 2024-09-10T11:31:14.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.818+0000 7fd3797fa640 1 -- 172.21.15.99:0/3385364664 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7fd37c05dc40 con 0x7fd38415c060 2024-09-10T11:31:14.820 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:14.820 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":31,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:13.834132+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.819+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 msgr2=0x7fd37006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 0x7fd37006e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd3800016e0 tx=0x7fd380009040 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 msgr2=0x7fd3841523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd37c007e40 tx=0x7fd37c00e4b0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 shutdown_connections 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd37006be60 0x7fd37006e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd38415c060 0x7fd3841523a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3840d1200 0x7fd384151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 --2- 172.21.15.99:0/3385364664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd38406ece0 0x7fd38413c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 >> 172.21.15.99:0/3385364664 conn(0x7fd38406bc10 msgr2=0x7fd38406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 shutdown_connections 2024-09-10T11:31:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.820+0000 7fd38ace6640 1 -- 172.21.15.99:0/3385364664 wait complete. 2024-09-10T11:31:14.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:14.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.931+0000 7fa451d5f640 1 Processor -- start 2024-09-10T11:31:14.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.931+0000 7fa451d5f640 1 -- start start 2024-09-10T11:31:14.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa451d5f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa44c0d1200 0x7fa44c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa451d5f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa44c06e5d0 0x7fa44c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa451d5f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c152e60 con 0x7fa44c15c060 2024-09-10T11:31:14.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa451d5f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c06ef80 con 0x7fa44c06e5d0 2024-09-10T11:31:14.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa451d5f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c0d15e0 con 0x7fa44c0d1200 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa44c06e5d0 0x7fa44c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa44c06e5d0 0x7fa44c06ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36770/0 (socket says 172.21.15.99:36770) 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa44c15c060 0x7fa44c15c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50062/0 (socket says 172.21.15.99:50062) 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 -- 172.21.15.99:0/97497387 learned_addr learned my addr 172.21.15.99:0/97497387 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44b7fe640 1 --2- 172.21.15.99:0/97497387 >> v2:172.21.15.196:3300/0 conn(0x7fa44c0d1200 0x7fa44c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 -- 172.21.15.99:0/97497387 >> v2:172.21.15.196:3300/0 conn(0x7fa44c0d1200 msgr2=0x7fa44c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 --2- 172.21.15.99:0/97497387 >> v2:172.21.15.196:3300/0 conn(0x7fa44c0d1200 0x7fa44c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 -- 172.21.15.99:0/97497387 >> v1:172.21.15.99:6789/0 conn(0x7fa44c15c060 legacy=0x7fa44c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44a7fc640 1 -- 172.21.15.99:0/97497387 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821437407 0 0) 0x7fa44c152e60 con 0x7fa44c15c060 2024-09-10T11:31:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.932+0000 7fa44affd640 1 -- 172.21.15.99:0/97497387 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c072680 con 0x7fa44c06e5d0 2024-09-10T11:31:14.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.933+0000 7fa44affd640 1 --2- 172.21.15.99:0/97497387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c06e5d0 0x7fa44c06ea40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0d1850 tx=0x7fa44002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce4a71a5b9d49a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.933+0000 7fa44a7fc640 1 -- 172.21.15.99:0/97497387 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440037070 con 0x7fa44c06e5d0 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.933+0000 7fa44a7fc640 1 -- 172.21.15.99:0/97497387 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440032440 con 0x7fa44c06e5d0 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.933+0000 7fa44a7fc640 1 -- 172.21.15.99:0/97497387 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440032810 con 0x7fa44c06e5d0 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 -- 172.21.15.99:0/97497387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c06e5d0 msgr2=0x7fa44c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 --2- 172.21.15.99:0/97497387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c06e5d0 0x7fa44c06ea40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0d1850 tx=0x7fa44002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 -- 172.21.15.99:0/97497387 shutdown_connections 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 --2- 172.21.15.99:0/97497387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c06e5d0 0x7fa44c06ea40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 --2- 172.21.15.99:0/97497387 >> v2:172.21.15.196:3300/0 conn(0x7fa44c0d1200 0x7fa44c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 -- 172.21.15.99:0/97497387 >> 172.21.15.99:0/97497387 conn(0x7fa44c06bc10 msgr2=0x7fa44c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 -- 172.21.15.99:0/97497387 shutdown_connections 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.934+0000 7fa451d5f640 1 -- 172.21.15.99:0/97497387 wait complete. 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.935+0000 7fa451d5f640 1 Processor -- start 2024-09-10T11:31:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.935+0000 7fa451d5f640 1 -- start start 2024-09-10T11:31:14.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa44c06e5d0 0x7fa44c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa44c0d1200 0x7fa44c1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c072f20 con 0x7fa44c15c060 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c072c40 con 0x7fa44c0d1200 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c15cd90 con 0x7fa44c06e5d0 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36784/0 (socket says 172.21.15.99:36784) 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 -- 172.21.15.99:0/4259913731 learned_addr learned my addr 172.21.15.99:0/4259913731 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44b7fe640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa44c06e5d0 0x7fa44c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44affd640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa44c0d1200 0x7fa44c1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 -- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa44c06e5d0 msgr2=0x7fa44c0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa44c06e5d0 0x7fa44c0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 -- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa44c0d1200 msgr2=0x7fa44c1522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa44c0d1200 0x7fa44c1522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 -- 172.21.15.99:0/4259913731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0c7bb0 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa44bfff640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c7a0 tx=0x7fa44400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444018070 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa44400a640 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c0c7e90 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44400aab0 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.936+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c0c8450 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.937+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c06ea40 con 0x7fa44c15c060 2024-09-10T11:31:14.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.938+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa44401b070 con 0x7fa44c15c060 2024-09-10T11:31:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.938+0000 7fa433fff640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 0x7fa43806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.938+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fa44408a330 con 0x7fa44c15c060 2024-09-10T11:31:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.938+0000 7fa44b7fe640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 0x7fa43806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.939+0000 7fa44b7fe640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 0x7fa43806e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0d00e0 tx=0x7fa43c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:14.943+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa444058720 con 0x7fa44c15c060 2024-09-10T11:31:15.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.109+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa44c152830 con 0x7fa44c15c060 2024-09-10T11:31:15.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.110+0000 7fa433fff640 1 -- 172.21.15.99:0/4259913731 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16682 (secure 0 0 0) 0x7fa44405ca20 con 0x7fa44c15c060 2024-09-10T11:31:15.112 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:15.112 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":31,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:13.834132+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:15.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 msgr2=0x7fa43806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 0x7fa43806e320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0d00e0 tx=0x7fa43c008040 comp rx=0 tx=0).stop 2024-09-10T11:31:15.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 msgr2=0x7fa44c0c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c7a0 tx=0x7fa44400cc60 comp rx=0 tx=0).stop 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 shutdown_connections 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa43806be60 0x7fa43806e320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa44c15c060 0x7fa44c0c9e50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa44c0d1200 0x7fa44c1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 --2- 172.21.15.99:0/4259913731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa44c06e5d0 0x7fa44c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 >> 172.21.15.99:0/4259913731 conn(0x7fa44c06bc10 msgr2=0x7fa44c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 shutdown_connections 2024-09-10T11:31:15.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.112+0000 7fa451d5f640 1 -- 172.21.15.99:0/4259913731 wait complete. 2024-09-10T11:31:15.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:31:15.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.223+0000 7fc1ca2ed640 1 Processor -- start 2024-09-10T11:31:15.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.223+0000 7fc1ca2ed640 1 -- start start 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1ca2ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 0x7fc1c4163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1ca2ed640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1c40b4930 0x7fc1c406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1ca2ed640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1c40b5140 con 0x7fc1c4160030 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1ca2ed640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c40b5420 con 0x7fc1c40b4930 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1ca2ed640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc1c4162b40 con 0x7fc1c406bc10 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1c888a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc1c40b4930 0x7fc1c406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1c37fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc1c4160030 0x7fc1c406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50074/0 (socket says 172.21.15.99:50074) 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.224+0000 7fc1c37fe640 1 -- 172.21.15.99:0/659227129 learned_addr learned my addr 172.21.15.99:0/659227129 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c888a640 1 -- 172.21.15.99:0/659227129 >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 msgr2=0x7fc1c4163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c3fff640 1 --2- 172.21.15.99:0/659227129 >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 0x7fc1c4163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c888a640 1 --2- 172.21.15.99:0/659227129 >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 0x7fc1c4163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c2ffd640 1 -- 172.21.15.99:0/659227129 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2652558843 0 0) 0x7fc1c40b5140 con 0x7fc1c4160030 2024-09-10T11:31:15.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c888a640 1 -- 172.21.15.99:0/659227129 >> v1:172.21.15.99:6789/0 conn(0x7fc1c4160030 legacy=0x7fc1c406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c888a640 1 -- 172.21.15.99:0/659227129 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c4162e20 con 0x7fc1c40b4930 2024-09-10T11:31:15.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c3fff640 1 --2- 172.21.15.99:0/659227129 >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 0x7fc1c4163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:15.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.225+0000 7fc1c888a640 1 --2- 172.21.15.99:0/659227129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c40b4930 0x7fc1c406e5d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4073870 tx=0x7fc1bc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59faac2f48c90fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1c2ffd640 1 -- 172.21.15.99:0/659227129 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007740 con 0x7fc1c40b4930 2024-09-10T11:31:15.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1c2ffd640 1 -- 172.21.15.99:0/659227129 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc0078e0 con 0x7fc1c40b4930 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1c2ffd640 1 -- 172.21.15.99:0/659227129 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007cb0 con 0x7fc1c40b4930 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/659227129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c40b4930 msgr2=0x7fc1c406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/659227129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c40b4930 0x7fc1c406e5d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4073870 tx=0x7fc1bc02a5f0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/659227129 shutdown_connections 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/659227129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c40b4930 0x7fc1c406e5d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/659227129 >> v2:172.21.15.196:3300/0 conn(0x7fc1c406bc10 0x7fc1c4163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/659227129 >> 172.21.15.99:0/659227129 conn(0x7fc1c40b3630 msgr2=0x7fc1c40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/659227129 shutdown_connections 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.226+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/659227129 wait complete. 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.227+0000 7fc1ca2ed640 1 Processor -- start 2024-09-10T11:31:15.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.227+0000 7fc1ca2ed640 1 -- start start 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1c406bc10 0x7fc1c4146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1c40b4930 0x7fc1c415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40d1560 con 0x7fc1c4160030 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c40b4dc0 con 0x7fc1c40b4930 2024-09-10T11:31:15.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1ca2ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c40d16e0 con 0x7fc1c406bc10 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1c888a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1c888a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36800/0 (socket says 172.21.15.99:36800) 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.228+0000 7fc1c888a640 1 -- 172.21.15.99:0/2974012627 learned_addr learned my addr 172.21.15.99:0/2974012627 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c37fe640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1c40b4930 0x7fc1c415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c3fff640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1c406bc10 0x7fc1c4146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 -- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1c406bc10 msgr2=0x7fc1c4146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1c406bc10 0x7fc1c4146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 -- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1c40b4930 msgr2=0x7fc1c415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1c40b4930 0x7fc1c415b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 -- 172.21.15.99:0/2974012627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c41f49a0 con 0x7fc1c4160030 2024-09-10T11:31:15.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c888a640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d60 tx=0x7fc1bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc03f070 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc031c90 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c415c890 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc043630 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.229+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c415ce50 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.230+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c40b5140 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.230+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1bc03b410 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.231+0000 7fc1c0ff9640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.231+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fc1bc0ac380 con 0x7fc1c4160030 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.231+0000 7fc1c3fff640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.231+0000 7fc1c3fff640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b840 tx=0x7fc1b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.235+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc1bc07ae80 con 0x7fc1c4160030 2024-09-10T11:31:15.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.432+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc1c4073d40 con 0x7fc1c4160030 2024-09-10T11:31:15.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.518+0000 7fc1c0ff9640 1 -- 172.21.15.99:0/2974012627 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7fc1bc07a780 con 0x7fc1c4160030 2024-09-10T11:31:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 msgr2=0x7fc1b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 0x7fc1b006e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400b840 tx=0x7fc1b4008040 comp rx=0 tx=0).stop 2024-09-10T11:31:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 msgr2=0x7fc1c40725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d60 tx=0x7fc1bc004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:15.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 shutdown_connections 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1b006be30 0x7fc1b006e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1c4160030 0x7fc1c40725b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1c40b4930 0x7fc1c415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 --2- 172.21.15.99:0/2974012627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1c406bc10 0x7fc1c4146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 >> 172.21.15.99:0/2974012627 conn(0x7fc1c40b3630 msgr2=0x7fc1c4148390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:15.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 shutdown_connections 2024-09-10T11:31:15.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.520+0000 7fc1ca2ed640 1 -- 172.21.15.99:0/2974012627 wait complete. 2024-09-10T11:31:15.533 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:31:15.533 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:31:15.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.633+0000 7f2949e82640 1 Processor -- start 2024-09-10T11:31:15.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.633+0000 7f2949e82640 1 -- start start 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.633+0000 7f2949e82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f294415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.633+0000 7f2949e82640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2944152e60 con 0x7f29440d1200 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.633+0000 7f2949e82640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29440d15e0 con 0x7f294406ece0 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f2949e82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2944072680 con 0x7f2944073440 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29437fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f29440d1200 0x7f294406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50090/0 (socket says 172.21.15.99:50090) 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29437fe640 1 -- 172.21.15.99:0/879048227 learned_addr learned my addr 172.21.15.99:0/879048227 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854042258 0 0) 0x7f2944152e60 con 0x7f29440d1200 2024-09-10T11:31:15.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f292c003520 con 0x7f29440d1200 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f2943fff640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f294415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 351474858 0 0) 0x7f292c003520 con 0x7f29440d1200 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 msgr2=0x7f294415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f294415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 >> v1:172.21.15.196:6789/0 conn(0x7f294406ece0 legacy=0x7f294415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2944072960 con 0x7f29440d1200 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f2934001670 con 0x7f29440d1200 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 >> v1:172.21.15.99:6789/0 conn(0x7f29440d1200 legacy=0x7f294406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f292c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f292c004010 0x7f292c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f292c004d40 0x7f292c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2944072680 con 0x7f292c004d40 2024-09-10T11:31:15.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f292c00bc00 con 0x7f292c004010 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f292c00be00 con 0x7f2944073440 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f2943fff640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f292c004d40 0x7f292c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f29437fe640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f292c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.634+0000 7f2942ffd640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f292c004010 0x7f292c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 -- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 msgr2=0x7f292c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2944073440 0x7f292c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 -- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f292c004010 msgr2=0x7f292c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f292c004010 0x7f292c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 -- 172.21.15.99:0/879048227 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c003520 con 0x7f292c004d40 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f2943fff640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f292c004d40 0x7f292c0051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f293c007360 tx=0x7f292c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409284e4245c5ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c009be0 con 0x7f292c004d40 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293c036600 con 0x7f292c004d40 2024-09-10T11:31:15.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.635+0000 7f29427fc640 1 -- 172.21.15.99:0/879048227 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0369d0 con 0x7f292c004d40 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 -- 172.21.15.99:0/879048227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f292c004d40 msgr2=0x7f292c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 --2- 172.21.15.99:0/879048227 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f292c004d40 0x7f292c0051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f293c007360 tx=0x7f292c005830 comp rx=0 tx=0).stop 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2943fff640 1 -- 172.21.15.99:0/879048227 reap_dead start 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 -- 172.21.15.99:0/879048227 shutdown_connections 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 -- 172.21.15.99:0/879048227 >> 172.21.15.99:0/879048227 conn(0x7f294406bc10 msgr2=0x7f294406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 -- 172.21.15.99:0/879048227 shutdown_connections 2024-09-10T11:31:15.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.636+0000 7f2949e82640 1 -- 172.21.15.99:0/879048227 wait complete. 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.637+0000 7f2949e82640 1 Processor -- start 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.637+0000 7f2949e82640 1 -- start start 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 0x7f29441e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 0x7f29440ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f294415cd90 con 0x7f29441422f0 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2944072f20 con 0x7f29441e71e0 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2949e82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f294415c770 con 0x7f29440add60 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36816/0 (socket says 172.21.15.99:36816) 2024-09-10T11:31:15.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 -- 172.21.15.99:0/4218599608 learned_addr learned my addr 172.21.15.99:0/4218599608 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2943fff640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 0x7f29440ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2942ffd640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 0x7f29441e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 -- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 msgr2=0x7f29440ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 0x7f29440ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 -- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 msgr2=0x7f29441e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 0x7f29441e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f29437fe640 1 -- 172.21.15.99:0/4218599608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29441426d0 con 0x7f29441422f0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2943fff640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 0x7f29440ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.638+0000 7f2942ffd640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 0x7f29441e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f29437fe640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2934007320 tx=0x7f29340077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293401a070 con 0x7f29441422f0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293400f960 con 0x7f29441422f0 2024-09-10T11:31:15.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29440aeab0 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293400fd50 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.639+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29440af070 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.640+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29441e7ba0 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.641+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f293401d070 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.641+0000 7f292bfff640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 0x7f293006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.641+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f293408a8a0 con 0x7f29441422f0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.641+0000 7f2942ffd640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 0x7f293006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:15.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.642+0000 7f2942ffd640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 0x7f293006e320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29441e7e70 tx=0x7f2938009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:15.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.646+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2934059250 con 0x7f29441422f0 2024-09-10T11:31:15.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:15.846+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f29440b22d0 con 0x7f29441422f0 2024-09-10T11:31:16.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.519+0000 7f292bfff640 1 -- 172.21.15.99:0/4218599608 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f2934058b50 con 0x7f29441422f0 2024-09-10T11:31:16.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 msgr2=0x7f293006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 0x7f293006e320 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29441e7e70 tx=0x7f2938009040 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 msgr2=0x7f29440afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2934007320 tx=0x7f29340077e0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 shutdown_connections 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f293006be60 0x7f293006e320 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.521+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29440add60 0x7f29440ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.522+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29441e71e0 0x7f29441e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.522+0000 7f2949e82640 1 --2- 172.21.15.99:0/4218599608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29441422f0 0x7f29440afb50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.522+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 >> 172.21.15.99:0/4218599608 conn(0x7f294406bc10 msgr2=0x7f2944151d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.522+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 shutdown_connections 2024-09-10T11:31:16.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.522+0000 7f2949e82640 1 -- 172.21.15.99:0/4218599608 wait complete. 2024-09-10T11:31:16.536 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:31:16.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.634+0000 7f3cfb59e640 1 Processor -- start 2024-09-10T11:31:16.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.634+0000 7f3cfb59e640 1 -- start start 2024-09-10T11:31:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3cfc0cfdf0 0x7f3cfc0c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfb59e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc0d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfb59e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cfc0d0d90 con 0x7f3cfc15c990 2024-09-10T11:31:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfb59e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3cfc0d1070 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfb59e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc0d1350 con 0x7f3cfc0d2790 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf9d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3cfc0cfdf0 0x7f3cfc0c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfa59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3cfc15c990 0x7f3cfc0d1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50094/0 (socket says 172.21.15.99:50094) 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cfa59c640 1 -- 172.21.15.99:0/2430346409 learned_addr learned my addr 172.21.15.99:0/2430346409 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf9d9b640 1 -- 172.21.15.99:0/2430346409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 msgr2=0x7f3cfc0d2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf9d9b640 1 --2- 172.21.15.99:0/2430346409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc0d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf9d9b640 1 -- 172.21.15.99:0/2430346409 >> v1:172.21.15.99:6789/0 conn(0x7f3cfc15c990 legacy=0x7f3cfc0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf959a640 1 -- 172.21.15.99:0/2430346409 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258083173 0 0) 0x7f3cfc0d0d90 con 0x7f3cfc15c990 2024-09-10T11:31:16.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.635+0000 7f3cf9d9b640 1 -- 172.21.15.99:0/2430346409 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc0c8760 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.636+0000 7f3cf9d9b640 1 --2- 172.21.15.99:0/2430346409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc0c8220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00097a0 tx=0x7f3cf002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f520e2e0af9e859e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.636+0000 7f3cf959a640 1 -- 172.21.15.99:0/2430346409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0037070 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.636+0000 7f3cf959a640 1 -- 172.21.15.99:0/2430346409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf0032440 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.636+0000 7f3cf959a640 1 -- 172.21.15.99:0/2430346409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0032810 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 -- 172.21.15.99:0/2430346409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 msgr2=0x7f3cfc0c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/2430346409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc0c8220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00097a0 tx=0x7f3cf002a180 comp rx=0 tx=0).stop 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 -- 172.21.15.99:0/2430346409 shutdown_connections 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/2430346409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc0d2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/2430346409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc0c8220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 -- 172.21.15.99:0/2430346409 >> 172.21.15.99:0/2430346409 conn(0x7f3cfc09ba30 msgr2=0x7f3cfc09be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 -- 172.21.15.99:0/2430346409 shutdown_connections 2024-09-10T11:31:16.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.637+0000 7f3cfb59e640 1 -- 172.21.15.99:0/2430346409 wait complete. 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.638+0000 7f3cfb59e640 1 Processor -- start 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.638+0000 7f3cfb59e640 1 -- start start 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 0x7f3cfc1f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc0b2b90 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cfc0b2a10 con 0x7f3cfc15c990 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfb59e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3cfc0b2d10 con 0x7f3cfc0d2790 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36832/0 (socket says 172.21.15.99:36832) 2024-09-10T11:31:16.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 -- 172.21.15.99:0/228338636 learned_addr learned my addr 172.21.15.99:0/228338636 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cf9d9b640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfad9d640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 0x7f3cfc1f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 -- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 msgr2=0x7f3cfc06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc06dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 -- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 msgr2=0x7f3cfc1f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 0x7f3cfc1f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfa59c640 1 -- 172.21.15.99:0/228338636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cfc06e500 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.639+0000 7f3cfad9d640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 0x7f3cfc1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3cfa59c640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c430 tx=0x7f3cec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec0074e0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cec0047e0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cfc06e7e0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec004bd0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.640+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cfc0b0cd0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.641+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cfc0d0d90 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.641+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cec011070 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.642+0000 7f3ceaffd640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 0x7f3cdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.642+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3cec08aaf0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.642+0000 7f3cf9d9b640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 0x7f3cdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.642+0000 7f3cf9d9b640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 0x7f3cdc06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0034000 tx=0x7f3cf002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.646+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3cec0594a0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.845+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cfc0b07a0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.846+0000 7f3ceaffd640 1 -- 172.21.15.99:0/228338636 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f3cec058da0 con 0x7f3cfc0cfdf0 2024-09-10T11:31:16.848 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:16.848 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 msgr2=0x7f3cdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 0x7f3cdc06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0034000 tx=0x7f3cf002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 msgr2=0x7f3cfc155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00c430 tx=0x7f3cec00c8f0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 shutdown_connections 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3cdc06be30 0x7f3cdc06e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cfc15c990 0x7f3cfc1f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cfc0d2790 0x7f3cfc06dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 --2- 172.21.15.99:0/228338636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cfc0cfdf0 0x7f3cfc155490 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 >> 172.21.15.99:0/228338636 conn(0x7f3cfc09ba30 msgr2=0x7f3cfc06f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 shutdown_connections 2024-09-10T11:31:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.848+0000 7f3cfb59e640 1 -- 172.21.15.99:0/228338636 wait complete. 2024-09-10T11:31:16.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:16.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.958+0000 7f21e976d640 1 Processor -- start 2024-09-10T11:31:16.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.958+0000 7f21e976d640 1 -- start start 2024-09-10T11:31:16.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e976d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f21e415b2d0 0x7f21e40b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e976d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 0x7f21e40714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e976d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21e40b0020 con 0x7f21e415aef0 2024-09-10T11:31:16.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e976d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f21e406ee10 con 0x7f21e415b2d0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e976d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f21e40cd9a0 con 0x7f21e40afbb0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e27fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f21e415b2d0 0x7f21e40b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e27fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f21e415b2d0 0x7f21e40b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36834/0 (socket says 172.21.15.99:36834) 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.959+0000 7f21e27fc640 1 -- 172.21.15.99:0/1929911307 learned_addr learned my addr 172.21.15.99:0/1929911307 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e27fc640 1 -- 172.21.15.99:0/1929911307 >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 msgr2=0x7f21e40714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e37fe640 1 --2- 172.21.15.99:0/1929911307 >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 0x7f21e40714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e27fc640 1 --2- 172.21.15.99:0/1929911307 >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 0x7f21e40714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e27fc640 1 -- 172.21.15.99:0/1929911307 >> v1:172.21.15.196:6789/0 conn(0x7f21e415aef0 legacy=0x7f21e40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e27fc640 1 -- 172.21.15.99:0/1929911307 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e40b0290 con 0x7f21e415b2d0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e37fe640 1 --2- 172.21.15.99:0/1929911307 >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 0x7f21e40714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e27fc640 1 --2- 172.21.15.99:0/1929911307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e40b4510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009870 tx=0x7f21d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a066f1cb0e5e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e1ffb640 1 -- 172.21.15.99:0/1929911307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8037070 con 0x7f21e415b2d0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e1ffb640 1 -- 172.21.15.99:0/1929911307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d8032440 con 0x7f21e415b2d0 2024-09-10T11:31:16.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.960+0000 7f21e1ffb640 1 -- 172.21.15.99:0/1929911307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8032810 con 0x7f21e415b2d0 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 -- 172.21.15.99:0/1929911307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 msgr2=0x7f21e40b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 --2- 172.21.15.99:0/1929911307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e40b4510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009870 tx=0x7f21d802a250 comp rx=0 tx=0).stop 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 -- 172.21.15.99:0/1929911307 shutdown_connections 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 --2- 172.21.15.99:0/1929911307 >> v2:172.21.15.196:3300/0 conn(0x7f21e40afbb0 0x7f21e40714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 --2- 172.21.15.99:0/1929911307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e40b4510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 -- 172.21.15.99:0/1929911307 >> 172.21.15.99:0/1929911307 conn(0x7f21e406bc10 msgr2=0x7f21e406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 -- 172.21.15.99:0/1929911307 shutdown_connections 2024-09-10T11:31:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.961+0000 7f21e976d640 1 -- 172.21.15.99:0/1929911307 wait complete. 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.962+0000 7f21e976d640 1 Processor -- start 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.962+0000 7f21e976d640 1 -- start start 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 0x7f21e4158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21e415aef0 0x7f21e406d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21e406eb30 con 0x7f21e415b2d0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21e406e850 con 0x7f21e40afbb0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e976d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f21e40719e0 con 0x7f21e415aef0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36848/0 (socket says 172.21.15.99:36848) 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 -- 172.21.15.99:0/3479909612 learned_addr learned my addr 172.21.15.99:0/3479909612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e2ffd640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 0x7f21e4158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e27fc640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21e415aef0 0x7f21e406d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 -- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21e415aef0 msgr2=0x7f21e406d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21e415aef0 0x7f21e406d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 -- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 msgr2=0x7f21e4158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 0x7f21e4158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e37fe640 1 -- 172.21.15.99:0/3479909612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e4149840 con 0x7f21e415b2d0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.963+0000 7f21e2ffd640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 0x7f21e4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21e37fe640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00cef0 tx=0x7f21dc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc01a070 con 0x7f21e415b2d0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21dc00d860 con 0x7f21e415b2d0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e4149ae0 con 0x7f21e415b2d0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00dc50 con 0x7f21e415b2d0 2024-09-10T11:31:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.964+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e414a3c0 con 0x7f21e415b2d0 2024-09-10T11:31:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.965+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e414b8f0 con 0x7f21e415b2d0 2024-09-10T11:31:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.965+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21dc01d070 con 0x7f21e415b2d0 2024-09-10T11:31:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.965+0000 7f21cb7fe640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 0x7f21d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.966+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f21dc08b650 con 0x7f21e415b2d0 2024-09-10T11:31:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.966+0000 7f21e2ffd640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 0x7f21d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:16.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.966+0000 7f21e2ffd640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 0x7f21d006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21e406e130 tx=0x7f21d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:16.970+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f21dc05a150 con 0x7f21e415b2d0 2024-09-10T11:31:17.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.136+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21e4071d70 con 0x7f21e415b2d0 2024-09-10T11:31:17.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.137+0000 7f21cb7fe640 1 -- 172.21.15.99:0/3479909612 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16669 (secure 0 0 0) 0x7f21dc059a50 con 0x7f21e415b2d0 2024-09-10T11:31:17.139 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:17.140 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":32,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:15.502561+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:17.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 msgr2=0x7f21d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 0x7f21d006e320 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21e406e130 tx=0x7f21d4009040 comp rx=0 tx=0).stop 2024-09-10T11:31:17.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 msgr2=0x7f21e414cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f21dc00cef0 tx=0x7f21dc00a710 comp rx=0 tx=0).stop 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 shutdown_connections 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f21d006be60 0x7f21d006e320 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21e415b2d0 0x7f21e414cf40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21e415aef0 0x7f21e406d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 --2- 172.21.15.99:0/3479909612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21e40afbb0 0x7f21e4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.139+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 >> 172.21.15.99:0/3479909612 conn(0x7f21e406bc10 msgr2=0x7f21e406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.140+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 shutdown_connections 2024-09-10T11:31:17.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.140+0000 7f21e976d640 1 -- 172.21.15.99:0/3479909612 wait complete. 2024-09-10T11:31:17.153 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:31:17.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.249+0000 7f9bfc423640 1 Processor -- start 2024-09-10T11:31:17.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.249+0000 7f9bfc423640 1 -- start start 2024-09-10T11:31:17.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfc423640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9bf40d1200 0x7f9bf4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfc423640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 0x7f9bf406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfc423640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf4152e60 con 0x7f9bf406e5d0 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfc423640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf406ef80 con 0x7f9bf40d1200 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfc423640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf40d15e0 con 0x7f9bf415c770 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bf99be640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9bf406e5d0 0x7f9bf415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50098/0 (socket says 172.21.15.99:50098) 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bf99be640 1 -- 172.21.15.99:0/3295274006 learned_addr learned my addr 172.21.15.99:0/3295274006 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/3295274006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 0x7f9bf406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bfa1bf640 1 --2- 172.21.15.99:0/3295274006 >> v2:172.21.15.196:3300/0 conn(0x7f9bf40d1200 0x7f9bf4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.250+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3074165025 0 0) 0x7f9bf4152e60 con 0x7f9bf406e5d0 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be0003520 con 0x7f9bf406e5d0 2024-09-10T11:31:17.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/3295274006 >> v2:172.21.15.196:3300/0 conn(0x7f9bf40d1200 msgr2=0x7f9bf4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2275369689 0 0) 0x7f9be0003520 con 0x7f9bf406e5d0 2024-09-10T11:31:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/3295274006 >> v2:172.21.15.196:3300/0 conn(0x7f9bf40d1200 0x7f9bf4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/3295274006 >> v1:172.21.15.99:6789/0 conn(0x7f9bf406e5d0 legacy=0x7f9bf415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/3295274006 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf4072680 con 0x7f9bf415c770 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.251+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/3295274006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 0x7f9bf406ea40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7af826ac87bf212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be4001670 con 0x7f9bf415c770 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec007540 con 0x7f9bf415c770 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bf91bd640 1 -- 172.21.15.99:0/3295274006 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0078f0 con 0x7f9bf415c770 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 -- 172.21.15.99:0/3295274006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 msgr2=0x7f9bf406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 --2- 172.21.15.99:0/3295274006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 0x7f9bf406ea40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009d90 tx=0x7f9bec02a560 comp rx=0 tx=0).stop 2024-09-10T11:31:17.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 -- 172.21.15.99:0/3295274006 shutdown_connections 2024-09-10T11:31:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 --2- 172.21.15.99:0/3295274006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf415c770 0x7f9bf406ea40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 --2- 172.21.15.99:0/3295274006 >> v2:172.21.15.196:3300/0 conn(0x7f9bf40d1200 0x7f9bf4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 -- 172.21.15.99:0/3295274006 >> 172.21.15.99:0/3295274006 conn(0x7f9bf406bc10 msgr2=0x7f9bf406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 -- 172.21.15.99:0/3295274006 shutdown_connections 2024-09-10T11:31:17.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.252+0000 7f9bfc423640 1 -- 172.21.15.99:0/3295274006 wait complete. 2024-09-10T11:31:17.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.253+0000 7f9bfc423640 1 Processor -- start 2024-09-10T11:31:17.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 -- start start 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf406e5d0 0x7f9bf4143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 0x7f9bf406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4072f20 con 0x7f9bf415c770 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf4072c40 con 0x7f9bf406e5d0 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfc423640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf415cd90 con 0x7f9bf40d1200 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfa9c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfa9c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36858/0 (socket says 172.21.15.99:36858) 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/1530918152 learned_addr learned my addr 172.21.15.99:0/1530918152 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bfa1bf640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf406e5d0 0x7f9bf4143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.254+0000 7f9bf99be640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 0x7f9bf406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 msgr2=0x7f9bf406dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 0x7f9bf406dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf406e5d0 msgr2=0x7f9bf4143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf406e5d0 0x7f9bf4143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 -- 172.21.15.99:0/1530918152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf41589f0 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bf99be640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 0x7f9bf406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfa9c0640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9bec02ab50 tx=0x7f9bec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec009450 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec004240 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf4158cd0 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0045f0 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.255+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf4159290 con 0x7f9bf415c770 2024-09-10T11:31:17.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.256+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf4073440 con 0x7f9bf415c770 2024-09-10T11:31:17.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.256+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bec03b410 con 0x7f9bf415c770 2024-09-10T11:31:17.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.257+0000 7f9beaffd640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.257+0000 7f9bfa1bf640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.257+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9bec0ac0c0 con 0x7f9bf415c770 2024-09-10T11:31:17.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.257+0000 7f9bfa1bf640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 0x7f9bd806e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009aa0 tx=0x7f9bf0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.261+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9bec03f030 con 0x7f9bf415c770 2024-09-10T11:31:17.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.423+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bf40b43b0 con 0x7f9bd806be60 2024-09-10T11:31:17.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.429+0000 7f9beaffd640 1 -- 172.21.15.99:0/1530918152 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247489 (secure 0 0 0) 0x7f9bf40b43b0 con 0x7f9bd806be60 2024-09-10T11:31:17.435 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:17.436 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.432+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 msgr2=0x7f9bd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.432+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 0x7f9bd806e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009aa0 tx=0x7f9bf0009190 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.432+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 msgr2=0x7f9bf4154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.432+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9bec02ab50 tx=0x7f9bec004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 shutdown_connections 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9bd806be60 0x7f9bd806e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bf415c770 0x7f9bf4154ab0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9bf40d1200 0x7f9bf406dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 --2- 172.21.15.99:0/1530918152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9bf406e5d0 0x7f9bf4143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 >> 172.21.15.99:0/1530918152 conn(0x7f9bf406bc10 msgr2=0x7f9bf41531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 shutdown_connections 2024-09-10T11:31:17.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.433+0000 7f9bfc423640 1 -- 172.21.15.99:0/1530918152 wait complete. 2024-09-10T11:31:17.445 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T11:31:16.920644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560254,"statfs":{"total":1600000000,"available":1597700132,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.424289"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063890+0000","last_change":"2024-09-10T11:31:14.520544+0000","last_active":"2024-09-10T11:31:16.063890+0000","last_peered":"2024-09-10T11:31:16.063890+0000","last_clean":"2024-09-10T11:31:16.063890+0000","last_became_active":"2024-09-10T11:31:14.520291+0000","last_became_peered":"2024-09-10T11:31:14.520291+0000","last_unstale":"2024-09-10T11:31:16.063890+0000","last_undegraded":"2024-09-10T11:31:16.063890+0000","last_fullsized":"2024-09-10T11:31:16.063890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:05.389775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513088+0000","last_change":"2024-09-10T11:31:14.515821+0000","last_active":"2024-09-10T11:31:15.513088+0000","last_peered":"2024-09-10T11:31:15.513088+0000","last_clean":"2024-09-10T11:31:15.513088+0000","last_became_active":"2024-09-10T11:31:14.515561+0000","last_became_peered":"2024-09-10T11:31:14.515561+0000","last_unstale":"2024-09-10T11:31:15.513088+0000","last_undegraded":"2024-09-10T11:31:15.513088+0000","last_fullsized":"2024-09-10T11:31:15.513088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:43.320611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063885+0000","last_change":"2024-09-10T11:31:14.520236+0000","last_active":"2024-09-10T11:31:16.063885+0000","last_peered":"2024-09-10T11:31:16.063885+0000","last_clean":"2024-09-10T11:31:16.063885+0000","last_became_active":"2024-09-10T11:31:14.514850+0000","last_became_peered":"2024-09-10T11:31:14.514850+0000","last_unstale":"2024-09-10T11:31:16.063885+0000","last_undegraded":"2024-09-10T11:31:16.063885+0000","last_fullsized":"2024-09-10T11:31:16.063885+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:59.636401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063766+0000","last_change":"2024-09-10T11:31:14.521746+0000","last_active":"2024-09-10T11:31:16.063766+0000","last_peered":"2024-09-10T11:31:16.063766+0000","last_clean":"2024-09-10T11:31:16.063766+0000","last_became_active":"2024-09-10T11:31:14.519927+0000","last_became_peered":"2024-09-10T11:31:14.519927+0000","last_unstale":"2024-09-10T11:31:16.063766+0000","last_undegraded":"2024-09-10T11:31:16.063766+0000","last_fullsized":"2024-09-10T11:31:16.063766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:50.231766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513204+0000","last_change":"2024-09-10T11:31:14.520758+0000","last_active":"2024-09-10T11:31:15.513204+0000","last_peered":"2024-09-10T11:31:15.513204+0000","last_clean":"2024-09-10T11:31:15.513204+0000","last_became_active":"2024-09-10T11:31:14.520477+0000","last_became_peered":"2024-09-10T11:31:14.520477+0000","last_unstale":"2024-09-10T11:31:15.513204+0000","last_undegraded":"2024-09-10T11:31:15.513204+0000","last_fullsized":"2024-09-10T11:31:15.513204+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:59.405732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523032+0000","last_change":"2024-09-10T11:31:14.517216+0000","last_active":"2024-09-10T11:31:15.523032+0000","last_peered":"2024-09-10T11:31:15.523032+0000","last_clean":"2024-09-10T11:31:15.523032+0000","last_became_active":"2024-09-10T11:31:14.516995+0000","last_became_peered":"2024-09-10T11:31:14.516995+0000","last_unstale":"2024-09-10T11:31:15.523032+0000","last_undegraded":"2024-09-10T11:31:15.523032+0000","last_fullsized":"2024-09-10T11:31:15.523032+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:19.270924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064509+0000","last_change":"2024-09-10T11:31:14.521880+0000","last_active":"2024-09-10T11:31:16.064509+0000","last_peered":"2024-09-10T11:31:16.064509+0000","last_clean":"2024-09-10T11:31:16.064509+0000","last_became_active":"2024-09-10T11:31:14.521574+0000","last_became_peered":"2024-09-10T11:31:14.521574+0000","last_unstale":"2024-09-10T11:31:16.064509+0000","last_undegraded":"2024-09-10T11:31:16.064509+0000","last_fullsized":"2024-09-10T11:31:16.064509+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:44.995070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063818+0000","last_change":"2024-09-10T11:31:14.523038+0000","last_active":"2024-09-10T11:31:16.063818+0000","last_peered":"2024-09-10T11:31:16.063818+0000","last_clean":"2024-09-10T11:31:16.063818+0000","last_became_active":"2024-09-10T11:31:14.521436+0000","last_became_peered":"2024-09-10T11:31:14.521436+0000","last_unstale":"2024-09-10T11:31:16.063818+0000","last_undegraded":"2024-09-10T11:31:16.063818+0000","last_fullsized":"2024-09-10T11:31:16.063818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:47.013779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064775+0000","last_change":"2024-09-10T11:31:14.513886+0000","last_active":"2024-09-10T11:31:16.064775+0000","last_peered":"2024-09-10T11:31:16.064775+0000","last_clean":"2024-09-10T11:31:16.064775+0000","last_became_active":"2024-09-10T11:31:14.513638+0000","last_became_peered":"2024-09-10T11:31:14.513638+0000","last_unstale":"2024-09-10T11:31:16.064775+0000","last_undegraded":"2024-09-10T11:31:16.064775+0000","last_fullsized":"2024-09-10T11:31:16.064775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:37.342836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063773+0000","last_change":"2024-09-10T11:31:14.516924+0000","last_active":"2024-09-10T11:31:16.063773+0000","last_peered":"2024-09-10T11:31:16.063773+0000","last_clean":"2024-09-10T11:31:16.063773+0000","last_became_active":"2024-09-10T11:31:14.516171+0000","last_became_peered":"2024-09-10T11:31:14.516171+0000","last_unstale":"2024-09-10T11:31:16.063773+0000","last_undegraded":"2024-09-10T11:31:16.063773+0000","last_fullsized":"2024-09-10T11:31:16.063773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:54.456682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513222+0000","last_change":"2024-09-10T11:31:14.516928+0000","last_active":"2024-09-10T11:31:15.513222+0000","last_peered":"2024-09-10T11:31:15.513222+0000","last_clean":"2024-09-10T11:31:15.513222+0000","last_became_active":"2024-09-10T11:31:14.516703+0000","last_became_peered":"2024-09-10T11:31:14.516703+0000","last_unstale":"2024-09-10T11:31:15.513222+0000","last_undegraded":"2024-09-10T11:31:15.513222+0000","last_fullsized":"2024-09-10T11:31:15.513222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:41.622401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064732+0000","last_change":"2024-09-10T11:31:14.520023+0000","last_active":"2024-09-10T11:31:16.064732+0000","last_peered":"2024-09-10T11:31:16.064732+0000","last_clean":"2024-09-10T11:31:16.064732+0000","last_became_active":"2024-09-10T11:31:14.515759+0000","last_became_peered":"2024-09-10T11:31:14.515759+0000","last_unstale":"2024-09-10T11:31:16.064732+0000","last_undegraded":"2024-09-10T11:31:16.064732+0000","last_fullsized":"2024-09-10T11:31:16.064732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:06.404563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515489+0000","last_change":"2024-09-10T11:31:14.517385+0000","last_active":"2024-09-10T11:31:15.515489+0000","last_peered":"2024-09-10T11:31:15.515489+0000","last_clean":"2024-09-10T11:31:15.515489+0000","last_became_active":"2024-09-10T11:31:14.515086+0000","last_became_peered":"2024-09-10T11:31:14.515086+0000","last_unstale":"2024-09-10T11:31:15.515489+0000","last_undegraded":"2024-09-10T11:31:15.515489+0000","last_fullsized":"2024-09-10T11:31:15.515489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:23.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064218+0000","last_change":"2024-09-10T11:31:14.520862+0000","last_active":"2024-09-10T11:31:16.064218+0000","last_peered":"2024-09-10T11:31:16.064218+0000","last_clean":"2024-09-10T11:31:16.064218+0000","last_became_active":"2024-09-10T11:31:14.518068+0000","last_became_peered":"2024-09-10T11:31:14.518068+0000","last_unstale":"2024-09-10T11:31:16.064218+0000","last_undegraded":"2024-09-10T11:31:16.064218+0000","last_fullsized":"2024-09-10T11:31:16.064218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:18.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.525155+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.524763+0000","last_became_peered":"2024-09-10T11:31:14.524763+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064291+0000","last_change":"2024-09-10T11:31:14.519244+0000","last_active":"2024-09-10T11:31:16.064291+0000","last_peered":"2024-09-10T11:31:16.064291+0000","last_clean":"2024-09-10T11:31:16.064291+0000","last_became_active":"2024-09-10T11:31:14.519024+0000","last_became_peered":"2024-09-10T11:31:14.519024+0000","last_unstale":"2024-09-10T11:31:16.064291+0000","last_undegraded":"2024-09-10T11:31:16.064291+0000","last_fullsized":"2024-09-10T11:31:16.064291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:13.389476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513155+0000","last_change":"2024-09-10T11:31:14.521724+0000","last_active":"2024-09-10T11:31:15.513155+0000","last_peered":"2024-09-10T11:31:15.513155+0000","last_clean":"2024-09-10T11:31:15.513155+0000","last_became_active":"2024-09-10T11:31:14.521159+0000","last_became_peered":"2024-09-10T11:31:14.521159+0000","last_unstale":"2024-09-10T11:31:15.513155+0000","last_undegraded":"2024-09-10T11:31:15.513155+0000","last_fullsized":"2024-09-10T11:31:15.513155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:28.750442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523039+0000","last_change":"2024-09-10T11:31:14.513624+0000","last_active":"2024-09-10T11:31:15.523039+0000","last_peered":"2024-09-10T11:31:15.523039+0000","last_clean":"2024-09-10T11:31:15.523039+0000","last_became_active":"2024-09-10T11:31:14.513389+0000","last_became_peered":"2024-09-10T11:31:14.513389+0000","last_unstale":"2024-09-10T11:31:15.523039+0000","last_undegraded":"2024-09-10T11:31:15.523039+0000","last_fullsized":"2024-09-10T11:31:15.523039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.159093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.522824+0000","last_change":"2024-09-10T11:31:14.510651+0000","last_active":"2024-09-10T11:31:15.522824+0000","last_peered":"2024-09-10T11:31:15.522824+0000","last_clean":"2024-09-10T11:31:15.522824+0000","last_became_active":"2024-09-10T11:31:14.510416+0000","last_became_peered":"2024-09-10T11:31:14.510416+0000","last_unstale":"2024-09-10T11:31:15.522824+0000","last_undegraded":"2024-09-10T11:31:15.522824+0000","last_fullsized":"2024-09-10T11:31:15.522824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.146570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515531+0000","last_change":"2024-09-10T11:31:14.521940+0000","last_active":"2024-09-10T11:31:15.515531+0000","last_peered":"2024-09-10T11:31:15.515531+0000","last_clean":"2024-09-10T11:31:15.515531+0000","last_became_active":"2024-09-10T11:31:14.521683+0000","last_became_peered":"2024-09-10T11:31:14.521683+0000","last_unstale":"2024-09-10T11:31:15.515531+0000","last_undegraded":"2024-09-10T11:31:15.515531+0000","last_fullsized":"2024-09-10T11:31:15.515531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:46.284242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064423+0000","last_change":"2024-09-10T11:31:14.522106+0000","last_active":"2024-09-10T11:31:16.064423+0000","last_peered":"2024-09-10T11:31:16.064423+0000","last_clean":"2024-09-10T11:31:16.064423+0000","last_became_active":"2024-09-10T11:31:14.521560+0000","last_became_peered":"2024-09-10T11:31:14.521560+0000","last_unstale":"2024-09-10T11:31:16.064423+0000","last_undegraded":"2024-09-10T11:31:16.064423+0000","last_fullsized":"2024-09-10T11:31:16.064423+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:29.933314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064579+0000","last_change":"2024-09-10T11:31:14.518944+0000","last_active":"2024-09-10T11:31:16.064579+0000","last_peered":"2024-09-10T11:31:16.064579+0000","last_clean":"2024-09-10T11:31:16.064579+0000","last_became_active":"2024-09-10T11:31:14.517995+0000","last_became_peered":"2024-09-10T11:31:14.517995+0000","last_unstale":"2024-09-10T11:31:16.064579+0000","last_undegraded":"2024-09-10T11:31:16.064579+0000","last_fullsized":"2024-09-10T11:31:16.064579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:40.011208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515615+0000","last_change":"2024-09-10T11:31:14.509386+0000","last_active":"2024-09-10T11:31:15.515615+0000","last_peered":"2024-09-10T11:31:15.515615+0000","last_clean":"2024-09-10T11:31:15.515615+0000","last_became_active":"2024-09-10T11:31:14.509109+0000","last_became_peered":"2024-09-10T11:31:14.509109+0000","last_unstale":"2024-09-10T11:31:15.515615+0000","last_undegraded":"2024-09-10T11:31:15.515615+0000","last_fullsized":"2024-09-10T11:31:15.515615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:46.687588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063963+0000","last_change":"2024-09-10T11:31:14.518236+0000","last_active":"2024-09-10T11:31:16.063963+0000","last_peered":"2024-09-10T11:31:16.063963+0000","last_clean":"2024-09-10T11:31:16.063963+0000","last_became_active":"2024-09-10T11:31:14.518030+0000","last_became_peered":"2024-09-10T11:31:14.518030+0000","last_unstale":"2024-09-10T11:31:16.063963+0000","last_undegraded":"2024-09-10T11:31:16.063963+0000","last_fullsized":"2024-09-10T11:31:16.063963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:32.832514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.516427+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.515330+0000","last_became_peered":"2024-09-10T11:31:14.515330+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:25.115249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064316+0000","last_change":"2024-09-10T11:31:14.511929+0000","last_active":"2024-09-10T11:31:16.064316+0000","last_peered":"2024-09-10T11:31:16.064316+0000","last_clean":"2024-09-10T11:31:16.064316+0000","last_became_active":"2024-09-10T11:31:14.511710+0000","last_became_peered":"2024-09-10T11:31:14.511710+0000","last_unstale":"2024-09-10T11:31:16.064316+0000","last_undegraded":"2024-09-10T11:31:16.064316+0000","last_fullsized":"2024-09-10T11:31:16.064316+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:07.427370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513218+0000","last_change":"2024-09-10T11:31:14.523316+0000","last_active":"2024-09-10T11:31:15.513218+0000","last_peered":"2024-09-10T11:31:15.513218+0000","last_clean":"2024-09-10T11:31:15.513218+0000","last_became_active":"2024-09-10T11:31:14.522736+0000","last_became_peered":"2024-09-10T11:31:14.522736+0000","last_unstale":"2024-09-10T11:31:15.513218+0000","last_undegraded":"2024-09-10T11:31:15.513218+0000","last_fullsized":"2024-09-10T11:31:15.513218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:39.936447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063964+0000","last_change":"2024-09-10T11:31:14.522018+0000","last_active":"2024-09-10T11:31:16.063964+0000","last_peered":"2024-09-10T11:31:16.063964+0000","last_clean":"2024-09-10T11:31:16.063964+0000","last_became_active":"2024-09-10T11:31:14.521149+0000","last_became_peered":"2024-09-10T11:31:14.521149+0000","last_unstale":"2024-09-10T11:31:16.063964+0000","last_undegraded":"2024-09-10T11:31:16.063964+0000","last_fullsized":"2024-09-10T11:31:16.063964+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:10.551902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063737+0000","last_change":"2024-09-10T11:31:14.522697+0000","last_active":"2024-09-10T11:31:16.063737+0000","last_peered":"2024-09-10T11:31:16.063737+0000","last_clean":"2024-09-10T11:31:16.063737+0000","last_became_active":"2024-09-10T11:31:14.522450+0000","last_became_peered":"2024-09-10T11:31:14.522450+0000","last_unstale":"2024-09-10T11:31:16.063737+0000","last_undegraded":"2024-09-10T11:31:16.063737+0000","last_fullsized":"2024-09-10T11:31:16.063737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:56.166552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064242+0000","last_change":"2024-09-10T11:31:14.522506+0000","last_active":"2024-09-10T11:31:16.064242+0000","last_peered":"2024-09-10T11:31:16.064242+0000","last_clean":"2024-09-10T11:31:16.064242+0000","last_became_active":"2024-09-10T11:31:14.520994+0000","last_became_peered":"2024-09-10T11:31:14.520994+0000","last_unstale":"2024-09-10T11:31:16.064242+0000","last_undegraded":"2024-09-10T11:31:16.064242+0000","last_fullsized":"2024-09-10T11:31:16.064242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:03.419687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064094+0000","last_change":"2024-09-10T11:31:14.514635+0000","last_active":"2024-09-10T11:31:16.064094+0000","last_peered":"2024-09-10T11:31:16.064094+0000","last_clean":"2024-09-10T11:31:16.064094+0000","last_became_active":"2024-09-10T11:31:14.514346+0000","last_became_peered":"2024-09-10T11:31:14.514346+0000","last_unstale":"2024-09-10T11:31:16.064094+0000","last_undegraded":"2024-09-10T11:31:16.064094+0000","last_fullsized":"2024-09-10T11:31:16.064094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:14.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513261+0000","last_change":"2024-09-10T11:31:14.521552+0000","last_active":"2024-09-10T11:31:15.513261+0000","last_peered":"2024-09-10T11:31:15.513261+0000","last_clean":"2024-09-10T11:31:15.513261+0000","last_became_active":"2024-09-10T11:31:14.520865+0000","last_became_peered":"2024-09-10T11:31:14.520865+0000","last_unstale":"2024-09-10T11:31:15.513261+0000","last_undegraded":"2024-09-10T11:31:15.513261+0000","last_fullsized":"2024-09-10T11:31:15.513261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:37.013148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515052+0000","last_change":"2024-09-10T11:31:14.515127+0000","last_active":"2024-09-10T11:31:15.515052+0000","last_peered":"2024-09-10T11:31:15.515052+0000","last_clean":"2024-09-10T11:31:15.515052+0000","last_became_active":"2024-09-10T11:31:14.514886+0000","last_became_peered":"2024-09-10T11:31:14.514886+0000","last_unstale":"2024-09-10T11:31:15.515052+0000","last_undegraded":"2024-09-10T11:31:15.515052+0000","last_fullsized":"2024-09-10T11:31:15.515052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:36:05.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513299+0000","last_change":"2024-09-10T11:31:13.520024+0000","last_active":"2024-09-10T11:31:15.513299+0000","last_peered":"2024-09-10T11:31:15.513299+0000","last_clean":"2024-09-10T11:31:15.513299+0000","last_became_active":"2024-09-10T11:31:13.519800+0000","last_became_peered":"2024-09-10T11:31:13.519800+0000","last_unstale":"2024-09-10T11:31:15.513299+0000","last_undegraded":"2024-09-10T11:31:15.513299+0000","last_fullsized":"2024-09-10T11:31:15.513299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:26.967099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064201+0000","last_change":"2024-09-10T11:31:14.523240+0000","last_active":"2024-09-10T11:31:16.064201+0000","last_peered":"2024-09-10T11:31:16.064201+0000","last_clean":"2024-09-10T11:31:16.064201+0000","last_became_active":"2024-09-10T11:31:14.521740+0000","last_became_peered":"2024-09-10T11:31:14.521740+0000","last_unstale":"2024-09-10T11:31:16.064201+0000","last_undegraded":"2024-09-10T11:31:16.064201+0000","last_fullsized":"2024-09-10T11:31:16.064201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:00.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064409+0000","last_change":"2024-09-10T11:31:13.516467+0000","last_active":"2024-09-10T11:31:16.064409+0000","last_peered":"2024-09-10T11:31:16.064409+0000","last_clean":"2024-09-10T11:31:16.064409+0000","last_became_active":"2024-09-10T11:31:13.515944+0000","last_became_peered":"2024-09-10T11:31:13.515944+0000","last_unstale":"2024-09-10T11:31:16.064409+0000","last_undegraded":"2024-09-10T11:31:16.064409+0000","last_fullsized":"2024-09-10T11:31:16.064409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:25.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064246+0000","last_change":"2024-09-10T11:31:14.516163+0000","last_active":"2024-09-10T11:31:16.064246+0000","last_peered":"2024-09-10T11:31:16.064246+0000","last_clean":"2024-09-10T11:31:16.064246+0000","last_became_active":"2024-09-10T11:31:14.515949+0000","last_became_peered":"2024-09-10T11:31:14.515949+0000","last_unstale":"2024-09-10T11:31:16.064246+0000","last_undegraded":"2024-09-10T11:31:16.064246+0000","last_fullsized":"2024-09-10T11:31:16.064246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:25.705984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523111+0000","last_change":"2024-09-10T11:31:13.517924+0000","last_active":"2024-09-10T11:31:15.523111+0000","last_peered":"2024-09-10T11:31:15.523111+0000","last_clean":"2024-09-10T11:31:15.523111+0000","last_became_active":"2024-09-10T11:31:13.517680+0000","last_became_peered":"2024-09-10T11:31:13.517680+0000","last_unstale":"2024-09-10T11:31:15.523111+0000","last_undegraded":"2024-09-10T11:31:15.523111+0000","last_fullsized":"2024-09-10T11:31:15.523111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:35.392288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064299+0000","last_change":"2024-09-10T11:31:14.516627+0000","last_active":"2024-09-10T11:31:16.064299+0000","last_peered":"2024-09-10T11:31:16.064299+0000","last_clean":"2024-09-10T11:31:16.064299+0000","last_became_active":"2024-09-10T11:31:14.515877+0000","last_became_peered":"2024-09-10T11:31:14.515877+0000","last_unstale":"2024-09-10T11:31:16.064299+0000","last_undegraded":"2024-09-10T11:31:16.064299+0000","last_fullsized":"2024-09-10T11:31:16.064299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:36.098774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064063+0000","last_change":"2024-09-10T11:31:13.513866+0000","last_active":"2024-09-10T11:31:16.064063+0000","last_peered":"2024-09-10T11:31:16.064063+0000","last_clean":"2024-09-10T11:31:16.064063+0000","last_became_active":"2024-09-10T11:31:13.510927+0000","last_became_peered":"2024-09-10T11:31:13.510927+0000","last_unstale":"2024-09-10T11:31:16.064063+0000","last_undegraded":"2024-09-10T11:31:16.064063+0000","last_fullsized":"2024-09-10T11:31:16.064063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:24.499517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064075+0000","last_change":"2024-09-10T11:31:14.520437+0000","last_active":"2024-09-10T11:31:16.064075+0000","last_peered":"2024-09-10T11:31:16.064075+0000","last_clean":"2024-09-10T11:31:16.064075+0000","last_became_active":"2024-09-10T11:31:14.515291+0000","last_became_peered":"2024-09-10T11:31:14.515291+0000","last_unstale":"2024-09-10T11:31:16.064075+0000","last_undegraded":"2024-09-10T11:31:16.064075+0000","last_fullsized":"2024-09-10T11:31:16.064075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:23.697849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064695+0000","last_change":"2024-09-10T11:31:13.519201+0000","last_active":"2024-09-10T11:31:16.064695+0000","last_peered":"2024-09-10T11:31:16.064695+0000","last_clean":"2024-09-10T11:31:16.064695+0000","last_became_active":"2024-09-10T11:31:13.518974+0000","last_became_peered":"2024-09-10T11:31:13.518974+0000","last_unstale":"2024-09-10T11:31:16.064695+0000","last_undegraded":"2024-09-10T11:31:16.064695+0000","last_fullsized":"2024-09-10T11:31:16.064695+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:48.077185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064646+0000","last_change":"2024-09-10T11:31:14.517590+0000","last_active":"2024-09-10T11:31:16.064646+0000","last_peered":"2024-09-10T11:31:16.064646+0000","last_clean":"2024-09-10T11:31:16.064646+0000","last_became_active":"2024-09-10T11:31:14.515618+0000","last_became_peered":"2024-09-10T11:31:14.515618+0000","last_unstale":"2024-09-10T11:31:16.064646+0000","last_undegraded":"2024-09-10T11:31:16.064646+0000","last_fullsized":"2024-09-10T11:31:16.064646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.870654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064428+0000","last_change":"2024-09-10T11:31:13.516663+0000","last_active":"2024-09-10T11:31:16.064428+0000","last_peered":"2024-09-10T11:31:16.064428+0000","last_clean":"2024-09-10T11:31:16.064428+0000","last_became_active":"2024-09-10T11:31:13.515737+0000","last_became_peered":"2024-09-10T11:31:13.515737+0000","last_unstale":"2024-09-10T11:31:16.064428+0000","last_undegraded":"2024-09-10T11:31:16.064428+0000","last_fullsized":"2024-09-10T11:31:16.064428+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:24.805568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063984+0000","last_change":"2024-09-10T11:31:14.524693+0000","last_active":"2024-09-10T11:31:16.063984+0000","last_peered":"2024-09-10T11:31:16.063984+0000","last_clean":"2024-09-10T11:31:16.063984+0000","last_became_active":"2024-09-10T11:31:14.524107+0000","last_became_peered":"2024-09-10T11:31:14.524107+0000","last_unstale":"2024-09-10T11:31:16.063984+0000","last_undegraded":"2024-09-10T11:31:16.063984+0000","last_fullsized":"2024-09-10T11:31:16.063984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:20.761540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523109+0000","last_change":"2024-09-10T11:31:13.513183+0000","last_active":"2024-09-10T11:31:15.523109+0000","last_peered":"2024-09-10T11:31:15.523109+0000","last_clean":"2024-09-10T11:31:15.523109+0000","last_became_active":"2024-09-10T11:31:13.512547+0000","last_became_peered":"2024-09-10T11:31:13.512547+0000","last_unstale":"2024-09-10T11:31:15.523109+0000","last_undegraded":"2024-09-10T11:31:15.523109+0000","last_fullsized":"2024-09-10T11:31:15.523109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:11.447778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063602+0000","last_change":"2024-09-10T11:31:14.520435+0000","last_active":"2024-09-10T11:31:16.063602+0000","last_peered":"2024-09-10T11:31:16.063602+0000","last_clean":"2024-09-10T11:31:16.063602+0000","last_became_active":"2024-09-10T11:31:14.517253+0000","last_became_peered":"2024-09-10T11:31:14.517253+0000","last_unstale":"2024-09-10T11:31:16.063602+0000","last_undegraded":"2024-09-10T11:31:16.063602+0000","last_fullsized":"2024-09-10T11:31:16.063602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:34.856235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064117+0000","last_change":"2024-09-10T11:31:13.516670+0000","last_active":"2024-09-10T11:31:16.064117+0000","last_peered":"2024-09-10T11:31:16.064117+0000","last_clean":"2024-09-10T11:31:16.064117+0000","last_became_active":"2024-09-10T11:31:13.516441+0000","last_became_peered":"2024-09-10T11:31:13.516441+0000","last_unstale":"2024-09-10T11:31:16.064117+0000","last_undegraded":"2024-09-10T11:31:16.064117+0000","last_fullsized":"2024-09-10T11:31:16.064117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:38.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513500+0000","last_change":"2024-09-10T11:31:14.514509+0000","last_active":"2024-09-10T11:31:15.513500+0000","last_peered":"2024-09-10T11:31:15.513500+0000","last_clean":"2024-09-10T11:31:15.513500+0000","last_became_active":"2024-09-10T11:31:14.513904+0000","last_became_peered":"2024-09-10T11:31:14.513904+0000","last_unstale":"2024-09-10T11:31:15.513500+0000","last_undegraded":"2024-09-10T11:31:15.513500+0000","last_fullsized":"2024-09-10T11:31:15.513500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:00.742503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513185+0000","last_change":"2024-09-10T11:31:13.519004+0000","last_active":"2024-09-10T11:31:15.513185+0000","last_peered":"2024-09-10T11:31:15.513185+0000","last_clean":"2024-09-10T11:31:15.513185+0000","last_became_active":"2024-09-10T11:31:13.518744+0000","last_became_peered":"2024-09-10T11:31:13.518744+0000","last_unstale":"2024-09-10T11:31:15.513185+0000","last_undegraded":"2024-09-10T11:31:15.513185+0000","last_fullsized":"2024-09-10T11:31:15.513185+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:12.285035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063800+0000","last_change":"2024-09-10T11:31:14.515923+0000","last_active":"2024-09-10T11:31:16.063800+0000","last_peered":"2024-09-10T11:31:16.063800+0000","last_clean":"2024-09-10T11:31:16.063800+0000","last_became_active":"2024-09-10T11:31:14.515177+0000","last_became_peered":"2024-09-10T11:31:14.515177+0000","last_unstale":"2024-09-10T11:31:16.063800+0000","last_undegraded":"2024-09-10T11:31:16.063800+0000","last_fullsized":"2024-09-10T11:31:16.063800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:47.104953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513425+0000","last_change":"2024-09-10T11:31:13.518158+0000","last_active":"2024-09-10T11:31:15.513425+0000","last_peered":"2024-09-10T11:31:15.513425+0000","last_clean":"2024-09-10T11:31:15.513425+0000","last_became_active":"2024-09-10T11:31:13.517660+0000","last_became_peered":"2024-09-10T11:31:13.517660+0000","last_unstale":"2024-09-10T11:31:15.513425+0000","last_undegraded":"2024-09-10T11:31:15.513425+0000","last_fullsized":"2024-09-10T11:31:15.513425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:05.641010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064486+0000","last_change":"2024-09-10T11:31:14.522063+0000","last_active":"2024-09-10T11:31:16.064486+0000","last_peered":"2024-09-10T11:31:16.064486+0000","last_clean":"2024-09-10T11:31:16.064486+0000","last_became_active":"2024-09-10T11:31:14.520217+0000","last_became_peered":"2024-09-10T11:31:14.520217+0000","last_unstale":"2024-09-10T11:31:16.064486+0000","last_undegraded":"2024-09-10T11:31:16.064486+0000","last_fullsized":"2024-09-10T11:31:16.064486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:43.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513488+0000","last_change":"2024-09-10T11:31:13.511626+0000","last_active":"2024-09-10T11:31:15.513488+0000","last_peered":"2024-09-10T11:31:15.513488+0000","last_clean":"2024-09-10T11:31:15.513488+0000","last_became_active":"2024-09-10T11:31:13.511367+0000","last_became_peered":"2024-09-10T11:31:13.511367+0000","last_unstale":"2024-09-10T11:31:15.513488+0000","last_undegraded":"2024-09-10T11:31:15.513488+0000","last_fullsized":"2024-09-10T11:31:15.513488+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:14.937108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513207+0000","last_change":"2024-09-10T11:31:14.517040+0000","last_active":"2024-09-10T11:31:15.513207+0000","last_peered":"2024-09-10T11:31:15.513207+0000","last_clean":"2024-09-10T11:31:15.513207+0000","last_became_active":"2024-09-10T11:31:14.516816+0000","last_became_peered":"2024-09-10T11:31:14.516816+0000","last_unstale":"2024-09-10T11:31:15.513207+0000","last_undegraded":"2024-09-10T11:31:15.513207+0000","last_fullsized":"2024-09-10T11:31:15.513207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:43.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515435+0000","last_change":"2024-09-10T11:31:13.513864+0000","last_active":"2024-09-10T11:31:15.515435+0000","last_peered":"2024-09-10T11:31:15.515435+0000","last_clean":"2024-09-10T11:31:15.515435+0000","last_became_active":"2024-09-10T11:31:13.513392+0000","last_became_peered":"2024-09-10T11:31:13.513392+0000","last_unstale":"2024-09-10T11:31:15.515435+0000","last_undegraded":"2024-09-10T11:31:15.515435+0000","last_fullsized":"2024-09-10T11:31:15.515435+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:32.396996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515362+0000","last_change":"2024-09-10T11:31:14.510565+0000","last_active":"2024-09-10T11:31:15.515362+0000","last_peered":"2024-09-10T11:31:15.515362+0000","last_clean":"2024-09-10T11:31:15.515362+0000","last_became_active":"2024-09-10T11:31:14.510323+0000","last_became_peered":"2024-09-10T11:31:14.510323+0000","last_unstale":"2024-09-10T11:31:15.515362+0000","last_undegraded":"2024-09-10T11:31:15.515362+0000","last_fullsized":"2024-09-10T11:31:15.515362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.783845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064217+0000","last_change":"2024-09-10T11:31:13.519970+0000","last_active":"2024-09-10T11:31:16.064217+0000","last_peered":"2024-09-10T11:31:16.064217+0000","last_clean":"2024-09-10T11:31:16.064217+0000","last_became_active":"2024-09-10T11:31:13.519444+0000","last_became_peered":"2024-09-10T11:31:13.519444+0000","last_unstale":"2024-09-10T11:31:16.064217+0000","last_undegraded":"2024-09-10T11:31:16.064217+0000","last_fullsized":"2024-09-10T11:31:16.064217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:39.129313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064076+0000","last_change":"2024-09-10T11:31:14.510319+0000","last_active":"2024-09-10T11:31:16.064076+0000","last_peered":"2024-09-10T11:31:16.064076+0000","last_clean":"2024-09-10T11:31:16.064076+0000","last_became_active":"2024-09-10T11:31:14.510057+0000","last_became_peered":"2024-09-10T11:31:14.510057+0000","last_unstale":"2024-09-10T11:31:16.064076+0000","last_undegraded":"2024-09-10T11:31:16.064076+0000","last_fullsized":"2024-09-10T11:31:16.064076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:46.501405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064418+0000","last_change":"2024-09-10T11:31:13.516451+0000","last_active":"2024-09-10T11:31:16.064418+0000","last_peered":"2024-09-10T11:31:16.064418+0000","last_clean":"2024-09-10T11:31:16.064418+0000","last_became_active":"2024-09-10T11:31:13.515947+0000","last_became_peered":"2024-09-10T11:31:13.515947+0000","last_unstale":"2024-09-10T11:31:16.064418+0000","last_undegraded":"2024-09-10T11:31:16.064418+0000","last_fullsized":"2024-09-10T11:31:16.064418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:08.085889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064276+0000","last_change":"2024-09-10T11:31:14.514288+0000","last_active":"2024-09-10T11:31:16.064276+0000","last_peered":"2024-09-10T11:31:16.064276+0000","last_clean":"2024-09-10T11:31:16.064276+0000","last_became_active":"2024-09-10T11:31:14.514037+0000","last_became_peered":"2024-09-10T11:31:14.514037+0000","last_unstale":"2024-09-10T11:31:16.064276+0000","last_undegraded":"2024-09-10T11:31:16.064276+0000","last_fullsized":"2024-09-10T11:31:16.064276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:32.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063736+0000","last_change":"2024-09-10T11:31:13.512395+0000","last_active":"2024-09-10T11:31:16.063736+0000","last_peered":"2024-09-10T11:31:16.063736+0000","last_clean":"2024-09-10T11:31:16.063736+0000","last_became_active":"2024-09-10T11:31:13.512042+0000","last_became_peered":"2024-09-10T11:31:13.512042+0000","last_unstale":"2024-09-10T11:31:16.063736+0000","last_undegraded":"2024-09-10T11:31:16.063736+0000","last_fullsized":"2024-09-10T11:31:16.063736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:28.809166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513334+0000","last_change":"2024-09-10T11:31:14.521372+0000","last_active":"2024-09-10T11:31:15.513334+0000","last_peered":"2024-09-10T11:31:15.513334+0000","last_clean":"2024-09-10T11:31:15.513334+0000","last_became_active":"2024-09-10T11:31:14.520563+0000","last_became_peered":"2024-09-10T11:31:14.520563+0000","last_unstale":"2024-09-10T11:31:15.513334+0000","last_undegraded":"2024-09-10T11:31:15.513334+0000","last_fullsized":"2024-09-10T11:31:15.513334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:41.141250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523252+0000","last_change":"2024-09-10T11:31:13.512302+0000","last_active":"2024-09-10T11:31:15.523252+0000","last_peered":"2024-09-10T11:31:15.523252+0000","last_clean":"2024-09-10T11:31:15.523252+0000","last_became_active":"2024-09-10T11:31:13.512062+0000","last_became_peered":"2024-09-10T11:31:13.512062+0000","last_unstale":"2024-09-10T11:31:15.523252+0000","last_undegraded":"2024-09-10T11:31:15.523252+0000","last_fullsized":"2024-09-10T11:31:15.523252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:10.103139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513512+0000","last_change":"2024-09-10T11:31:14.517992+0000","last_active":"2024-09-10T11:31:15.513512+0000","last_peered":"2024-09-10T11:31:15.513512+0000","last_clean":"2024-09-10T11:31:15.513512+0000","last_became_active":"2024-09-10T11:31:14.517243+0000","last_became_peered":"2024-09-10T11:31:14.517243+0000","last_unstale":"2024-09-10T11:31:15.513512+0000","last_undegraded":"2024-09-10T11:31:15.513512+0000","last_fullsized":"2024-09-10T11:31:15.513512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:28.492681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523216+0000","last_change":"2024-09-10T11:31:13.511780+0000","last_active":"2024-09-10T11:31:15.523216+0000","last_peered":"2024-09-10T11:31:15.523216+0000","last_clean":"2024-09-10T11:31:15.523216+0000","last_became_active":"2024-09-10T11:31:13.511513+0000","last_became_peered":"2024-09-10T11:31:13.511513+0000","last_unstale":"2024-09-10T11:31:15.523216+0000","last_undegraded":"2024-09-10T11:31:15.523216+0000","last_fullsized":"2024-09-10T11:31:15.523216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:56.401271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064841+0000","last_change":"2024-09-10T11:31:14.516007+0000","last_active":"2024-09-10T11:31:16.064841+0000","last_peered":"2024-09-10T11:31:16.064841+0000","last_clean":"2024-09-10T11:31:16.064841+0000","last_became_active":"2024-09-10T11:31:14.515747+0000","last_became_peered":"2024-09-10T11:31:14.515747+0000","last_unstale":"2024-09-10T11:31:16.064841+0000","last_undegraded":"2024-09-10T11:31:16.064841+0000","last_fullsized":"2024-09-10T11:31:16.064841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:37.712553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064327+0000","last_change":"2024-09-10T11:31:13.519713+0000","last_active":"2024-09-10T11:31:16.064327+0000","last_peered":"2024-09-10T11:31:16.064327+0000","last_clean":"2024-09-10T11:31:16.064327+0000","last_became_active":"2024-09-10T11:31:13.519090+0000","last_became_peered":"2024-09-10T11:31:13.519090+0000","last_unstale":"2024-09-10T11:31:16.064327+0000","last_undegraded":"2024-09-10T11:31:16.064327+0000","last_fullsized":"2024-09-10T11:31:16.064327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:09.363236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513322+0000","last_change":"2024-09-10T11:31:14.522994+0000","last_active":"2024-09-10T11:31:15.513322+0000","last_peered":"2024-09-10T11:31:15.513322+0000","last_clean":"2024-09-10T11:31:15.513322+0000","last_became_active":"2024-09-10T11:31:14.522196+0000","last_became_peered":"2024-09-10T11:31:14.522196+0000","last_unstale":"2024-09-10T11:31:15.513322+0000","last_undegraded":"2024-09-10T11:31:15.513322+0000","last_fullsized":"2024-09-10T11:31:15.513322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:24.388027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513308+0000","last_change":"2024-09-10T11:31:13.515294+0000","last_active":"2024-09-10T11:31:15.513308+0000","last_peered":"2024-09-10T11:31:15.513308+0000","last_clean":"2024-09-10T11:31:15.513308+0000","last_became_active":"2024-09-10T11:31:13.514887+0000","last_became_peered":"2024-09-10T11:31:13.514887+0000","last_unstale":"2024-09-10T11:31:15.513308+0000","last_undegraded":"2024-09-10T11:31:15.513308+0000","last_fullsized":"2024-09-10T11:31:15.513308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:50.109398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523187+0000","last_change":"2024-09-10T11:31:14.520452+0000","last_active":"2024-09-10T11:31:15.523187+0000","last_peered":"2024-09-10T11:31:15.523187+0000","last_clean":"2024-09-10T11:31:15.523187+0000","last_became_active":"2024-09-10T11:31:14.520132+0000","last_became_peered":"2024-09-10T11:31:14.520132+0000","last_unstale":"2024-09-10T11:31:15.523187+0000","last_undegraded":"2024-09-10T11:31:15.523187+0000","last_fullsized":"2024-09-10T11:31:15.523187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:38.896738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064892+0000","last_change":"2024-09-10T11:31:13.512607+0000","last_active":"2024-09-10T11:31:16.064892+0000","last_peered":"2024-09-10T11:31:16.064892+0000","last_clean":"2024-09-10T11:31:16.064892+0000","last_became_active":"2024-09-10T11:31:13.511603+0000","last_became_peered":"2024-09-10T11:31:13.511603+0000","last_unstale":"2024-09-10T11:31:16.064892+0000","last_undegraded":"2024-09-10T11:31:16.064892+0000","last_fullsized":"2024-09-10T11:31:16.064892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:23:54.656308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515238+0000","last_change":"2024-09-10T11:31:14.516178+0000","last_active":"2024-09-10T11:31:15.515238+0000","last_peered":"2024-09-10T11:31:15.515238+0000","last_clean":"2024-09-10T11:31:15.515238+0000","last_became_active":"2024-09-10T11:31:14.515903+0000","last_became_peered":"2024-09-10T11:31:14.515903+0000","last_unstale":"2024-09-10T11:31:15.515238+0000","last_undegraded":"2024-09-10T11:31:15.515238+0000","last_fullsized":"2024-09-10T11:31:15.515238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:17.150776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513347+0000","last_change":"2024-09-10T11:31:13.520005+0000","last_active":"2024-09-10T11:31:15.513347+0000","last_peered":"2024-09-10T11:31:15.513347+0000","last_clean":"2024-09-10T11:31:15.513347+0000","last_became_active":"2024-09-10T11:31:13.519498+0000","last_became_peered":"2024-09-10T11:31:13.519498+0000","last_unstale":"2024-09-10T11:31:15.513347+0000","last_undegraded":"2024-09-10T11:31:15.513347+0000","last_fullsized":"2024-09-10T11:31:15.513347+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:53.759850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513152+0000","last_change":"2024-09-10T11:31:14.522205+0000","last_active":"2024-09-10T11:31:15.513152+0000","last_peered":"2024-09-10T11:31:15.513152+0000","last_clean":"2024-09-10T11:31:15.513152+0000","last_became_active":"2024-09-10T11:31:14.521101+0000","last_became_peered":"2024-09-10T11:31:14.521101+0000","last_unstale":"2024-09-10T11:31:15.513152+0000","last_undegraded":"2024-09-10T11:31:15.513152+0000","last_fullsized":"2024-09-10T11:31:15.513152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:11.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064077+0000","last_change":"2024-09-10T11:31:13.518647+0000","last_active":"2024-09-10T11:31:16.064077+0000","last_peered":"2024-09-10T11:31:16.064077+0000","last_clean":"2024-09-10T11:31:16.064077+0000","last_became_active":"2024-09-10T11:31:13.518413+0000","last_became_peered":"2024-09-10T11:31:13.518413+0000","last_unstale":"2024-09-10T11:31:16.064077+0000","last_undegraded":"2024-09-10T11:31:16.064077+0000","last_fullsized":"2024-09-10T11:31:16.064077+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:36:54.845425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515515+0000","last_change":"2024-09-10T11:31:14.515196+0000","last_active":"2024-09-10T11:31:15.515515+0000","last_peered":"2024-09-10T11:31:15.515515+0000","last_clean":"2024-09-10T11:31:15.515515+0000","last_became_active":"2024-09-10T11:31:14.514349+0000","last_became_peered":"2024-09-10T11:31:14.514349+0000","last_unstale":"2024-09-10T11:31:15.515515+0000","last_undegraded":"2024-09-10T11:31:15.515515+0000","last_fullsized":"2024-09-10T11:31:15.515515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:43.860920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064484+0000","last_change":"2024-09-10T11:31:13.518516+0000","last_active":"2024-09-10T11:31:16.064484+0000","last_peered":"2024-09-10T11:31:16.064484+0000","last_clean":"2024-09-10T11:31:16.064484+0000","last_became_active":"2024-09-10T11:31:13.518280+0000","last_became_peered":"2024-09-10T11:31:13.518280+0000","last_unstale":"2024-09-10T11:31:16.064484+0000","last_undegraded":"2024-09-10T11:31:16.064484+0000","last_fullsized":"2024-09-10T11:31:16.064484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:50.257425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064385+0000","last_change":"2024-09-10T11:31:14.524423+0000","last_active":"2024-09-10T11:31:16.064385+0000","last_peered":"2024-09-10T11:31:16.064385+0000","last_clean":"2024-09-10T11:31:16.064385+0000","last_became_active":"2024-09-10T11:31:14.523773+0000","last_became_peered":"2024-09-10T11:31:14.523773+0000","last_unstale":"2024-09-10T11:31:16.064385+0000","last_undegraded":"2024-09-10T11:31:16.064385+0000","last_fullsized":"2024-09-10T11:31:16.064385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:10.048931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513396+0000","last_change":"2024-09-10T11:31:13.515100+0000","last_active":"2024-09-10T11:31:15.513396+0000","last_peered":"2024-09-10T11:31:15.513396+0000","last_clean":"2024-09-10T11:31:15.513396+0000","last_became_active":"2024-09-10T11:31:13.514586+0000","last_became_peered":"2024-09-10T11:31:13.514586+0000","last_unstale":"2024-09-10T11:31:15.513396+0000","last_undegraded":"2024-09-10T11:31:15.513396+0000","last_fullsized":"2024-09-10T11:31:15.513396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:36.506041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523188+0000","last_change":"2024-09-10T11:31:14.519010+0000","last_active":"2024-09-10T11:31:15.523188+0000","last_peered":"2024-09-10T11:31:15.523188+0000","last_clean":"2024-09-10T11:31:15.523188+0000","last_became_active":"2024-09-10T11:31:14.518709+0000","last_became_peered":"2024-09-10T11:31:14.518709+0000","last_unstale":"2024-09-10T11:31:15.523188+0000","last_undegraded":"2024-09-10T11:31:15.523188+0000","last_fullsized":"2024-09-10T11:31:15.523188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:57.491771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515205+0000","last_change":"2024-09-10T11:31:13.513825+0000","last_active":"2024-09-10T11:31:15.515205+0000","last_peered":"2024-09-10T11:31:15.515205+0000","last_clean":"2024-09-10T11:31:15.515205+0000","last_became_active":"2024-09-10T11:31:13.513564+0000","last_became_peered":"2024-09-10T11:31:13.513564+0000","last_unstale":"2024-09-10T11:31:15.515205+0000","last_undegraded":"2024-09-10T11:31:15.515205+0000","last_fullsized":"2024-09-10T11:31:15.515205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:45.716071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513409+0000","last_change":"2024-09-10T11:31:14.514277+0000","last_active":"2024-09-10T11:31:15.513409+0000","last_peered":"2024-09-10T11:31:15.513409+0000","last_clean":"2024-09-10T11:31:15.513409+0000","last_became_active":"2024-09-10T11:31:14.513594+0000","last_became_peered":"2024-09-10T11:31:14.513594+0000","last_unstale":"2024-09-10T11:31:15.513409+0000","last_undegraded":"2024-09-10T11:31:15.513409+0000","last_fullsized":"2024-09-10T11:31:15.513409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:25.578992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523069+0000","last_change":"2024-09-10T11:31:13.519852+0000","last_active":"2024-09-10T11:31:15.523069+0000","last_peered":"2024-09-10T11:31:15.523069+0000","last_clean":"2024-09-10T11:31:15.523069+0000","last_became_active":"2024-09-10T11:31:13.519088+0000","last_became_peered":"2024-09-10T11:31:13.519088+0000","last_unstale":"2024-09-10T11:31:15.523069+0000","last_undegraded":"2024-09-10T11:31:15.523069+0000","last_fullsized":"2024-09-10T11:31:15.523069+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:50.538366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.522997+0000","last_change":"2024-09-10T11:31:14.513437+0000","last_active":"2024-09-10T11:31:15.522997+0000","last_peered":"2024-09-10T11:31:15.522997+0000","last_clean":"2024-09-10T11:31:15.522997+0000","last_became_active":"2024-09-10T11:31:14.513267+0000","last_became_peered":"2024-09-10T11:31:14.513267+0000","last_unstale":"2024-09-10T11:31:15.522997+0000","last_undegraded":"2024-09-10T11:31:15.522997+0000","last_fullsized":"2024-09-10T11:31:15.522997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:06.292104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515609+0000","last_change":"2024-09-10T11:31:13.510861+0000","last_active":"2024-09-10T11:31:15.515609+0000","last_peered":"2024-09-10T11:31:15.515609+0000","last_clean":"2024-09-10T11:31:15.515609+0000","last_became_active":"2024-09-10T11:31:13.510601+0000","last_became_peered":"2024-09-10T11:31:13.510601+0000","last_unstale":"2024-09-10T11:31:15.515609+0000","last_undegraded":"2024-09-10T11:31:15.515609+0000","last_fullsized":"2024-09-10T11:31:15.515609+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:08.504847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064693+0000","last_change":"2024-09-10T11:31:14.518976+0000","last_active":"2024-09-10T11:31:16.064693+0000","last_peered":"2024-09-10T11:31:16.064693+0000","last_clean":"2024-09-10T11:31:16.064693+0000","last_became_active":"2024-09-10T11:31:14.518757+0000","last_became_peered":"2024-09-10T11:31:14.518757+0000","last_unstale":"2024-09-10T11:31:16.064693+0000","last_undegraded":"2024-09-10T11:31:16.064693+0000","last_fullsized":"2024-09-10T11:31:16.064693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.058041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515668+0000","last_change":"2024-09-10T11:31:13.512970+0000","last_active":"2024-09-10T11:31:15.515668+0000","last_peered":"2024-09-10T11:31:15.515668+0000","last_clean":"2024-09-10T11:31:15.515668+0000","last_became_active":"2024-09-10T11:31:13.512699+0000","last_became_peered":"2024-09-10T11:31:13.512699+0000","last_unstale":"2024-09-10T11:31:15.515668+0000","last_undegraded":"2024-09-10T11:31:15.515668+0000","last_fullsized":"2024-09-10T11:31:15.515668+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:15.117198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063951+0000","last_change":"2024-09-10T11:31:14.516102+0000","last_active":"2024-09-10T11:31:16.063951+0000","last_peered":"2024-09-10T11:31:16.063951+0000","last_clean":"2024-09-10T11:31:16.063951+0000","last_became_active":"2024-09-10T11:31:14.515535+0000","last_became_peered":"2024-09-10T11:31:14.515535+0000","last_unstale":"2024-09-10T11:31:16.063951+0000","last_undegraded":"2024-09-10T11:31:16.063951+0000","last_fullsized":"2024-09-10T11:31:16.063951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:30.490092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515403+0000","last_change":"2024-09-10T11:31:13.515604+0000","last_active":"2024-09-10T11:31:15.515403+0000","last_peered":"2024-09-10T11:31:15.515403+0000","last_clean":"2024-09-10T11:31:15.515403+0000","last_became_active":"2024-09-10T11:31:13.515355+0000","last_became_peered":"2024-09-10T11:31:13.515355+0000","last_unstale":"2024-09-10T11:31:15.515403+0000","last_undegraded":"2024-09-10T11:31:15.515403+0000","last_fullsized":"2024-09-10T11:31:15.515403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:15.981692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.521550+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.519610+0000","last_became_peered":"2024-09-10T11:31:14.519610+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:11.170200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064530+0000","last_change":"2024-09-10T11:31:13.517588+0000","last_active":"2024-09-10T11:31:16.064530+0000","last_peered":"2024-09-10T11:31:16.064530+0000","last_clean":"2024-09-10T11:31:16.064530+0000","last_became_active":"2024-09-10T11:31:13.517351+0000","last_became_peered":"2024-09-10T11:31:13.517351+0000","last_unstale":"2024-09-10T11:31:16.064530+0000","last_undegraded":"2024-09-10T11:31:16.064530+0000","last_fullsized":"2024-09-10T11:31:16.064530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:08.110011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513426+0000","last_change":"2024-09-10T11:31:14.514305+0000","last_active":"2024-09-10T11:31:15.513426+0000","last_peered":"2024-09-10T11:31:15.513426+0000","last_clean":"2024-09-10T11:31:15.513426+0000","last_became_active":"2024-09-10T11:31:14.513992+0000","last_became_peered":"2024-09-10T11:31:14.513992+0000","last_unstale":"2024-09-10T11:31:15.513426+0000","last_undegraded":"2024-09-10T11:31:15.513426+0000","last_fullsized":"2024-09-10T11:31:15.513426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:04.655999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064537+0000","last_change":"2024-09-10T11:31:13.516709+0000","last_active":"2024-09-10T11:31:16.064537+0000","last_peered":"2024-09-10T11:31:16.064537+0000","last_clean":"2024-09-10T11:31:16.064537+0000","last_became_active":"2024-09-10T11:31:13.516254+0000","last_became_peered":"2024-09-10T11:31:13.516254+0000","last_unstale":"2024-09-10T11:31:16.064537+0000","last_undegraded":"2024-09-10T11:31:16.064537+0000","last_fullsized":"2024-09-10T11:31:16.064537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:09.219617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064381+0000","last_change":"2024-09-10T11:31:14.521249+0000","last_active":"2024-09-10T11:31:16.064381+0000","last_peered":"2024-09-10T11:31:16.064381+0000","last_clean":"2024-09-10T11:31:16.064381+0000","last_became_active":"2024-09-10T11:31:14.520629+0000","last_became_peered":"2024-09-10T11:31:14.520629+0000","last_unstale":"2024-09-10T11:31:16.064381+0000","last_undegraded":"2024-09-10T11:31:16.064381+0000","last_fullsized":"2024-09-10T11:31:16.064381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:31.363139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513156+0000","last_change":"2024-09-10T11:31:13.516168+0000","last_active":"2024-09-10T11:31:15.513156+0000","last_peered":"2024-09-10T11:31:15.513156+0000","last_clean":"2024-09-10T11:31:15.513156+0000","last_became_active":"2024-09-10T11:31:13.515961+0000","last_became_peered":"2024-09-10T11:31:13.515961+0000","last_unstale":"2024-09-10T11:31:15.513156+0000","last_undegraded":"2024-09-10T11:31:15.513156+0000","last_fullsized":"2024-09-10T11:31:15.513156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:57.520071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064928+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:16.064928+0000","last_peered":"2024-09-10T11:31:16.064928+0000","last_clean":"2024-09-10T11:31:16.064928+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:16.064928+0000","last_undegraded":"2024-09-10T11:31:16.064928+0000","last_fullsized":"2024-09-10T11:31:16.064928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860689,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199859099,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":713,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199269761,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199859077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859935,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270273,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:17.451 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:17.494 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:31:17.492+0000 7f3da71d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:31:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.583+0000 7f522c2c0640 1 Processor -- start 2024-09-10T11:31:17.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.583+0000 7f522c2c0640 1 -- start start 2024-09-10T11:31:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522c2c0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f5224073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522c2c0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5224152e60 con 0x7f522406e5d0 2024-09-10T11:31:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522c2c0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52240d15e0 con 0x7f522415c770 2024-09-10T11:31:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522c2c0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5224072680 con 0x7f52240d1200 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522985b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f522406e5d0 0x7f522415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50102/0 (socket says 172.21.15.99:50102) 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522985b640 1 -- 172.21.15.99:0/1535187545 learned_addr learned my addr 172.21.15.99:0/1535187545 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522a05c640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f5224073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 296267804 0 0) 0x7f5224152e60 con 0x7f522406e5d0 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5210003520 con 0x7f522406e5d0 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3149165867 0 0) 0x7f5210003520 con 0x7f522406e5d0 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 msgr2=0x7f5224073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f5224073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 >> v1:172.21.15.196:6789/0 conn(0x7f522415c770 legacy=0x7f5224159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5224072960 con 0x7f522406e5d0 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5214001670 con 0x7f522406e5d0 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 >> v1:172.21.15.99:6789/0 conn(0x7f522406e5d0 legacy=0x7f522415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f52100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5210003f60 0x7f5210004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.584+0000 7f522905a640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210004d40 0x7f52100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5224072680 con 0x7f5210003f60 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f521000bb50 con 0x7f5210004d40 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f521000bd50 con 0x7f52240d1200 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5210003f60 0x7f5210004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522a05c640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f52100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522a85d640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210004d40 0x7f52100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 -- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 msgr2=0x7f52100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52240d1200 0x7f52100039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 -- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210004d40 msgr2=0x7f52100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210004d40 0x7f52100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522985b640 1 -- 172.21.15.99:0/1535187545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5210003520 con 0x7f5210003f60 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.585+0000 7f522a85d640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210004d40 0x7f52100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522985b640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5210003f60 0x7f5210004800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52240b4200 tx=0x7f5210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41b5fd9f1f9ff22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214007d30 con 0x7f5210003f60 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5214005af0 con 0x7f5210003f60 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522905a640 1 -- 172.21.15.99:0/1535187545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214007070 con 0x7f5210003f60 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 -- 172.21.15.99:0/1535187545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5210003f60 msgr2=0x7f5210004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 --2- 172.21.15.99:0/1535187545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5210003f60 0x7f5210004800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52240b4200 tx=0x7f5210005830 comp rx=0 tx=0).stop 2024-09-10T11:31:17.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522a85d640 1 -- 172.21.15.99:0/1535187545 reap_dead start 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 -- 172.21.15.99:0/1535187545 shutdown_connections 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 -- 172.21.15.99:0/1535187545 >> 172.21.15.99:0/1535187545 conn(0x7f522406bc10 msgr2=0x7f522406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 -- 172.21.15.99:0/1535187545 shutdown_connections 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.586+0000 7f522c2c0640 1 -- 172.21.15.99:0/1535187545 wait complete. 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.587+0000 7f522c2c0640 1 Processor -- start 2024-09-10T11:31:17.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 -- start start 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 0x7f5224147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f522414d630 0x7f522414da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f522415cd90 con 0x7f522414dfd0 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5224072f20 con 0x7f5210003f60 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522c2c0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5224073440 con 0x7f522414d630 2024-09-10T11:31:17.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522a85d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522a85d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36868/0 (socket says 172.21.15.99:36868) 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.588+0000 7f522a85d640 1 -- 172.21.15.99:0/28934885 learned_addr learned my addr 172.21.15.99:0/28934885 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a05c640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 0x7f5224147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522985b640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f522414d630 0x7f522414da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 -- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f522414d630 msgr2=0x7f522414da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f522414d630 0x7f522414da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 -- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 msgr2=0x7f5224147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 0x7f5224147010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 -- 172.21.15.99:0/28934885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5224147f60 con 0x7f522414dfd0 2024-09-10T11:31:17.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a05c640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 0x7f5224147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522a85d640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52240cd5a0 tx=0x7f521c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c006b50 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c006d10 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c003970 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522414e450 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.589+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522414acc0 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.590+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522406ea10 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.590+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f521c01a020 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.591+0000 7f521affd640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 0x7f520806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.591+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f521c08a470 con 0x7f522414dfd0 2024-09-10T11:31:17.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.591+0000 7f522a05c640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 0x7f520806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.591+0000 7f522a05c640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 0x7f520806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5220009bc0 tx=0x7f5220008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.595+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52240d15e0 con 0x7f522414dfd0 2024-09-10T11:31:17.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.756+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f522415d390 con 0x7f520806be60 2024-09-10T11:31:17.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.762+0000 7f521affd640 1 -- 172.21.15.99:0/28934885 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247489 (secure 0 0 0) 0x7f522415d390 con 0x7f520806be60 2024-09-10T11:31:17.767 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:17.769 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:17.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 msgr2=0x7f520806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 0x7f520806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5220009bc0 tx=0x7f5220008040 comp rx=0 tx=0).stop 2024-09-10T11:31:17.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 msgr2=0x7f5224154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f52240cd5a0 tx=0x7f521c00eb90 comp rx=0 tx=0).stop 2024-09-10T11:31:17.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 shutdown_connections 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f520806be60 0x7f520806e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f522414dfd0 0x7f5224154350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f522414d630 0x7f522414da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 --2- 172.21.15.99:0/28934885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5210003f60 0x7f5224147010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 >> 172.21.15.99:0/28934885 conn(0x7f522406bc10 msgr2=0x7f522415dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 shutdown_connections 2024-09-10T11:31:17.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.765+0000 7f522c2c0640 1 -- 172.21.15.99:0/28934885 wait complete. 2024-09-10T11:31:17.777 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-10T11:31:16.920644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560254,"statfs":{"total":1600000000,"available":1597700132,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.424289"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063890+0000","last_change":"2024-09-10T11:31:14.520544+0000","last_active":"2024-09-10T11:31:16.063890+0000","last_peered":"2024-09-10T11:31:16.063890+0000","last_clean":"2024-09-10T11:31:16.063890+0000","last_became_active":"2024-09-10T11:31:14.520291+0000","last_became_peered":"2024-09-10T11:31:14.520291+0000","last_unstale":"2024-09-10T11:31:16.063890+0000","last_undegraded":"2024-09-10T11:31:16.063890+0000","last_fullsized":"2024-09-10T11:31:16.063890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:05.389775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513088+0000","last_change":"2024-09-10T11:31:14.515821+0000","last_active":"2024-09-10T11:31:15.513088+0000","last_peered":"2024-09-10T11:31:15.513088+0000","last_clean":"2024-09-10T11:31:15.513088+0000","last_became_active":"2024-09-10T11:31:14.515561+0000","last_became_peered":"2024-09-10T11:31:14.515561+0000","last_unstale":"2024-09-10T11:31:15.513088+0000","last_undegraded":"2024-09-10T11:31:15.513088+0000","last_fullsized":"2024-09-10T11:31:15.513088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:28:43.320611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063885+0000","last_change":"2024-09-10T11:31:14.520236+0000","last_active":"2024-09-10T11:31:16.063885+0000","last_peered":"2024-09-10T11:31:16.063885+0000","last_clean":"2024-09-10T11:31:16.063885+0000","last_became_active":"2024-09-10T11:31:14.514850+0000","last_became_peered":"2024-09-10T11:31:14.514850+0000","last_unstale":"2024-09-10T11:31:16.063885+0000","last_undegraded":"2024-09-10T11:31:16.063885+0000","last_fullsized":"2024-09-10T11:31:16.063885+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:22:59.636401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063766+0000","last_change":"2024-09-10T11:31:14.521746+0000","last_active":"2024-09-10T11:31:16.063766+0000","last_peered":"2024-09-10T11:31:16.063766+0000","last_clean":"2024-09-10T11:31:16.063766+0000","last_became_active":"2024-09-10T11:31:14.519927+0000","last_became_peered":"2024-09-10T11:31:14.519927+0000","last_unstale":"2024-09-10T11:31:16.063766+0000","last_undegraded":"2024-09-10T11:31:16.063766+0000","last_fullsized":"2024-09-10T11:31:16.063766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:50.231766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513204+0000","last_change":"2024-09-10T11:31:14.520758+0000","last_active":"2024-09-10T11:31:15.513204+0000","last_peered":"2024-09-10T11:31:15.513204+0000","last_clean":"2024-09-10T11:31:15.513204+0000","last_became_active":"2024-09-10T11:31:14.520477+0000","last_became_peered":"2024-09-10T11:31:14.520477+0000","last_unstale":"2024-09-10T11:31:15.513204+0000","last_undegraded":"2024-09-10T11:31:15.513204+0000","last_fullsized":"2024-09-10T11:31:15.513204+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:59.405732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523032+0000","last_change":"2024-09-10T11:31:14.517216+0000","last_active":"2024-09-10T11:31:15.523032+0000","last_peered":"2024-09-10T11:31:15.523032+0000","last_clean":"2024-09-10T11:31:15.523032+0000","last_became_active":"2024-09-10T11:31:14.516995+0000","last_became_peered":"2024-09-10T11:31:14.516995+0000","last_unstale":"2024-09-10T11:31:15.523032+0000","last_undegraded":"2024-09-10T11:31:15.523032+0000","last_fullsized":"2024-09-10T11:31:15.523032+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:19.270924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064509+0000","last_change":"2024-09-10T11:31:14.521880+0000","last_active":"2024-09-10T11:31:16.064509+0000","last_peered":"2024-09-10T11:31:16.064509+0000","last_clean":"2024-09-10T11:31:16.064509+0000","last_became_active":"2024-09-10T11:31:14.521574+0000","last_became_peered":"2024-09-10T11:31:14.521574+0000","last_unstale":"2024-09-10T11:31:16.064509+0000","last_undegraded":"2024-09-10T11:31:16.064509+0000","last_fullsized":"2024-09-10T11:31:16.064509+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:44.995070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063818+0000","last_change":"2024-09-10T11:31:14.523038+0000","last_active":"2024-09-10T11:31:16.063818+0000","last_peered":"2024-09-10T11:31:16.063818+0000","last_clean":"2024-09-10T11:31:16.063818+0000","last_became_active":"2024-09-10T11:31:14.521436+0000","last_became_peered":"2024-09-10T11:31:14.521436+0000","last_unstale":"2024-09-10T11:31:16.063818+0000","last_undegraded":"2024-09-10T11:31:16.063818+0000","last_fullsized":"2024-09-10T11:31:16.063818+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:47.013779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064775+0000","last_change":"2024-09-10T11:31:14.513886+0000","last_active":"2024-09-10T11:31:16.064775+0000","last_peered":"2024-09-10T11:31:16.064775+0000","last_clean":"2024-09-10T11:31:16.064775+0000","last_became_active":"2024-09-10T11:31:14.513638+0000","last_became_peered":"2024-09-10T11:31:14.513638+0000","last_unstale":"2024-09-10T11:31:16.064775+0000","last_undegraded":"2024-09-10T11:31:16.064775+0000","last_fullsized":"2024-09-10T11:31:16.064775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:37.342836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063773+0000","last_change":"2024-09-10T11:31:14.516924+0000","last_active":"2024-09-10T11:31:16.063773+0000","last_peered":"2024-09-10T11:31:16.063773+0000","last_clean":"2024-09-10T11:31:16.063773+0000","last_became_active":"2024-09-10T11:31:14.516171+0000","last_became_peered":"2024-09-10T11:31:14.516171+0000","last_unstale":"2024-09-10T11:31:16.063773+0000","last_undegraded":"2024-09-10T11:31:16.063773+0000","last_fullsized":"2024-09-10T11:31:16.063773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:54.456682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513222+0000","last_change":"2024-09-10T11:31:14.516928+0000","last_active":"2024-09-10T11:31:15.513222+0000","last_peered":"2024-09-10T11:31:15.513222+0000","last_clean":"2024-09-10T11:31:15.513222+0000","last_became_active":"2024-09-10T11:31:14.516703+0000","last_became_peered":"2024-09-10T11:31:14.516703+0000","last_unstale":"2024-09-10T11:31:15.513222+0000","last_undegraded":"2024-09-10T11:31:15.513222+0000","last_fullsized":"2024-09-10T11:31:15.513222+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:41.622401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064732+0000","last_change":"2024-09-10T11:31:14.520023+0000","last_active":"2024-09-10T11:31:16.064732+0000","last_peered":"2024-09-10T11:31:16.064732+0000","last_clean":"2024-09-10T11:31:16.064732+0000","last_became_active":"2024-09-10T11:31:14.515759+0000","last_became_peered":"2024-09-10T11:31:14.515759+0000","last_unstale":"2024-09-10T11:31:16.064732+0000","last_undegraded":"2024-09-10T11:31:16.064732+0000","last_fullsized":"2024-09-10T11:31:16.064732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:06.404563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515489+0000","last_change":"2024-09-10T11:31:14.517385+0000","last_active":"2024-09-10T11:31:15.515489+0000","last_peered":"2024-09-10T11:31:15.515489+0000","last_clean":"2024-09-10T11:31:15.515489+0000","last_became_active":"2024-09-10T11:31:14.515086+0000","last_became_peered":"2024-09-10T11:31:14.515086+0000","last_unstale":"2024-09-10T11:31:15.515489+0000","last_undegraded":"2024-09-10T11:31:15.515489+0000","last_fullsized":"2024-09-10T11:31:15.515489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:23.696361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064218+0000","last_change":"2024-09-10T11:31:14.520862+0000","last_active":"2024-09-10T11:31:16.064218+0000","last_peered":"2024-09-10T11:31:16.064218+0000","last_clean":"2024-09-10T11:31:16.064218+0000","last_became_active":"2024-09-10T11:31:14.518068+0000","last_became_peered":"2024-09-10T11:31:14.518068+0000","last_unstale":"2024-09-10T11:31:16.064218+0000","last_undegraded":"2024-09-10T11:31:16.064218+0000","last_fullsized":"2024-09-10T11:31:16.064218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:18.966305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.525155+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.524763+0000","last_became_peered":"2024-09-10T11:31:14.524763+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.825777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064291+0000","last_change":"2024-09-10T11:31:14.519244+0000","last_active":"2024-09-10T11:31:16.064291+0000","last_peered":"2024-09-10T11:31:16.064291+0000","last_clean":"2024-09-10T11:31:16.064291+0000","last_became_active":"2024-09-10T11:31:14.519024+0000","last_became_peered":"2024-09-10T11:31:14.519024+0000","last_unstale":"2024-09-10T11:31:16.064291+0000","last_undegraded":"2024-09-10T11:31:16.064291+0000","last_fullsized":"2024-09-10T11:31:16.064291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:13.389476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513155+0000","last_change":"2024-09-10T11:31:14.521724+0000","last_active":"2024-09-10T11:31:15.513155+0000","last_peered":"2024-09-10T11:31:15.513155+0000","last_clean":"2024-09-10T11:31:15.513155+0000","last_became_active":"2024-09-10T11:31:14.521159+0000","last_became_peered":"2024-09-10T11:31:14.521159+0000","last_unstale":"2024-09-10T11:31:15.513155+0000","last_undegraded":"2024-09-10T11:31:15.513155+0000","last_fullsized":"2024-09-10T11:31:15.513155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:28.750442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523039+0000","last_change":"2024-09-10T11:31:14.513624+0000","last_active":"2024-09-10T11:31:15.523039+0000","last_peered":"2024-09-10T11:31:15.523039+0000","last_clean":"2024-09-10T11:31:15.523039+0000","last_became_active":"2024-09-10T11:31:14.513389+0000","last_became_peered":"2024-09-10T11:31:14.513389+0000","last_unstale":"2024-09-10T11:31:15.523039+0000","last_undegraded":"2024-09-10T11:31:15.523039+0000","last_fullsized":"2024-09-10T11:31:15.523039+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:11.159093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.522824+0000","last_change":"2024-09-10T11:31:14.510651+0000","last_active":"2024-09-10T11:31:15.522824+0000","last_peered":"2024-09-10T11:31:15.522824+0000","last_clean":"2024-09-10T11:31:15.522824+0000","last_became_active":"2024-09-10T11:31:14.510416+0000","last_became_peered":"2024-09-10T11:31:14.510416+0000","last_unstale":"2024-09-10T11:31:15.522824+0000","last_undegraded":"2024-09-10T11:31:15.522824+0000","last_fullsized":"2024-09-10T11:31:15.522824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:34.146570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515531+0000","last_change":"2024-09-10T11:31:14.521940+0000","last_active":"2024-09-10T11:31:15.515531+0000","last_peered":"2024-09-10T11:31:15.515531+0000","last_clean":"2024-09-10T11:31:15.515531+0000","last_became_active":"2024-09-10T11:31:14.521683+0000","last_became_peered":"2024-09-10T11:31:14.521683+0000","last_unstale":"2024-09-10T11:31:15.515531+0000","last_undegraded":"2024-09-10T11:31:15.515531+0000","last_fullsized":"2024-09-10T11:31:15.515531+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:46.284242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064423+0000","last_change":"2024-09-10T11:31:14.522106+0000","last_active":"2024-09-10T11:31:16.064423+0000","last_peered":"2024-09-10T11:31:16.064423+0000","last_clean":"2024-09-10T11:31:16.064423+0000","last_became_active":"2024-09-10T11:31:14.521560+0000","last_became_peered":"2024-09-10T11:31:14.521560+0000","last_unstale":"2024-09-10T11:31:16.064423+0000","last_undegraded":"2024-09-10T11:31:16.064423+0000","last_fullsized":"2024-09-10T11:31:16.064423+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:29.933314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064579+0000","last_change":"2024-09-10T11:31:14.518944+0000","last_active":"2024-09-10T11:31:16.064579+0000","last_peered":"2024-09-10T11:31:16.064579+0000","last_clean":"2024-09-10T11:31:16.064579+0000","last_became_active":"2024-09-10T11:31:14.517995+0000","last_became_peered":"2024-09-10T11:31:14.517995+0000","last_unstale":"2024-09-10T11:31:16.064579+0000","last_undegraded":"2024-09-10T11:31:16.064579+0000","last_fullsized":"2024-09-10T11:31:16.064579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:40.011208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515615+0000","last_change":"2024-09-10T11:31:14.509386+0000","last_active":"2024-09-10T11:31:15.515615+0000","last_peered":"2024-09-10T11:31:15.515615+0000","last_clean":"2024-09-10T11:31:15.515615+0000","last_became_active":"2024-09-10T11:31:14.509109+0000","last_became_peered":"2024-09-10T11:31:14.509109+0000","last_unstale":"2024-09-10T11:31:15.515615+0000","last_undegraded":"2024-09-10T11:31:15.515615+0000","last_fullsized":"2024-09-10T11:31:15.515615+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:46.687588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063963+0000","last_change":"2024-09-10T11:31:14.518236+0000","last_active":"2024-09-10T11:31:16.063963+0000","last_peered":"2024-09-10T11:31:16.063963+0000","last_clean":"2024-09-10T11:31:16.063963+0000","last_became_active":"2024-09-10T11:31:14.518030+0000","last_became_peered":"2024-09-10T11:31:14.518030+0000","last_unstale":"2024-09-10T11:31:16.063963+0000","last_undegraded":"2024-09-10T11:31:16.063963+0000","last_fullsized":"2024-09-10T11:31:16.063963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:25:32.832514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.516427+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.515330+0000","last_became_peered":"2024-09-10T11:31:14.515330+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:25.115249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064316+0000","last_change":"2024-09-10T11:31:14.511929+0000","last_active":"2024-09-10T11:31:16.064316+0000","last_peered":"2024-09-10T11:31:16.064316+0000","last_clean":"2024-09-10T11:31:16.064316+0000","last_became_active":"2024-09-10T11:31:14.511710+0000","last_became_peered":"2024-09-10T11:31:14.511710+0000","last_unstale":"2024-09-10T11:31:16.064316+0000","last_undegraded":"2024-09-10T11:31:16.064316+0000","last_fullsized":"2024-09-10T11:31:16.064316+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:42:07.427370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513218+0000","last_change":"2024-09-10T11:31:14.523316+0000","last_active":"2024-09-10T11:31:15.513218+0000","last_peered":"2024-09-10T11:31:15.513218+0000","last_clean":"2024-09-10T11:31:15.513218+0000","last_became_active":"2024-09-10T11:31:14.522736+0000","last_became_peered":"2024-09-10T11:31:14.522736+0000","last_unstale":"2024-09-10T11:31:15.513218+0000","last_undegraded":"2024-09-10T11:31:15.513218+0000","last_fullsized":"2024-09-10T11:31:15.513218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:39.936447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063964+0000","last_change":"2024-09-10T11:31:14.522018+0000","last_active":"2024-09-10T11:31:16.063964+0000","last_peered":"2024-09-10T11:31:16.063964+0000","last_clean":"2024-09-10T11:31:16.063964+0000","last_became_active":"2024-09-10T11:31:14.521149+0000","last_became_peered":"2024-09-10T11:31:14.521149+0000","last_unstale":"2024-09-10T11:31:16.063964+0000","last_undegraded":"2024-09-10T11:31:16.063964+0000","last_fullsized":"2024-09-10T11:31:16.063964+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:10.551902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063737+0000","last_change":"2024-09-10T11:31:14.522697+0000","last_active":"2024-09-10T11:31:16.063737+0000","last_peered":"2024-09-10T11:31:16.063737+0000","last_clean":"2024-09-10T11:31:16.063737+0000","last_became_active":"2024-09-10T11:31:14.522450+0000","last_became_peered":"2024-09-10T11:31:14.522450+0000","last_unstale":"2024-09-10T11:31:16.063737+0000","last_undegraded":"2024-09-10T11:31:16.063737+0000","last_fullsized":"2024-09-10T11:31:16.063737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:51:56.166552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064242+0000","last_change":"2024-09-10T11:31:14.522506+0000","last_active":"2024-09-10T11:31:16.064242+0000","last_peered":"2024-09-10T11:31:16.064242+0000","last_clean":"2024-09-10T11:31:16.064242+0000","last_became_active":"2024-09-10T11:31:14.520994+0000","last_became_peered":"2024-09-10T11:31:14.520994+0000","last_unstale":"2024-09-10T11:31:16.064242+0000","last_undegraded":"2024-09-10T11:31:16.064242+0000","last_fullsized":"2024-09-10T11:31:16.064242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:03.419687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064094+0000","last_change":"2024-09-10T11:31:14.514635+0000","last_active":"2024-09-10T11:31:16.064094+0000","last_peered":"2024-09-10T11:31:16.064094+0000","last_clean":"2024-09-10T11:31:16.064094+0000","last_became_active":"2024-09-10T11:31:14.514346+0000","last_became_peered":"2024-09-10T11:31:14.514346+0000","last_unstale":"2024-09-10T11:31:16.064094+0000","last_undegraded":"2024-09-10T11:31:16.064094+0000","last_fullsized":"2024-09-10T11:31:16.064094+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:14.719377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513261+0000","last_change":"2024-09-10T11:31:14.521552+0000","last_active":"2024-09-10T11:31:15.513261+0000","last_peered":"2024-09-10T11:31:15.513261+0000","last_clean":"2024-09-10T11:31:15.513261+0000","last_became_active":"2024-09-10T11:31:14.520865+0000","last_became_peered":"2024-09-10T11:31:14.520865+0000","last_unstale":"2024-09-10T11:31:15.513261+0000","last_undegraded":"2024-09-10T11:31:15.513261+0000","last_fullsized":"2024-09-10T11:31:15.513261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:38:37.013148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515052+0000","last_change":"2024-09-10T11:31:14.515127+0000","last_active":"2024-09-10T11:31:15.515052+0000","last_peered":"2024-09-10T11:31:15.515052+0000","last_clean":"2024-09-10T11:31:15.515052+0000","last_became_active":"2024-09-10T11:31:14.514886+0000","last_became_peered":"2024-09-10T11:31:14.514886+0000","last_unstale":"2024-09-10T11:31:15.515052+0000","last_undegraded":"2024-09-10T11:31:15.515052+0000","last_fullsized":"2024-09-10T11:31:15.515052+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:36:05.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513299+0000","last_change":"2024-09-10T11:31:13.520024+0000","last_active":"2024-09-10T11:31:15.513299+0000","last_peered":"2024-09-10T11:31:15.513299+0000","last_clean":"2024-09-10T11:31:15.513299+0000","last_became_active":"2024-09-10T11:31:13.519800+0000","last_became_peered":"2024-09-10T11:31:13.519800+0000","last_unstale":"2024-09-10T11:31:15.513299+0000","last_undegraded":"2024-09-10T11:31:15.513299+0000","last_fullsized":"2024-09-10T11:31:15.513299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:26.967099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064201+0000","last_change":"2024-09-10T11:31:14.523240+0000","last_active":"2024-09-10T11:31:16.064201+0000","last_peered":"2024-09-10T11:31:16.064201+0000","last_clean":"2024-09-10T11:31:16.064201+0000","last_became_active":"2024-09-10T11:31:14.521740+0000","last_became_peered":"2024-09-10T11:31:14.521740+0000","last_unstale":"2024-09-10T11:31:16.064201+0000","last_undegraded":"2024-09-10T11:31:16.064201+0000","last_fullsized":"2024-09-10T11:31:16.064201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:00.924364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064409+0000","last_change":"2024-09-10T11:31:13.516467+0000","last_active":"2024-09-10T11:31:16.064409+0000","last_peered":"2024-09-10T11:31:16.064409+0000","last_clean":"2024-09-10T11:31:16.064409+0000","last_became_active":"2024-09-10T11:31:13.515944+0000","last_became_peered":"2024-09-10T11:31:13.515944+0000","last_unstale":"2024-09-10T11:31:16.064409+0000","last_undegraded":"2024-09-10T11:31:16.064409+0000","last_fullsized":"2024-09-10T11:31:16.064409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:25.734819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064246+0000","last_change":"2024-09-10T11:31:14.516163+0000","last_active":"2024-09-10T11:31:16.064246+0000","last_peered":"2024-09-10T11:31:16.064246+0000","last_clean":"2024-09-10T11:31:16.064246+0000","last_became_active":"2024-09-10T11:31:14.515949+0000","last_became_peered":"2024-09-10T11:31:14.515949+0000","last_unstale":"2024-09-10T11:31:16.064246+0000","last_undegraded":"2024-09-10T11:31:16.064246+0000","last_fullsized":"2024-09-10T11:31:16.064246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:25.705984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523111+0000","last_change":"2024-09-10T11:31:13.517924+0000","last_active":"2024-09-10T11:31:15.523111+0000","last_peered":"2024-09-10T11:31:15.523111+0000","last_clean":"2024-09-10T11:31:15.523111+0000","last_became_active":"2024-09-10T11:31:13.517680+0000","last_became_peered":"2024-09-10T11:31:13.517680+0000","last_unstale":"2024-09-10T11:31:15.523111+0000","last_undegraded":"2024-09-10T11:31:15.523111+0000","last_fullsized":"2024-09-10T11:31:15.523111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:35.392288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064299+0000","last_change":"2024-09-10T11:31:14.516627+0000","last_active":"2024-09-10T11:31:16.064299+0000","last_peered":"2024-09-10T11:31:16.064299+0000","last_clean":"2024-09-10T11:31:16.064299+0000","last_became_active":"2024-09-10T11:31:14.515877+0000","last_became_peered":"2024-09-10T11:31:14.515877+0000","last_unstale":"2024-09-10T11:31:16.064299+0000","last_undegraded":"2024-09-10T11:31:16.064299+0000","last_fullsized":"2024-09-10T11:31:16.064299+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:36.098774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064063+0000","last_change":"2024-09-10T11:31:13.513866+0000","last_active":"2024-09-10T11:31:16.064063+0000","last_peered":"2024-09-10T11:31:16.064063+0000","last_clean":"2024-09-10T11:31:16.064063+0000","last_became_active":"2024-09-10T11:31:13.510927+0000","last_became_peered":"2024-09-10T11:31:13.510927+0000","last_unstale":"2024-09-10T11:31:16.064063+0000","last_undegraded":"2024-09-10T11:31:16.064063+0000","last_fullsized":"2024-09-10T11:31:16.064063+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:24.499517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064075+0000","last_change":"2024-09-10T11:31:14.520437+0000","last_active":"2024-09-10T11:31:16.064075+0000","last_peered":"2024-09-10T11:31:16.064075+0000","last_clean":"2024-09-10T11:31:16.064075+0000","last_became_active":"2024-09-10T11:31:14.515291+0000","last_became_peered":"2024-09-10T11:31:14.515291+0000","last_unstale":"2024-09-10T11:31:16.064075+0000","last_undegraded":"2024-09-10T11:31:16.064075+0000","last_fullsized":"2024-09-10T11:31:16.064075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:23.697849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064695+0000","last_change":"2024-09-10T11:31:13.519201+0000","last_active":"2024-09-10T11:31:16.064695+0000","last_peered":"2024-09-10T11:31:16.064695+0000","last_clean":"2024-09-10T11:31:16.064695+0000","last_became_active":"2024-09-10T11:31:13.518974+0000","last_became_peered":"2024-09-10T11:31:13.518974+0000","last_unstale":"2024-09-10T11:31:16.064695+0000","last_undegraded":"2024-09-10T11:31:16.064695+0000","last_fullsized":"2024-09-10T11:31:16.064695+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:04:48.077185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064646+0000","last_change":"2024-09-10T11:31:14.517590+0000","last_active":"2024-09-10T11:31:16.064646+0000","last_peered":"2024-09-10T11:31:16.064646+0000","last_clean":"2024-09-10T11:31:16.064646+0000","last_became_active":"2024-09-10T11:31:14.515618+0000","last_became_peered":"2024-09-10T11:31:14.515618+0000","last_unstale":"2024-09-10T11:31:16.064646+0000","last_undegraded":"2024-09-10T11:31:16.064646+0000","last_fullsized":"2024-09-10T11:31:16.064646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.870654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064428+0000","last_change":"2024-09-10T11:31:13.516663+0000","last_active":"2024-09-10T11:31:16.064428+0000","last_peered":"2024-09-10T11:31:16.064428+0000","last_clean":"2024-09-10T11:31:16.064428+0000","last_became_active":"2024-09-10T11:31:13.515737+0000","last_became_peered":"2024-09-10T11:31:13.515737+0000","last_unstale":"2024-09-10T11:31:16.064428+0000","last_undegraded":"2024-09-10T11:31:16.064428+0000","last_fullsized":"2024-09-10T11:31:16.064428+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:24.805568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063984+0000","last_change":"2024-09-10T11:31:14.524693+0000","last_active":"2024-09-10T11:31:16.063984+0000","last_peered":"2024-09-10T11:31:16.063984+0000","last_clean":"2024-09-10T11:31:16.063984+0000","last_became_active":"2024-09-10T11:31:14.524107+0000","last_became_peered":"2024-09-10T11:31:14.524107+0000","last_unstale":"2024-09-10T11:31:16.063984+0000","last_undegraded":"2024-09-10T11:31:16.063984+0000","last_fullsized":"2024-09-10T11:31:16.063984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:20.761540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523109+0000","last_change":"2024-09-10T11:31:13.513183+0000","last_active":"2024-09-10T11:31:15.523109+0000","last_peered":"2024-09-10T11:31:15.523109+0000","last_clean":"2024-09-10T11:31:15.523109+0000","last_became_active":"2024-09-10T11:31:13.512547+0000","last_became_peered":"2024-09-10T11:31:13.512547+0000","last_unstale":"2024-09-10T11:31:15.523109+0000","last_undegraded":"2024-09-10T11:31:15.523109+0000","last_fullsized":"2024-09-10T11:31:15.523109+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:11.447778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063602+0000","last_change":"2024-09-10T11:31:14.520435+0000","last_active":"2024-09-10T11:31:16.063602+0000","last_peered":"2024-09-10T11:31:16.063602+0000","last_clean":"2024-09-10T11:31:16.063602+0000","last_became_active":"2024-09-10T11:31:14.517253+0000","last_became_peered":"2024-09-10T11:31:14.517253+0000","last_unstale":"2024-09-10T11:31:16.063602+0000","last_undegraded":"2024-09-10T11:31:16.063602+0000","last_fullsized":"2024-09-10T11:31:16.063602+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:34.856235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064117+0000","last_change":"2024-09-10T11:31:13.516670+0000","last_active":"2024-09-10T11:31:16.064117+0000","last_peered":"2024-09-10T11:31:16.064117+0000","last_clean":"2024-09-10T11:31:16.064117+0000","last_became_active":"2024-09-10T11:31:13.516441+0000","last_became_peered":"2024-09-10T11:31:13.516441+0000","last_unstale":"2024-09-10T11:31:16.064117+0000","last_undegraded":"2024-09-10T11:31:16.064117+0000","last_fullsized":"2024-09-10T11:31:16.064117+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:38.508209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513500+0000","last_change":"2024-09-10T11:31:14.514509+0000","last_active":"2024-09-10T11:31:15.513500+0000","last_peered":"2024-09-10T11:31:15.513500+0000","last_clean":"2024-09-10T11:31:15.513500+0000","last_became_active":"2024-09-10T11:31:14.513904+0000","last_became_peered":"2024-09-10T11:31:14.513904+0000","last_unstale":"2024-09-10T11:31:15.513500+0000","last_undegraded":"2024-09-10T11:31:15.513500+0000","last_fullsized":"2024-09-10T11:31:15.513500+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:00.742503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513185+0000","last_change":"2024-09-10T11:31:13.519004+0000","last_active":"2024-09-10T11:31:15.513185+0000","last_peered":"2024-09-10T11:31:15.513185+0000","last_clean":"2024-09-10T11:31:15.513185+0000","last_became_active":"2024-09-10T11:31:13.518744+0000","last_became_peered":"2024-09-10T11:31:13.518744+0000","last_unstale":"2024-09-10T11:31:15.513185+0000","last_undegraded":"2024-09-10T11:31:15.513185+0000","last_fullsized":"2024-09-10T11:31:15.513185+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:12.285035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063800+0000","last_change":"2024-09-10T11:31:14.515923+0000","last_active":"2024-09-10T11:31:16.063800+0000","last_peered":"2024-09-10T11:31:16.063800+0000","last_clean":"2024-09-10T11:31:16.063800+0000","last_became_active":"2024-09-10T11:31:14.515177+0000","last_became_peered":"2024-09-10T11:31:14.515177+0000","last_unstale":"2024-09-10T11:31:16.063800+0000","last_undegraded":"2024-09-10T11:31:16.063800+0000","last_fullsized":"2024-09-10T11:31:16.063800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:47.104953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513425+0000","last_change":"2024-09-10T11:31:13.518158+0000","last_active":"2024-09-10T11:31:15.513425+0000","last_peered":"2024-09-10T11:31:15.513425+0000","last_clean":"2024-09-10T11:31:15.513425+0000","last_became_active":"2024-09-10T11:31:13.517660+0000","last_became_peered":"2024-09-10T11:31:13.517660+0000","last_unstale":"2024-09-10T11:31:15.513425+0000","last_undegraded":"2024-09-10T11:31:15.513425+0000","last_fullsized":"2024-09-10T11:31:15.513425+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:05.641010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064486+0000","last_change":"2024-09-10T11:31:14.522063+0000","last_active":"2024-09-10T11:31:16.064486+0000","last_peered":"2024-09-10T11:31:16.064486+0000","last_clean":"2024-09-10T11:31:16.064486+0000","last_became_active":"2024-09-10T11:31:14.520217+0000","last_became_peered":"2024-09-10T11:31:14.520217+0000","last_unstale":"2024-09-10T11:31:16.064486+0000","last_undegraded":"2024-09-10T11:31:16.064486+0000","last_fullsized":"2024-09-10T11:31:16.064486+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:43.697667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513488+0000","last_change":"2024-09-10T11:31:13.511626+0000","last_active":"2024-09-10T11:31:15.513488+0000","last_peered":"2024-09-10T11:31:15.513488+0000","last_clean":"2024-09-10T11:31:15.513488+0000","last_became_active":"2024-09-10T11:31:13.511367+0000","last_became_peered":"2024-09-10T11:31:13.511367+0000","last_unstale":"2024-09-10T11:31:15.513488+0000","last_undegraded":"2024-09-10T11:31:15.513488+0000","last_fullsized":"2024-09-10T11:31:15.513488+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:14.937108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513207+0000","last_change":"2024-09-10T11:31:14.517040+0000","last_active":"2024-09-10T11:31:15.513207+0000","last_peered":"2024-09-10T11:31:15.513207+0000","last_clean":"2024-09-10T11:31:15.513207+0000","last_became_active":"2024-09-10T11:31:14.516816+0000","last_became_peered":"2024-09-10T11:31:14.516816+0000","last_unstale":"2024-09-10T11:31:15.513207+0000","last_undegraded":"2024-09-10T11:31:15.513207+0000","last_fullsized":"2024-09-10T11:31:15.513207+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:43.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515435+0000","last_change":"2024-09-10T11:31:13.513864+0000","last_active":"2024-09-10T11:31:15.515435+0000","last_peered":"2024-09-10T11:31:15.515435+0000","last_clean":"2024-09-10T11:31:15.515435+0000","last_became_active":"2024-09-10T11:31:13.513392+0000","last_became_peered":"2024-09-10T11:31:13.513392+0000","last_unstale":"2024-09-10T11:31:15.515435+0000","last_undegraded":"2024-09-10T11:31:15.515435+0000","last_fullsized":"2024-09-10T11:31:15.515435+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:32.396996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515362+0000","last_change":"2024-09-10T11:31:14.510565+0000","last_active":"2024-09-10T11:31:15.515362+0000","last_peered":"2024-09-10T11:31:15.515362+0000","last_clean":"2024-09-10T11:31:15.515362+0000","last_became_active":"2024-09-10T11:31:14.510323+0000","last_became_peered":"2024-09-10T11:31:14.510323+0000","last_unstale":"2024-09-10T11:31:15.515362+0000","last_undegraded":"2024-09-10T11:31:15.515362+0000","last_fullsized":"2024-09-10T11:31:15.515362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.783845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064217+0000","last_change":"2024-09-10T11:31:13.519970+0000","last_active":"2024-09-10T11:31:16.064217+0000","last_peered":"2024-09-10T11:31:16.064217+0000","last_clean":"2024-09-10T11:31:16.064217+0000","last_became_active":"2024-09-10T11:31:13.519444+0000","last_became_peered":"2024-09-10T11:31:13.519444+0000","last_unstale":"2024-09-10T11:31:16.064217+0000","last_undegraded":"2024-09-10T11:31:16.064217+0000","last_fullsized":"2024-09-10T11:31:16.064217+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:39.129313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064076+0000","last_change":"2024-09-10T11:31:14.510319+0000","last_active":"2024-09-10T11:31:16.064076+0000","last_peered":"2024-09-10T11:31:16.064076+0000","last_clean":"2024-09-10T11:31:16.064076+0000","last_became_active":"2024-09-10T11:31:14.510057+0000","last_became_peered":"2024-09-10T11:31:14.510057+0000","last_unstale":"2024-09-10T11:31:16.064076+0000","last_undegraded":"2024-09-10T11:31:16.064076+0000","last_fullsized":"2024-09-10T11:31:16.064076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:46.501405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064418+0000","last_change":"2024-09-10T11:31:13.516451+0000","last_active":"2024-09-10T11:31:16.064418+0000","last_peered":"2024-09-10T11:31:16.064418+0000","last_clean":"2024-09-10T11:31:16.064418+0000","last_became_active":"2024-09-10T11:31:13.515947+0000","last_became_peered":"2024-09-10T11:31:13.515947+0000","last_unstale":"2024-09-10T11:31:16.064418+0000","last_undegraded":"2024-09-10T11:31:16.064418+0000","last_fullsized":"2024-09-10T11:31:16.064418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:08.085889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064276+0000","last_change":"2024-09-10T11:31:14.514288+0000","last_active":"2024-09-10T11:31:16.064276+0000","last_peered":"2024-09-10T11:31:16.064276+0000","last_clean":"2024-09-10T11:31:16.064276+0000","last_became_active":"2024-09-10T11:31:14.514037+0000","last_became_peered":"2024-09-10T11:31:14.514037+0000","last_unstale":"2024-09-10T11:31:16.064276+0000","last_undegraded":"2024-09-10T11:31:16.064276+0000","last_fullsized":"2024-09-10T11:31:16.064276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:32.614234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063736+0000","last_change":"2024-09-10T11:31:13.512395+0000","last_active":"2024-09-10T11:31:16.063736+0000","last_peered":"2024-09-10T11:31:16.063736+0000","last_clean":"2024-09-10T11:31:16.063736+0000","last_became_active":"2024-09-10T11:31:13.512042+0000","last_became_peered":"2024-09-10T11:31:13.512042+0000","last_unstale":"2024-09-10T11:31:16.063736+0000","last_undegraded":"2024-09-10T11:31:16.063736+0000","last_fullsized":"2024-09-10T11:31:16.063736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:28.809166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513334+0000","last_change":"2024-09-10T11:31:14.521372+0000","last_active":"2024-09-10T11:31:15.513334+0000","last_peered":"2024-09-10T11:31:15.513334+0000","last_clean":"2024-09-10T11:31:15.513334+0000","last_became_active":"2024-09-10T11:31:14.520563+0000","last_became_peered":"2024-09-10T11:31:14.520563+0000","last_unstale":"2024-09-10T11:31:15.513334+0000","last_undegraded":"2024-09-10T11:31:15.513334+0000","last_fullsized":"2024-09-10T11:31:15.513334+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:41.141250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523252+0000","last_change":"2024-09-10T11:31:13.512302+0000","last_active":"2024-09-10T11:31:15.523252+0000","last_peered":"2024-09-10T11:31:15.523252+0000","last_clean":"2024-09-10T11:31:15.523252+0000","last_became_active":"2024-09-10T11:31:13.512062+0000","last_became_peered":"2024-09-10T11:31:13.512062+0000","last_unstale":"2024-09-10T11:31:15.523252+0000","last_undegraded":"2024-09-10T11:31:15.523252+0000","last_fullsized":"2024-09-10T11:31:15.523252+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:10.103139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513512+0000","last_change":"2024-09-10T11:31:14.517992+0000","last_active":"2024-09-10T11:31:15.513512+0000","last_peered":"2024-09-10T11:31:15.513512+0000","last_clean":"2024-09-10T11:31:15.513512+0000","last_became_active":"2024-09-10T11:31:14.517243+0000","last_became_peered":"2024-09-10T11:31:14.517243+0000","last_unstale":"2024-09-10T11:31:15.513512+0000","last_undegraded":"2024-09-10T11:31:15.513512+0000","last_fullsized":"2024-09-10T11:31:15.513512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:28.492681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523216+0000","last_change":"2024-09-10T11:31:13.511780+0000","last_active":"2024-09-10T11:31:15.523216+0000","last_peered":"2024-09-10T11:31:15.523216+0000","last_clean":"2024-09-10T11:31:15.523216+0000","last_became_active":"2024-09-10T11:31:13.511513+0000","last_became_peered":"2024-09-10T11:31:13.511513+0000","last_unstale":"2024-09-10T11:31:15.523216+0000","last_undegraded":"2024-09-10T11:31:15.523216+0000","last_fullsized":"2024-09-10T11:31:15.523216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:56.401271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064841+0000","last_change":"2024-09-10T11:31:14.516007+0000","last_active":"2024-09-10T11:31:16.064841+0000","last_peered":"2024-09-10T11:31:16.064841+0000","last_clean":"2024-09-10T11:31:16.064841+0000","last_became_active":"2024-09-10T11:31:14.515747+0000","last_became_peered":"2024-09-10T11:31:14.515747+0000","last_unstale":"2024-09-10T11:31:16.064841+0000","last_undegraded":"2024-09-10T11:31:16.064841+0000","last_fullsized":"2024-09-10T11:31:16.064841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:37.712553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064327+0000","last_change":"2024-09-10T11:31:13.519713+0000","last_active":"2024-09-10T11:31:16.064327+0000","last_peered":"2024-09-10T11:31:16.064327+0000","last_clean":"2024-09-10T11:31:16.064327+0000","last_became_active":"2024-09-10T11:31:13.519090+0000","last_became_peered":"2024-09-10T11:31:13.519090+0000","last_unstale":"2024-09-10T11:31:16.064327+0000","last_undegraded":"2024-09-10T11:31:16.064327+0000","last_fullsized":"2024-09-10T11:31:16.064327+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:09.363236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513322+0000","last_change":"2024-09-10T11:31:14.522994+0000","last_active":"2024-09-10T11:31:15.513322+0000","last_peered":"2024-09-10T11:31:15.513322+0000","last_clean":"2024-09-10T11:31:15.513322+0000","last_became_active":"2024-09-10T11:31:14.522196+0000","last_became_peered":"2024-09-10T11:31:14.522196+0000","last_unstale":"2024-09-10T11:31:15.513322+0000","last_undegraded":"2024-09-10T11:31:15.513322+0000","last_fullsized":"2024-09-10T11:31:15.513322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:24.388027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513308+0000","last_change":"2024-09-10T11:31:13.515294+0000","last_active":"2024-09-10T11:31:15.513308+0000","last_peered":"2024-09-10T11:31:15.513308+0000","last_clean":"2024-09-10T11:31:15.513308+0000","last_became_active":"2024-09-10T11:31:13.514887+0000","last_became_peered":"2024-09-10T11:31:13.514887+0000","last_unstale":"2024-09-10T11:31:15.513308+0000","last_undegraded":"2024-09-10T11:31:15.513308+0000","last_fullsized":"2024-09-10T11:31:15.513308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:50.109398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523187+0000","last_change":"2024-09-10T11:31:14.520452+0000","last_active":"2024-09-10T11:31:15.523187+0000","last_peered":"2024-09-10T11:31:15.523187+0000","last_clean":"2024-09-10T11:31:15.523187+0000","last_became_active":"2024-09-10T11:31:14.520132+0000","last_became_peered":"2024-09-10T11:31:14.520132+0000","last_unstale":"2024-09-10T11:31:15.523187+0000","last_undegraded":"2024-09-10T11:31:15.523187+0000","last_fullsized":"2024-09-10T11:31:15.523187+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:38.896738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064892+0000","last_change":"2024-09-10T11:31:13.512607+0000","last_active":"2024-09-10T11:31:16.064892+0000","last_peered":"2024-09-10T11:31:16.064892+0000","last_clean":"2024-09-10T11:31:16.064892+0000","last_became_active":"2024-09-10T11:31:13.511603+0000","last_became_peered":"2024-09-10T11:31:13.511603+0000","last_unstale":"2024-09-10T11:31:16.064892+0000","last_undegraded":"2024-09-10T11:31:16.064892+0000","last_fullsized":"2024-09-10T11:31:16.064892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:23:54.656308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515238+0000","last_change":"2024-09-10T11:31:14.516178+0000","last_active":"2024-09-10T11:31:15.515238+0000","last_peered":"2024-09-10T11:31:15.515238+0000","last_clean":"2024-09-10T11:31:15.515238+0000","last_became_active":"2024-09-10T11:31:14.515903+0000","last_became_peered":"2024-09-10T11:31:14.515903+0000","last_unstale":"2024-09-10T11:31:15.515238+0000","last_undegraded":"2024-09-10T11:31:15.515238+0000","last_fullsized":"2024-09-10T11:31:15.515238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:17.150776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513347+0000","last_change":"2024-09-10T11:31:13.520005+0000","last_active":"2024-09-10T11:31:15.513347+0000","last_peered":"2024-09-10T11:31:15.513347+0000","last_clean":"2024-09-10T11:31:15.513347+0000","last_became_active":"2024-09-10T11:31:13.519498+0000","last_became_peered":"2024-09-10T11:31:13.519498+0000","last_unstale":"2024-09-10T11:31:15.513347+0000","last_undegraded":"2024-09-10T11:31:15.513347+0000","last_fullsized":"2024-09-10T11:31:15.513347+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:53.759850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513152+0000","last_change":"2024-09-10T11:31:14.522205+0000","last_active":"2024-09-10T11:31:15.513152+0000","last_peered":"2024-09-10T11:31:15.513152+0000","last_clean":"2024-09-10T11:31:15.513152+0000","last_became_active":"2024-09-10T11:31:14.521101+0000","last_became_peered":"2024-09-10T11:31:14.521101+0000","last_unstale":"2024-09-10T11:31:15.513152+0000","last_undegraded":"2024-09-10T11:31:15.513152+0000","last_fullsized":"2024-09-10T11:31:15.513152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:11.017372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064077+0000","last_change":"2024-09-10T11:31:13.518647+0000","last_active":"2024-09-10T11:31:16.064077+0000","last_peered":"2024-09-10T11:31:16.064077+0000","last_clean":"2024-09-10T11:31:16.064077+0000","last_became_active":"2024-09-10T11:31:13.518413+0000","last_became_peered":"2024-09-10T11:31:13.518413+0000","last_unstale":"2024-09-10T11:31:16.064077+0000","last_undegraded":"2024-09-10T11:31:16.064077+0000","last_fullsized":"2024-09-10T11:31:16.064077+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:36:54.845425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515515+0000","last_change":"2024-09-10T11:31:14.515196+0000","last_active":"2024-09-10T11:31:15.515515+0000","last_peered":"2024-09-10T11:31:15.515515+0000","last_clean":"2024-09-10T11:31:15.515515+0000","last_became_active":"2024-09-10T11:31:14.514349+0000","last_became_peered":"2024-09-10T11:31:14.514349+0000","last_unstale":"2024-09-10T11:31:15.515515+0000","last_undegraded":"2024-09-10T11:31:15.515515+0000","last_fullsized":"2024-09-10T11:31:15.515515+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:43.860920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064484+0000","last_change":"2024-09-10T11:31:13.518516+0000","last_active":"2024-09-10T11:31:16.064484+0000","last_peered":"2024-09-10T11:31:16.064484+0000","last_clean":"2024-09-10T11:31:16.064484+0000","last_became_active":"2024-09-10T11:31:13.518280+0000","last_became_peered":"2024-09-10T11:31:13.518280+0000","last_unstale":"2024-09-10T11:31:16.064484+0000","last_undegraded":"2024-09-10T11:31:16.064484+0000","last_fullsized":"2024-09-10T11:31:16.064484+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:50.257425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064385+0000","last_change":"2024-09-10T11:31:14.524423+0000","last_active":"2024-09-10T11:31:16.064385+0000","last_peered":"2024-09-10T11:31:16.064385+0000","last_clean":"2024-09-10T11:31:16.064385+0000","last_became_active":"2024-09-10T11:31:14.523773+0000","last_became_peered":"2024-09-10T11:31:14.523773+0000","last_unstale":"2024-09-10T11:31:16.064385+0000","last_undegraded":"2024-09-10T11:31:16.064385+0000","last_fullsized":"2024-09-10T11:31:16.064385+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:10.048931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513396+0000","last_change":"2024-09-10T11:31:13.515100+0000","last_active":"2024-09-10T11:31:15.513396+0000","last_peered":"2024-09-10T11:31:15.513396+0000","last_clean":"2024-09-10T11:31:15.513396+0000","last_became_active":"2024-09-10T11:31:13.514586+0000","last_became_peered":"2024-09-10T11:31:13.514586+0000","last_unstale":"2024-09-10T11:31:15.513396+0000","last_undegraded":"2024-09-10T11:31:15.513396+0000","last_fullsized":"2024-09-10T11:31:15.513396+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:36.506041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523188+0000","last_change":"2024-09-10T11:31:14.519010+0000","last_active":"2024-09-10T11:31:15.523188+0000","last_peered":"2024-09-10T11:31:15.523188+0000","last_clean":"2024-09-10T11:31:15.523188+0000","last_became_active":"2024-09-10T11:31:14.518709+0000","last_became_peered":"2024-09-10T11:31:14.518709+0000","last_unstale":"2024-09-10T11:31:15.523188+0000","last_undegraded":"2024-09-10T11:31:15.523188+0000","last_fullsized":"2024-09-10T11:31:15.523188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:57.491771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515205+0000","last_change":"2024-09-10T11:31:13.513825+0000","last_active":"2024-09-10T11:31:15.515205+0000","last_peered":"2024-09-10T11:31:15.515205+0000","last_clean":"2024-09-10T11:31:15.515205+0000","last_became_active":"2024-09-10T11:31:13.513564+0000","last_became_peered":"2024-09-10T11:31:13.513564+0000","last_unstale":"2024-09-10T11:31:15.515205+0000","last_undegraded":"2024-09-10T11:31:15.515205+0000","last_fullsized":"2024-09-10T11:31:15.515205+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:45.716071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513409+0000","last_change":"2024-09-10T11:31:14.514277+0000","last_active":"2024-09-10T11:31:15.513409+0000","last_peered":"2024-09-10T11:31:15.513409+0000","last_clean":"2024-09-10T11:31:15.513409+0000","last_became_active":"2024-09-10T11:31:14.513594+0000","last_became_peered":"2024-09-10T11:31:14.513594+0000","last_unstale":"2024-09-10T11:31:15.513409+0000","last_undegraded":"2024-09-10T11:31:15.513409+0000","last_fullsized":"2024-09-10T11:31:15.513409+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:25.578992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.523069+0000","last_change":"2024-09-10T11:31:13.519852+0000","last_active":"2024-09-10T11:31:15.523069+0000","last_peered":"2024-09-10T11:31:15.523069+0000","last_clean":"2024-09-10T11:31:15.523069+0000","last_became_active":"2024-09-10T11:31:13.519088+0000","last_became_peered":"2024-09-10T11:31:13.519088+0000","last_unstale":"2024-09-10T11:31:15.523069+0000","last_undegraded":"2024-09-10T11:31:15.523069+0000","last_fullsized":"2024-09-10T11:31:15.523069+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:50.538366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.522997+0000","last_change":"2024-09-10T11:31:14.513437+0000","last_active":"2024-09-10T11:31:15.522997+0000","last_peered":"2024-09-10T11:31:15.522997+0000","last_clean":"2024-09-10T11:31:15.522997+0000","last_became_active":"2024-09-10T11:31:14.513267+0000","last_became_peered":"2024-09-10T11:31:14.513267+0000","last_unstale":"2024-09-10T11:31:15.522997+0000","last_undegraded":"2024-09-10T11:31:15.522997+0000","last_fullsized":"2024-09-10T11:31:15.522997+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:06.292104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515609+0000","last_change":"2024-09-10T11:31:13.510861+0000","last_active":"2024-09-10T11:31:15.515609+0000","last_peered":"2024-09-10T11:31:15.515609+0000","last_clean":"2024-09-10T11:31:15.515609+0000","last_became_active":"2024-09-10T11:31:13.510601+0000","last_became_peered":"2024-09-10T11:31:13.510601+0000","last_unstale":"2024-09-10T11:31:15.515609+0000","last_undegraded":"2024-09-10T11:31:15.515609+0000","last_fullsized":"2024-09-10T11:31:15.515609+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:08.504847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064693+0000","last_change":"2024-09-10T11:31:14.518976+0000","last_active":"2024-09-10T11:31:16.064693+0000","last_peered":"2024-09-10T11:31:16.064693+0000","last_clean":"2024-09-10T11:31:16.064693+0000","last_became_active":"2024-09-10T11:31:14.518757+0000","last_became_peered":"2024-09-10T11:31:14.518757+0000","last_unstale":"2024-09-10T11:31:16.064693+0000","last_undegraded":"2024-09-10T11:31:16.064693+0000","last_fullsized":"2024-09-10T11:31:16.064693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:17.058041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515668+0000","last_change":"2024-09-10T11:31:13.512970+0000","last_active":"2024-09-10T11:31:15.515668+0000","last_peered":"2024-09-10T11:31:15.515668+0000","last_clean":"2024-09-10T11:31:15.515668+0000","last_became_active":"2024-09-10T11:31:13.512699+0000","last_became_peered":"2024-09-10T11:31:13.512699+0000","last_unstale":"2024-09-10T11:31:15.515668+0000","last_undegraded":"2024-09-10T11:31:15.515668+0000","last_fullsized":"2024-09-10T11:31:15.515668+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:15.117198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063951+0000","last_change":"2024-09-10T11:31:14.516102+0000","last_active":"2024-09-10T11:31:16.063951+0000","last_peered":"2024-09-10T11:31:16.063951+0000","last_clean":"2024-09-10T11:31:16.063951+0000","last_became_active":"2024-09-10T11:31:14.515535+0000","last_became_peered":"2024-09-10T11:31:14.515535+0000","last_unstale":"2024-09-10T11:31:16.063951+0000","last_undegraded":"2024-09-10T11:31:16.063951+0000","last_fullsized":"2024-09-10T11:31:16.063951+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:30.490092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.515403+0000","last_change":"2024-09-10T11:31:13.515604+0000","last_active":"2024-09-10T11:31:15.515403+0000","last_peered":"2024-09-10T11:31:15.515403+0000","last_clean":"2024-09-10T11:31:15.515403+0000","last_became_active":"2024-09-10T11:31:13.515355+0000","last_became_peered":"2024-09-10T11:31:13.515355+0000","last_unstale":"2024-09-10T11:31:15.515403+0000","last_undegraded":"2024-09-10T11:31:15.515403+0000","last_fullsized":"2024-09-10T11:31:15.515403+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:15.981692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.063929+0000","last_change":"2024-09-10T11:31:14.521550+0000","last_active":"2024-09-10T11:31:16.063929+0000","last_peered":"2024-09-10T11:31:16.063929+0000","last_clean":"2024-09-10T11:31:16.063929+0000","last_became_active":"2024-09-10T11:31:14.519610+0000","last_became_peered":"2024-09-10T11:31:14.519610+0000","last_unstale":"2024-09-10T11:31:16.063929+0000","last_undegraded":"2024-09-10T11:31:16.063929+0000","last_fullsized":"2024-09-10T11:31:16.063929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:11.170200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064530+0000","last_change":"2024-09-10T11:31:13.517588+0000","last_active":"2024-09-10T11:31:16.064530+0000","last_peered":"2024-09-10T11:31:16.064530+0000","last_clean":"2024-09-10T11:31:16.064530+0000","last_became_active":"2024-09-10T11:31:13.517351+0000","last_became_peered":"2024-09-10T11:31:13.517351+0000","last_unstale":"2024-09-10T11:31:16.064530+0000","last_undegraded":"2024-09-10T11:31:16.064530+0000","last_fullsized":"2024-09-10T11:31:16.064530+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:08.110011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513426+0000","last_change":"2024-09-10T11:31:14.514305+0000","last_active":"2024-09-10T11:31:15.513426+0000","last_peered":"2024-09-10T11:31:15.513426+0000","last_clean":"2024-09-10T11:31:15.513426+0000","last_became_active":"2024-09-10T11:31:14.513992+0000","last_became_peered":"2024-09-10T11:31:14.513992+0000","last_unstale":"2024-09-10T11:31:15.513426+0000","last_undegraded":"2024-09-10T11:31:15.513426+0000","last_fullsized":"2024-09-10T11:31:15.513426+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:04.655999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064537+0000","last_change":"2024-09-10T11:31:13.516709+0000","last_active":"2024-09-10T11:31:16.064537+0000","last_peered":"2024-09-10T11:31:16.064537+0000","last_clean":"2024-09-10T11:31:16.064537+0000","last_became_active":"2024-09-10T11:31:13.516254+0000","last_became_peered":"2024-09-10T11:31:13.516254+0000","last_unstale":"2024-09-10T11:31:16.064537+0000","last_undegraded":"2024-09-10T11:31:16.064537+0000","last_fullsized":"2024-09-10T11:31:16.064537+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:09.219617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064381+0000","last_change":"2024-09-10T11:31:14.521249+0000","last_active":"2024-09-10T11:31:16.064381+0000","last_peered":"2024-09-10T11:31:16.064381+0000","last_clean":"2024-09-10T11:31:16.064381+0000","last_became_active":"2024-09-10T11:31:14.520629+0000","last_became_peered":"2024-09-10T11:31:14.520629+0000","last_unstale":"2024-09-10T11:31:16.064381+0000","last_undegraded":"2024-09-10T11:31:16.064381+0000","last_fullsized":"2024-09-10T11:31:16.064381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:13.496197+0000","last_clean_scrub_stamp":"2024-09-10T11:31:13.496197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:31.363139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:15.513156+0000","last_change":"2024-09-10T11:31:13.516168+0000","last_active":"2024-09-10T11:31:15.513156+0000","last_peered":"2024-09-10T11:31:15.513156+0000","last_clean":"2024-09-10T11:31:15.513156+0000","last_became_active":"2024-09-10T11:31:13.515961+0000","last_became_peered":"2024-09-10T11:31:13.515961+0000","last_unstale":"2024-09-10T11:31:15.513156+0000","last_undegraded":"2024-09-10T11:31:15.513156+0000","last_fullsized":"2024-09-10T11:31:15.513156+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:12.491421+0000","last_clean_scrub_stamp":"2024-09-10T11:31:12.491421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:57.520071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T11:31:16.064928+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:16.064928+0000","last_peered":"2024-09-10T11:31:16.064928+0000","last_clean":"2024-09-10T11:31:16.064928+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:16.064928+0000","last_undegraded":"2024-09-10T11:31:16.064928+0000","last_fullsized":"2024-09-10T11:31:16.064928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860689,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199859099,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":713,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199269761,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":137,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195174,"statfs":{"total":200000000,"available":199859077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195175,"statfs":{"total":200000000,"available":199859935,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":712,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194599,"statfs":{"total":200000000,"available":199270273,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":136,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195176,"statfs":{"total":200000000,"available":199860621,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:17.781 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:31:17.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:31:17.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.921+0000 7f815b3b7640 1 Processor -- start 2024-09-10T11:31:17.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 -- start start 2024-09-10T11:31:17.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f815415b2d0 0x7f81540b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81540714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81540b0020 con 0x7f815415aef0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f815406ee10 con 0x7f815415b2d0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.922+0000 7f815b3b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f81540cd9a0 con 0x7f81540afbb0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159153640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f815415aef0 0x7f81540cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50110/0 (socket says 172.21.15.99:50110) 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159153640 1 -- 172.21.15.99:0/3529733740 learned_addr learned my addr 172.21.15.99:0/3529733740 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 --2- 172.21.15.99:0/3529733740 >> v2:172.21.15.196:3300/0 conn(0x7f815415b2d0 0x7f81540b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657728498 0 0) 0x7f81540b0020 con 0x7f815415aef0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8140004630 con 0x7f815415aef0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159954640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81540714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 47821809 0 0) 0x7f8140004630 con 0x7f815415aef0 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 msgr2=0x7f81540714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81540714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 >> v2:172.21.15.196:3300/0 conn(0x7f815415b2d0 msgr2=0x7f81540b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 --2- 172.21.15.99:0/3529733740 >> v2:172.21.15.196:3300/0 conn(0x7f815415b2d0 0x7f81540b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81540b0290 con 0x7f815415aef0 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8150001670 con 0x7f815415aef0 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159954640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81540714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 >> v1:172.21.15.99:6789/0 conn(0x7f815415aef0 legacy=0x7f81540cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140004ac0 0x7f8140004ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f8140005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81400062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f815406ee10 con 0x7f8140005910 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81540cd9a0 con 0x7f8140004ac0 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8140003520 con 0x7f81540afbb0 2024-09-10T11:31:17.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f8140005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159954640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81400062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159153640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140004ac0 0x7f8140004ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 -- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 msgr2=0x7f81400062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81400062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 -- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140004ac0 msgr2=0x7f8140004ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140004ac0 0x7f8140004ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8158952640 1 -- 172.21.15.99:0/3529733740 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140004630 con 0x7f8140005910 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159954640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81540afbb0 0x7f81400062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.923+0000 7f8159153640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8140004ac0 0x7f8140004ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.924+0000 7f8158952640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f8140005d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f814400bd60 tx=0x7f8140006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0159f2296b23eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.924+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814403a070 con 0x7f8140005910 2024-09-10T11:31:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.924+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8144035440 con 0x7f8140005910 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.924+0000 7f814bfff640 1 -- 172.21.15.99:0/3529733740 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400b930 con 0x7f8140005910 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 -- 172.21.15.99:0/3529733740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 msgr2=0x7f8140005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 --2- 172.21.15.99:0/3529733740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f8140005d90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f814400bd60 tx=0x7f8140006940 comp rx=0 tx=0).stop 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f8159954640 1 -- 172.21.15.99:0/3529733740 reap_dead start 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 -- 172.21.15.99:0/3529733740 shutdown_connections 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 -- 172.21.15.99:0/3529733740 >> 172.21.15.99:0/3529733740 conn(0x7f815406bc10 msgr2=0x7f815406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 -- 172.21.15.99:0/3529733740 shutdown_connections 2024-09-10T11:31:17.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.925+0000 7f815b3b7640 1 -- 172.21.15.99:0/3529733740 wait complete. 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.926+0000 7f815b3b7640 1 Processor -- start 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.926+0000 7f815b3b7640 1 -- start start 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81540ae970 0x7f81540aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 0x7f81540b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f815406eb30 con 0x7f8140005910 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f815406e850 con 0x7f81540ae970 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f815b3b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f81540d07b0 con 0x7f8154156330 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36888/0 (socket says 172.21.15.99:36888) 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 -- 172.21.15.99:0/1258731205 learned_addr learned my addr 172.21.15.99:0/1258731205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8158952640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81540ae970 0x7f81540aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159954640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 0x7f81540b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 -- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 msgr2=0x7f81540b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 0x7f81540b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 -- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81540ae970 msgr2=0x7f81540aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81540ae970 0x7f81540aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 -- 172.21.15.99:0/1258731205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81540aedf0 con 0x7f8140005910 2024-09-10T11:31:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159954640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 0x7f81540b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.927+0000 7f8159153640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f815000e8f0 tx=0x7f815000edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.928+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150019070 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.928+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150005240 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.928+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500055f0 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.928+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81540af090 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.928+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81540af650 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.929+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81540d0b70 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.929+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81500047e0 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.930+0000 7f8149ffb640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 0x7f813806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.930+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f815008a6b0 con 0x7f8140005910 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.930+0000 7f8158952640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 0x7f813806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:17.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.930+0000 7f8158952640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 0x7f813806e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8144037000 tx=0x7f8144031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:17.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:17.934+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8150058b60 con 0x7f8140005910 2024-09-10T11:31:18.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.158+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f815406c0c0 con 0x7f8140005910 2024-09-10T11:31:18.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.159+0000 7f8149ffb640 1 -- 172.21.15.99:0/1258731205 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f815005ce60 con 0x7f8140005910 2024-09-10T11:31:18.161 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:31:18.161 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:31:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:31:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:31:18.162 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:31:18.162 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:31:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 msgr2=0x7f813806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 0x7f813806e320 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8144037000 tx=0x7f8144031000 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 msgr2=0x7f81541f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f815000e8f0 tx=0x7f815000edb0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 shutdown_connections 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813806be60 0x7f813806e320 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8154156330 0x7f81540b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81540ae970 0x7f81540aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 --2- 172.21.15.99:0/1258731205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8140005910 0x7f81541f02f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.162+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 >> 172.21.15.99:0/1258731205 conn(0x7f815406bc10 msgr2=0x7f81541573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.163+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 shutdown_connections 2024-09-10T11:31:18.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.163+0000 7f815b3b7640 1 -- 172.21.15.99:0/1258731205 wait complete. 2024-09-10T11:31:18.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:18.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 Processor -- start 2024-09-10T11:31:18.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 -- start start 2024-09-10T11:31:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f16ac1626c0 0x7f16ac0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 0x7f16ac0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16ac15ff70 con 0x7f16ac15eb80 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f16ac0d30c0 con 0x7f16ac1626c0 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.272+0000 7f16b28e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f16ac0d1620 con 0x7f16ac0d4e40 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f16ac1626c0 0x7f16ac0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16abfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f16ac15eb80 0x7f16ac0d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50120/0 (socket says 172.21.15.99:50120) 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16abfff640 1 -- 172.21.15.99:0/2999390944 learned_addr learned my addr 172.21.15.99:0/2999390944 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2999390944 >> v2:172.21.15.99:3300/0 conn(0x7f16ac1626c0 0x7f16ac0d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36904/0 (socket says 172.21.15.99:36904) 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 -- 172.21.15.99:0/2999390944 >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 msgr2=0x7f16ac0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:18.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16b0e81640 1 --2- 172.21.15.99:0/2999390944 >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 0x7f16ac0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2999390944 >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 0x7f16ac0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 -- 172.21.15.99:0/2999390944 >> v1:172.21.15.99:6789/0 conn(0x7f16ac15eb80 legacy=0x7f16ac0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16aaffd640 1 -- 172.21.15.99:0/2999390944 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502997044 0 0) 0x7f16ac15ff70 con 0x7f16ac15eb80 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16ab7fe640 1 -- 172.21.15.99:0/2999390944 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac162bc0 con 0x7f16ac1626c0 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.273+0000 7f16b0e81640 1 --2- 172.21.15.99:0/2999390944 >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 0x7f16ac0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2999390944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac1626c0 0x7f16ac0d4900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16a00097a0 tx=0x7f16a002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c278f9a21ed59c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16aaffd640 1 -- 172.21.15.99:0/2999390944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0037070 con 0x7f16ac1626c0 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16aaffd640 1 -- 172.21.15.99:0/2999390944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a002ae50 con 0x7f16ac1626c0 2024-09-10T11:31:18.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16aaffd640 1 -- 172.21.15.99:0/2999390944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0032660 con 0x7f16ac1626c0 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 -- 172.21.15.99:0/2999390944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac1626c0 msgr2=0x7f16ac0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2999390944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac1626c0 0x7f16ac0d4900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16a00097a0 tx=0x7f16a002a140 comp rx=0 tx=0).stop 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 -- 172.21.15.99:0/2999390944 shutdown_connections 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2999390944 >> v2:172.21.15.196:3300/0 conn(0x7f16ac0d4e40 0x7f16ac0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2999390944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac1626c0 0x7f16ac0d4900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 -- 172.21.15.99:0/2999390944 >> 172.21.15.99:0/2999390944 conn(0x7f16ac08e680 msgr2=0x7f16ac09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 -- 172.21.15.99:0/2999390944 shutdown_connections 2024-09-10T11:31:18.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.274+0000 7f16b28e4640 1 -- 172.21.15.99:0/2999390944 wait complete. 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.275+0000 7f16b28e4640 1 Processor -- start 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 -- start start 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16ac15eb80 0x7f16ac0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 0x7f16ac0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac163460 con 0x7f16ac0d4e40 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac163180 con 0x7f16ac1626c0 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16b28e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f16ac0d5290 con 0x7f16ac15eb80 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16abfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16abfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36918/0 (socket says 172.21.15.99:36918) 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.276+0000 7f16abfff640 1 -- 172.21.15.99:0/2782934867 learned_addr learned my addr 172.21.15.99:0/2782934867 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16b0e81640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 0x7f16ac0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16ac15eb80 0x7f16ac0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 -- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16ac15eb80 msgr2=0x7f16ac0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16ac15eb80 0x7f16ac0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 -- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 msgr2=0x7f16ac0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 0x7f16ac0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 -- 172.21.15.99:0/2782934867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac0cb0d0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16b0e81640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 0x7f16ac0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16abfff640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c430 tx=0x7f169c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0074e0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169c0047e0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac0cb3b0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c004bd0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.277+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac0b1cf0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.278+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0d4900 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.279+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f169c011070 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.279+0000 7f16a8ff9640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 0x7f169806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.279+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f169c08ab00 con 0x7f16ac0d4e40 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.279+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 0x7f169806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:18.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.280+0000 7f16ab7fe640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 0x7f169806e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f16a0034000 tx=0x7f16a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:18.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.283+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f169c0594b0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.480+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16ac0b1f30 con 0x7f16ac0d4e40 2024-09-10T11:31:18.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.481+0000 7f16a8ff9640 1 -- 172.21.15.99:0/2782934867 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f169c058db0 con 0x7f16ac0d4e40 2024-09-10T11:31:18.483 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:18.483 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:18.483 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:18.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 msgr2=0x7f169806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 0x7f169806e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f16a0034000 tx=0x7f16a002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:18.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 msgr2=0x7f16ac161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:18.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c430 tx=0x7f169c00c8f0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 shutdown_connections 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f169806be30 0x7f169806e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16ac1626c0 0x7f16ac0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16ac15eb80 0x7f16ac0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 --2- 172.21.15.99:0/2782934867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ac0d4e40 0x7f16ac161940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 >> 172.21.15.99:0/2782934867 conn(0x7f16ac08e680 msgr2=0x7f16ac0cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 shutdown_connections 2024-09-10T11:31:18.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:18.483+0000 7f16b28e4640 1 -- 172.21.15.99:0/2782934867 wait complete. 2024-09-10T11:31:18.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:31:13.834494+0000', 'modified': '2024-09-10T11:31:15.848619+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4839}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4839': {'gid': 4839, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6837/3405308472', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3405308472}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3405308472}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4839, 'qdb_cluster': [4839]} 2024-09-10T11:31:18.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:31:18.898 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:31:19.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.000+0000 7f97c1ad0640 1 Processor -- start 2024-09-10T11:31:19.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.000+0000 7f97c1ad0640 1 -- start start 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97c1ad0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97bc0d1180 0x7f97bc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97c1ad0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f97bc06ee30 0x7f97bc06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97c1ad0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97bc0ceb10 con 0x7f97bc15c7b0 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97c1ad0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc06f280 con 0x7f97bc0d1180 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97c1ad0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f97bc0b3840 con 0x7f97bc06ee30 2024-09-10T11:31:19.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97bc0d1180 0x7f97bc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97bc0d1180 0x7f97bc06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36922/0 (socket says 172.21.15.99:36922) 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 -- 172.21.15.99:0/3770919028 learned_addr learned my addr 172.21.15.99:0/3770919028 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 -- 172.21.15.99:0/3770919028 >> v2:172.21.15.196:3300/0 conn(0x7f97bc06ee30 msgr2=0x7f97bc06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 --2- 172.21.15.99:0/3770919028 >> v2:172.21.15.196:3300/0 conn(0x7f97bc06ee30 0x7f97bc06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 -- 172.21.15.99:0/3770919028 >> v1:172.21.15.196:6789/0 conn(0x7f97bc15c7b0 legacy=0x7f97bc0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.001+0000 7f97baffd640 1 -- 172.21.15.99:0/3770919028 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc0b3a90 con 0x7f97bc0d1180 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.002+0000 7f97baffd640 1 --2- 172.21.15.99:0/3770919028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc0d1180 0x7f97bc06e8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0cd7f0 tx=0x7f97b002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56efc79754739811 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.002+0000 7f97ba7fc640 1 -- 172.21.15.99:0/3770919028 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0037070 con 0x7f97bc0d1180 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.002+0000 7f97ba7fc640 1 -- 172.21.15.99:0/3770919028 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0032440 con 0x7f97bc0d1180 2024-09-10T11:31:19.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.002+0000 7f97ba7fc640 1 -- 172.21.15.99:0/3770919028 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0032810 con 0x7f97bc0d1180 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 -- 172.21.15.99:0/3770919028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc0d1180 msgr2=0x7f97bc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/3770919028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc0d1180 0x7f97bc06e8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0cd7f0 tx=0x7f97b002a230 comp rx=0 tx=0).stop 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 -- 172.21.15.99:0/3770919028 shutdown_connections 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/3770919028 >> v2:172.21.15.196:3300/0 conn(0x7f97bc06ee30 0x7f97bc06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/3770919028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc0d1180 0x7f97bc06e8f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 -- 172.21.15.99:0/3770919028 >> 172.21.15.99:0/3770919028 conn(0x7f97bc06bc10 msgr2=0x7f97bc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 -- 172.21.15.99:0/3770919028 shutdown_connections 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.003+0000 7f97c1ad0640 1 -- 172.21.15.99:0/3770919028 wait complete. 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.004+0000 7f97c1ad0640 1 Processor -- start 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.004+0000 7f97c1ad0640 1 -- start start 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97bc06ee30 0x7f97bc156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97bc0d1180 0x7f97bc13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b4330 con 0x7f97bc15c7b0 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0b4050 con 0x7f97bc0d1180 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97c1ad0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0b4610 con 0x7f97bc06ee30 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36934/0 (socket says 172.21.15.99:36934) 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 -- 172.21.15.99:0/2841489662 learned_addr learned my addr 172.21.15.99:0/2841489662 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97bc06ee30 0x7f97bc156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 -- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97bc06ee30 msgr2=0x7f97bc156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97baffd640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97bc0d1180 0x7f97bc13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97bc06ee30 0x7f97bc156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 -- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97bc0d1180 msgr2=0x7f97bc13fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97bc0d1180 0x7f97bc13fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 -- 172.21.15.99:0/2841489662 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc1500b0 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.005+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b970 tx=0x7f97b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.006+0000 7f97a3fff640 1 -- 172.21.15.99:0/2841489662 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4019070 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.006+0000 7f97a3fff640 1 -- 172.21.15.99:0/2841489662 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b400cb60 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.006+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc1424b0 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.006+0000 7f97a3fff640 1 -- 172.21.15.99:0/2841489662 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4014440 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.006+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc142a70 con 0x7f97bc15c7b0 2024-09-10T11:31:19.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.007+0000 7f97a3fff640 1 -- 172.21.15.99:0/2841489662 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97b401c070 con 0x7f97bc15c7b0 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.008+0000 7f97a3fff640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.008+0000 7f97a3fff640 1 -- 172.21.15.99:0/2841489662 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97b408abd0 con 0x7f97bc15c7b0 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.008+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.008+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 0x7f97a806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00a730 tx=0x7f97ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.008+0000 7f97c486b740 1 Processor -- start 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 -- start start 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663e2c0 0x55f276683a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663fa50 0x55f2766388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f2765a0520 con 0x55f27663ba80 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2765483e0 con 0x55f27663e2c0 2024-09-10T11:31:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.009+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f27659c4e0 con 0x55f27663fa50 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36938/0 (socket says 172.21.15.99:36938) 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 -- 172.21.15.99:0/2979198742 learned_addr learned my addr 172.21.15.99:0/2979198742 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663fa50 0x55f2766388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663e2c0 0x55f276683a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 -- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663fa50 msgr2=0x55f2766388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663fa50 0x55f2766388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 -- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663e2c0 msgr2=0x55f276683a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663e2c0 0x55f276683a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 -- 172.21.15.99:0/2979198742 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2765632f0 con 0x55f27663ba80 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97baffd640 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f97b002a820 tx=0x7f97b002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884099a8714ffb83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97a17fa640 1 -- 172.21.15.99:0/2979198742 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0037040 con 0x55f27663ba80 2024-09-10T11:31:19.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97a17fa640 1 -- 172.21.15.99:0/2979198742 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0032740 con 0x55f27663ba80 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.010+0000 7f97a17fa640 1 -- 172.21.15.99:0/2979198742 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0032b10 con 0x55f27663ba80 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- 172.21.15.99:0/2979198742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 msgr2=0x55f27663e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f97b002a820 tx=0x7f97b002c000 comp rx=0 tx=0).stop 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- 172.21.15.99:0/2979198742 shutdown_connections 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663fa50 0x55f2766388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663ba80 0x55f27663e6a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 --2- 172.21.15.99:0/2979198742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663e2c0 0x55f276683a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- 172.21.15.99:0/2979198742 >> 172.21.15.99:0/2979198742 conn(0x55f27663aa80 msgr2=0x55f276682f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- 172.21.15.99:0/2979198742 shutdown_connections 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- 172.21.15.99:0/2979198742 wait complete. 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.011+0000 7f97c486b740 1 -- start start 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 Processor -- start 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f2766660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f27667cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f276639f20 con 0x55f2766970e0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f276529450 con 0x55f27663fa50 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f276684290 con 0x55f27663e2c0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97c486b740 10 client.0 _ll_register_callbacks cb 0x55f2762b2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36940/0 (socket says 172.21.15.99:36940) 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97bbfff640 1 -- 172.21.15.99:0/2715887795 learned_addr learned my addr 172.21.15.99:0/2715887795 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.012+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f2766660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97baffd640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f27667cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 -- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 msgr2=0x55f2766660f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f2766660f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 -- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 msgr2=0x55f27667cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f27667cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 -- 172.21.15.99:0/2715887795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2762b32d0 con 0x55f2766970e0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97baffd640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f27667cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f97b4058b80 tx=0x7f97b405d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cf89690b52c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40582d0 con 0x55f2766970e0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97b4058470 con 0x55f2766970e0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b0032740 con 0x55f2766970e0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f97b405f430 con 0x55f2766970e0 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 10 client.5001 fetch_fsmap learned FSMap version 14 2024-09-10T11:31:19.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55f27637d6b0 con 0x55f2766970e0 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0032740 con 0x55f2766970e0 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f97b405f5d0 con 0x55f2766970e0 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 10 client.5001 fetch_fsmap finished waiting for FSMap version 14 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 10 client.5001 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 20 client.5001 populate_metadata read hostname 'smithi099' 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f276693040 0x55f276693420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 4 client.5001 mds_command: new command op to 4839 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 --> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2765adc50 con 0x55f276693040 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.013+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f276693040 0x55f276693420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.014+0000 7f979a7fc640 10 client.5001 ms_handle_connect on v2:172.21.15.99:6836/3405308472 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.014+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f276693040 0x55f276693420 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5622f6f0cab4083 server_cookie=97530e8b3e7680f in_seq=0 out_seq=0 2024-09-10T11:31:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.014+0000 7f979a7fc640 1 -- 172.21.15.99:0/2715887795 <== mds.0 v2:172.21.15.99:6836/3405308472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2765adc50 con 0x55f276693040 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.014+0000 7f979a7fc640 10 client.5001 handle_command_reply: tid=0 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.014+0000 7f97c486b740 1 client.5001 shutdown 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 msgr2=0x55f27667fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f97b4058b80 tx=0x7f97b405d300 comp rx=0 tx=0).stop 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 shutdown_connections 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f276693040 0x55f276693420 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f2766970e0 0x55f27667fe60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f27667cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f979a7fc640 0 client.5001 ms_handle_reset on v2:172.21.15.99:6836/3405308472 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 --2- 172.21.15.99:0/2715887795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f2766660f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 >> 172.21.15.99:0/2715887795 conn(0x55f27663ba80 msgr2=0x55f2766824e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 shutdown_connections 2024-09-10T11:31:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 1 -- 172.21.15.99:0/2715887795 wait complete. 2024-09-10T11:31:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.015+0000 7f97c486b740 20 client.5001 trim_cache size 0 max 16384 2024-09-10T11:31:19.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.032+0000 7f97c486b740 1 Processor -- start 2024-09-10T11:31:19.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.032+0000 7f97c486b740 1 -- start start 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f276665ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2766970e0 0x55f27667f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc2004c0 con 0x55f27663fa50 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc0720a0 con 0x55f2766970e0 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.033+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97bc0726f0 con 0x55f27663e2c0 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36946/0 (socket says 172.21.15.99:36946) 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 -- 172.21.15.99:0/2868835000 learned_addr learned my addr 172.21.15.99:0/2868835000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f276665ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2766970e0 0x55f27667f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 -- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 msgr2=0x55f276665ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f276665ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 -- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2766970e0 msgr2=0x55f27667f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2766970e0 0x55f27667f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 -- 172.21.15.99:0/2868835000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2765da800 con 0x55f27663fa50 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97baffd640 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f97b0000c00 tx=0x7f97b0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d31f3e640403633 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97a0ff9640 1 -- 172.21.15.99:0/2868835000 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b004a070 con 0x55f27663fa50 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97a0ff9640 1 -- 172.21.15.99:0/2868835000 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0004380 con 0x55f27663fa50 2024-09-10T11:31:19.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.034+0000 7f97a0ff9640 1 -- 172.21.15.99:0/2868835000 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b0037040 con 0x55f27663fa50 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- 172.21.15.99:0/2868835000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 msgr2=0x55f2766661e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f97b0000c00 tx=0x7f97b0009240 comp rx=0 tx=0).stop 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- 172.21.15.99:0/2868835000 shutdown_connections 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f2766970e0 0x55f27667f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663fa50 0x55f2766661e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 --2- 172.21.15.99:0/2868835000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f27663e2c0 0x55f276665ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- 172.21.15.99:0/2868835000 >> 172.21.15.99:0/2868835000 conn(0x55f27663ba80 msgr2=0x55f276669af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- 172.21.15.99:0/2868835000 shutdown_connections 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- 172.21.15.99:0/2868835000 wait complete. 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 -- start start 2024-09-10T11:31:19.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.035+0000 7f97c486b740 1 Processor -- start 2024-09-10T11:31:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f276701c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f2766970e0 0x55f276681530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f2766679e0 con 0x55f27663e2c0 2024-09-10T11:31:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f2766674c0 con 0x55f27663fa50 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f27667d750 con 0x55f2766970e0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97c486b740 10 client.0 _ll_register_callbacks cb 0x7f97b40588d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36958/0 (socket says 172.21.15.99:36958) 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 -- 172.21.15.99:0/2598779268 learned_addr learned my addr 172.21.15.99:0/2598779268 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97baffd640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f276701c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bbfff640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f2766970e0 0x55f276681530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 -- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f2766970e0 msgr2=0x55f276681530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f2766970e0 0x55f276681530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 -- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 msgr2=0x55f276701c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f276701c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.036+0000 7f97bb7fe640 1 -- 172.21.15.99:0/2598779268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2766a5a70 con 0x55f27663e2c0 2024-09-10T11:31:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97bb7fe640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f27667dbe0 tx=0x7f97ac00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de130b0af02b4676 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac02b070 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f97ac013600 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b0004380 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f97ac036050 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 10 client.5007 fetch_fsmap learned FSMap version 14 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55f2766a5ff0 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0004380 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f97ac039020 con 0x55f27663e2c0 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 10 client.5007 fetch_fsmap finished waiting for FSMap version 14 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 10 client.5007 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 20 client.5007 populate_metadata read hostname 'smithi099' 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f2766661d0 0x55f276663f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 4 client.5007 mds_command: new command op to 4839 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 --> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] -- command(tid 0: {"prefix": "status"}) -- 0x55f2765a7fa0 con 0x55f2766661d0 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.037+0000 7f97baffd640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f2766661d0 0x55f276663f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f979a7fc640 10 client.5007 ms_handle_connect on v2:172.21.15.99:6836/3405308472 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97baffd640 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f2766661d0 0x55f276663f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f8ce4235779d76e server_cookie=dee7787a6192f80b in_seq=0 out_seq=0 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f979a7fc640 1 -- 172.21.15.99:0/2598779268 <== mds.0 v2:172.21.15.99:6836/3405308472 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55f2765a7fa0 con 0x55f2766661d0 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f979a7fc640 10 client.5007 handle_command_reply: tid=0 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 client.5007 shutdown 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 msgr2=0x55f2766ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f27667dbe0 tx=0x7f97ac00f000 comp rx=0 tx=0).stop 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 shutdown_connections 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:6836/3405308472,v1:172.21.15.99:6837/3405308472] conn(0x55f2766661d0 0x55f276663f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f2766970e0 0x55f276681530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f979a7fc640 0 client.5007 ms_handle_reset on v2:172.21.15.99:6836/3405308472 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f27663fa50 0x55f276701c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 --2- 172.21.15.99:0/2598779268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f27663e2c0 0x55f2766ff1c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 >> 172.21.15.99:0/2598779268 conn(0x55f27663ba80 msgr2=0x55f2766826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 shutdown_connections 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 1 -- 172.21.15.99:0/2598779268 wait complete. 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.038+0000 7f97c486b740 20 client.5007 trim_cache size 0 max 16384 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4839, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5222705330000004, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 14, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 32, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 32, 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 12.467047858000001 2024-09-10T11:31:19.048 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 msgr2=0x7f97a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 0x7f97a806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00a730 tx=0x7f97ac008040 comp rx=0 tx=0).stop 2024-09-10T11:31:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 msgr2=0x7f97bc14f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97b400b970 tx=0x7f97b400be30 comp rx=0 tx=0).stop 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 shutdown_connections 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f97a806be30 0x7f97a806e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97bc15c7b0 0x7f97bc14f7f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97bc0d1180 0x7f97bc13fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 --2- 172.21.15.99:0/2841489662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97bc06ee30 0x7f97bc156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 >> 172.21.15.99:0/2841489662 conn(0x7f97bc06bc10 msgr2=0x7f97bc06e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 shutdown_connections 2024-09-10T11:31:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.039+0000 7f97c1ad0640 1 -- 172.21.15.99:0/2841489662 wait complete. 2024-09-10T11:31:19.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:31:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 Processor -- start 2024-09-10T11:31:19.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 -- start start 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 0x7f0dd00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd00b3620 con 0x7f0dd00b3160 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd006d960 con 0x7f0dd00b3960 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.148+0000 7f0dd60f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd006dc40 con 0x7f0dd0163580 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0dd00b3160 0x7f0dd006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50128/0 (socket says 172.21.15.99:50128) 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcffff640 1 -- 172.21.15.99:0/1402803869 learned_addr learned my addr 172.21.15.99:0/1402803869 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1402803869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 0x7f0dd00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485987579 0 0) 0x7f0dd00b3620 con 0x7f0dd00b3160 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0db8003520 con 0x7f0dd00b3160 2024-09-10T11:31:19.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260375307 0 0) 0x7f0dd006d960 con 0x7f0dd00b3960 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1402803869 >> v1:172.21.15.196:6789/0 conn(0x7f0dd00b3960 legacy=0x7f0dd00d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1402803869 >> v1:172.21.15.99:6789/0 conn(0x7f0dd00b3160 legacy=0x7f0dd006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.149+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1402803869 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd006df20 con 0x7f0dd0163580 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1402803869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 0x7f0dd00b5090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097d0 tx=0x7f0dc002a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4ca866805556c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0037070 con 0x7f0dd0163580 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc002aae0 con 0x7f0dd0163580 2024-09-10T11:31:19.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1402803869 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80019f0 con 0x7f0dd0163580 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1402803869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 msgr2=0x7f0dd00b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.150+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1402803869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 0x7f0dd00b5090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097d0 tx=0x7f0dc002a050 comp rx=0 tx=0).stop 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.151+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1402803869 shutdown_connections 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.151+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1402803869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd0163580 0x7f0dd00b5090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.151+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1402803869 >> 172.21.15.99:0/1402803869 conn(0x7f0dd00b4570 msgr2=0x7f0dd00b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.151+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1402803869 shutdown_connections 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.151+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1402803869 wait complete. 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.152+0000 7f0dd60f1640 1 Processor -- start 2024-09-10T11:31:19.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.152+0000 7f0dd60f1640 1 -- start start 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00b3960 0x7f0dd0162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd006d5b0 con 0x7f0dd0163580 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd006d430 con 0x7f0dd00b3960 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd00ce7b0 con 0x7f0dd00b3160 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36970/0 (socket says 172.21.15.99:36970) 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:50514/0 (socket says 172.21.15.99:50514) 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 -- 172.21.15.99:0/1838527246 learned_addr learned my addr 172.21.15.99:0/1838527246 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dceffd640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00b3960 0x7f0dd0162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 -- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 msgr2=0x7f0dd00b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 -- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00b3960 msgr2=0x7f0dd0162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00b3960 0x7f0dd0162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 -- 172.21.15.99:0/1838527246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd01f5430 con 0x7f0dd0163580 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dcffff640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800caf0 tx=0x7f0dc800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8018070 con 0x7f0dd0163580 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc800a7c0 con 0x7f0dd0163580 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd00cea70 con 0x7f0dd0163580 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc800abb0 con 0x7f0dd0163580 2024-09-10T11:31:19.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.153+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd0162880 con 0x7f0dd0163580 2024-09-10T11:31:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.155+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd00761a0 con 0x7f0dd0163580 2024-09-10T11:31:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.155+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0dc801b070 con 0x7f0dd0163580 2024-09-10T11:31:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.155+0000 7f0db7fff640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 0x7f0dbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:19.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.156+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0dc808a6a0 con 0x7f0dd0163580 2024-09-10T11:31:19.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.156+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 0x7f0dbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:19.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.156+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 0x7f0dbc06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0034000 tx=0x7f0dc002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:19.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.160+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0dc8058a00 con 0x7f0dd0163580 2024-09-10T11:31:19.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.361+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0dd00ce5b0 con 0x7f0dd0163580 2024-09-10T11:31:19.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.361+0000 7f0db7fff640 1 -- 172.21.15.99:0/1838527246 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f0dc805cd00 con 0x7f0dd0163580 2024-09-10T11:31:19.363 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:31:19.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 msgr2=0x7f0dbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 0x7f0dbc06e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0034000 tx=0x7f0dc002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 msgr2=0x7f0dd00c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800caf0 tx=0x7f0dc800cef0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 shutdown_connections 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dbc06be30 0x7f0dbc06e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0163580 0x7f0dd00c8cf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00b3960 0x7f0dd0162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 --2- 172.21.15.99:0/1838527246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd00b3160 0x7f0dd00b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:19.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 >> 172.21.15.99:0/1838527246 conn(0x7f0dd00b4570 msgr2=0x7f0dd00d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:19.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 shutdown_connections 2024-09-10T11:31:19.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19.363+0000 7f0dd60f1640 1 -- 172.21.15.99:0/1838527246 wait complete. 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:19.377 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:31:19.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:31:19.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:31:19.378 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:19.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.379 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:31:19.395 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:31:19.395 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:19.396 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42770sec preferred_lft 42770sec 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.397 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:19.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.398 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:31:19.459 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:31:19.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.461 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:31:19.517 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:31:19.517 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:31:19.517 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:31:19.518 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:31:19.518 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:31:19.518 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:31:19.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:31:19.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:19.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.644 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:31:19.659 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:31:19.659 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:31:19.659 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:19.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.660 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:31:19.725 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:31:19.725 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:19.725 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:19.780 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:19.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:19.780 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:19.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:31:19.903 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:19.959 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:31:19.959 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:31:19.959 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:31:19.959 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:31:19.959 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:31:19.960 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:31:19.961 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:19.962 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:19.962 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:19.962 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:19.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:19.962 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:31:20.015 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:31:20.015 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:31:20.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:31:20.081 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:31:20.138 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:20.139 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:20.140 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:20.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.141 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:31:20.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:31:20.136+0000 7fdded31a480 -1 init, newargv = 0x555daf534160 newargc=15 2024-09-10T11:31:20.141 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[38443]: starting ceph client 2024-09-10T11:31:20.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[38443]: starting fuse 2024-09-10T11:31:20.156 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:31:20.156 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:31:20.156 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:20.156 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:20.157 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:20.158 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:20.274 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:20.284 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38443.asok status 2024-09-10T11:31:20.417 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:20.417 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "38443", 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:20.418 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5022, 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5022 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:20.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5022 192.168.0.1:0/3773731081", 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3773731081", 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 14, 2024-09-10T11:31:20.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 32, 2024-09-10T11:31:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:31:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:20.421 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:20.421 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:20.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.426 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:31:20.426 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:20.442 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:31:20.442 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:20.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:20.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:31:20.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.644+0000 7f8bf4352640 1 Processor -- start 2024-09-10T11:31:20.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.644+0000 7f8bf4352640 1 -- start start 2024-09-10T11:31:20.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf4352640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8bec0d1200 0x7f8bec073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf4352640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf4352640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bec152e60 con 0x7f8bec06e5d0 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf4352640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8bec06ef80 con 0x7f8bec0d1200 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf4352640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8bec0d15e0 con 0x7f8bec15c770 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf28ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8bec0d1200 0x7f8bec073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf18ed640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f8bec06e5d0 0x7f8bec15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:49192/0 (socket says 172.21.15.99:49192) 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8bec0d1200 0x7f8bec073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:45012/0 (socket says 172.21.15.99:45012) 2024-09-10T11:31:20.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf28ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:50520/0 (socket says 172.21.15.99:50520) 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf18ed640 1 -- 172.21.15.99:0/1423435987 learned_addr learned my addr 172.21.15.99:0/1423435987 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 -- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 msgr2=0x7f8bec06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 -- 172.21.15.99:0/1423435987 >> v1:172.21.15.196:6789/0 conn(0x7f8bec06e5d0 legacy=0x7f8bec15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf10ec640 1 -- 172.21.15.99:0/1423435987 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679044093 0 0) 0x7f8bec152e60 con 0x7f8bec06e5d0 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.645+0000 7f8bf20ee640 1 -- 172.21.15.99:0/1423435987 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec072680 con 0x7f8bec0d1200 2024-09-10T11:31:20.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.646+0000 7f8bf28ef640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.646+0000 7f8bf20ee640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec0d1200 0x7f8bec073440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ba40 tx=0x7f8be802c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e51ab834e60a1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.646+0000 7f8bf10ec640 1 -- 172.21.15.99:0/1423435987 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8031710 con 0x7f8bec0d1200 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.646+0000 7f8bf10ec640 1 -- 172.21.15.99:0/1423435987 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be80318b0 con 0x7f8bec0d1200 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.646+0000 7f8bf10ec640 1 -- 172.21.15.99:0/1423435987 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8031c80 con 0x7f8bec0d1200 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 -- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec0d1200 msgr2=0x7f8bec073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec0d1200 0x7f8bec073440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8be800ba40 tx=0x7f8be802c7b0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 -- 172.21.15.99:0/1423435987 shutdown_connections 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec15c770 0x7f8bec06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 --2- 172.21.15.99:0/1423435987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec0d1200 0x7f8bec073440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 -- 172.21.15.99:0/1423435987 >> 172.21.15.99:0/1423435987 conn(0x7f8bec06bc10 msgr2=0x7f8bec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 -- 172.21.15.99:0/1423435987 shutdown_connections 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.647+0000 7f8bf4352640 1 -- 172.21.15.99:0/1423435987 wait complete. 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.648+0000 7f8bf4352640 1 Processor -- start 2024-09-10T11:31:20.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.648+0000 7f8bf4352640 1 -- start start 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec0d1200 0x7f8bec151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec15c770 0x7f8bec154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec072f20 con 0x7f8bec06e5d0 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8bec072c40 con 0x7f8bec15c770 2024-09-10T11:31:20.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf4352640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8bec15cd90 con 0x7f8bec0d1200 2024-09-10T11:31:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36978/0 (socket says 172.21.15.99:36978) 2024-09-10T11:31:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 -- 172.21.15.99:0/150172433 learned_addr learned my addr 172.21.15.99:0/150172433 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:20.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf28ef640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec15c770 0x7f8bec154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf18ed640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec0d1200 0x7f8bec151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 -- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec0d1200 msgr2=0x7f8bec151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec0d1200 0x7f8bec151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 -- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec15c770 msgr2=0x7f8bec154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec15c770 0x7f8bec154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.649+0000 7f8bf20ee640 1 -- 172.21.15.99:0/150172433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bec152100 con 0x7f8bec06e5d0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8bf20ee640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8be8000c00 tx=0x7f8be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8041070 con 0x7f8bec06e5d0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8033c40 con 0x7f8bec06e5d0 2024-09-10T11:31:20.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bec157ee0 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be8006e10 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.650+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bec1584a0 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.651+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bec071c50 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.651+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8be8004030 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.652+0000 7f8be2ffd640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 0x7f8bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.652+0000 7f8bf18ed640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 0x7f8bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.652+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8be80ae000 con 0x7f8bec06e5d0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.652+0000 7f8bf18ed640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 0x7f8bd006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bec159360 tx=0x7f8bdc007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:20.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.656+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bec071c50 con 0x7f8bec06e5d0 2024-09-10T11:31:20.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.857+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8bec073440 con 0x7f8bec06e5d0 2024-09-10T11:31:20.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.857+0000 7f8be2ffd640 1 -- 172.21.15.99:0/150172433 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f8be807c9b0 con 0x7f8bec06e5d0 2024-09-10T11:31:20.859 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 msgr2=0x7f8bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 0x7f8bd006e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8bec159360 tx=0x7f8bdc007de0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 msgr2=0x7f8bec15a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8be8000c00 tx=0x7f8be80046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 shutdown_connections 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8bd006be30 0x7f8bd006e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bec15c770 0x7f8bec154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bec0d1200 0x7f8bec151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 --2- 172.21.15.99:0/150172433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bec06e5d0 0x7f8bec15a660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:20.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 >> 172.21.15.99:0/150172433 conn(0x7f8bec06bc10 msgr2=0x7f8bec15da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:20.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 shutdown_connections 2024-09-10T11:31:20.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:20.860+0000 7f8bf4352640 1 -- 172.21.15.99:0/150172433 wait complete. 2024-09-10T11:31:20.872 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:31:20.872 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:31:20.873 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:20.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.873 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:31:20.893 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:31:20.893 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:31:20.893 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:31:20.893 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:31:20.894 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42783sec preferred_lft 42783sec 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:20.895 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:20.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.896 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:31:20.962 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:31:20.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:20.964 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:31:21.023 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:31:21.023 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:31:21.024 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:31:21.024 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:31:21.024 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:31:21.024 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:31:21.117 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:31:21.150 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:21.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.156 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:31:21.171 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:31:21.172 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:31:21.172 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:21.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.172 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:31:21.237 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:31:21.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:21.238 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:21.294 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:21.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:21.294 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:21.349 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:31:21.416 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:21.472 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:31:21.472 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:31:21.472 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:31:21.472 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:31:21.473 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:21.474 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:21.475 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:21.475 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:21.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.475 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:31:21.528 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:31:21.528 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:31:21.571 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:31:21.596 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:31:21.654 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:21.655 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:21.656 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:21.657 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:21.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.657 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:31:21.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:31:21.655+0000 7f71b9720480 -1 init, newargv = 0x55ae13c72310 newargc=15 2024-09-10T11:31:21.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34176]: starting ceph client 2024-09-10T11:31:21.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34176]: starting fuse 2024-09-10T11:31:21.674 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:31:21.674 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:31:21.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:31:21.675 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:31:21.676 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:31:21.794 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.34176.asok 2024-09-10T11:31:21.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:21.806 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34176.asok 2024-09-10T11:31:21.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.806 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34176.asok status 2024-09-10T11:31:21.941 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:31:21.941 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:31:21.941 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:21.941 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:21.941 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "34176", 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4697, 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4697 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:21.942 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2431960259 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2431960259 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4697 192.168.0.1:0/2431960259", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/2431960259", 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:31:21.943 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 14, 2024-09-10T11:31:21.944 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 32, 2024-09-10T11:31:21.944 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:31:21.944 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:31:21.944 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:31:21.944 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:31:21.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.950 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:31:21.950 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:21.965 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:31:21.965 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:21.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:21.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:22.031 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.141+0000 7f20b6601640 1 Processor -- start 2024-09-10T11:31:22.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.141+0000 7f20b6601640 1 -- start start 2024-09-10T11:31:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b6601640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 0x7f20b0167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b6601640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00ceb10 con 0x7f20b00d1180 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b6601640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00b3a90 con 0x7f20b015c7b0 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b6601640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20b00b3840 con 0x7f20b006e4d0 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20af7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f20b00d1180 0x7f20b006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50132/0 (socket says 172.21.15.99:50132) 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20af7fe640 1 -- 172.21.15.99:0/1510788488 learned_addr learned my addr 172.21.15.99:0/1510788488 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b4b9e640 1 --2- 172.21.15.99:0/1510788488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 0x7f20b0167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226730501 0 0) 0x7f20a4001110 con 0x7f20b00d1180 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2098003520 con 0x7f20b00d1180 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012528081 0 0) 0x7f20b00b3a90 con 0x7f20b015c7b0 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f20a4001110 con 0x7f20b015c7b0 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b4b9e640 1 -- 172.21.15.99:0/1510788488 >> v1:172.21.15.196:6789/0 conn(0x7f20b015c7b0 legacy=0x7f20b00d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.142+0000 7f20b4b9e640 1 -- 172.21.15.99:0/1510788488 >> v1:172.21.15.99:6789/0 conn(0x7f20b00d1180 legacy=0x7f20b006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.143+0000 7f20b4b9e640 1 -- 172.21.15.99:0/1510788488 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b00b3d70 con 0x7f20b006e4d0 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.143+0000 7f20b4b9e640 1 --2- 172.21.15.99:0/1510788488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 0x7f20b0167610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b39c0 tx=0x7f20a802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfbc94968c8f5c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.143+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8007740 con 0x7f20b006e4d0 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.143+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a80078e0 con 0x7f20b006e4d0 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.143+0000 7f20aeffd640 1 -- 172.21.15.99:0/1510788488 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8007cb0 con 0x7f20b006e4d0 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 -- 172.21.15.99:0/1510788488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 msgr2=0x7f20b0167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 --2- 172.21.15.99:0/1510788488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 0x7f20b0167610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b39c0 tx=0x7f20a802a6c0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 -- 172.21.15.99:0/1510788488 shutdown_connections 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 --2- 172.21.15.99:0/1510788488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006e4d0 0x7f20b0167610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 -- 172.21.15.99:0/1510788488 >> 172.21.15.99:0/1510788488 conn(0x7f20b006bc10 msgr2=0x7f20b006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 -- 172.21.15.99:0/1510788488 shutdown_connections 2024-09-10T11:31:22.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.144+0000 7f20b6601640 1 -- 172.21.15.99:0/1510788488 wait complete. 2024-09-10T11:31:22.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.145+0000 7f20b6601640 1 Processor -- start 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 -- start start 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00d1180 0x7f20b01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015c7b0 0x7f20b0151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00b4610 con 0x7f20b006e4d0 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00b4370 con 0x7f20b015c7b0 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b6601640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20b015cb90 con 0x7f20b00d1180 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20affff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36986/0 (socket says 172.21.15.99:36986) 2024-09-10T11:31:22.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20affff640 1 -- 172.21.15.99:0/1573195944 learned_addr learned my addr 172.21.15.99:0/1573195944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20b4b9e640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015c7b0 0x7f20b0151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.146+0000 7f20af7fe640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00d1180 0x7f20b01548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 -- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00d1180 msgr2=0x7f20b01548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00d1180 0x7f20b01548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 -- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015c7b0 msgr2=0x7f20b0151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015c7b0 0x7f20b0151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 -- 172.21.15.99:0/1573195944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0153b00 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20affff640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c430 tx=0x7f20a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0017070 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a00052f0 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b0153de0 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0003830 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.147+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01523a0 con 0x7f20b006e4d0 2024-09-10T11:31:22.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.148+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b00d0a70 con 0x7f20b006e4d0 2024-09-10T11:31:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.149+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20a001a070 con 0x7f20b006e4d0 2024-09-10T11:31:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.149+0000 7f20acff9640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 0x7f209c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.149+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f20a008a1a0 con 0x7f20b006e4d0 2024-09-10T11:31:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.149+0000 7f20af7fe640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 0x7f209c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.149+0000 7f20af7fe640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 0x7f209c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20b01f5dc0 tx=0x7f20a4007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.153+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f20a0058500 con 0x7f20b006e4d0 2024-09-10T11:31:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.350+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20b0158ef0 con 0x7f20b006e4d0 2024-09-10T11:31:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.350+0000 7f20acff9640 1 -- 172.21.15.99:0/1573195944 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f20a005c800 con 0x7f20b006e4d0 2024-09-10T11:31:22.352 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:22.352 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 msgr2=0x7f209c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 0x7f209c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f20b01f5dc0 tx=0x7f20a4007d30 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 msgr2=0x7f20b013cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20a000c430 tx=0x7f20a000c8f0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 shutdown_connections 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f209c06be30 0x7f209c06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015c7b0 0x7f20b0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00d1180 0x7f20b01548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 --2- 172.21.15.99:0/1573195944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20b006e4d0 0x7f20b013cd50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.352+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 >> 172.21.15.99:0/1573195944 conn(0x7f20b006bc10 msgr2=0x7f20b006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.353+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 shutdown_connections 2024-09-10T11:31:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.353+0000 7f20b6601640 1 -- 172.21.15.99:0/1573195944 wait complete. 2024-09-10T11:31:22.366 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:31:22.481 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":4839,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9549911760000001,"mdsmap_epoch":14,"osdmap_epoch":32,"osdmap_epoch_barrier":32,"uptime":15.899770386} 2024-09-10T11:31:22.481 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 4839, "mdsmap_epoch": 14, "osdmap_epoch": 32, "osdmap_epoch_barrier": 32, "rank_uptime": 7.954991176, "state": "up:active", "uptime": 15.899770386, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:31:22.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-09-10T11:31:22.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.579+0000 7f649f12c640 1 Processor -- start 2024-09-10T11:31:22.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.579+0000 7f649f12c640 1 -- start start 2024-09-10T11:31:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f649f12c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6498073440 0x7f649815c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f649f12c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6498152e60 con 0x7f649806ece0 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f649f12c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64980d15e0 con 0x7f64980d1200 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f649f12c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6498072680 con 0x7f6498073440 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f6497fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f649806ece0 0x7f649815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58964/0 (socket says 172.21.15.99:58964) 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f6497fff640 1 -- 172.21.15.99:0/2988233792 learned_addr learned my addr 172.21.15.99:0/2988233792 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f649d6c9640 1 --2- 172.21.15.99:0/2988233792 >> v2:172.21.15.196:3300/0 conn(0x7f6498073440 0x7f649815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715390218 0 0) 0x7f6498152e60 con 0x7f649806ece0 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6488003520 con 0x7f649806ece0 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143827898 0 0) 0x7f648c001110 con 0x7f64980d1200 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6498152e60 con 0x7f64980d1200 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1465309022 0 0) 0x7f6488003520 con 0x7f649806ece0 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 >> v2:172.21.15.196:3300/0 conn(0x7f6498073440 msgr2=0x7f649815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 --2- 172.21.15.99:0/2988233792 >> v2:172.21.15.196:3300/0 conn(0x7f6498073440 0x7f649815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 >> v1:172.21.15.196:6789/0 conn(0x7f64980d1200 legacy=0x7f649806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6498072960 con 0x7f649806ece0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f6484001670 con 0x7f649806ece0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 >> v1:172.21.15.99:6789/0 conn(0x7f649806ece0 legacy=0x7f649815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6488003b40 0x7f6488003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f6488004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64880051c0 0x7f648800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6498072680 con 0x7f6488004800 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f648800c0f0 con 0x7f64880051c0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f648800c2f0 con 0x7f6488003b40 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.580+0000 7f6497fff640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f6488004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f649cec8640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6488003b40 0x7f6488003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f649d6c9640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64880051c0 0x7f648800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 -- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6488003b40 msgr2=0x7f6488003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6488003b40 0x7f6488003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 -- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64880051c0 msgr2=0x7f648800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64880051c0 0x7f648800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 -- 172.21.15.99:0/2988233792 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6488003520 con 0x7f6488004800 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f649d6c9640 1 -- 172.21.15.99:0/2988233792 reap_dead start 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.581+0000 7f6497fff640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f6488004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f648400c190 tx=0x7f6488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2dc5b3316775f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484007cc0 con 0x7f6488004800 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6484005ad0 con 0x7f6488004800 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f64977fe640 1 -- 172.21.15.99:0/2988233792 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484007070 con 0x7f6488004800 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 -- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 msgr2=0x7f6488004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f6488004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f648400c190 tx=0x7f6488005830 comp rx=0 tx=0).stop 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 -- 172.21.15.99:0/2988233792 shutdown_connections 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 --2- 172.21.15.99:0/2988233792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f6488004c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 -- 172.21.15.99:0/2988233792 >> 172.21.15.99:0/2988233792 conn(0x7f649806bc10 msgr2=0x7f649806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 -- 172.21.15.99:0/2988233792 shutdown_connections 2024-09-10T11:31:22.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.582+0000 7f649f12c640 1 -- 172.21.15.99:0/2988233792 wait complete. 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.583+0000 7f649f12c640 1 Processor -- start 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.583+0000 7f649f12c640 1 -- start start 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6498151930 0x7f649815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64980c57b0 0x7f6498169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f649815cd90 con 0x7f6488004800 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6498072f20 con 0x7f64980c57b0 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649f12c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f649815c060 con 0x7f6498151930 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649cec8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649cec8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41864/0 (socket says 172.21.15.99:41864) 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649cec8640 1 -- 172.21.15.99:0/1952081218 learned_addr learned my addr 172.21.15.99:0/1952081218 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f649d6c9640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64980c57b0 0x7f6498169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.584+0000 7f6497fff640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6498151930 0x7f649815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 -- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6498151930 msgr2=0x7f649815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6498151930 0x7f649815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 -- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64980c57b0 msgr2=0x7f6498169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64980c57b0 0x7f6498169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 -- 172.21.15.99:0/1952081218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64981508d0 con 0x7f6488004800 2024-09-10T11:31:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649cec8640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f648c00f7a0 tx=0x7f648c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c01a070 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c00fe50 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6498150b70 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c007050 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.585+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6498151130 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.586+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649806eaf0 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.586+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f648c01d070 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.587+0000 7f64957fa640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.587+0000 7f6497fff640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.587+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f648c08a550 con 0x7f6488004800 2024-09-10T11:31:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.587+0000 7f6497fff640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 0x7f647c06e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6484032000 tx=0x7f6484009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.591+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64980d15e0 con 0x7f6488004800 2024-09-10T11:31:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.817+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) -- 0x7f649815c3f0 con 0x7f6488004800 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.818+0000 7f64957fa640 1 -- 172.21.15.99:0/1952081218 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) ==== 75+0+1620 (secure 0 0 0) 0x7f648c058940 con 0x7f6488004800 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "total_bytes": 1600000000, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "total_avail_bytes": 1597700132, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_bytes": 2299868, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_bytes": 2299868, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_ratio": 0.0014374174643307924, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "num_osds": 8, 2024-09-10T11:31:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "num_per_pool_osds": 0, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "num_per_pool_omap_osds": 0 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "stats_by_class": { 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "hdd": { 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "total_bytes": 1600000000, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "total_avail_bytes": 1597700132, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_bytes": 2299868, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_bytes": 2299868, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "total_used_raw_ratio": 0.0014374174643307924 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:31:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "name": ".mgr", 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 1, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 590368, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 2, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 577, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 590368, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 0.00052959844470024109, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557078080 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephfs_metadata", 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 2410, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 22, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 3, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 2410, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 2.1630673927575117e-06, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557078080 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "name": "cephfs_data", 2024-09-10T11:31:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "stats": { 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "stored": 0, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "objects": 0, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "kb_used": 0, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "bytes_used": 0, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "percent_used": 0, 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "max_avail": 557078080 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:22.825 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:22.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 msgr2=0x7f647c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 0x7f647c06e2f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6484032000 tx=0x7f6484009030 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 msgr2=0x7f64980c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f648c00f7a0 tx=0x7f648c00fc60 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 shutdown_connections 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f647c06be30 0x7f647c06e2f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64980c57b0 0x7f6498169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6498151930 0x7f649815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 --2- 172.21.15.99:0/1952081218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6488004800 0x7f64980c6e90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 >> 172.21.15.99:0/1952081218 conn(0x7f649806bc10 msgr2=0x7f649815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 shutdown_connections 2024-09-10T11:31:22.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22.820+0000 7f649f12c640 1 -- 172.21.15.99:0/1952081218 wait complete. 2024-09-10T11:31:22.833 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:31:22.833 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:22.833 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:31:22.833 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:31:22.833 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:31:22.834 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:31:22.834 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:22.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:22.912 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:22.912 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:22.912 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:22.912 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:22.912 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:22.913 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:22.914 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:23.030 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:23.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:23.040 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:23.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:23.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38443.asok status 2024-09-10T11:31:23.174 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:23.174 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "38443", 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 1, 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5022, 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:23.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5022 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:23.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5022 192.168.0.1:0/3773731081", 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3773731081", 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 2, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 14, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 32, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 32, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:23.177 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:23.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-09-10T11:31:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.286+0000 7fcf97fcf640 1 Processor -- start 2024-09-10T11:31:23.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.287+0000 7fcf97fcf640 1 -- start start 2024-09-10T11:31:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.287+0000 7fcf97fcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf90073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf97fcf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcf9006e5d0 0x7fcf9006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf97fcf640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf90152e60 con 0x7fcf9015c060 2024-09-10T11:31:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf97fcf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9006ef50 con 0x7fcf9006e5d0 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf97fcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf900d15e0 con 0x7fcf900d1200 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf9656c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf9015c060 0x7fcf9015c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58974/0 (socket says 172.21.15.99:58974) 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf9656c640 1 -- 172.21.15.99:0/3966569701 learned_addr learned my addr 172.21.15.99:0/3966569701 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf9556a640 1 --2- 172.21.15.99:0/3966569701 >> v2:172.21.15.196:3300/0 conn(0x7fcf9006e5d0 0x7fcf9006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291832440 0 0) 0x7fcf90152e60 con 0x7fcf9015c060 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf95d6b640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf90073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf7c003520 con 0x7fcf9015c060 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3228716915 0 0) 0x7fcf7c003520 con 0x7fcf9015c060 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 msgr2=0x7fcf90073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf90073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 >> v2:172.21.15.196:3300/0 conn(0x7fcf9006e5d0 msgr2=0x7fcf9006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 --2- 172.21.15.99:0/3966569701 >> v2:172.21.15.196:3300/0 conn(0x7fcf9006e5d0 0x7fcf9006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90072680 con 0x7fcf9015c060 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fcf880019f0 con 0x7fcf9015c060 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 >> v1:172.21.15.99:6789/0 conn(0x7fcf9015c060 legacy=0x7fcf9015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9006ef50 con 0x7fcf7c004d40 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf900d15e0 con 0x7fcf7c0039b0 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c00b9c0 con 0x7fcf900d1200 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf9656c640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf95d6b640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.288+0000 7fcf9556a640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 -- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 msgr2=0x7fcf7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900d1200 0x7fcf7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 -- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c0039b0 msgr2=0x7fcf7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 -- 172.21.15.99:0/3966569701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c003520 con 0x7fcf7c004d40 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf9656c640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcf880013d0 tx=0x7fcf7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd782171de00db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.289+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800a4e0 con 0x7fcf7c004d40 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8802f070 con 0x7fcf7c004d40 2024-09-10T11:31:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf94d69640 1 -- 172.21.15.99:0/3966569701 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8802bce0 con 0x7fcf7c004d40 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 -- 172.21.15.99:0/3966569701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7c004d40 msgr2=0x7fcf7c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/3966569701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcf880013d0 tx=0x7fcf7c005830 comp rx=0 tx=0).stop 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf9656c640 1 -- 172.21.15.99:0/3966569701 reap_dead start 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 -- 172.21.15.99:0/3966569701 shutdown_connections 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 -- 172.21.15.99:0/3966569701 >> 172.21.15.99:0/3966569701 conn(0x7fcf9006bc10 msgr2=0x7fcf9006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 -- 172.21.15.99:0/3966569701 shutdown_connections 2024-09-10T11:31:23.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.290+0000 7fcf97fcf640 1 -- 172.21.15.99:0/3966569701 wait complete. 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.291+0000 7fcf97fcf640 1 Processor -- start 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.291+0000 7fcf97fcf640 1 -- start start 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c004d40 0x7fcf901502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900cc050 0x7fcf90139e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90072f20 con 0x7fcf900aa6b0 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf90072c40 con 0x7fcf7c004d40 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf97fcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9015cd90 con 0x7fcf900cc050 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41882/0 (socket says 172.21.15.99:41882) 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 -- 172.21.15.99:0/4026990160 learned_addr learned my addr 172.21.15.99:0/4026990160 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9656c640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900cc050 0x7fcf90139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf95d6b640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c004d40 0x7fcf901502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 -- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900cc050 msgr2=0x7fcf90139e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900cc050 0x7fcf90139e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 -- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c004d40 msgr2=0x7fcf901502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c004d40 0x7fcf901502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.292+0000 7fcf9556a640 1 -- 172.21.15.99:0/4026990160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf901507e0 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf9556a640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c011980 tx=0x7fcf8c011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c00fa30 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c00fbd0 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf90138dd0 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c014440 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.293+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf90139390 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.294+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9006ea10 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.294+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf8c012040 con 0x7fcf900aa6b0 2024-09-10T11:31:23.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.295+0000 7fcf827fc640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 0x7fcf7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.295+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcf8c08b430 con 0x7fcf900aa6b0 2024-09-10T11:31:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.295+0000 7fcf95d6b640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 0x7fcf7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.295+0000 7fcf95d6b640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 0x7fcf7406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf900b1a00 tx=0x7fcf84009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.299+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf8c019030 con 0x7fcf900aa6b0 2024-09-10T11:31:23.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.472+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7fcf9015dca0 con 0x7fcf900aa6b0 2024-09-10T11:31:23.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.540+0000 7fcf827fc640 1 -- 172.21.15.99:0/4026990160 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v33) ==== 93+0+0 (secure 0 0 0) 0x7fcf8c059de0 con 0x7fcf900aa6b0 2024-09-10T11:31:23.542 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is set 2024-09-10T11:31:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 msgr2=0x7fcf7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 0x7fcf7406e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf900b1a00 tx=0x7fcf84009190 comp rx=0 tx=0).stop 2024-09-10T11:31:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 msgr2=0x7fcf9013af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c011980 tx=0x7fcf8c011d80 comp rx=0 tx=0).stop 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 shutdown_connections 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf7406be30 0x7fcf7406e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf900cc050 0x7fcf90139e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf900aa6b0 0x7fcf9013af30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 --2- 172.21.15.99:0/4026990160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c004d40 0x7fcf901502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 >> 172.21.15.99:0/4026990160 conn(0x7fcf9006bc10 msgr2=0x7fcf9015fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 shutdown_connections 2024-09-10T11:31:23.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.542+0000 7fcf97fcf640 1 -- 172.21.15.99:0/4026990160 wait complete. 2024-09-10T11:31:23.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-09-10T11:31:23.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.652+0000 7fe97040e640 1 Processor -- start 2024-09-10T11:31:23.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.652+0000 7fe97040e640 1 -- start start 2024-09-10T11:31:23.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe97040e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9680d1200 0x7fe968073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe97040e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 0x7fe96806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe97040e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe968152e60 con 0x7fe96815c060 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe97040e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe96806ef50 con 0x7fe9680d1200 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe97040e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe9680d15e0 con 0x7fe96806e5d0 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9680d1200 0x7fe968073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e9ab640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe96815c060 0x7fe96815c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58988/0 (socket says 172.21.15.99:58988) 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9680d1200 0x7fe968073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41886/0 (socket says 172.21.15.99:41886) 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e9ab640 1 -- 172.21.15.99:0/2038249500 learned_addr learned my addr 172.21.15.99:0/2038249500 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96d9a9640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 0x7fe96806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 -- 172.21.15.99:0/2038249500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 msgr2=0x7fe96806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 0x7fe96806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 -- 172.21.15.99:0/2038249500 >> v1:172.21.15.99:6789/0 conn(0x7fe96815c060 legacy=0x7fe96815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96d1a8640 1 -- 172.21.15.99:0/2038249500 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2597098543 0 0) 0x7fe968152e60 con 0x7fe96815c060 2024-09-10T11:31:23.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96e1aa640 1 -- 172.21.15.99:0/2038249500 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe968072680 con 0x7fe9680d1200 2024-09-10T11:31:23.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.653+0000 7fe96d9a9640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 0x7fe96806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.654+0000 7fe96e1aa640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9680d1200 0x7fe968073440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe9640097a0 tx=0x7fe96402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932d452cb5b4846 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.654+0000 7fe96d1a8640 1 -- 172.21.15.99:0/2038249500 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037070 con 0x7fe9680d1200 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.654+0000 7fe96d1a8640 1 -- 172.21.15.99:0/2038249500 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96402ae50 con 0x7fe9680d1200 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.654+0000 7fe96d1a8640 1 -- 172.21.15.99:0/2038249500 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964032660 con 0x7fe9680d1200 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 -- 172.21.15.99:0/2038249500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9680d1200 msgr2=0x7fe968073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9680d1200 0x7fe968073440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe9640097a0 tx=0x7fe96402a100 comp rx=0 tx=0).stop 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 -- 172.21.15.99:0/2038249500 shutdown_connections 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe96806e5d0 0x7fe96806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 --2- 172.21.15.99:0/2038249500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9680d1200 0x7fe968073440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 -- 172.21.15.99:0/2038249500 >> 172.21.15.99:0/2038249500 conn(0x7fe96806bc10 msgr2=0x7fe96806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 -- 172.21.15.99:0/2038249500 shutdown_connections 2024-09-10T11:31:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.655+0000 7fe97040e640 1 -- 172.21.15.99:0/2038249500 wait complete. 2024-09-10T11:31:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.656+0000 7fe97040e640 1 Processor -- start 2024-09-10T11:31:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.656+0000 7fe97040e640 1 -- start start 2024-09-10T11:31:23.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9680d1200 0x7fe9681679a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe96815c060 0x7fe968152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe968072f20 con 0x7fe96806e5d0 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe968072c40 con 0x7fe96815c060 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe96815cd90 con 0x7fe9680d1200 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41892/0 (socket says 172.21.15.99:41892) 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 -- 172.21.15.99:0/3893374730 learned_addr learned my addr 172.21.15.99:0/3893374730 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96d9a9640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9680d1200 0x7fe9681679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e9ab640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe96815c060 0x7fe968152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 -- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9680d1200 msgr2=0x7fe9681679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9680d1200 0x7fe9681679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 -- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe96815c060 msgr2=0x7fe968152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe96815c060 0x7fe968152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9681f0590 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe96e1aa640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe964035040 tx=0x7fe9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964041070 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96403c670 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037040 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.657+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9681f0870 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.658+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681f0e30 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.658+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96806ea10 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.659+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe96403a070 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.659+0000 7fe95effd640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 0x7fe94c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.659+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe9640ab2a0 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.659+0000 7fe96d9a9640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 0x7fe94c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.660+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=34+}) -- 0x7fe94c0739c0 con 0x7fe96806e5d0 2024-09-10T11:31:23.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.660+0000 7fe96d9a9640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 0x7fe94c06e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe9680b1820 tx=0x7fe958008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:23.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.665+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe964079c50 con 0x7fe96806e5d0 2024-09-10T11:31:23.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:23.838+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7fe96815d530 con 0x7fe96806e5d0 2024-09-10T11:31:24.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.545+0000 7fe95effd640 1 -- 172.21.15.99:0/3893374730 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v34) ==== 97+0+0 (secure 0 0 0) 0x7fe9640ab0c0 con 0x7fe96806e5d0 2024-09-10T11:31:24.547 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is unset 2024-09-10T11:31:24.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 msgr2=0x7fe94c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 0x7fe94c06e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe9680b1820 tx=0x7fe958008040 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 msgr2=0x7fe9681effb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe964035040 tx=0x7fe9640046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 shutdown_connections 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe94c06be30 0x7fe94c06e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.548+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe96815c060 0x7fe968152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.549+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9680d1200 0x7fe9681679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.549+0000 7fe97040e640 1 --2- 172.21.15.99:0/3893374730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe96806e5d0 0x7fe9681effb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.549+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 >> 172.21.15.99:0/3893374730 conn(0x7fe96806bc10 msgr2=0x7fe96815e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.549+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 shutdown_connections 2024-09-10T11:31:24.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.549+0000 7fe97040e640 1 -- 172.21.15.99:0/3893374730 wait complete. 2024-09-10T11:31:24.563 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:24.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.662+0000 7f7ad0511640 1 Processor -- start 2024-09-10T11:31:24.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.662+0000 7f7ad0511640 1 -- start start 2024-09-10T11:31:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7ad0511640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ac80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7ad0511640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac80b40e0 con 0x7f7ac80b0090 2024-09-10T11:31:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7ad0511640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ac80b5ab0 con 0x7f7ac815ee00 2024-09-10T11:31:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7ad0511640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac8073c50 con 0x7f7ac815f1e0 2024-09-10T11:31:24.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7aceaae640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7ac80b0090 0x7f7ac80b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59002/0 (socket says 172.21.15.99:59002) 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7aceaae640 1 -- 172.21.15.99:0/3913381397 learned_addr learned my addr 172.21.15.99:0/3913381397 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7acdaac640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ac80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207900236 0 0) 0x7f7ac80b40e0 con 0x7f7ac80b0090 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ab0003520 con 0x7f7ac80b0090 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.663+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3654131672 0 0) 0x7f7ab0003520 con 0x7f7ac80b0090 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 msgr2=0x7f7ac80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ac80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 >> v1:172.21.15.196:6789/0 conn(0x7f7ac815ee00 legacy=0x7f7ac80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac80b4380 con 0x7f7ac80b0090 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7ab8001670 con 0x7f7ac80b0090 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 >> v1:172.21.15.99:6789/0 conn(0x7f7ac80b0090 legacy=0x7f7ac80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ab00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab0003f20 0x7f7ab0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ab0004d40 0x7f7ab00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac8073c50 con 0x7f7ab0003f20 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab000ba20 con 0x7f7ab0004d40 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab000bc20 con 0x7f7ac815f1e0 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab0003f20 0x7f7ab0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7aceaae640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ab0004d40 0x7f7ab00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7ace2ad640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ab00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 -- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 msgr2=0x7f7ab00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ac815f1e0 0x7f7ab00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 -- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ab0004d40 msgr2=0x7f7ab00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ab0004d40 0x7f7ab00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7acdaac640 1 -- 172.21.15.99:0/3913381397 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab0003520 con 0x7f7ab0003f20 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.664+0000 7f7aceaae640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ab0004d40 0x7f7ab00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7acdaac640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab0003f20 0x7f7ab0004800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ac806da30 tx=0x7f7ab0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d9df5750476c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:24.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc036070 con 0x7f7ab0003f20 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7abc031600 con 0x7f7ab0003f20 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7acd2ab640 1 -- 172.21.15.99:0/3913381397 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7abc009c10 con 0x7f7ab0003f20 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 -- 172.21.15.99:0/3913381397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab0003f20 msgr2=0x7f7ab0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 --2- 172.21.15.99:0/3913381397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ab0003f20 0x7f7ab0004800 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ac806da30 tx=0x7f7ab0005830 comp rx=0 tx=0).stop 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7aceaae640 1 -- 172.21.15.99:0/3913381397 reap_dead start 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 -- 172.21.15.99:0/3913381397 shutdown_connections 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 -- 172.21.15.99:0/3913381397 >> 172.21.15.99:0/3913381397 conn(0x7f7ac80730c0 msgr2=0x7f7ac80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 -- 172.21.15.99:0/3913381397 shutdown_connections 2024-09-10T11:31:24.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.665+0000 7f7ad0511640 1 -- 172.21.15.99:0/3913381397 wait complete. 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.666+0000 7f7ad0511640 1 Processor -- start 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 -- start start 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ab0003f20 0x7f7ac809da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ac81f5d70 0x7f7ac809dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac80b0600 con 0x7f7ac81f81a0 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac80afd60 con 0x7f7ac81f5d70 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ad0511640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac80b0780 con 0x7f7ab0003f20 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7aceaae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7aceaae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41900/0 (socket says 172.21.15.99:41900) 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7aceaae640 1 -- 172.21.15.99:0/2759452873 learned_addr learned my addr 172.21.15.99:0/2759452873 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7ace2ad640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ab0003f20 0x7f7ac809da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.667+0000 7f7acdaac640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ac81f5d70 0x7f7ac809dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 -- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ab0003f20 msgr2=0x7f7ac809da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ab0003f20 0x7f7ac809da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 -- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ac81f5d70 msgr2=0x7f7ac809dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ac81f5d70 0x7f7ac809dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 -- 172.21.15.99:0/2759452873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac81590e0 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7aceaae640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8070b20 tx=0x7f7ab800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800c590 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8018070 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac809b450 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800c990 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.668+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac80cb4b0 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.669+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac815e320 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.669+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ab800cb30 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.670+0000 7f7ac6ffd640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.670+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7ab808a890 con 0x7f7ac81f81a0 2024-09-10T11:31:24.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.670+0000 7f7ace2ad640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:24.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.670+0000 7f7ace2ad640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 0x7f7ab406e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007ac0 tx=0x7f7ac0007a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:24.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.674+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ab8058c80 con 0x7f7ac81f81a0 2024-09-10T11:31:24.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.839+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ac815e040 con 0x7f7ac81f81a0 2024-09-10T11:31:24.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.840+0000 7f7ac6ffd640 1 -- 172.21.15.99:0/2759452873 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16669 (secure 0 0 0) 0x7f7ab805cf80 con 0x7f7ac81f81a0 2024-09-10T11:31:24.842 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:24.842 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:24.542163+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 msgr2=0x7f7ab406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 0x7f7ab406e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0007ac0 tx=0x7f7ac0007a50 comp rx=0 tx=0).stop 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 msgr2=0x7f7ac81f8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8070b20 tx=0x7f7ab800eb90 comp rx=0 tx=0).stop 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 shutdown_connections 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ab406be30 0x7f7ab406e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ac81f81a0 0x7f7ac81f8630 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ac81f5d70 0x7f7ac809dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 --2- 172.21.15.99:0/2759452873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ab0003f20 0x7f7ac809da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 >> 172.21.15.99:0/2759452873 conn(0x7f7ac80730c0 msgr2=0x7f7ac80ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 shutdown_connections 2024-09-10T11:31:24.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24.842+0000 7f7ad0511640 1 -- 172.21.15.99:0/2759452873 wait complete. 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:31:24.855 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:31:24.856 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:24.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:24.931 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:24.932 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:24.933 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:25.049 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:25.059 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:25.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:25.059 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38443.asok status 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:25.193 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "38443", 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 1, 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5022, 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5022 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:25.194 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5022 192.168.0.1:0/3773731081", 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3773731081", 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 2, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 14, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 32, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 32, 2024-09-10T11:31:25.195 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:25.196 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:25.196 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:25.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:25.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.298+0000 7fb73264c640 1 Processor -- start 2024-09-10T11:31:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.298+0000 7fb73264c640 1 -- start start 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.298+0000 7fb73264c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c0d1200 0x7fb72c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb73264c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb72c06e5d0 0x7fb72c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb73264c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb72c152e60 con 0x7fb72c15c060 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb73264c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb72c06ef50 con 0x7fb72c06e5d0 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb73264c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c0d15e0 con 0x7fb72c0d1200 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb72c06e5d0 0x7fb72c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb72c06e5d0 0x7fb72c06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41912/0 (socket says 172.21.15.99:41912) 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 -- 172.21.15.99:0/4273898375 learned_addr learned my addr 172.21.15.99:0/4273898375 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 -- 172.21.15.99:0/4273898375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c0d1200 msgr2=0x7fb72c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 --2- 172.21.15.99:0/4273898375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c0d1200 0x7fb72c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 -- 172.21.15.99:0/4273898375 >> v1:172.21.15.196:6789/0 conn(0x7fb72c15c060 legacy=0x7fb72c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.299+0000 7fb72b7fe640 1 -- 172.21.15.99:0/4273898375 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c072680 con 0x7fb72c06e5d0 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb72b7fe640 1 --2- 172.21.15.99:0/4273898375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c06e5d0 0x7fb72c06ea10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb72002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958f63539b4958f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb72affd640 1 -- 172.21.15.99:0/4273898375 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720037070 con 0x7fb72c06e5d0 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb72affd640 1 -- 172.21.15.99:0/4273898375 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb720032440 con 0x7fb72c06e5d0 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb72affd640 1 -- 172.21.15.99:0/4273898375 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720032810 con 0x7fb72c06e5d0 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 -- 172.21.15.99:0/4273898375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c06e5d0 msgr2=0x7fb72c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 --2- 172.21.15.99:0/4273898375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c06e5d0 0x7fb72c06ea10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb72002a180 comp rx=0 tx=0).stop 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 -- 172.21.15.99:0/4273898375 shutdown_connections 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 --2- 172.21.15.99:0/4273898375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c06e5d0 0x7fb72c06ea10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 --2- 172.21.15.99:0/4273898375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c0d1200 0x7fb72c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 -- 172.21.15.99:0/4273898375 >> 172.21.15.99:0/4273898375 conn(0x7fb72c06bc10 msgr2=0x7fb72c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:25.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 -- 172.21.15.99:0/4273898375 shutdown_connections 2024-09-10T11:31:25.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.300+0000 7fb73264c640 1 -- 172.21.15.99:0/4273898375 wait complete. 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 Processor -- start 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 -- start start 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72c06e5d0 0x7fb72c14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c15c060 0x7fb72c14dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c072f20 con 0x7fb72c0d1200 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c072c40 con 0x7fb72c06e5d0 2024-09-10T11:31:25.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.302+0000 7fb73264c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c15cd90 con 0x7fb72c15c060 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41924/0 (socket says 172.21.15.99:41924) 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 -- 172.21.15.99:0/819193475 learned_addr learned my addr 172.21.15.99:0/819193475 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb730be9640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c15c060 0x7fb72c14dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72bfff640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72c06e5d0 0x7fb72c14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 -- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c15c060 msgr2=0x7fb72c14dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c15c060 0x7fb72c14dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 -- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72c06e5d0 msgr2=0x7fb72c14b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72c06e5d0 0x7fb72c14b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 -- 172.21.15.99:0/819193475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c1549c0 con 0x7fb72c0d1200 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb72b7fe640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb7200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720041070 con 0x7fb72c0d1200 2024-09-10T11:31:25.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72003c870 con 0x7fb72c0d1200 2024-09-10T11:31:25.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c154ca0 con 0x7fb72c0d1200 2024-09-10T11:31:25.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb720037040 con 0x7fb72c0d1200 2024-09-10T11:31:25.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.303+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c155260 con 0x7fb72c0d1200 2024-09-10T11:31:25.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.304+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c06ea10 con 0x7fb72c0d1200 2024-09-10T11:31:25.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.305+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb720044070 con 0x7fb72c0d1200 2024-09-10T11:31:25.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.305+0000 7fb728ff9640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 0x7fb71806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:25.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.305+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb7200ab9b0 con 0x7fb72c0d1200 2024-09-10T11:31:25.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.305+0000 7fb72bfff640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 0x7fb71806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:25.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.306+0000 7fb72bfff640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 0x7fb71806e320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b810 tx=0x7fb71c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:25.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.309+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb72007a360 con 0x7fb72c0d1200 2024-09-10T11:31:25.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.507+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72c0b20c0 con 0x7fb72c0d1200 2024-09-10T11:31:25.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.507+0000 7fb728ff9640 1 -- 172.21.15.99:0/819193475 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7fb720079c60 con 0x7fb72c0d1200 2024-09-10T11:31:25.509 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:25.509 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:25.510 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 msgr2=0x7fb71806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 0x7fb71806e320 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b810 tx=0x7fb71c008040 comp rx=0 tx=0).stop 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 msgr2=0x7fb72c153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb720009870 tx=0x7fb7200046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 shutdown_connections 2024-09-10T11:31:25.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb71806be60 0x7fb71806e320 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb72c15c060 0x7fb72c14dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb72c0d1200 0x7fb72c153a40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 --2- 172.21.15.99:0/819193475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb72c06e5d0 0x7fb72c14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 >> 172.21.15.99:0/819193475 conn(0x7fb72c06bc10 msgr2=0x7fb72c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 shutdown_connections 2024-09-10T11:31:25.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25.509+0000 7fb73264c640 1 -- 172.21.15.99:0/819193475 wait complete. 2024-09-10T11:31:25.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 34 2024-09-10T11:31:25.639 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:31:25.639 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:31:25.640 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:25.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:25.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:25.717 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:25.718 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:25.719 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:25.720 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:25.720 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.720 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:25.720 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:25.835 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:25.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:25.846 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:25.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:25.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38443.asok status 2024-09-10T11:31:25.978 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:25.978 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "38443", 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 2, 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5022, 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:25.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5022 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:25.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5022 192.168.0.1:0/3773731081", 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3773731081", 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 3, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 14, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 34, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 34, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:25.981 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:25.987 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:25.988 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:25.989 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:25.989 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:26.066 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:26.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:26.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38443.asok 2024-09-10T11:31:26.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:26.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38443.asok status 2024-09-10T11:31:26.210 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:26.210 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:26.210 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:26.210 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:26.210 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "38443", 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 2, 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5022, 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5022 2024-09-10T11:31:26.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3773731081 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5022 192.168.0.1:0/3773731081", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3773731081", 2024-09-10T11:31:26.212 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 3, 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 14, 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 34, 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 34, 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:26.213 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:26.218 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:31:26.219 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:31:26.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:31:26.267 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:31:26.309 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:31:26.309 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:31:26.309 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:26.309 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:26.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:31:26.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.462+0000 7feb3c3a1640 1 Processor -- start 2024-09-10T11:31:26.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.462+0000 7feb3c3a1640 1 -- start start 2024-09-10T11:31:26.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb340b3960 0x7feb340d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feb3406cd20 0x7feb3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7feb340b3620 con 0x7feb3406cd20 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7feb3406d690 con 0x7feb340b3960 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3c3a1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feb3406d960 con 0x7feb34163580 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feb3406cd20 0x7feb3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feb3406cd20 0x7feb3406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41926/0 (socket says 172.21.15.99:41926) 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 -- 172.21.15.99:0/832072877 learned_addr learned my addr 172.21.15.99:0/832072877 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 -- 172.21.15.99:0/832072877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 msgr2=0x7feb340b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 --2- 172.21.15.99:0/832072877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 -- 172.21.15.99:0/832072877 >> v2:172.21.15.196:3300/0 conn(0x7feb340b3960 msgr2=0x7feb340d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3993c640 1 --2- 172.21.15.99:0/832072877 >> v2:172.21.15.196:3300/0 conn(0x7feb340b3960 0x7feb340d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 --2- 172.21.15.99:0/832072877 >> v2:172.21.15.196:3300/0 conn(0x7feb340b3960 0x7feb340d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.463+0000 7feb3a93e640 1 -- 172.21.15.99:0/832072877 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3406dc40 con 0x7feb3406cd20 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.464+0000 7feb3a93e640 1 --2- 172.21.15.99:0/832072877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3406cd20 0x7feb3406d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb340d3a80 tx=0x7feb2c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e043ed1f97abb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.464+0000 7feb3913b640 1 -- 172.21.15.99:0/832072877 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c036070 con 0x7feb3406cd20 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.464+0000 7feb3913b640 1 -- 172.21.15.99:0/832072877 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c031600 con 0x7feb3406cd20 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.464+0000 7feb3913b640 1 -- 172.21.15.99:0/832072877 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0319d0 con 0x7feb3406cd20 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 -- 172.21.15.99:0/832072877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3406cd20 msgr2=0x7feb3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/832072877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3406cd20 0x7feb3406d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb340d3a80 tx=0x7feb2c02a610 comp rx=0 tx=0).stop 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 -- 172.21.15.99:0/832072877 shutdown_connections 2024-09-10T11:31:26.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/832072877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb3406cd20 0x7feb3406d150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/832072877 >> v2:172.21.15.196:3300/0 conn(0x7feb340b3960 0x7feb340d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/832072877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 -- 172.21.15.99:0/832072877 >> 172.21.15.99:0/832072877 conn(0x7feb340b4570 msgr2=0x7feb340b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 -- 172.21.15.99:0/832072877 shutdown_connections 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.465+0000 7feb3c3a1640 1 -- 172.21.15.99:0/832072877 wait complete. 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.466+0000 7feb3c3a1640 1 Processor -- start 2024-09-10T11:31:26.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.466+0000 7feb3c3a1640 1 -- start start 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb3406cd20 0x7feb34154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb340d2120 con 0x7feb340b3960 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feb340d16c0 con 0x7feb3406cd20 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3c3a1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feb340af850 con 0x7feb34163580 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41930/0 (socket says 172.21.15.99:41930) 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 -- 172.21.15.99:0/3601788158 learned_addr learned my addr 172.21.15.99:0/3601788158 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3a93e640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3a13d640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb3406cd20 0x7feb34154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 -- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 msgr2=0x7feb340cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 -- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb3406cd20 msgr2=0x7feb34154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb3406cd20 0x7feb34154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.467+0000 7feb3993c640 1 -- 172.21.15.99:0/3601788158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb34161090 con 0x7feb340b3960 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb3993c640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c7f0 tx=0x7feb2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24015070 con 0x7feb340b3960 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb24004220 con 0x7feb340b3960 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb34161370 con 0x7feb340b3960 2024-09-10T11:31:26.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24007620 con 0x7feb340b3960 2024-09-10T11:31:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.468+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb34161930 con 0x7feb340b3960 2024-09-10T11:31:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.469+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3406d150 con 0x7feb340b3960 2024-09-10T11:31:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.469+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb240079d0 con 0x7feb340b3960 2024-09-10T11:31:26.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.470+0000 7feb2affd640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 0x7feb1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.470+0000 7feb3a13d640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 0x7feb1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.470+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7feb2408b290 con 0x7feb340b3960 2024-09-10T11:31:26.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.470+0000 7feb3a13d640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 0x7feb1806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b810 tx=0x7feb30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.474+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb24059c40 con 0x7feb340b3960 2024-09-10T11:31:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.653+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb340afe70 con 0x7feb340b3960 2024-09-10T11:31:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.653+0000 7feb2affd640 1 -- 172.21.15.99:0/3601788158 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7feb24059540 con 0x7feb340b3960 2024-09-10T11:31:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.655+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 msgr2=0x7feb1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.655+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 0x7feb1806e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb3000b810 tx=0x7feb30008040 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.655+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 msgr2=0x7feb3415dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.655+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7feb2400c7f0 tx=0x7feb2400ccb0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 shutdown_connections 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feb1806be30 0x7feb1806e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb34163580 0x7feb340cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb340b3960 0x7feb3415dcd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 --2- 172.21.15.99:0/3601788158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb3406cd20 0x7feb34154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 >> 172.21.15.99:0/3601788158 conn(0x7feb340b4570 msgr2=0x7feb340b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 shutdown_connections 2024-09-10T11:31:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.656+0000 7feb3c3a1640 1 -- 172.21.15.99:0/3601788158 wait complete. 2024-09-10T11:31:26.668 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:31:26.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:26.668 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:26.692 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[38443]: fuse finished with error 0 and tester_r 0 2024-09-10T11:31:26.694 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:31:26.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:26.694 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:31:26.790 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:31:26.791 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:31:26.791 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:26.810 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:31:26.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.928+0000 7ff85234b640 1 Processor -- start 2024-09-10T11:31:26.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.928+0000 7ff85234b640 1 -- start start 2024-09-10T11:31:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff84c15c7b0 0x7ff84c0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 0x7ff84c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c06ee30 0x7ff84c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff84c0d1420 con 0x7ff84c15c7b0 2024-09-10T11:31:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff84c06f280 con 0x7ff84c0d0fb0 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff85234b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff84c0b3840 con 0x7ff84c06ee30 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff84c15c7b0 0x7ff84c0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff84c15c7b0 0x7ff84c0d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41934/0 (socket says 172.21.15.99:41934) 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 -- 172.21.15.99:0/511416958 learned_addr learned my addr 172.21.15.99:0/511416958 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84b7fe640 1 --2- 172.21.15.99:0/511416958 >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 0x7ff84c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff8508e8640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c06ee30 0x7ff84c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 -- 172.21.15.99:0/511416958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c06ee30 msgr2=0x7ff84c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c06ee30 0x7ff84c1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 -- 172.21.15.99:0/511416958 >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 msgr2=0x7ff84c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 --2- 172.21.15.99:0/511416958 >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 0x7ff84c06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84bfff640 1 -- 172.21.15.99:0/511416958 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c0ceb10 con 0x7ff84c15c7b0 2024-09-10T11:31:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.929+0000 7ff84b7fe640 1 --2- 172.21.15.99:0/511416958 >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 0x7ff84c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.930+0000 7ff84bfff640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c0d0a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0d0250 tx=0x7ff83c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f75247fe98e78f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.930+0000 7ff84affd640 1 -- 172.21.15.99:0/511416958 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c037070 con 0x7ff84c15c7b0 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.930+0000 7ff84affd640 1 -- 172.21.15.99:0/511416958 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83c032440 con 0x7ff84c15c7b0 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.930+0000 7ff84affd640 1 -- 172.21.15.99:0/511416958 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c032810 con 0x7ff84c15c7b0 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 -- 172.21.15.99:0/511416958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 msgr2=0x7ff84c0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c0d0a70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff84c0d0250 tx=0x7ff83c02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 -- 172.21.15.99:0/511416958 shutdown_connections 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c06ee30 0x7ff84c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 --2- 172.21.15.99:0/511416958 >> v2:172.21.15.196:3300/0 conn(0x7ff84c0d0fb0 0x7ff84c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 --2- 172.21.15.99:0/511416958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c0d0a70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 -- 172.21.15.99:0/511416958 >> 172.21.15.99:0/511416958 conn(0x7ff84c06bc10 msgr2=0x7ff84c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 -- 172.21.15.99:0/511416958 shutdown_connections 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.931+0000 7ff85234b640 1 -- 172.21.15.99:0/511416958 wait complete. 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.932+0000 7ff85234b640 1 Processor -- start 2024-09-10T11:31:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.932+0000 7ff85234b640 1 -- start start 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c06ee30 0x7ff84c1f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c0d0fb0 0x7ff84c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0b4050 con 0x7ff84c15c7b0 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0b3d70 con 0x7ff84c06ee30 2024-09-10T11:31:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff85234b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff84c0b4330 con 0x7ff84c0d0fb0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41946/0 (socket says 172.21.15.99:41946) 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 -- 172.21.15.99:0/3613768206 learned_addr learned my addr 172.21.15.99:0/3613768206 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff84bfff640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c06ee30 0x7ff84c1f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff84b7fe640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c0d0fb0 0x7ff84c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 -- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c0d0fb0 msgr2=0x7ff84c154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c0d0fb0 0x7ff84c154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 -- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c06ee30 msgr2=0x7ff84c1f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c06ee30 0x7ff84c1f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 -- 172.21.15.99:0/3613768206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c154ce0 con 0x7ff84c15c7b0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.933+0000 7ff8508e8640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff84400cd70 tx=0x7ff84400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.934+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844003b90 con 0x7ff84c15c7b0 2024-09-10T11:31:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.934+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff844003d30 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.934+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c150030 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.934+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844004240 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.934+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1505f0 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.935+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c15cb90 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.935+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8440043e0 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.935+0000 7ff848ff9640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 0x7ff83806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.936+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff84408b7c0 con 0x7ff84c15c7b0 2024-09-10T11:31:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.936+0000 7ff84bfff640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 0x7ff83806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:26.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.936+0000 7ff84bfff640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 0x7ff83806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff83c034000 tx=0x7ff83c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:26.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:26.940+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff844059bb0 con 0x7ff84c15c7b0 2024-09-10T11:31:27.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.121+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff84c15bfd0 con 0x7ff84c15c7b0 2024-09-10T11:31:27.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.122+0000 7ff848ff9640 1 -- 172.21.15.99:0/3613768206 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7ff84405deb0 con 0x7ff84c15c7b0 2024-09-10T11:31:27.124 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:31:27.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 msgr2=0x7ff83806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 0x7ff83806e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff83c034000 tx=0x7ff83c02e000 comp rx=0 tx=0).stop 2024-09-10T11:31:27.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 msgr2=0x7ff84c1546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff84400cd70 tx=0x7ff84400a660 comp rx=0 tx=0).stop 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 shutdown_connections 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83806be30 0x7ff83806e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff84c15c7b0 0x7ff84c1546a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff84c0d0fb0 0x7ff84c154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 --2- 172.21.15.99:0/3613768206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c06ee30 0x7ff84c1f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 >> 172.21.15.99:0/3613768206 conn(0x7ff84c06bc10 msgr2=0x7ff84c0b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 shutdown_connections 2024-09-10T11:31:27.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.124+0000 7ff85234b640 1 -- 172.21.15.99:0/3613768206 wait complete. 2024-09-10T11:31:27.137 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:31:27.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:27.137 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:27.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34176]: fuse finished with error 0 and tester_r 0 2024-09-10T11:31:27.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:31:27.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:27.169 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:31:27.250 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:31:27.250 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:27.250 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:27.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:31:27.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:31:27.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:27.267 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:31:27.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:31:27.322 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:31:27.322 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:31:27.322 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:31:27.323 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:27.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:31:27.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:31:27.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:27.340 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:31:27.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:31:27.397 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:31:27.398 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.494+0000 7fbefc8e6640 1 Processor -- start 2024-09-10T11:31:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.494+0000 7fbefc8e6640 1 -- start start 2024-09-10T11:31:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbefc8e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbef80b3660 0x7fbef80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbefc8e6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbef80d1410 0x7fbef80cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbefc8e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef806df80 con 0x7fbef80cfb90 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbefc8e6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80d18a0 con 0x7fbef80d1410 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbefc8e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80d2120 con 0x7fbef80b3660 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbef6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbef80d1410 0x7fbef80cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbef6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbef80d1410 0x7fbef80cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41950/0 (socket says 172.21.15.99:41950) 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.495+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1313370522 learned_addr learned my addr 172.21.15.99:0/1313370522 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1313370522 >> v2:172.21.15.196:3300/0 conn(0x7fbef80b3660 msgr2=0x7fbef80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef6d9d640 1 --2- 172.21.15.99:0/1313370522 >> v2:172.21.15.196:3300/0 conn(0x7fbef80b3660 0x7fbef80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1313370522 >> v1:172.21.15.196:6789/0 conn(0x7fbef80cfb90 legacy=0x7fbef80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:27.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1313370522 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80d1ad0 con 0x7fbef80d1410 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef6d9d640 1 --2- 172.21.15.99:0/1313370522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef80cde00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbef8162ba0 tx=0x7fbee802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fb42879bad36cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef559a640 1 -- 172.21.15.99:0/1313370522 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbef80d1410 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.496+0000 7fbef559a640 1 -- 172.21.15.99:0/1313370522 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbef80d1410 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbef559a640 1 -- 172.21.15.99:0/1313370522 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8032810 con 0x7fbef80d1410 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1313370522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 msgr2=0x7fbef80cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1313370522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef80cde00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbef8162ba0 tx=0x7fbee802a1a0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1313370522 shutdown_connections 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1313370522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef80cde00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1313370522 >> v2:172.21.15.196:3300/0 conn(0x7fbef80b3660 0x7fbef80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1313370522 >> 172.21.15.99:0/1313370522 conn(0x7fbef80b4650 msgr2=0x7fbef80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1313370522 shutdown_connections 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.497+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1313370522 wait complete. 2024-09-10T11:31:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.498+0000 7fbefc8e6640 1 Processor -- start 2024-09-10T11:31:27.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 -- start start 2024-09-10T11:31:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 0x7fbef80b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef80cfb90 0x7fbef80b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbef806dd30 con 0x7fbef80d1410 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbef806e260 con 0x7fbef80b3660 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbefc8e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80cff70 con 0x7fbef80cfb90 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbef6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbef6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41962/0 (socket says 172.21.15.99:41962) 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.499+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1940545782 learned_addr learned my addr 172.21.15.99:0/1940545782 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef659c640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 0x7fbef80b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef80cfb90 msgr2=0x7fbef80b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef5d9b640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef80cfb90 0x7fbef80b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef80cfb90 0x7fbef80b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 msgr2=0x7fbef80b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 0x7fbef80b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 -- 172.21.15.99:0/1940545782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80cb720 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef659c640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 0x7fbef80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbef6d9d640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbee80090b0 tx=0x7fbee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8041070 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee803c950 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037040 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef80cba00 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.500+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef80cbfc0 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.501+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef80b5170 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.501+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbee8044070 con 0x7fbef80d1410 2024-09-10T11:31:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.502+0000 7fbeeeffd640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.502+0000 7fbef659c640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.502+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fbee80abae0 con 0x7fbef80d1410 2024-09-10T11:31:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.503+0000 7fbef659c640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbef0007eb0 tx=0x7fbef00073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.506+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbee807a640 con 0x7fbef80d1410 2024-09-10T11:31:27.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.707+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef80cd210 con 0x7fbef80d1410 2024-09-10T11:31:27.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.708+0000 7fbeeeffd640 1 -- 172.21.15.99:0/1940545782 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7fbee8079f40 con 0x7fbef80d1410 2024-09-10T11:31:27.710 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:27.710 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:27.710 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.710+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 msgr2=0x7fbed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.710+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 0x7fbed806e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbef0007eb0 tx=0x7fbef00073c0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.710+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 msgr2=0x7fbef81f7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.710+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbee80090b0 tx=0x7fbee80046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 shutdown_connections 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbed806be30 0x7fbed806e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbef80d1410 0x7fbef81f7890 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbef80cfb90 0x7fbef80b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 --2- 172.21.15.99:0/1940545782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbef80b3660 0x7fbef80b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 >> 172.21.15.99:0/1940545782 conn(0x7fbef80b4650 msgr2=0x7fbef80d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 shutdown_connections 2024-09-10T11:31:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.711+0000 7fbefc8e6640 1 -- 172.21.15.99:0/1940545782 wait complete. 2024-09-10T11:31:27.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 Processor -- start 2024-09-10T11:31:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 -- start start 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f906c15eb80 0x7f906c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c0d5010 0x7f906c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f906c15ff70 con 0x7f906c1626c0 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f906c0d1620 con 0x7f906c15eb80 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.820+0000 7f9071db7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f906c162bc0 con 0x7f906c0d5010 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906affd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f906c1626c0 0x7f906c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59012/0 (socket says 172.21.15.99:59012) 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f906c15eb80 0x7f906c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906affd640 1 -- 172.21.15.99:0/3895394664 learned_addr learned my addr 172.21.15.99:0/3895394664 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906bfff640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c0d5010 0x7f906c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906b7fe640 1 -- 172.21.15.99:0/3895394664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c0d5010 msgr2=0x7f906c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906b7fe640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c0d5010 0x7f906c0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906b7fe640 1 -- 172.21.15.99:0/3895394664 >> v1:172.21.15.99:6789/0 conn(0x7f906c1626c0 legacy=0x7f906c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906b7fe640 1 -- 172.21.15.99:0/3895394664 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c162ea0 con 0x7f906c15eb80 2024-09-10T11:31:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.821+0000 7f906a7fc640 1 -- 172.21.15.99:0/3895394664 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996884465 0 0) 0x7f906c15ff70 con 0x7f906c1626c0 2024-09-10T11:31:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f906b7fe640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c15eb80 0x7f906c0d29b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f905c0097d0 tx=0x7f905c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d02dc6979c8313 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f906a7fc640 1 -- 172.21.15.99:0/3895394664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c037070 con 0x7f906c15eb80 2024-09-10T11:31:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f906a7fc640 1 -- 172.21.15.99:0/3895394664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c02ae40 con 0x7f906c15eb80 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f906a7fc640 1 -- 172.21.15.99:0/3895394664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c032660 con 0x7f906c15eb80 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 -- 172.21.15.99:0/3895394664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c15eb80 msgr2=0x7f906c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c15eb80 0x7f906c0d29b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f905c0097d0 tx=0x7f905c02a130 comp rx=0 tx=0).stop 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 -- 172.21.15.99:0/3895394664 shutdown_connections 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c0d5010 0x7f906c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 --2- 172.21.15.99:0/3895394664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c15eb80 0x7f906c0d29b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 -- 172.21.15.99:0/3895394664 >> 172.21.15.99:0/3895394664 conn(0x7f906c08e680 msgr2=0x7f906c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 -- 172.21.15.99:0/3895394664 shutdown_connections 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.822+0000 7f9071db7640 1 -- 172.21.15.99:0/3895394664 wait complete. 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.823+0000 7f9071db7640 1 Processor -- start 2024-09-10T11:31:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 -- start start 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 0x7f906c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c1626c0 0x7f906c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0d2ef0 con 0x7f906c0d5010 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f906c163460 con 0x7f906c15eb80 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f9071db7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f906c0d3070 con 0x7f906c1626c0 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.824+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41978/0 (socket says 172.21.15.99:41978) 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 -- 172.21.15.99:0/1315290753 learned_addr learned my addr 172.21.15.99:0/1315290753 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906bfff640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c1626c0 0x7f906c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906affd640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 0x7f906c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 -- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c1626c0 msgr2=0x7f906c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c1626c0 0x7f906c0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 -- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 msgr2=0x7f906c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 0x7f906c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 -- 172.21.15.99:0/1315290753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c0b94f0 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906affd640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 0x7f906c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f906b7fe640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c041020 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c03c670 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c1602a0 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c037040 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.825+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c160860 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.826+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c0d29b0 con 0x7f906c0d5010 2024-09-10T11:31:27.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.827+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f905c044020 con 0x7f906c0d5010 2024-09-10T11:31:27.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.827+0000 7f9053fff640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 0x7f905806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:27.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.827+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f905c0ab4c0 con 0x7f906c0d5010 2024-09-10T11:31:27.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.827+0000 7f906affd640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 0x7f905806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:27.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.827+0000 7f906affd640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 0x7f905806e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f906c0cf180 tx=0x7f9060007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:27.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:27.831+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f905c04b030 con 0x7f906c0d5010 2024-09-10T11:31:27.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:31:27.890 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:31:28.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.034+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f906c15e060 con 0x7f906c0d5010 2024-09-10T11:31:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.034+0000 7f9053fff640 1 -- 172.21.15.99:0/1315290753 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f905c079e70 con 0x7f906c0d5010 2024-09-10T11:31:28.036 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:28.036 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:28.037 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:28.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 msgr2=0x7f905806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 0x7f905806e320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f906c0cf180 tx=0x7f9060007ca0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 msgr2=0x7f906c0b8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f905c0098f0 tx=0x7f905c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 shutdown_connections 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f905806be60 0x7f905806e320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906c1626c0 0x7f906c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f906c15eb80 0x7f906c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 --2- 172.21.15.99:0/1315290753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f906c0d5010 0x7f906c0b8fb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 >> 172.21.15.99:0/1315290753 conn(0x7f906c08e680 msgr2=0x7f906c0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 shutdown_connections 2024-09-10T11:31:28.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.036+0000 7f9071db7640 1 -- 172.21.15.99:0/1315290753 wait complete. 2024-09-10T11:31:28.049 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:28.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.146+0000 7f3e89bfd640 1 Processor -- start 2024-09-10T11:31:28.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.146+0000 7f3e89bfd640 1 -- start start 2024-09-10T11:31:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e89bfd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3e8406e4d0 0x7f3e84167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e89bfd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e840ceb10 con 0x7f3e840d1180 2024-09-10T11:31:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e89bfd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e840b3a90 con 0x7f3e8415c7b0 2024-09-10T11:31:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e89bfd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3e840b3840 con 0x7f3e8406e4d0 2024-09-10T11:31:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e83fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3e8406e4d0 0x7f3e84167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e82ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3e840d1180 0x7f3e8406e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59026/0 (socket says 172.21.15.99:59026) 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e82ffd640 1 -- 172.21.15.99:0/3620738706 learned_addr learned my addr 172.21.15.99:0/3620738706 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e83fff640 1 -- 172.21.15.99:0/3620738706 >> v1:172.21.15.196:6789/0 conn(0x7f3e8415c7b0 legacy=0x7f3e840d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e83fff640 1 -- 172.21.15.99:0/3620738706 >> v1:172.21.15.99:6789/0 conn(0x7f3e840d1180 legacy=0x7f3e8406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e827fc640 1 -- 172.21.15.99:0/3620738706 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919218198 0 0) 0x7f3e78001110 con 0x7f3e840d1180 2024-09-10T11:31:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.147+0000 7f3e83fff640 1 -- 172.21.15.99:0/3620738706 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e840b3d70 con 0x7f3e8406e4d0 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e83fff640 1 --2- 172.21.15.99:0/3620738706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8406e4d0 0x7f3e84167610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009d90 tx=0x7f3e7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114e03daec8681b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e827fc640 1 -- 172.21.15.99:0/3620738706 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c007740 con 0x7f3e8406e4d0 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e827fc640 1 -- 172.21.15.99:0/3620738706 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7c0078e0 con 0x7f3e8406e4d0 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e827fc640 1 -- 172.21.15.99:0/3620738706 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c007cb0 con 0x7f3e8406e4d0 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e89bfd640 1 -- 172.21.15.99:0/3620738706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8406e4d0 msgr2=0x7f3e84167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.148+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/3620738706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8406e4d0 0x7f3e84167610 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009d90 tx=0x7f3e7c02a6a0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.149+0000 7f3e89bfd640 1 -- 172.21.15.99:0/3620738706 shutdown_connections 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.149+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/3620738706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8406e4d0 0x7f3e84167610 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.149+0000 7f3e89bfd640 1 -- 172.21.15.99:0/3620738706 >> 172.21.15.99:0/3620738706 conn(0x7f3e8406bc10 msgr2=0x7f3e8406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.149+0000 7f3e89bfd640 1 -- 172.21.15.99:0/3620738706 shutdown_connections 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.149+0000 7f3e89bfd640 1 -- 172.21.15.99:0/3620738706 wait complete. 2024-09-10T11:31:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.150+0000 7f3e89bfd640 1 Processor -- start 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.150+0000 7f3e89bfd640 1 -- start start 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e8406e4d0 0x7f3e8415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e840d1180 0x7f3e84149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840b4610 con 0x7f3e8415c7b0 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e840b4370 con 0x7f3e840d1180 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e8415cb90 con 0x7f3e8406e4d0 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41996/0 (socket says 172.21.15.99:41996) 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 -- 172.21.15.99:0/2667003827 learned_addr learned my addr 172.21.15.99:0/2667003827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e82ffd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e840d1180 0x7f3e84149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e837fe640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e8406e4d0 0x7f3e8415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 -- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e8406e4d0 msgr2=0x7f3e8415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e8406e4d0 0x7f3e8415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 -- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e840d1180 msgr2=0x7f3e84149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e840d1180 0x7f3e84149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 -- 172.21.15.99:0/2667003827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8415d400 con 0x7f3e8415c7b0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e83fff640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009eb0 tx=0x7f3e7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c040070 con 0x7f3e8415c7b0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7c031c90 con 0x7f3e8415c7b0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8415d6a0 con 0x7f3e8415c7b0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.151+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c045680 con 0x7f3e8415c7b0 2024-09-10T11:31:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.152+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e8415dc60 con 0x7f3e8415c7b0 2024-09-10T11:31:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.153+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e840d0a70 con 0x7f3e8415c7b0 2024-09-10T11:31:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.153+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e7c039040 con 0x7f3e8415c7b0 2024-09-10T11:31:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.153+0000 7f3e6bfff640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.154+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3e7c0ac7f0 con 0x7f3e8415c7b0 2024-09-10T11:31:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.154+0000 7f3e837fe640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.154+0000 7f3e837fe640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 0x7f3e7006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006fd0 tx=0x7f3e74009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.158+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e7c07ab80 con 0x7f3e8415c7b0 2024-09-10T11:31:28.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.323+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e8415b260 con 0x7f3e8415c7b0 2024-09-10T11:31:28.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.325+0000 7f3e6bfff640 1 -- 172.21.15.99:0/2667003827 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16669 (secure 0 0 0) 0x7f3e7c07ee80 con 0x7f3e8415c7b0 2024-09-10T11:31:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:28.327 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:24.542163+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 msgr2=0x7f3e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 0x7f3e7006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e74006fd0 tx=0x7f3e74009060 comp rx=0 tx=0).stop 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 msgr2=0x7f3e8415e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009eb0 tx=0x7f3e7c004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 shutdown_connections 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e7006be30 0x7f3e7006e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e8415c7b0 0x7f3e8415e740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e840d1180 0x7f3e84149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 --2- 172.21.15.99:0/2667003827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e8406e4d0 0x7f3e8415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 >> 172.21.15.99:0/2667003827 conn(0x7f3e8406bc10 msgr2=0x7f3e840b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 shutdown_connections 2024-09-10T11:31:28.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.327+0000 7f3e89bfd640 1 -- 172.21.15.99:0/2667003827 wait complete. 2024-09-10T11:31:28.340 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:31:28.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 Processor -- start 2024-09-10T11:31:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 -- start start 2024-09-10T11:31:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f462815eb80 0x7f46280d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462815ff70 con 0x7f46281626c0 2024-09-10T11:31:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46280d1620 con 0x7f46280d5010 2024-09-10T11:31:28.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.437+0000 7f462d518640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4628162bc0 con 0x7f462815eb80 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46267fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f46281626c0 0x7f46280d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59034/0 (socket says 172.21.15.99:59034) 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46267fc640 1 -- 172.21.15.99:0/220942586 learned_addr learned my addr 172.21.15.99:0/220942586 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4626ffd640 1 --2- 172.21.15.99:0/220942586 >> v2:172.21.15.196:3300/0 conn(0x7f462815eb80 0x7f46280d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1726353086 0 0) 0x7f462815ff70 con 0x7f46281626c0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4614003520 con 0x7f46281626c0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 296713247 0 0) 0x7f4614003520 con 0x7f46281626c0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 >> v2:172.21.15.196:3300/0 conn(0x7f462815eb80 msgr2=0x7f46280d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 --2- 172.21.15.99:0/220942586 >> v2:172.21.15.196:3300/0 conn(0x7f462815eb80 0x7f46280d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 >> v1:172.21.15.196:6789/0 conn(0x7f46280d5010 legacy=0x7f46280d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628162ea0 con 0x7f46281626c0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4610001670 con 0x7f46281626c0 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 >> v1:172.21.15.99:6789/0 conn(0x7f46281626c0 legacy=0x7f46280d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140039b0 0x7f4614003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4614004800 0x7f4614004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 0x7f461400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4628162bc0 con 0x7f46140051c0 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f461400c160 con 0x7f46140039b0 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f461400c360 con 0x7f4614004800 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46277fe640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 0x7f461400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f4626ffd640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140039b0 0x7f4614003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46267fc640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4614004800 0x7f4614004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46277fe640 1 -- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4614004800 msgr2=0x7f4614004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46277fe640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4614004800 0x7f4614004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.438+0000 7f46277fe640 1 -- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140039b0 msgr2=0x7f4614003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f46277fe640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140039b0 0x7f4614003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f46277fe640 1 -- 172.21.15.99:0/220942586 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614003520 con 0x7f46140051c0 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f46277fe640 1 -- 172.21.15.99:0/220942586 reap_dead start 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f46277fe640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 0x7f461400b9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4620001e20 tx=0x7f4614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d2b21128098b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620001a70 con 0x7f46140051c0 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462002ae40 con 0x7f46140051c0 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.439+0000 7f4625ffb640 1 -- 172.21.15.99:0/220942586 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003bb0 con 0x7f46140051c0 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 -- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 msgr2=0x7f461400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 0x7f461400b9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4620001e20 tx=0x7f4614005830 comp rx=0 tx=0).stop 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 -- 172.21.15.99:0/220942586 shutdown_connections 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 --2- 172.21.15.99:0/220942586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46140051c0 0x7f461400b9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 -- 172.21.15.99:0/220942586 >> 172.21.15.99:0/220942586 conn(0x7f462808e680 msgr2=0x7f462809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 -- 172.21.15.99:0/220942586 shutdown_connections 2024-09-10T11:31:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.440+0000 7f462d518640 1 -- 172.21.15.99:0/220942586 wait complete. 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.441+0000 7f462d518640 1 Processor -- start 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.441+0000 7f462d518640 1 -- start start 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 0x7f46280b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 0x7f46280d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46280d4900 con 0x7f46280d0de0 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4628163460 con 0x7f46140051c0 2024-09-10T11:31:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f462d518640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46280d4a80 con 0x7f4628157430 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42014/0 (socket says 172.21.15.99:42014) 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 -- 172.21.15.99:0/125122390 learned_addr learned my addr 172.21.15.99:0/125122390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46267fc640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 0x7f46280d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f4626ffd640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 0x7f46280b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 -- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 msgr2=0x7f46280d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 0x7f46280d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 -- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 msgr2=0x7f46280b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 0x7f46280b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46277fe640 1 -- 172.21.15.99:0/125122390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46280baa80 con 0x7f46280d0de0 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f4626ffd640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 0x7f46280b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.442+0000 7f46267fc640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 0x7f46280d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f46277fe640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4620001f40 tx=0x7f46200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620007dd0 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4620003e10 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46280b21c0 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620006b50 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.443+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46280b2780 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.444+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46280d1260 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.444+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4620032440 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.445+0000 7f461f7fe640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 0x7f460806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.445+0000 7f4626ffd640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 0x7f460806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.445+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f46200aa170 con 0x7f46280d0de0 2024-09-10T11:31:28.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.445+0000 7f4626ffd640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 0x7f460806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4618009b40 tx=0x7f4618008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.449+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4620078bb0 con 0x7f46280d0de0 2024-09-10T11:31:28.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.650+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46280d29b0 con 0x7f46280d0de0 2024-09-10T11:31:28.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.650+0000 7f461f7fe640 1 -- 172.21.15.99:0/125122390 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f46200784b0 con 0x7f46280d0de0 2024-09-10T11:31:28.652 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:28.652 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:28.653 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:28.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 msgr2=0x7f460806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 0x7f460806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4618009b40 tx=0x7f4618008040 comp rx=0 tx=0).stop 2024-09-10T11:31:28.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 msgr2=0x7f46280b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4620001f40 tx=0x7f46200014a0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 shutdown_connections 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f460806be60 0x7f460806e320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46280d0de0 0x7f46280b9e30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4628157430 0x7f46280d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 --2- 172.21.15.99:0/125122390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46140051c0 0x7f46280b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 >> 172.21.15.99:0/125122390 conn(0x7f462808e680 msgr2=0x7f4628160200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 shutdown_connections 2024-09-10T11:31:28.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.652+0000 7f462d518640 1 -- 172.21.15.99:0/125122390 wait complete. 2024-09-10T11:31:28.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:28.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.761+0000 7f7f22ffd640 1 Processor -- start 2024-09-10T11:31:28.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.761+0000 7f7f22ffd640 1 -- start start 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7f1c0d1200 0x7f7f1c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c06e5d0 0x7f7f1c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 0x7f7f1c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f7f1c06ef50 con 0x7f7f1c0d1200 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7f1c152e60 con 0x7f7f1c15c060 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f22ffd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1c0d15e0 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f20d99640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7f1c0d1200 0x7f7f1c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f20d99640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7f1c0d1200 0x7f7f1c073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42028/0 (socket says 172.21.15.99:42028) 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f20d99640 1 -- 172.21.15.99:0/3316702640 learned_addr learned my addr 172.21.15.99:0/3316702640 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:28.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.762+0000 7f7f2159a640 1 --2- 172.21.15.99:0/3316702640 >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 0x7f7f1c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f1bfff640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c06e5d0 0x7f7f1c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 -- 172.21.15.99:0/3316702640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c06e5d0 msgr2=0x7f7f1c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c06e5d0 0x7f7f1c06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 -- 172.21.15.99:0/3316702640 >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 msgr2=0x7f7f1c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3316702640 >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 0x7f7f1c15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 -- 172.21.15.99:0/3316702640 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c072680 con 0x7f7f1c0d1200 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f2159a640 1 --2- 172.21.15.99:0/3316702640 >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 0x7f7f1c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c0d1200 0x7f7f1c073440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c071b80 tx=0x7f7f0c02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1158946b57b84b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f1b7fe640 1 -- 172.21.15.99:0/3316702640 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c039070 con 0x7f7f1c0d1200 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f1b7fe640 1 -- 172.21.15.99:0/3316702640 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c009600 con 0x7f7f1c0d1200 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.763+0000 7f7f1b7fe640 1 -- 172.21.15.99:0/3316702640 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0099f0 con 0x7f7f1c0d1200 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3316702640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c0d1200 msgr2=0x7f7f1c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c0d1200 0x7f7f1c073440 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c071b80 tx=0x7f7f0c02c810 comp rx=0 tx=0).stop 2024-09-10T11:31:28.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3316702640 shutdown_connections 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3316702640 >> v2:172.21.15.196:3300/0 conn(0x7f7f1c15c060 0x7f7f1c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c06e5d0 0x7f7f1c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3316702640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c0d1200 0x7f7f1c073440 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3316702640 >> 172.21.15.99:0/3316702640 conn(0x7f7f1c06bc10 msgr2=0x7f7f1c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3316702640 shutdown_connections 2024-09-10T11:31:28.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.764+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3316702640 wait complete. 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.765+0000 7f7f22ffd640 1 Processor -- start 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.765+0000 7f7f22ffd640 1 -- start start 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c0d1200 0x7f7f1c142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c15c060 0x7f7f1c142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c072f20 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c072c40 con 0x7f7f1c15c060 2024-09-10T11:31:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f22ffd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7f1c15cd90 con 0x7f7f1c0d1200 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42034/0 (socket says 172.21.15.99:42034) 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 -- 172.21.15.99:0/3410713995 learned_addr learned my addr 172.21.15.99:0/3410713995 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f1bfff640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c0d1200 0x7f7f1c142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f2159a640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c15c060 0x7f7f1c142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 -- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c0d1200 msgr2=0x7f7f1c142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c0d1200 0x7f7f1c142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 -- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c15c060 msgr2=0x7f7f1c142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c15c060 0x7f7f1c142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.766+0000 7f7f20d99640 1 -- 172.21.15.99:0/3410713995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c0ad5c0 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f20d99640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d240 tx=0x7f7f0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c039040 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c044070 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c0ad8a0 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00b720 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.767+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c0ade60 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.768+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c06ea10 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.768+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f0c004040 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.769+0000 7f7f197fa640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 0x7f7f0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.769+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7f0c0ad700 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.769+0000 7f7f1bfff640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 0x7f7f0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.769+0000 7f7f1bfff640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 0x7f7f0806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.773+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f0c04e020 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.973+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f1c142f50 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.974+0000 7f7f197fa640 1 -- 172.21.15.99:0/3410713995 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5216 (secure 0 0 0) 0x7f7f0c07c140 con 0x7f7f1c06e5d0 2024-09-10T11:31:28.976 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:28.976 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":14,"btime":"2024-09-10T11:31:16:519607+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:13.834494+0000","modified":"2024-09-10T11:31:15.848619+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4839},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4839":{"gid":4839,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/3405308472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3405308472},{"type":"v1","addr":"172.21.15.99:6837","nonce":3405308472}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4839,"qdb_cluster":[4839]},"id":2}]} 2024-09-10T11:31:28.976 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 14 2024-09-10T11:31:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 msgr2=0x7f7f0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 0x7f7f0806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f10006fd0 tx=0x7f7f10008040 comp rx=0 tx=0).stop 2024-09-10T11:31:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 msgr2=0x7f7f1c0acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00d240 tx=0x7f7f0c004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 shutdown_connections 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7f0806be60 0x7f7f0806e320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1c15c060 0x7f7f1c142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f1c0d1200 0x7f7f1c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 --2- 172.21.15.99:0/3410713995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f1c06e5d0 0x7f7f1c0acfe0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 >> 172.21.15.99:0/3410713995 conn(0x7f7f1c06bc10 msgr2=0x7f7f1c15de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 shutdown_connections 2024-09-10T11:31:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:28.976+0000 7f7f22ffd640 1 -- 172.21.15.99:0/3410713995 wait complete. 2024-09-10T11:31:28.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.086+0000 7f2a6279f640 1 Processor -- start 2024-09-10T11:31:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.086+0000 7f2a6279f640 1 -- start start 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5c0d1290 0x7f2a5c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a5c0cf660 0x7f2a5c0cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0d34a0 con 0x7f2a5c0d1290 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c0d3780 con 0x7f2a5c0cf660 2024-09-10T11:31:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6279f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c0b4550 con 0x7f2a5c0d1bb0 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5c0d1290 0x7f2a5c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2a5c0d1290 0x7f2a5c0d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42040/0 (socket says 172.21.15.99:42040) 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 -- 172.21.15.99:0/2260222144 learned_addr learned my addr 172.21.15.99:0/2260222144 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 -- 172.21.15.99:0/2260222144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 msgr2=0x7f2a5c0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 --2- 172.21.15.99:0/2260222144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 -- 172.21.15.99:0/2260222144 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c0cf660 msgr2=0x7f2a5c0cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 --2- 172.21.15.99:0/2260222144 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c0cf660 0x7f2a5c0cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.087+0000 7f2a6179d640 1 -- 172.21.15.99:0/2260222144 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0b4830 con 0x7f2a5c0d1290 2024-09-10T11:31:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6179d640 1 --2- 172.21.15.99:0/2260222144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0d1670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009e50 tx=0x7f2a5802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae2a9440223795b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a53fff640 1 -- 172.21.15.99:0/2260222144 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58036070 con 0x7f2a5c0d1290 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a53fff640 1 -- 172.21.15.99:0/2260222144 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a58031600 con 0x7f2a5c0d1290 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a53fff640 1 -- 172.21.15.99:0/2260222144 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580319d0 con 0x7f2a5c0d1290 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 -- 172.21.15.99:0/2260222144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0d1290 msgr2=0x7f2a5c0d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 --2- 172.21.15.99:0/2260222144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0d1670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009e50 tx=0x7f2a5802a6c0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 -- 172.21.15.99:0/2260222144 shutdown_connections 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 --2- 172.21.15.99:0/2260222144 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c0cf660 0x7f2a5c0cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 --2- 172.21.15.99:0/2260222144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c0d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 --2- 172.21.15.99:0/2260222144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0d1670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.088+0000 7f2a6279f640 1 -- 172.21.15.99:0/2260222144 >> 172.21.15.99:0/2260222144 conn(0x7f2a5c0d4b50 msgr2=0x7f2a5c0d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.089+0000 7f2a6279f640 1 -- 172.21.15.99:0/2260222144 shutdown_connections 2024-09-10T11:31:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.089+0000 7f2a6279f640 1 -- 172.21.15.99:0/2260222144 wait complete. 2024-09-10T11:31:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 Processor -- start 2024-09-10T11:31:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 -- start start 2024-09-10T11:31:29.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0b96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d3130 con 0x7f2a5c0cf660 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c0d3a60 con 0x7f2a5c0d1290 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.090+0000 7f2a6279f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c15f000 con 0x7f2a5c0d1bb0 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42048/0 (socket says 172.21.15.99:42048) 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 -- 172.21.15.99:0/1655697610 learned_addr learned my addr 172.21.15.99:0/1655697610 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 -- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 msgr2=0x7f2a5c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a61f9e640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a60f9c640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0b96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 -- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 msgr2=0x7f2a5c0b96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0b96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 -- 172.21.15.99:0/1655697610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c0c8680 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a61f9e640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a60f9c640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6179d640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a58034040 tx=0x7f2a58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58041020 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a58007820 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c0c8960 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58036040 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.091+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c0c8f20 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.093+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c0d1670 con 0x7f2a5c0cf660 2024-09-10T11:31:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.093+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a58044020 con 0x7f2a5c0cf660 2024-09-10T11:31:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.093+0000 7f2a51ffb640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 0x7f2a4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.093+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2a580abd80 con 0x7f2a5c0cf660 2024-09-10T11:31:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.093+0000 7f2a60f9c640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 0x7f2a4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.094+0000 7f2a60f9c640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 0x7f2a4006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c158b60 tx=0x7f2a4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.097+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a5804b030 con 0x7f2a5c0cf660 2024-09-10T11:31:29.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.263+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a5c0b4290 con 0x7f2a5c0cf660 2024-09-10T11:31:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.264+0000 7f2a51ffb640 1 -- 172.21.15.99:0/1655697610 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16669 (secure 0 0 0) 0x7f2a5807a730 con 0x7f2a5c0cf660 2024-09-10T11:31:29.266 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:29.266 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":34,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:24.542163+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:11.651963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:12.819619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 msgr2=0x7f2a4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 0x7f2a4006e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c158b60 tx=0x7f2a4c008040 comp rx=0 tx=0).stop 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 msgr2=0x7f2a5c156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a58034040 tx=0x7f2a58004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 shutdown_connections 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2a4006be30 0x7f2a4006e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0d1bb0 0x7f2a5c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c0d1290 0x7f2a5c0b96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 --2- 172.21.15.99:0/1655697610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a5c0cf660 0x7f2a5c156100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 >> 172.21.15.99:0/1655697610 conn(0x7f2a5c0d4b50 msgr2=0x7f2a5c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 shutdown_connections 2024-09-10T11:31:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.266+0000 7f2a6279f640 1 -- 172.21.15.99:0/1655697610 wait complete. 2024-09-10T11:31:29.279 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:31:29.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.376+0000 7f1455867640 1 Processor -- start 2024-09-10T11:31:29.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.376+0000 7f1455867640 1 -- start start 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f1455867640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f145015f020 0x7f14500d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f1455867640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14500d3b40 con 0x7f14500d1d10 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f1455867640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14500d3e20 con 0x7f14500d32f0 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f1455867640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f14500b4810 con 0x7f145015f020 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f145015f020 0x7f14500d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144f7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f14500d1d10 0x7f14500b52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59040/0 (socket says 172.21.15.99:59040) 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f145015f020 0x7f14500d5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42064/0 (socket says 172.21.15.99:42064) 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144f7fe640 1 -- 172.21.15.99:0/3428716804 learned_addr learned my addr 172.21.15.99:0/3428716804 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:29.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144effd640 1 -- 172.21.15.99:0/3428716804 >> v1:172.21.15.196:6789/0 conn(0x7f14500d32f0 legacy=0x7f14500d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144effd640 1 -- 172.21.15.99:0/3428716804 >> v1:172.21.15.99:6789/0 conn(0x7f14500d1d10 legacy=0x7f14500b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144effd640 1 -- 172.21.15.99:0/3428716804 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14500b4af0 con 0x7f145015f020 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.377+0000 7f144dffb640 1 -- 172.21.15.99:0/3428716804 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462758800 0 0) 0x7f14500d3b40 con 0x7f14500d1d10 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.378+0000 7f144effd640 1 --2- 172.21.15.99:0/3428716804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500d5850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14400097a0 tx=0x7f144002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c815718ed19c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.378+0000 7f144dffb640 1 -- 172.21.15.99:0/3428716804 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440037070 con 0x7f145015f020 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.378+0000 7f144dffb640 1 -- 172.21.15.99:0/3428716804 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144002ae50 con 0x7f145015f020 2024-09-10T11:31:29.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.378+0000 7f144dffb640 1 -- 172.21.15.99:0/3428716804 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440032660 con 0x7f145015f020 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 -- 172.21.15.99:0/3428716804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 msgr2=0x7f14500d5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 --2- 172.21.15.99:0/3428716804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500d5850 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f14400097a0 tx=0x7f144002a140 comp rx=0 tx=0).stop 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 -- 172.21.15.99:0/3428716804 shutdown_connections 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 --2- 172.21.15.99:0/3428716804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500d5850 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 -- 172.21.15.99:0/3428716804 >> 172.21.15.99:0/3428716804 conn(0x7f145009dfa0 msgr2=0x7f14500d4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 -- 172.21.15.99:0/3428716804 shutdown_connections 2024-09-10T11:31:29.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.379+0000 7f1455867640 1 -- 172.21.15.99:0/3428716804 wait complete. 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.380+0000 7f1455867640 1 Processor -- start 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.380+0000 7f1455867640 1 -- start start 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14500d1d10 0x7f14500ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 0x7f14500c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14500d38e0 con 0x7f145015f020 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14500d3760 con 0x7f14500d32f0 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f1455867640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14500b5a00 con 0x7f14500d1d10 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42080/0 (socket says 172.21.15.99:42080) 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 -- 172.21.15.99:0/2389520819 learned_addr learned my addr 172.21.15.99:0/2389520819 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144effd640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14500d1d10 0x7f14500ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144e7fc640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 0x7f14500c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 -- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14500d1d10 msgr2=0x7f14500ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14500d1d10 0x7f14500ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 -- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 msgr2=0x7f14500c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 0x7f14500c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 -- 172.21.15.99:0/2389520819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14500c8da0 con 0x7f145015f020 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144e7fc640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 0x7f14500c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.381+0000 7f144f7fe640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1448001560 tx=0x7f144800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.382+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448018070 con 0x7f145015f020 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.382+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144800a640 con 0x7f145015f020 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.382+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14500c9080 con 0x7f145015f020 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.382+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144800aa10 con 0x7f145015f020 2024-09-10T11:31:29.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.382+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14500c9640 con 0x7f145015f020 2024-09-10T11:31:29.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.383+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14500b3d70 con 0x7f145015f020 2024-09-10T11:31:29.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.383+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f144801b070 con 0x7f145015f020 2024-09-10T11:31:29.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.383+0000 7f1437fff640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 0x7f143c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.384+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f144808a390 con 0x7f145015f020 2024-09-10T11:31:29.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.384+0000 7f144effd640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 0x7f143c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.384+0000 7f144effd640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 0x7f143c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1440034000 tx=0x7f144002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.388+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1448058780 con 0x7f145015f020 2024-09-10T11:31:29.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.589+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f14500b4450 con 0x7f145015f020 2024-09-10T11:31:29.591 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:31:29.590+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:31:29.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.591+0000 7f1437fff640 1 -- 172.21.15.99:0/2389520819 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7f144805ca80 con 0x7f145015f020 2024-09-10T11:31:29.593 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:31:29.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.594+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 msgr2=0x7f143c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.594+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 0x7f143c06e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1440034000 tx=0x7f144002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.594+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 msgr2=0x7f14500cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.594+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1448001560 tx=0x7f144800cc70 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 shutdown_connections 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f143c06be30 0x7f143c06e2f0 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1440034000 tx=0x7f144002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f145015f020 0x7f14500cc0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14500d32f0 0x7f14500c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 --2- 172.21.15.99:0/2389520819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14500d1d10 0x7f14500ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.595+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 >> 172.21.15.99:0/2389520819 conn(0x7f145009dfa0 msgr2=0x7f14500b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:29.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.596+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 shutdown_connections 2024-09-10T11:31:29.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.596+0000 7f1455867640 1 -- 172.21.15.99:0/2389520819 wait complete. 2024-09-10T11:31:29.598 INFO:tasks.ceph.mds.a.smithi099.stderr: -3115> 2024-09-10T11:31:17.492+0000 7f3da71d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:31:29.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:31:29.626 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:31:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.708+0000 7f9d5dfa8640 1 Processor -- start 2024-09-10T11:31:29.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 -- start start 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d580d52d0 con 0x7f9d580d4e40 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9d5815ff70 con 0x7f9d581626c0 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.709+0000 7f9d5dfa8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d580d1620 con 0x7f9d5815eb80 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d4ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:49722/0 (socket says 172.21.15.99:49722) 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d4ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:51910/0 (socket says 172.21.15.99:51910) 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 -- 172.21.15.99:0/3296810846 learned_addr learned my addr 172.21.15.99:0/3296810846 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:29.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d57fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9d580d4e40 0x7f9d580d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:55872/0 (socket says 172.21.15.99:55872) 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 -- 172.21.15.99:0/3296810846 >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 msgr2=0x7f9d580d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 --2- 172.21.15.99:0/3296810846 >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 -- 172.21.15.99:0/3296810846 >> v1:172.21.15.196:6789/0 conn(0x7f9d580d4e40 legacy=0x7f9d580d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d577fe640 1 -- 172.21.15.99:0/3296810846 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d58162bc0 con 0x7f9d5815eb80 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.710+0000 7f9d4ffff640 1 --2- 172.21.15.99:0/3296810846 >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.711+0000 7f9d577fe640 1 --2- 172.21.15.99:0/3296810846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009870 tx=0x7f9d4802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474acba8e1d767ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.711+0000 7f9d56ffd640 1 -- 172.21.15.99:0/3296810846 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48037070 con 0x7f9d5815eb80 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.711+0000 7f9d56ffd640 1 -- 172.21.15.99:0/3296810846 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d48032440 con 0x7f9d5815eb80 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.711+0000 7f9d56ffd640 1 -- 172.21.15.99:0/3296810846 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48032810 con 0x7f9d5815eb80 2024-09-10T11:31:29.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/3296810846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 msgr2=0x7f9d580d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/3296810846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009870 tx=0x7f9d4802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/3296810846 shutdown_connections 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/3296810846 >> v2:172.21.15.196:3300/0 conn(0x7f9d581626c0 0x7f9d580d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/3296810846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d5815eb80 0x7f9d580d29b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/3296810846 >> 172.21.15.99:0/3296810846 conn(0x7f9d5808e680 msgr2=0x7f9d5809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/3296810846 shutdown_connections 2024-09-10T11:31:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.712+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/3296810846 wait complete. 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.713+0000 7f9d5dfa8640 1 Processor -- start 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.713+0000 7f9d5dfa8640 1 -- start start 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 0x7f9d580ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d581626c0 0x7f9d580ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58163460 con 0x7f9d580d4e40 2024-09-10T11:31:29.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58163180 con 0x7f9d5815eb80 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d580d2ef0 con 0x7f9d581626c0 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42098/0 (socket says 172.21.15.99:42098) 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 -- 172.21.15.99:0/2568930603 learned_addr learned my addr 172.21.15.99:0/2568930603 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d57fff640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d581626c0 0x7f9d580ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d4ffff640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 0x7f9d580ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 -- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d581626c0 msgr2=0x7f9d580ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d581626c0 0x7f9d580ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 -- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 msgr2=0x7f9d580ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 0x7f9d580ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 -- 172.21.15.99:0/2568930603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5815a5c0 con 0x7f9d580d4e40 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d4ffff640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 0x7f9d580ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:29.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d577fe640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48037040 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d48042070 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d580b30f0 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48030880 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.714+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5815b7b0 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.715+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d580babb0 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.716+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d4802ad10 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.716+0000 7f9d54ff9640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 0x7f9d3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.716+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f9d480ab4b0 con 0x7f9d580d4e40 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.716+0000 7f9d4ffff640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 0x7f9d3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:29.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.717+0000 7f9d4ffff640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 0x7f9d3806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d580b5b40 tx=0x7f9d40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:29.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.720+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d48079d70 con 0x7f9d580d4e40 2024-09-10T11:31:29.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:29.922+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9d580d29b0 con 0x7f9d580d4e40 2024-09-10T11:31:30.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.592+0000 7f9d54ff9640 1 -- 172.21.15.99:0/2568930603 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f9d48079670 con 0x7f9d580d4e40 2024-09-10T11:31:30.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 msgr2=0x7f9d3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 0x7f9d3806e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d580b5b40 tx=0x7f9d40008040 comp rx=0 tx=0).stop 2024-09-10T11:31:30.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 msgr2=0x7f9d5815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9d48000c00 tx=0x7f9d480046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 shutdown_connections 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d3806be30 0x7f9d3806e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d581626c0 0x7f9d580ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d5815eb80 0x7f9d580ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 --2- 172.21.15.99:0/2568930603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d580d4e40 0x7f9d5815a080 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 >> 172.21.15.99:0/2568930603 conn(0x7f9d5808e680 msgr2=0x7f9d5815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 shutdown_connections 2024-09-10T11:31:30.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.596+0000 7f9d5dfa8640 1 -- 172.21.15.99:0/2568930603 wait complete. 2024-09-10T11:31:30.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:31:30.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.754+0000 7f563af28640 1 Processor -- start 2024-09-10T11:31:30.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f563af28640 1 -- start start 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f563af28640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f563415e340 0x7f56340cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f563af28640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340cf1d0 con 0x7f56340d30f0 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f563af28640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56340b3e40 con 0x7f5634157160 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f563af28640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f56340b4120 con 0x7f563415e340 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f5633fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f56340d30f0 0x7f563415dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59064/0 (socket says 172.21.15.99:59064) 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.755+0000 7f5633fff640 1 -- 172.21.15.99:0/4276618842 learned_addr learned my addr 172.21.15.99:0/4276618842 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749779586 0 0) 0x7f56340cf1d0 con 0x7f56340d30f0 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56394c5640 1 --2- 172.21.15.99:0/4276618842 >> v2:172.21.15.196:3300/0 conn(0x7f563415e340 0x7f56340cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5620003520 con 0x7f56340d30f0 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1713702464 0 0) 0x7f5620003520 con 0x7f56340d30f0 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 >> v2:172.21.15.196:3300/0 conn(0x7f563415e340 msgr2=0x7f56340cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 --2- 172.21.15.99:0/4276618842 >> v2:172.21.15.196:3300/0 conn(0x7f563415e340 0x7f56340cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 >> v1:172.21.15.196:6789/0 conn(0x7f5634157160 legacy=0x7f56340d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563415e750 con 0x7f56340d30f0 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f561c001670 con 0x7f56340d30f0 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 >> v1:172.21.15.99:6789/0 conn(0x7f56340d30f0 legacy=0x7f563415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5620003a00 0x7f5620003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56200051c0 0x7f562000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56340b4120 con 0x7f5620004800 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f562000c0c0 con 0x7f56200051c0 2024-09-10T11:31:30.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f562000c2c0 con 0x7f5620003a00 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56394c5640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56200051c0 0x7f562000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5638cc4640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5620003a00 0x7f5620003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 -- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5620003a00 msgr2=0x7f5620003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5620003a00 0x7f5620003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 -- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56200051c0 msgr2=0x7f562000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56200051c0 0x7f562000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f56394c5640 1 -- 172.21.15.99:0/4276618842 reap_dead start 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.756+0000 7f5633fff640 1 -- 172.21.15.99:0/4276618842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5620003520 con 0x7f5620004800 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.757+0000 7f5633fff640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 0x7f5620004c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c110 tx=0x7f5620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89631f856ffb7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.757+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c007c40 con 0x7f5620004800 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.757+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f561c005af0 con 0x7f5620004800 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.757+0000 7f56337fe640 1 -- 172.21.15.99:0/4276618842 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c007070 con 0x7f5620004800 2024-09-10T11:31:30.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 -- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 msgr2=0x7f5620004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 0x7f5620004c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f561c00c110 tx=0x7f5620005830 comp rx=0 tx=0).stop 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 -- 172.21.15.99:0/4276618842 shutdown_connections 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 --2- 172.21.15.99:0/4276618842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620004800 0x7f5620004c80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 -- 172.21.15.99:0/4276618842 >> 172.21.15.99:0/4276618842 conn(0x7f5634073cd0 msgr2=0x7f563409ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 -- 172.21.15.99:0/4276618842 shutdown_connections 2024-09-10T11:31:30.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.758+0000 7f563af28640 1 -- 172.21.15.99:0/4276618842 wait complete. 2024-09-10T11:31:30.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.759+0000 7f563af28640 1 Processor -- start 2024-09-10T11:31:30.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.759+0000 7f563af28640 1 -- start start 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5620004800 0x7f56341f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56341f01e0 0x7f56340afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f56340d0ad0 con 0x7f5634157b20 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56340d0950 con 0x7f5620004800 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f563af28640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f56340d0d30 con 0x7f56341f01e0 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42124/0 (socket says 172.21.15.99:42124) 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 -- 172.21.15.99:0/1558048025 learned_addr learned my addr 172.21.15.99:0/1558048025 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f5633fff640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56341f01e0 0x7f56340afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f5638cc4640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5620004800 0x7f56341f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 -- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56341f01e0 msgr2=0x7f56340afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56341f01e0 0x7f56340afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 -- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5620004800 msgr2=0x7f56341f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5620004800 0x7f56341f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.760+0000 7f56394c5640 1 -- 172.21.15.99:0/1558048025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56341f30c0 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f56394c5640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f56340d0690 tx=0x7f562c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c004030 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c01c070 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0043f0 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56341f33a0 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.761+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340b6a50 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.762+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634157540 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.762+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f562c012040 con 0x7f5634157b20 2024-09-10T11:31:30.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.763+0000 7f56317fa640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 0x7f561406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.763+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f562c08add0 con 0x7f5634157b20 2024-09-10T11:31:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.763+0000 7f5638cc4640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 0x7f561406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:30.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.763+0000 7f5638cc4640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 0x7f561406e320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5624000f80 tx=0x7f5624009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:30.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.767+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f562c0590d0 con 0x7f5634157b20 2024-09-10T11:31:30.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:30.936+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f563415a2b0 con 0x7f5634157b20 2024-09-10T11:31:31.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.594+0000 7f56317fa640 1 -- 172.21.15.99:0/1558048025 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f562c05d3d0 con 0x7f5634157b20 2024-09-10T11:31:31.596 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 msgr2=0x7f561406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 0x7f561406e320 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5624000f80 tx=0x7f5624009040 comp rx=0 tx=0).stop 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 msgr2=0x7f5634070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f56340d0690 tx=0x7f562c011cf0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 shutdown_connections 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f561406be60 0x7f561406e320 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5634157b20 0x7f5634070b20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56341f01e0 0x7f56340afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 --2- 172.21.15.99:0/1558048025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5620004800 0x7f56341f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 >> 172.21.15.99:0/1558048025 conn(0x7f5634073cd0 msgr2=0x7f563406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 shutdown_connections 2024-09-10T11:31:31.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.598+0000 7f563af28640 1 -- 172.21.15.99:0/1558048025 wait complete. 2024-09-10T11:31:31.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:31:31.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.717+0000 7fdf286af640 1 Processor -- start 2024-09-10T11:31:31.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.717+0000 7fdf286af640 1 -- start start 2024-09-10T11:31:31.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf286af640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdf200af930 0x7fdf201627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf286af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 0x7fdf2006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf286af640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf200afdc0 con 0x7fdf2015ee10 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf286af640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdf2006d550 con 0x7fdf200af930 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf286af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf201635c0 con 0x7fdf20162d20 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdf200af930 0x7fdf201627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdf200af930 0x7fdf201627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42138/0 (socket says 172.21.15.99:42138) 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 -- 172.21.15.99:0/3679140372 learned_addr learned my addr 172.21.15.99:0/3679140372 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf26c4c640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 0x7fdf2006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 -- 172.21.15.99:0/3679140372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 msgr2=0x7fdf2006d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 0x7fdf2006d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 -- 172.21.15.99:0/3679140372 >> v1:172.21.15.196:6789/0 conn(0x7fdf2015ee10 legacy=0x7fdf200b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf25c4a640 1 -- 172.21.15.99:0/3679140372 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf200b7b50 con 0x7fdf200af930 2024-09-10T11:31:31.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.718+0000 7fdf26c4c640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 0x7fdf2006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:31.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.719+0000 7fdf25c4a640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf200af930 0x7fdf201627e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009870 tx=0x7fdf1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9de671709219fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.719+0000 7fdf25449640 1 -- 172.21.15.99:0/3679140372 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10037070 con 0x7fdf200af930 2024-09-10T11:31:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.719+0000 7fdf25449640 1 -- 172.21.15.99:0/3679140372 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf10032440 con 0x7fdf200af930 2024-09-10T11:31:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.719+0000 7fdf25449640 1 -- 172.21.15.99:0/3679140372 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10032810 con 0x7fdf200af930 2024-09-10T11:31:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 -- 172.21.15.99:0/3679140372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf200af930 msgr2=0x7fdf201627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf200af930 0x7fdf201627e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009870 tx=0x7fdf1002a270 comp rx=0 tx=0).stop 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 -- 172.21.15.99:0/3679140372 shutdown_connections 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf20162d20 0x7fdf2006d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 --2- 172.21.15.99:0/3679140372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf200af930 0x7fdf201627e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 -- 172.21.15.99:0/3679140372 >> 172.21.15.99:0/3679140372 conn(0x7fdf20073c60 msgr2=0x7fdf2009ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 -- 172.21.15.99:0/3679140372 shutdown_connections 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.720+0000 7fdf286af640 1 -- 172.21.15.99:0/3679140372 wait complete. 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.721+0000 7fdf286af640 1 Processor -- start 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.721+0000 7fdf286af640 1 -- start start 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 0x7fdf201500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf2015ee10 0x7fdf200aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdf200b76f0 con 0x7fdf20162d20 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2006d1e0 con 0x7fdf200af930 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf200b7870 con 0x7fdf2015ee10 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42140/0 (socket says 172.21.15.99:42140) 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 -- 172.21.15.99:0/4121017766 learned_addr learned my addr 172.21.15.99:0/4121017766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf2644b640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 0x7fdf201500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf25c4a640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf2015ee10 0x7fdf200aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 -- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf2015ee10 msgr2=0x7fdf200aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf2015ee10 0x7fdf200aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 -- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 msgr2=0x7fdf201500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 0x7fdf201500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 -- 172.21.15.99:0/4121017766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201546d0 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf2644b640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 0x7fdf201500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf26c4c640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800cd50 tx=0x7fdf1800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800f680 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf1800f840 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800fc30 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201549b0 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.722+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf20154f70 con 0x7fdf20162d20 2024-09-10T11:31:31.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.723+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf201627e0 con 0x7fdf20162d20 2024-09-10T11:31:31.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.724+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf180213f0 con 0x7fdf20162d20 2024-09-10T11:31:31.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.724+0000 7fdf16ffd640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 0x7fdf0406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:31.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.724+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7fdf1808bb60 con 0x7fdf20162d20 2024-09-10T11:31:31.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.724+0000 7fdf2644b640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 0x7fdf0406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:31.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.725+0000 7fdf2644b640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 0x7fdf0406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c001700 tx=0x7fdf1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:31.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.728+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf18014030 con 0x7fdf20162d20 2024-09-10T11:31:31.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:31.894+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf2006d9d0 con 0x7fdf20162d20 2024-09-10T11:31:32.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.600+0000 7fdf16ffd640 1 -- 172.21.15.99:0/4121017766 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7fdf18059f90 con 0x7fdf20162d20 2024-09-10T11:31:32.602 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 msgr2=0x7fdf0406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 0x7fdf0406e320 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c001700 tx=0x7fdf1c009040 comp rx=0 tx=0).stop 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 msgr2=0x7fdf200b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800cd50 tx=0x7fdf1800a650 comp rx=0 tx=0).stop 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 shutdown_connections 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf0406be60 0x7fdf0406e320 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf20162d20 0x7fdf200b04d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf2015ee10 0x7fdf200aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 --2- 172.21.15.99:0/4121017766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf200af930 0x7fdf201500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 >> 172.21.15.99:0/4121017766 conn(0x7fdf20073c60 msgr2=0x7fdf2006e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 shutdown_connections 2024-09-10T11:31:32.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.604+0000 7fdf286af640 1 -- 172.21.15.99:0/4121017766 wait complete. 2024-09-10T11:31:32.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:31:32.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.722+0000 7f0cbcbf1640 1 Processor -- start 2024-09-10T11:31:32.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.722+0000 7f0cbcbf1640 1 -- start start 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cb815ee00 0x7f0cb80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0cb80b0090 0x7f0cb80b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb80b40e0 con 0x7f0cb815ee00 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb80b5ab0 con 0x7f0cb80b0090 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cbcbf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb8073c50 con 0x7f0cb815f1e0 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cb815ee00 0x7f0cb80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cb815ee00 0x7f0cb80b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36478/0 (socket says 172.21.15.99:36478) 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 -- 172.21.15.99:0/753583836 learned_addr learned my addr 172.21.15.99:0/753583836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb5d9b640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb6d9d640 1 --2- 172.21.15.99:0/753583836 >> v2:172.21.15.196:3300/0 conn(0x7f0cb80b0090 0x7f0cb80b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 -- 172.21.15.99:0/753583836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 msgr2=0x7f0cb80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 -- 172.21.15.99:0/753583836 >> v2:172.21.15.196:3300/0 conn(0x7f0cb80b0090 msgr2=0x7f0cb80b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 --2- 172.21.15.99:0/753583836 >> v2:172.21.15.196:3300/0 conn(0x7f0cb80b0090 0x7f0cb80b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.723+0000 7f0cb659c640 1 -- 172.21.15.99:0/753583836 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb80b4380 con 0x7f0cb815ee00 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.724+0000 7f0cb659c640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb80b5f90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8070d70 tx=0x7f0ca802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a53a8aceb80ac42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.724+0000 7f0cb559a640 1 -- 172.21.15.99:0/753583836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8037070 con 0x7f0cb815ee00 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.724+0000 7f0cb559a640 1 -- 172.21.15.99:0/753583836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca8032440 con 0x7f0cb815ee00 2024-09-10T11:31:32.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.724+0000 7f0cb559a640 1 -- 172.21.15.99:0/753583836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8032810 con 0x7f0cb815ee00 2024-09-10T11:31:32.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/753583836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 msgr2=0x7f0cb80b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb80b5f90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8070d70 tx=0x7f0ca802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/753583836 shutdown_connections 2024-09-10T11:31:32.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/753583836 >> v2:172.21.15.196:3300/0 conn(0x7f0cb80b0090 0x7f0cb80b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/753583836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb80b5f90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/753583836 >> 172.21.15.99:0/753583836 conn(0x7f0cb80730c0 msgr2=0x7f0cb80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/753583836 shutdown_connections 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.725+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/753583836 wait complete. 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.726+0000 7f0cbcbf1640 1 Processor -- start 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.726+0000 7f0cbcbf1640 1 -- start start 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb80b0090 0x7f0cb8155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80b1fe0 con 0x7f0cb815ee00 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb80afd60 con 0x7f0cb80b0090 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cbcbf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb80b2160 con 0x7f0cb815f1e0 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36480/0 (socket says 172.21.15.99:36480) 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 -- 172.21.15.99:0/3009271382 learned_addr learned my addr 172.21.15.99:0/3009271382 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb659c640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb80b0090 0x7f0cb8155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb6d9d640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 -- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 msgr2=0x7f0cb80ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 -- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb80b0090 msgr2=0x7f0cb8155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb80b0090 0x7f0cb8155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.727+0000 7f0cb5d9b640 1 -- 172.21.15.99:0/3009271382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb8158070 con 0x7f0cb815ee00 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0cb5d9b640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00e870 tx=0x7f0cac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cb815ee00 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cac00ee60 con 0x7f0cb815ee00 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb8158350 con 0x7f0cb815ee00 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00cb60 con 0x7f0cb815ee00 2024-09-10T11:31:32.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.728+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb8158910 con 0x7f0cb815ee00 2024-09-10T11:31:32.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.729+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb8158ae0 con 0x7f0cb815ee00 2024-09-10T11:31:32.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.729+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cac01a070 con 0x7f0cb815ee00 2024-09-10T11:31:32.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.730+0000 7f0ca6ffd640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 0x7f0c9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:32.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.730+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f0cac08aba0 con 0x7f0cb815ee00 2024-09-10T11:31:32.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.730+0000 7f0cb659c640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 0x7f0c9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:32.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.730+0000 7f0cb659c640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 0x7f0c9806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8034000 tx=0x7f0ca802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:32.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.734+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0cac059290 con 0x7f0cb815ee00 2024-09-10T11:31:32.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.900+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f0cb80b0600 con 0x7f0cb815ee00 2024-09-10T11:31:32.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.903+0000 7f0ca6ffd640 1 -- 172.21.15.99:0/3009271382 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) ==== 104+0+0 (secure 0 0 0) 0x7f0cac05d590 con 0x7f0cb815ee00 2024-09-10T11:31:32.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 msgr2=0x7f0c9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 0x7f0c9806e2f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8034000 tx=0x7f0ca802e000 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 msgr2=0x7f0cb815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00e870 tx=0x7f0cac00ed30 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 shutdown_connections 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0c9806be30 0x7f0c9806e2f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cb815f1e0 0x7f0cb80ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cb815ee00 0x7f0cb815f5c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 --2- 172.21.15.99:0/3009271382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cb80b0090 0x7f0cb8155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 >> 172.21.15.99:0/3009271382 conn(0x7f0cb80730c0 msgr2=0x7f0cb815e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 shutdown_connections 2024-09-10T11:31:32.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:32.905+0000 7f0cbcbf1640 1 -- 172.21.15.99:0/3009271382 wait complete. 2024-09-10T11:31:32.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-09-10T11:31:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.015+0000 7f22f8617640 1 Processor -- start 2024-09-10T11:31:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.015+0000 7f22f8617640 1 -- start start 2024-09-10T11:31:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f8617640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22f0073440 0x7f22f015c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f8617640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f0152e60 con 0x7f22f00d1200 2024-09-10T11:31:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f8617640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f00d15e0 con 0x7f22f006ece0 2024-09-10T11:31:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f8617640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f22f0072680 con 0x7f22f0073440 2024-09-10T11:31:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f63b3640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f22f00d1200 0x7f22f006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40542/0 (socket says 172.21.15.99:40542) 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f63b3640 1 -- 172.21.15.99:0/98759735 learned_addr learned my addr 172.21.15.99:0/98759735 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 433712299 0 0) 0x7f22f0152e60 con 0x7f22f00d1200 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22dc003520 con 0x7f22f00d1200 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> v2:172.21.15.196:3300/0 conn(0x7f22f0073440 0x7f22f015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2951153270 0 0) 0x7f22dc003520 con 0x7f22f00d1200 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 >> v2:172.21.15.196:3300/0 conn(0x7f22f0073440 msgr2=0x7f22f015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 --2- 172.21.15.99:0/98759735 >> v2:172.21.15.196:3300/0 conn(0x7f22f0073440 0x7f22f015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 >> v1:172.21.15.196:6789/0 conn(0x7f22f006ece0 legacy=0x7f22f015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f0072960 con 0x7f22f00d1200 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f22ec001670 con 0x7f22f00d1200 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> v2:172.21.15.196:3300/0 conn(0x7f22f0073440 0x7f22f015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:33.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 >> v1:172.21.15.99:6789/0 conn(0x7f22f00d1200 legacy=0x7f22f006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.016+0000 7f22f53b1640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 0x7f22dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0072680 con 0x7f22dc0052e0 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc00c1b0 con 0x7f22dc0039b0 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc00c3b0 con 0x7f22dc004800 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 0x7f22dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f5bb2640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f63b3640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 -- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22dc004800 msgr2=0x7f22dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22dc004800 0x7f22dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 -- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0039b0 msgr2=0x7f22dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0039b0 0x7f22dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 -- 172.21.15.99:0/98759735 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc003520 con 0x7f22dc0052e0 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.017+0000 7f22f6bb4640 1 -- 172.21.15.99:0/98759735 reap_dead start 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 0x7f22dc00b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22e80073b0 tx=0x7f22dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf345f0d11e520d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8009c80 con 0x7f22dc0052e0 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e802c650 con 0x7f22dc0052e0 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f53b1640 1 -- 172.21.15.99:0/98759735 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e802ca40 con 0x7f22dc0052e0 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 -- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 msgr2=0x7f22dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 0x7f22dc00b9c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f22e80073b0 tx=0x7f22dc005830 comp rx=0 tx=0).stop 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 -- 172.21.15.99:0/98759735 shutdown_connections 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 --2- 172.21.15.99:0/98759735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22dc0052e0 0x7f22dc00b9c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 -- 172.21.15.99:0/98759735 >> 172.21.15.99:0/98759735 conn(0x7f22f006bc10 msgr2=0x7f22f006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 -- 172.21.15.99:0/98759735 shutdown_connections 2024-09-10T11:31:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.018+0000 7f22f8617640 1 -- 172.21.15.99:0/98759735 wait complete. 2024-09-10T11:31:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.019+0000 7f22f8617640 1 Processor -- start 2024-09-10T11:31:33.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 -- start start 2024-09-10T11:31:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 0x7f22f01f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22f014f360 0x7f22f01f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22f015cd90 con 0x7f22f01f36c0 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22f0072f20 con 0x7f22dc0052e0 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f8617640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22f006e5d0 con 0x7f22f014f360 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f6bb4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f6bb4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36490/0 (socket says 172.21.15.99:36490) 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.020+0000 7f22f6bb4640 1 -- 172.21.15.99:0/3450028026 learned_addr learned my addr 172.21.15.99:0/3450028026 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f63b3640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 0x7f22f01f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f5bb2640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22f014f360 0x7f22f01f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 -- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22f014f360 msgr2=0x7f22f01f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22f014f360 0x7f22f01f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 -- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 msgr2=0x7f22f01f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 0x7f22f01f3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 -- 172.21.15.99:0/3450028026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f01f3b60 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f63b3640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 0x7f22f01f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f6bb4640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22e80074d0 tx=0x7f22e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e8009c80 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e802d9f0 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f01f3e40 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e802ddc0 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.021+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f01f4400 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.022+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f006e960 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.022+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22e8036400 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.023+0000 7f22e6ffd640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.023+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f22e80ab130 con 0x7f22f01f36c0 2024-09-10T11:31:33.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.023+0000 7f22f63b3640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.023+0000 7f22f63b3640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 0x7f22d406e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f22ec001eb0 tx=0x7f22ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.027+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f22e8032030 con 0x7f22f01f36c0 2024-09-10T11:31:33.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.259+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7f22f015fed0 con 0x7f22f01f36c0 2024-09-10T11:31:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.602+0000 7f22e6ffd640 1 -- 172.21.15.99:0/3450028026 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v76) ==== 130+0+0 (secure 0 0 0) 0x7f22e8079790 con 0x7f22f01f36c0 2024-09-10T11:31:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 msgr2=0x7f22d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 0x7f22d406e2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f22ec001eb0 tx=0x7f22ec009040 comp rx=0 tx=0).stop 2024-09-10T11:31:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 msgr2=0x7f22f01f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22e80074d0 tx=0x7f22e8004ab0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 shutdown_connections 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f22d406be30 0x7f22d406e2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22f01f36c0 0x7f22f01f5140 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22f014f360 0x7f22f01f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 --2- 172.21.15.99:0/3450028026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22dc0052e0 0x7f22f01f3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 >> 172.21.15.99:0/3450028026 conn(0x7f22f006bc10 msgr2=0x7f22f015e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 shutdown_connections 2024-09-10T11:31:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.604+0000 7f22f8617640 1 -- 172.21.15.99:0/3450028026 wait complete. 2024-09-10T11:31:33.618 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-09-10T11:31:33.618 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-09-10T11:31:33.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:33.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.719+0000 7f6f065e8640 1 Processor -- start 2024-09-10T11:31:33.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.719+0000 7f6f065e8640 1 -- start start 2024-09-10T11:31:33.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f065e8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f065e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f065e8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f000b3620 con 0x7f6f000b3960 2024-09-10T11:31:33.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f065e8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0006d960 con 0x7f6f00163580 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f065e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0006dc40 con 0x7f6f000b3160 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f04b85640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6efffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6efffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:37756/0 (socket says 172.21.15.99:37756) 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6f04b85640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:32872/0 (socket says 172.21.15.99:32872) 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6efffff640 1 -- 172.21.15.99:0/18755660 learned_addr learned my addr 172.21.15.99:0/18755660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.720+0000 7f6eff7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f6f000b3960 0x7f6f000d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37658/0 (socket says 172.21.15.99:37658) 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6f04b85640 1 -- 172.21.15.99:0/18755660 >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 msgr2=0x7f6f000b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6f04b85640 1 --2- 172.21.15.99:0/18755660 >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6f04b85640 1 -- 172.21.15.99:0/18755660 >> v1:172.21.15.196:6789/0 conn(0x7f6f000b3960 legacy=0x7f6f000d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6f04b85640 1 -- 172.21.15.99:0/18755660 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0006df20 con 0x7f6f000b3160 2024-09-10T11:31:33.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6efffff640 1 --2- 172.21.15.99:0/18755660 >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.721+0000 7f6f04b85640 1 --2- 172.21.15.99:0/18755660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009d90 tx=0x7f6ef802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=176a2f3a1adcae6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6efeffd640 1 -- 172.21.15.99:0/18755660 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8007740 con 0x7f6f000b3160 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6efeffd640 1 -- 172.21.15.99:0/18755660 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef80078e0 con 0x7f6f000b3160 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6efeffd640 1 -- 172.21.15.99:0/18755660 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8007cb0 con 0x7f6f000b3160 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 -- 172.21.15.99:0/18755660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 msgr2=0x7f6f0006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 --2- 172.21.15.99:0/18755660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009d90 tx=0x7f6ef802a530 comp rx=0 tx=0).stop 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 -- 172.21.15.99:0/18755660 shutdown_connections 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 --2- 172.21.15.99:0/18755660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f0006cd20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 --2- 172.21.15.99:0/18755660 >> v2:172.21.15.196:3300/0 conn(0x7f6f00163580 0x7f6f000b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 -- 172.21.15.99:0/18755660 >> 172.21.15.99:0/18755660 conn(0x7f6f000b4570 msgr2=0x7f6f000b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 -- 172.21.15.99:0/18755660 shutdown_connections 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.722+0000 7f6f065e8640 1 -- 172.21.15.99:0/18755660 wait complete. 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.723+0000 7f6f065e8640 1 Processor -- start 2024-09-10T11:31:33.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 -- start start 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f000cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f000b3960 0x7f6f000cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0006d3e0 con 0x7f6f00163580 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0006d260 con 0x7f6f000b3960 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f065e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0006d560 con 0x7f6f000b3160 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f04b85640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f04b85640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36500/0 (socket says 172.21.15.99:36500) 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6f04b85640 1 -- 172.21.15.99:0/1635684996 learned_addr learned my addr 172.21.15.99:0/1635684996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:33.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6efffff640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f000cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.724+0000 7f6eff7fe640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f000b3960 0x7f6f000cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 -- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 msgr2=0x7f6f000cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f000cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 -- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f000b3960 msgr2=0x7f6f000cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f000b3960 0x7f6f000cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 -- 172.21.15.99:0/1635684996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f000cb7b0 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f04b85640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009eb0 tx=0x7f6ef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef803f070 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef8031bc0 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f000cba90 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8009450 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.725+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f000cc050 con 0x7f6f00163580 2024-09-10T11:31:33.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.726+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0006c860 con 0x7f6f00163580 2024-09-10T11:31:33.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.727+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ef8044460 con 0x7f6f00163580 2024-09-10T11:31:33.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.727+0000 7f6efcff9640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 0x7f6eec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:33.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.727+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6ef80ac1b0 con 0x7f6f00163580 2024-09-10T11:31:33.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.727+0000 7f6efffff640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 0x7f6eec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:33.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.727+0000 7f6efffff640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 0x7f6eec06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0006ca0 tx=0x7f6ef0009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:33.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.731+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f0006c860 con 0x7f6f00163580 2024-09-10T11:31:33.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.897+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f000d0790 con 0x7f6f00163580 2024-09-10T11:31:33.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.898+0000 7f6efcff9640 1 -- 172.21.15.99:0/1635684996 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13032 (secure 0 0 0) 0x7f6f000d0790 con 0x7f6f00163580 2024-09-10T11:31:33.899 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:33.899 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":37,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:32.599346+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6837/3405308472":"2024-09-11T11:31:29.591170+0000","172.21.15.99:6836/3405308472":"2024-09-11T11:31:29.591170+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 msgr2=0x7f6eec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 0x7f6eec06e2f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0006ca0 tx=0x7f6ef0009240 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 msgr2=0x7f6f001f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009eb0 tx=0x7f6ef8004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 shutdown_connections 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6eec06be30 0x7f6eec06e2f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00163580 0x7f6f001f4ff0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f000b3960 0x7f6f000cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 --2- 172.21.15.99:0/1635684996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f000b3160 0x7f6f000cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 >> 172.21.15.99:0/1635684996 conn(0x7f6f000b4570 msgr2=0x7f6f0015ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 shutdown_connections 2024-09-10T11:31:33.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:33.900+0000 7f6f065e8640 1 -- 172.21.15.99:0/1635684996 wait complete. 2024-09-10T11:31:33.913 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:31:33.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:31:34.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.013+0000 7f87c0911640 1 Processor -- start 2024-09-10T11:31:34.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.013+0000 7f87c0911640 1 -- start start 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87c0911640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f87bc0cf410 0x7f87bc0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87c0911640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87bc15cc30 0x7f87bc0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87c0911640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87bc0cea00 con 0x7f87bc0cf7f0 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87c0911640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f87bc0d1540 con 0x7f87bc15cc30 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87c0911640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f87bc0d1770 con 0x7f87bc0cf410 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b9d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f87bc15cc30 0x7f87bc0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87bad9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f87bc0cf7f0 0x7f87bc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40552/0 (socket says 172.21.15.99:40552) 2024-09-10T11:31:34.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87bad9d640 1 -- 172.21.15.99:0/2474759198 learned_addr learned my addr 172.21.15.99:0/2474759198 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b9d9b640 1 -- 172.21.15.99:0/2474759198 >> v2:172.21.15.196:3300/0 conn(0x7f87bc0cf410 msgr2=0x7f87bc0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87ba59c640 1 --2- 172.21.15.99:0/2474759198 >> v2:172.21.15.196:3300/0 conn(0x7f87bc0cf410 0x7f87bc0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b9d9b640 1 --2- 172.21.15.99:0/2474759198 >> v2:172.21.15.196:3300/0 conn(0x7f87bc0cf410 0x7f87bc0cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b9d9b640 1 -- 172.21.15.99:0/2474759198 >> v1:172.21.15.99:6789/0 conn(0x7f87bc0cf7f0 legacy=0x7f87bc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b959a640 1 -- 172.21.15.99:0/2474759198 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2668216266 0 0) 0x7f87bc0cea00 con 0x7f87bc0cf7f0 2024-09-10T11:31:34.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.014+0000 7f87b9d9b640 1 -- 172.21.15.99:0/2474759198 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc0cd810 con 0x7f87bc15cc30 2024-09-10T11:31:34.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87b9d9b640 1 --2- 172.21.15.99:0/2474759198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc15cc30 0x7f87bc0d1000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87bc072600 tx=0x7f87ac02a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952fa83102da15bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87b959a640 1 -- 172.21.15.99:0/2474759198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac037070 con 0x7f87bc15cc30 2024-09-10T11:31:34.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87b959a640 1 -- 172.21.15.99:0/2474759198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ac02add0 con 0x7f87bc15cc30 2024-09-10T11:31:34.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87b959a640 1 -- 172.21.15.99:0/2474759198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac032660 con 0x7f87bc15cc30 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87c0911640 1 -- 172.21.15.99:0/2474759198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc15cc30 msgr2=0x7f87bc0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.015+0000 7f87c0911640 1 --2- 172.21.15.99:0/2474759198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc15cc30 0x7f87bc0d1000 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f87bc072600 tx=0x7f87ac02a080 comp rx=0 tx=0).stop 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 -- 172.21.15.99:0/2474759198 shutdown_connections 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 --2- 172.21.15.99:0/2474759198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc15cc30 0x7f87bc0d1000 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 --2- 172.21.15.99:0/2474759198 >> v2:172.21.15.196:3300/0 conn(0x7f87bc0cf410 0x7f87bc0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 -- 172.21.15.99:0/2474759198 >> 172.21.15.99:0/2474759198 conn(0x7f87bc0b4570 msgr2=0x7f87bc0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 -- 172.21.15.99:0/2474759198 shutdown_connections 2024-09-10T11:31:34.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.016+0000 7f87c0911640 1 -- 172.21.15.99:0/2474759198 wait complete. 2024-09-10T11:31:34.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.017+0000 7f87c0911640 1 Processor -- start 2024-09-10T11:31:34.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.017+0000 7f87c0911640 1 -- start start 2024-09-10T11:31:34.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87bc0cf7f0 0x7f87bc0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87bc15cc30 0x7f87bc1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0cfc00 con 0x7f87bc0cf410 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0cd4c0 con 0x7f87bc15cc30 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc0cfd80 con 0x7f87bc0cf7f0 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36510/0 (socket says 172.21.15.99:36510) 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 -- 172.21.15.99:0/1296776923 learned_addr learned my addr 172.21.15.99:0/1296776923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87bad9d640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87bc15cc30 0x7f87bc1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87b9d9b640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87bc0cf7f0 0x7f87bc0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 -- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87bc0cf7f0 msgr2=0x7f87bc0af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87bc0cf7f0 0x7f87bc0af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 -- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87bc15cc30 msgr2=0x7f87bc1f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87bc15cc30 0x7f87bc1f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 -- 172.21.15.99:0/1296776923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc0b0130 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87ba59c640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c820 tx=0x7f87a400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4015070 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4004020 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc0b0410 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a40043f0 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.018+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc0b09d0 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.019+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc0b0e10 con 0x7f87bc0cf410 2024-09-10T11:31:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.020+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87a4007830 con 0x7f87bc0cf410 2024-09-10T11:31:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.020+0000 7f87b2ffd640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.021+0000 7f87b9d9b640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.021+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f87a408a990 con 0x7f87bc0cf410 2024-09-10T11:31:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.021+0000 7f87b9d9b640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 0x7f879c06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87ac034000 tx=0x7f87ac02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.025+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87a4059640 con 0x7f87bc0cf410 2024-09-10T11:31:34.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.188+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87bc0b89a0 con 0x7f87bc0cf410 2024-09-10T11:31:34.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.189+0000 7f87b2ffd640 1 -- 172.21.15.99:0/1296776923 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f87a4058f40 con 0x7f87bc0cf410 2024-09-10T11:31:34.190 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 msgr2=0x7f879c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 0x7f879c06e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f87ac034000 tx=0x7f87ac02e000 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 msgr2=0x7f87bc0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c820 tx=0x7f87a400cce0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 shutdown_connections 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f879c06be30 0x7f879c06e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f87bc15cc30 0x7f87bc1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f87bc0cf7f0 0x7f87bc0af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 --2- 172.21.15.99:0/1296776923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87bc0cf410 0x7f87bc0b2460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 >> 172.21.15.99:0/1296776923 conn(0x7f87bc0b4570 msgr2=0x7f87bc0b7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:34.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 shutdown_connections 2024-09-10T11:31:34.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.191+0000 7f87c0911640 1 -- 172.21.15.99:0/1296776923 wait complete. 2024-09-10T11:31:34.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-09-10T11:31:34.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.300+0000 7fb078f66640 1 Processor -- start 2024-09-10T11:31:34.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.300+0000 7fb078f66640 1 -- start start 2024-09-10T11:31:34.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.300+0000 7fb078f66640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0740d1200 0x7fb074073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb078f66640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb07406e5d0 0x7fb07406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb078f66640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb078f66640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb07406ef80 con 0x7fb0740d1200 2024-09-10T11:31:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb078f66640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb074152e60 con 0x7fb07406e5d0 2024-09-10T11:31:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb078f66640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740d15e0 con 0x7fb07415c060 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0740d1200 0x7fb074073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0740d1200 0x7fb074073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36526/0 (socket says 172.21.15.99:36526) 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 -- 172.21.15.99:0/3859859570 learned_addr learned my addr 172.21.15.99:0/3859859570 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 -- 172.21.15.99:0/3859859570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 msgr2=0x7fb07415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 --2- 172.21.15.99:0/3859859570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 -- 172.21.15.99:0/3859859570 >> v2:172.21.15.196:3300/0 conn(0x7fb07406e5d0 msgr2=0x7fb07406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 --2- 172.21.15.99:0/3859859570 >> v2:172.21.15.196:3300/0 conn(0x7fb07406e5d0 0x7fb07406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.301+0000 7fb07259c640 1 -- 172.21.15.99:0/3859859570 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb074072680 con 0x7fb0740d1200 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb07259c640 1 --2- 172.21.15.99:0/3859859570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0740d1200 0x7fb074073440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb074071bb0 tx=0x7fb06402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fb6e28ba0d2a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb071d9b640 1 -- 172.21.15.99:0/3859859570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064032700 con 0x7fb0740d1200 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb071d9b640 1 -- 172.21.15.99:0/3859859570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0640328a0 con 0x7fb0740d1200 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb071d9b640 1 -- 172.21.15.99:0/3859859570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064032c70 con 0x7fb0740d1200 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 -- 172.21.15.99:0/3859859570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0740d1200 msgr2=0x7fb074073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 --2- 172.21.15.99:0/3859859570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0740d1200 0x7fb074073440 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb074071bb0 tx=0x7fb06402c800 comp rx=0 tx=0).stop 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 -- 172.21.15.99:0/3859859570 shutdown_connections 2024-09-10T11:31:34.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 --2- 172.21.15.99:0/3859859570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 --2- 172.21.15.99:0/3859859570 >> v2:172.21.15.196:3300/0 conn(0x7fb07406e5d0 0x7fb07406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 --2- 172.21.15.99:0/3859859570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0740d1200 0x7fb074073440 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 -- 172.21.15.99:0/3859859570 >> 172.21.15.99:0/3859859570 conn(0x7fb07406bc10 msgr2=0x7fb07406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 -- 172.21.15.99:0/3859859570 shutdown_connections 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.302+0000 7fb078f66640 1 -- 172.21.15.99:0/3859859570 wait complete. 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.304+0000 7fb078f66640 1 Processor -- start 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.304+0000 7fb078f66640 1 -- start start 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 0x7fb07415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb074072f20 con 0x7fb07406e5d0 2024-09-10T11:31:34.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb074072c40 con 0x7fb0740d1200 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb07415cd90 con 0x7fb07415c060 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36534/0 (socket says 172.21.15.99:36534) 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 -- 172.21.15.99:0/1611936545 learned_addr learned my addr 172.21.15.99:0/1611936545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb072d9d640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb069d9b640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 0x7fb07415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 -- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 msgr2=0x7fb07415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 -- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 msgr2=0x7fb07415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 0x7fb07415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 -- 172.21.15.99:0/1611936545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07413f700 con 0x7fb07406e5d0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb072d9d640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb069d9b640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 0x7fb07415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb07259c640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb06402c840 tx=0x7fb0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064034c30 con 0x7fb07406e5d0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064034dd0 con 0x7fb07406e5d0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0740cdc40 con 0x7fb07406e5d0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064048650 con 0x7fb07406e5d0 2024-09-10T11:31:34.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.305+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0740ce200 con 0x7fb07406e5d0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.306+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07406ea40 con 0x7fb07406e5d0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.307+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb064046020 con 0x7fb07406e5d0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.307+0000 7fb06b7fe640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.307+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fb0640adf80 con 0x7fb07406e5d0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.307+0000 7fb069d9b640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.308+0000 7fb069d9b640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 0x7fb05406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb07414a9d0 tx=0x7fb05c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.312+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb07413f700 con 0x7fb07406e5d0 2024-09-10T11:31:34.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.539+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7fb074151990 con 0x7fb07406e5d0 2024-09-10T11:31:34.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.603+0000 7fb06b7fe640 1 -- 172.21.15.99:0/1611936545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v77) ==== 145+0+0 (secure 0 0 0) 0x7fb064031070 con 0x7fb07406e5d0 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 msgr2=0x7fb05406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 0x7fb05406e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb07414a9d0 tx=0x7fb05c008040 comp rx=0 tx=0).stop 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 msgr2=0x7fb07413f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb06402c840 tx=0x7fb0640046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 shutdown_connections 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb05406be30 0x7fb05406e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb07415c060 0x7fb07415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0740d1200 0x7fb07415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 --2- 172.21.15.99:0/1611936545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb07406e5d0 0x7fb07413f1c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 >> 172.21.15.99:0/1611936545 conn(0x7fb07406bc10 msgr2=0x7fb07415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 shutdown_connections 2024-09-10T11:31:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.605+0000 7fb078f66640 1 -- 172.21.15.99:0/1611936545 wait complete. 2024-09-10T11:31:34.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-09-10T11:31:34.726 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2024-09-10T11:31:34.726 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-09-10T11:31:34.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:31:34.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.826+0000 7fef64742640 1 Processor -- start 2024-09-10T11:31:34.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.826+0000 7fef64742640 1 -- start start 2024-09-10T11:31:34.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef64742640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef5c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef64742640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef5c15c770 0x7fef5c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef64742640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef5c152e60 con 0x7fef5c06e5d0 2024-09-10T11:31:34.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef64742640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fef5c06ef50 con 0x7fef5c15c770 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef64742640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef5c0d15e0 con 0x7fef5c0d1200 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef61cdd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fef5c06e5d0 0x7fef5c15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40556/0 (socket says 172.21.15.99:40556) 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.827+0000 7fef61cdd640 1 -- 172.21.15.99:0/2726432590 learned_addr learned my addr 172.21.15.99:0/2726432590 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef624de640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef5c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef62cdf640 1 --2- 172.21.15.99:0/2726432590 >> v2:172.21.15.196:3300/0 conn(0x7fef5c15c770 0x7fef5c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852726931 0 0) 0x7fef5c152e60 con 0x7fef5c06e5d0 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef48003520 con 0x7fef5c06e5d0 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2078957805 0 0) 0x7fef48003520 con 0x7fef5c06e5d0 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 msgr2=0x7fef5c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef5c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 >> v2:172.21.15.196:3300/0 conn(0x7fef5c15c770 msgr2=0x7fef5c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 --2- 172.21.15.99:0/2726432590 >> v2:172.21.15.196:3300/0 conn(0x7fef5c15c770 0x7fef5c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c072680 con 0x7fef5c06e5d0 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fef4c001670 con 0x7fef5c06e5d0 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef624de640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef5c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 >> v1:172.21.15.99:6789/0 conn(0x7fef5c06e5d0 legacy=0x7fef5c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef48003ef0 0x7fef48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48004d40 0x7fef480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c06ef50 con 0x7fef48003ef0 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c0d15e0 con 0x7fef48004d40 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef4800b9c0 con 0x7fef5c0d1200 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef62cdf640 1 --2- 172.21.15.99:0/2726432590 >> v2:172.21.15.196:3300/0 conn(0x7fef5c15c770 0x7fef5c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef48003ef0 0x7fef48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef624de640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef62cdf640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48004d40 0x7fef480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 -- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 msgr2=0x7fef480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c0d1200 0x7fef480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 -- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48004d40 msgr2=0x7fef480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48004d40 0x7fef480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef61cdd640 1 -- 172.21.15.99:0/2726432590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef48003520 con 0x7fef48003ef0 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.828+0000 7fef62cdf640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48004d40 0x7fef480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.829+0000 7fef61cdd640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef48003ef0 0x7fef48004800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea0d5ca27e4cd1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.829+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c007d50 con 0x7fef48003ef0 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.829+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c005af0 con 0x7fef48003ef0 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.829+0000 7fef614dc640 1 -- 172.21.15.99:0/2726432590 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c007070 con 0x7fef48003ef0 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 -- 172.21.15.99:0/2726432590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef48003ef0 msgr2=0x7fef48004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 --2- 172.21.15.99:0/2726432590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef48003ef0 0x7fef48004800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef48005830 comp rx=0 tx=0).stop 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef62cdf640 1 -- 172.21.15.99:0/2726432590 reap_dead start 2024-09-10T11:31:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 -- 172.21.15.99:0/2726432590 shutdown_connections 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 -- 172.21.15.99:0/2726432590 >> 172.21.15.99:0/2726432590 conn(0x7fef5c06bc10 msgr2=0x7fef5c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 -- 172.21.15.99:0/2726432590 shutdown_connections 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.830+0000 7fef64742640 1 -- 172.21.15.99:0/2726432590 wait complete. 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.831+0000 7fef64742640 1 Processor -- start 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.831+0000 7fef64742640 1 -- start start 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48003ef0 0x7fef5c1f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 0x7fef5c1f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c072f20 con 0x7fef5c15a280 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c072c40 con 0x7fef48003ef0 2024-09-10T11:31:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef5c15cd90 con 0x7fef5c1f2d40 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36538/0 (socket says 172.21.15.99:36538) 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 -- 172.21.15.99:0/3234248044 learned_addr learned my addr 172.21.15.99:0/3234248044 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef624de640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48003ef0 0x7fef5c1f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef62cdf640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 0x7fef5c1f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 -- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 msgr2=0x7fef5c1f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 0x7fef5c1f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 -- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48003ef0 msgr2=0x7fef5c1f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48003ef0 0x7fef5c1f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 -- 172.21.15.99:0/3234248044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c0aa6a0 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef62cdf640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 0x7fef5c1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef61cdd640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02ae20 tx=0x7fef4c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c042020 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef4c004bf0 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c0aa980 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.832+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c001c90 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.833+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c0aaf40 con 0x7fef5c15a280 2024-09-10T11:31:34.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.833+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c06ea10 con 0x7fef5c15a280 2024-09-10T11:31:34.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.834+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef4c045020 con 0x7fef5c15a280 2024-09-10T11:31:34.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.834+0000 7fef52ffd640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 0x7fef4006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:34.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.834+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fef4c0ab3b0 con 0x7fef5c15a280 2024-09-10T11:31:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.834+0000 7fef624de640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 0x7fef4006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.835+0000 7fef624de640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 0x7fef4006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58009ad0 tx=0x7fef58009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:34.839+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef4c04c020 con 0x7fef5c15a280 2024-09-10T11:31:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.006+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fef5c156370 con 0x7fef5c15a280 2024-09-10T11:31:35.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.009+0000 7fef52ffd640 1 -- 172.21.15.99:0/3234248044 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) ==== 122+0+0 (secure 0 0 0) 0x7fef4c07a0f0 con 0x7fef5c15a280 2024-09-10T11:31:35.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 msgr2=0x7fef4006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 0x7fef4006e320 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fef58009ad0 tx=0x7fef58009190 comp rx=0 tx=0).stop 2024-09-10T11:31:35.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 msgr2=0x7fef5c15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02ae20 tx=0x7fef4c001a40 comp rx=0 tx=0).stop 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 shutdown_connections 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef4006be60 0x7fef4006e320 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef5c1f2d40 0x7fef5c1f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef5c15a280 0x7fef5c15a700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 --2- 172.21.15.99:0/3234248044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef48003ef0 0x7fef5c1f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 >> 172.21.15.99:0/3234248044 conn(0x7fef5c06bc10 msgr2=0x7fef5c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 shutdown_connections 2024-09-10T11:31:35.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.011+0000 7fef64742640 1 -- 172.21.15.99:0/3234248044 wait complete. 2024-09-10T11:31:35.023 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:31:35.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:31:35.066 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:31:35.112 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:31:35.112 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:35.112 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:35.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:31:35.128 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:31:35.128 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:35.146 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:31:35.146 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:35.146 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:35.183 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:31:35.183 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:31:35.183 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:35.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:31:35.204 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:31:35.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.319+0000 7fa5a0630640 1 Processor -- start 2024-09-10T11:31:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.319+0000 7fa5a0630640 1 -- start start 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa5a0630640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 0x7fa5980b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa5a0630640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5980afbb0 0x7fa5980714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa5a0630640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa59806ee10 con 0x7fa59815aef0 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa5a0630640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa5980afff0 con 0x7fa5980afbb0 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa5a0630640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5980cd9a0 con 0x7fa59815b2d0 2024-09-10T11:31:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa59ebcd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5980afbb0 0x7fa5980714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa59ebcd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5980afbb0 0x7fa5980714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36542/0 (socket says 172.21.15.99:36542) 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.320+0000 7fa59ebcd640 1 -- 172.21.15.99:0/3206477852 learned_addr learned my addr 172.21.15.99:0/3206477852 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59ebcd640 1 -- 172.21.15.99:0/3206477852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 msgr2=0x7fa5980b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59dbcb640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 0x7fa5980b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59ebcd640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 0x7fa5980b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59ebcd640 1 -- 172.21.15.99:0/3206477852 >> v1:172.21.15.196:6789/0 conn(0x7fa59815aef0 legacy=0x7fa5980cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59ebcd640 1 -- 172.21.15.99:0/3206477852 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5980b0290 con 0x7fa5980afbb0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59dbcb640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 0x7fa5980b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59ebcd640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5980afbb0 0x7fa5980714a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa59000bb10 tx=0x7fa59002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b546a0d78adeadd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59d3ca640 1 -- 172.21.15.99:0/3206477852 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590032790 con 0x7fa5980afbb0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59d3ca640 1 -- 172.21.15.99:0/3206477852 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590038070 con 0x7fa5980afbb0 2024-09-10T11:31:35.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.321+0000 7fa59d3ca640 1 -- 172.21.15.99:0/3206477852 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590032b50 con 0x7fa5980afbb0 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 -- 172.21.15.99:0/3206477852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5980afbb0 msgr2=0x7fa5980714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5980afbb0 0x7fa5980714a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa59000bb10 tx=0x7fa59002c8d0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 -- 172.21.15.99:0/3206477852 shutdown_connections 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5980afbb0 0x7fa5980714a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 --2- 172.21.15.99:0/3206477852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa59815b2d0 0x7fa5980b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 -- 172.21.15.99:0/3206477852 >> 172.21.15.99:0/3206477852 conn(0x7fa59806bc10 msgr2=0x7fa59806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 -- 172.21.15.99:0/3206477852 shutdown_connections 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.322+0000 7fa5a0630640 1 -- 172.21.15.99:0/3206477852 wait complete. 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.323+0000 7fa5a0630640 1 Processor -- start 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.323+0000 7fa5a0630640 1 -- start start 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 0x7fa598157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa59815aef0 0x7fa59815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa59806eb30 con 0x7fa59815b2d0 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa59806e850 con 0x7fa59815aef0 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa5a0630640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5980719e0 con 0x7fa5980afbb0 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36550/0 (socket says 172.21.15.99:36550) 2024-09-10T11:31:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 -- 172.21.15.99:0/4065666639 learned_addr learned my addr 172.21.15.99:0/4065666639 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59e3cc640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 0x7fa598157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59dbcb640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa59815aef0 0x7fa59815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 -- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 msgr2=0x7fa598157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 0x7fa598157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 -- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa59815aef0 msgr2=0x7fa59815e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa59815aef0 0x7fa59815e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 -- 172.21.15.99:0/4065666639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59815e770 con 0x7fa59815b2d0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59e3cc640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 0x7fa598157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.324+0000 7fa59ebcd640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa59000bb10 tx=0x7fa5900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.325+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590033620 con 0x7fa59815b2d0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.325+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5900337c0 con 0x7fa59815b2d0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.325+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5981582c0 con 0x7fa59815b2d0 2024-09-10T11:31:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.325+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590033bb0 con 0x7fa59815b2d0 2024-09-10T11:31:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.325+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa598158880 con 0x7fa59815b2d0 2024-09-10T11:31:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.326+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa598071dc0 con 0x7fa59815b2d0 2024-09-10T11:31:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.326+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa59004d060 con 0x7fa59815b2d0 2024-09-10T11:31:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.326+0000 7fa596ffd640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 0x7fa58406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.327+0000 7fa59e3cc640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 0x7fa58406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.327+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fa5900ad390 con 0x7fa59815b2d0 2024-09-10T11:31:35.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.327+0000 7fa59e3cc640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 0x7fa58406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa5880045b0 tx=0x7fa588009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.331+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa590041070 con 0x7fa59815b2d0 2024-09-10T11:31:35.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.529+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa59806c240 con 0x7fa59815b2d0 2024-09-10T11:31:35.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.530+0000 7fa596ffd640 1 -- 172.21.15.99:0/4065666639 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3904 (secure 0 0 0) 0x7fa59007c040 con 0x7fa59815b2d0 2024-09-10T11:31:35.531 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:35.531 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"btime":"2024-09-10T11:31:30:592672+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/270939242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":270939242},{"type":"v1","addr":"172.21.15.196:6835","nonce":270939242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3989471203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3989471203},{"type":"v1","addr":"172.21.15.196:6834","nonce":3989471203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4815,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/3898571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3898571},{"type":"v1","addr":"172.21.15.99:6835","nonce":3898571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":5139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4155015571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":4155015571},{"type":"v1","addr":"172.21.15.99:6837","nonce":4155015571}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-09-10T11:31:35.531 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 16 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 msgr2=0x7fa58406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 0x7fa58406e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa5880045b0 tx=0x7fa588009040 comp rx=0 tx=0).stop 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 msgr2=0x7fa598156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa59000bb10 tx=0x7fa5900046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 shutdown_connections 2024-09-10T11:31:35.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58406be30 0x7fa58406e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa59815b2d0 0x7fa598156100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa59815aef0 0x7fa59815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 --2- 172.21.15.99:0/4065666639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5980afbb0 0x7fa598157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 >> 172.21.15.99:0/4065666639 conn(0x7fa59806bc10 msgr2=0x7fa598072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 shutdown_connections 2024-09-10T11:31:35.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.532+0000 7fa5a0630640 1 -- 172.21.15.99:0/4065666639 wait complete. 2024-09-10T11:31:35.544 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:31:35.544 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:31:35.545 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:31:35.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:35.545 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:31:35.545 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:31:35.545 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:31:35.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:35.545 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:31:35.545 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:31:35.546 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:31:35.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:35.546 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:31:35.546 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:31:35.546 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:31:35.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:31:35.554 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:31:35.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:31:35.555 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:31:35.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:31:35.558 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:31:35.558 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:31:35.559 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:31:35.559 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:31:35.597 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:31:35.597 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:35.598 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:31:35.598 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:35.601 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:31:35.602 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:35.602 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:31:35.602 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:35.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:31:35.674 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:31:35.674 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:31:35.685 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:31:35.685 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:31:35.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.706+0000 7f99a42ba640 1 Processor -- start 2024-09-10T11:31:35.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.706+0000 7f99a42ba640 1 -- start start 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a42ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c15eb80 0x7f999c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a42ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f999c0d5010 0x7f999c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a42ba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f999c15ff70 con 0x7f999c1626c0 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a42ba640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f999c0d1620 con 0x7f999c0d5010 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a42ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f999c162bc0 con 0x7f999c15eb80 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a2857640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f999c0d5010 0x7f999c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a2857640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f999c0d5010 0x7f999c0d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36586/0 (socket says 172.21.15.99:36586) 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.707+0000 7f99a2857640 1 -- 172.21.15.99:0/1665292992 learned_addr learned my addr 172.21.15.99:0/1665292992 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2056640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c15eb80 0x7f999c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2857640 1 -- 172.21.15.99:0/1665292992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c15eb80 msgr2=0x7f999c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2857640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c15eb80 0x7f999c0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2857640 1 -- 172.21.15.99:0/1665292992 >> v1:172.21.15.196:6789/0 conn(0x7f999c1626c0 legacy=0x7f999c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2857640 1 -- 172.21.15.99:0/1665292992 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c162ea0 con 0x7f999c0d5010 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.708+0000 7f99a2857640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c0d5010 0x7f999c0d1b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9994009e50 tx=0x7f999402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9895d9aa49b592d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a1054640 1 -- 172.21.15.99:0/1665292992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994036070 con 0x7f999c0d5010 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a1054640 1 -- 172.21.15.99:0/1665292992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9994031600 con 0x7f999c0d5010 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a1054640 1 -- 172.21.15.99:0/1665292992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99940319d0 con 0x7f999c0d5010 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 -- 172.21.15.99:0/1665292992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c0d5010 msgr2=0x7f999c0d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c0d5010 0x7f999c0d1b20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9994009e50 tx=0x7f999402a680 comp rx=0 tx=0).stop 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 -- 172.21.15.99:0/1665292992 shutdown_connections 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c0d5010 0x7f999c0d1b20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 --2- 172.21.15.99:0/1665292992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c15eb80 0x7f999c0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 -- 172.21.15.99:0/1665292992 >> 172.21.15.99:0/1665292992 conn(0x7f999c08e680 msgr2=0x7f999c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 -- 172.21.15.99:0/1665292992 shutdown_connections 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.709+0000 7f99a42ba640 1 -- 172.21.15.99:0/1665292992 wait complete. 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.710+0000 7f99a42ba640 1 Processor -- start 2024-09-10T11:31:35.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.710+0000 7f99a42ba640 1 -- start start 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c0d5010 0x7f999c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f999c15eb80 0x7f999c1f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0d2ef0 con 0x7f999c1626c0 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f999c163460 con 0x7f999c15eb80 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a42ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f999c0d3070 con 0x7f999c0d5010 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36600/0 (socket says 172.21.15.99:36600) 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 -- 172.21.15.99:0/2557258709 learned_addr learned my addr 172.21.15.99:0/2557258709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2056640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c0d5010 0x7f999c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a1855640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f999c15eb80 0x7f999c1f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 -- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c0d5010 msgr2=0x7f999c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c0d5010 0x7f999c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 -- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f999c15eb80 msgr2=0x7f999c1f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f999c15eb80 0x7f999c1f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.711+0000 7f99a2857640 1 -- 172.21.15.99:0/2557258709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c0b5730 con 0x7f999c1626c0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f99a2857640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9994034040 tx=0x7f9994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994041020 con 0x7f999c1626c0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99940077a0 con 0x7f999c1626c0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994036040 con 0x7f999c1626c0 2024-09-10T11:31:35.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c0b46b0 con 0x7f999c1626c0 2024-09-10T11:31:35.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.712+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c0b4c70 con 0x7f999c1626c0 2024-09-10T11:31:35.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.713+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c0d1b20 con 0x7f999c1626c0 2024-09-10T11:31:35.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.713+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9994044020 con 0x7f999c1626c0 2024-09-10T11:31:35.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.714+0000 7f9992ffd640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 0x7f998006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:35.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.714+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f99940ab750 con 0x7f999c1626c0 2024-09-10T11:31:35.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.714+0000 7f99a2056640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 0x7f998006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:35.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.714+0000 7f99a2056640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 0x7f998006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99980079a0 tx=0x7f9998008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:35.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.718+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f999404b030 con 0x7f999c1626c0 2024-09-10T11:31:35.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:35.897+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f999c09e040 con 0x7f999c1626c0 2024-09-10T11:31:36.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.616+0000 7f9992ffd640 1 -- 172.21.15.99:0/2557258709 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7f9994079db0 con 0x7f999c1626c0 2024-09-10T11:31:36.617 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:31:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 msgr2=0x7f998006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 0x7f998006e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f99980079a0 tx=0x7f9998008040 comp rx=0 tx=0).stop 2024-09-10T11:31:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 msgr2=0x7f999c1f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9994034040 tx=0x7f9994004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 shutdown_connections 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f998006be30 0x7f998006e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f999c1626c0 0x7f999c1f84c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f999c15eb80 0x7f999c1f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 --2- 172.21.15.99:0/2557258709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f999c0d5010 0x7f999c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 >> 172.21.15.99:0/2557258709 conn(0x7f999c08e680 msgr2=0x7f999c160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 shutdown_connections 2024-09-10T11:31:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.619+0000 7f99a42ba640 1 -- 172.21.15.99:0/2557258709 wait complete. 2024-09-10T11:31:36.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:31:36.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 Processor -- start 2024-09-10T11:31:36.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 -- start start 2024-09-10T11:31:36.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa1a81626c0 0x7fa1a80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a815ff70 con 0x7fa1a815eb80 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a80d30c0 con 0x7fa1a81626c0 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1ad18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a80d52d0 con 0x7fa1a80d4e40 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.733+0000 7fa1a659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa1a81626c0 0x7fa1a80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa1a81626c0 0x7fa1a80d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36612/0 (socket says 172.21.15.99:36612) 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 -- 172.21.15.99:0/2686932474 learned_addr learned my addr 172.21.15.99:0/2686932474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 -- 172.21.15.99:0/2686932474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 msgr2=0x7fa1a80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 --2- 172.21.15.99:0/2686932474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 -- 172.21.15.99:0/2686932474 >> v1:172.21.15.196:6789/0 conn(0x7fa1a815eb80 legacy=0x7fa1a80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 -- 172.21.15.99:0/2686932474 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a80d1620 con 0x7fa1a81626c0 2024-09-10T11:31:36.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.734+0000 7fa1a659c640 1 --2- 172.21.15.99:0/2686932474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a80d4900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0098a0 tx=0x7fa19c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3d8f924a44ed79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:36.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1a5d9b640 1 -- 172.21.15.99:0/2686932474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c037070 con 0x7fa1a81626c0 2024-09-10T11:31:36.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1a5d9b640 1 -- 172.21.15.99:0/2686932474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c032440 con 0x7fa1a81626c0 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1a5d9b640 1 -- 172.21.15.99:0/2686932474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c032810 con 0x7fa1a81626c0 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 -- 172.21.15.99:0/2686932474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 msgr2=0x7fa1a80d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/2686932474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a80d4900 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0098a0 tx=0x7fa19c02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 -- 172.21.15.99:0/2686932474 shutdown_connections 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/2686932474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a80d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/2686932474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a80d4900 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 -- 172.21.15.99:0/2686932474 >> 172.21.15.99:0/2686932474 conn(0x7fa1a808e680 msgr2=0x7fa1a809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 -- 172.21.15.99:0/2686932474 shutdown_connections 2024-09-10T11:31:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.735+0000 7fa1ad18d640 1 -- 172.21.15.99:0/2686932474 wait complete. 2024-09-10T11:31:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 Processor -- start 2024-09-10T11:31:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 -- start start 2024-09-10T11:31:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a8162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a815eb80 0x7fa1a80b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8163180 con 0x7fa1a81626c0 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8162ea0 con 0x7fa1a815eb80 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1ad18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8163460 con 0x7fa1a80d4e40 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.737+0000 7fa1a759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36622/0 (socket says 172.21.15.99:36622) 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 -- 172.21.15.99:0/476434493 learned_addr learned my addr 172.21.15.99:0/476434493 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a659c640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a815eb80 0x7fa1a80b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a6d9d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a8162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 -- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 msgr2=0x7fa1a8162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a8162030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 -- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a815eb80 msgr2=0x7fa1a80b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a815eb80 0x7fa1a80b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 -- 172.21.15.99:0/476434493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8160f90 con 0x7fa1a81626c0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1a759e640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b970 tx=0x7fa1a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0019070 con 0x7fa1a81626c0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a000cb60 con 0x7fa1a81626c0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a8161270 con 0x7fa1a81626c0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0014440 con 0x7fa1a81626c0 2024-09-10T11:31:36.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.738+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a8161830 con 0x7fa1a81626c0 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.739+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a80d4900 con 0x7fa1a81626c0 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.740+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1a001c070 con 0x7fa1a81626c0 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.740+0000 7fa18f7fe640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 0x7fa19406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.740+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fa1a008a860 con 0x7fa1a81626c0 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.740+0000 7fa1a6d9d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 0x7fa19406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:36.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.741+0000 7fa1a6d9d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 0x7fa19406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa198006fd0 tx=0x7fa198008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:36.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.744+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1a0059570 con 0x7fa1a81626c0 2024-09-10T11:31:36.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.972+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa1a80b3de0 con 0x7fa1a81626c0 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.973+0000 7fa18f7fe640 1 -- 172.21.15.99:0/476434493 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fa1a0058e70 con 0x7fa1a81626c0 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:31:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:31:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:31:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:31:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:31:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:31:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:31:36.988 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:31:36.989 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:31:36.990 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:31:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:36.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:36.991 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:36.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 msgr2=0x7fa19406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 0x7fa19406e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa198006fd0 tx=0x7fa198008040 comp rx=0 tx=0).stop 2024-09-10T11:31:36.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 msgr2=0x7fa1a81605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000b970 tx=0x7fa1a000be30 comp rx=0 tx=0).stop 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 shutdown_connections 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa19406be30 0x7fa19406e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1a81626c0 0x7fa1a81605d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a815eb80 0x7fa1a80b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 --2- 172.21.15.99:0/476434493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a80d4e40 0x7fa1a8162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 >> 172.21.15.99:0/476434493 conn(0x7fa1a808e680 msgr2=0x7fa1a80b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 shutdown_connections 2024-09-10T11:31:36.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:36.975+0000 7fa1ad18d640 1 -- 172.21.15.99:0/476434493 wait complete. 2024-09-10T11:31:36.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:31:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.090+0000 7f46f94c9640 1 Processor -- start 2024-09-10T11:31:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.090+0000 7f46f94c9640 1 -- start start 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f94c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46f40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f94c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46f40b3160 0x7f46f406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f94c9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f40b3620 con 0x7f46f40b3960 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f94c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f46f406d960 con 0x7f46f40b3160 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f94c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46f406dc40 con 0x7f46f4163580 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46ea7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f46f40b3960 0x7f46f40d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40576/0 (socket says 172.21.15.99:40576) 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46ea7fc640 1 -- 172.21.15.99:0/3341549839 learned_addr learned my addr 172.21.15.99:0/3341549839 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f37fe640 1 --2- 172.21.15.99:0/3341549839 >> v2:172.21.15.196:3300/0 conn(0x7f46f40b3160 0x7f46f406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46f40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.091+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965118813 0 0) 0x7f46f40b3620 con 0x7f46f40b3960 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e0003520 con 0x7f46f40b3960 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 773781427 0 0) 0x7f46e0003520 con 0x7f46f40b3960 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 msgr2=0x7f46f40b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46f40b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 >> v2:172.21.15.196:3300/0 conn(0x7f46f40b3160 msgr2=0x7f46f406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 --2- 172.21.15.99:0/3341549839 >> v2:172.21.15.196:3300/0 conn(0x7f46f40b3160 0x7f46f406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f406df20 con 0x7f46f40b3960 2024-09-10T11:31:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f46dc001670 con 0x7f46f40b3960 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 >> v1:172.21.15.99:6789/0 conn(0x7f46f40b3960 legacy=0x7f46f40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46e00039b0 0x7f46e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e0004d40 0x7f46e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406d960 con 0x7f46e00039b0 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46f406dc40 con 0x7f46e0004d40 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46e000ba40 con 0x7f46f4163580 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46e00039b0 0x7f46e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f37fe640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e0004d40 0x7f46e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46ea7fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 -- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 msgr2=0x7f46e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 -- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e0004d40 msgr2=0x7f46e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e0004d40 0x7f46e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f2ffd640 1 -- 172.21.15.99:0/3341549839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0003520 con 0x7f46e00039b0 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46f37fe640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e0004d40 0x7f46e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.092+0000 7f46ea7fc640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f4163580 0x7f46e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46e00039b0 0x7f46e0003dd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009ac0 tx=0x7f46e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358d7693de92bd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4036070 con 0x7f46e00039b0 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4031400 con 0x7f46e00039b0 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f27fc640 1 -- 172.21.15.99:0/3341549839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4009690 con 0x7f46e00039b0 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f94c9640 1 -- 172.21.15.99:0/3341549839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46e00039b0 msgr2=0x7f46e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f94c9640 1 --2- 172.21.15.99:0/3341549839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46e00039b0 0x7f46e0003dd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009ac0 tx=0x7f46e0005830 comp rx=0 tx=0).stop 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.093+0000 7f46f37fe640 1 -- 172.21.15.99:0/3341549839 reap_dead start 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.094+0000 7f46f94c9640 1 -- 172.21.15.99:0/3341549839 shutdown_connections 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.094+0000 7f46f94c9640 1 -- 172.21.15.99:0/3341549839 >> 172.21.15.99:0/3341549839 conn(0x7f46f40b4570 msgr2=0x7f46f40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.094+0000 7f46f94c9640 1 -- 172.21.15.99:0/3341549839 shutdown_connections 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.094+0000 7f46f94c9640 1 -- 172.21.15.99:0/3341549839 wait complete. 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.095+0000 7f46f94c9640 1 Processor -- start 2024-09-10T11:31:37.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.095+0000 7f46f94c9640 1 -- start start 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e00039b0 0x7f46f415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f40b20d0 0x7f46f415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40b5210 con 0x7f46f40b69f0 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40b5090 con 0x7f46e00039b0 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40b5390 con 0x7f46f40b20d0 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36632/0 (socket says 172.21.15.99:36632) 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 -- 172.21.15.99:0/2994186149 learned_addr learned my addr 172.21.15.99:0/2994186149 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:37.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46ea7fc640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f40b20d0 0x7f46f415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e00039b0 0x7f46f415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 -- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f40b20d0 msgr2=0x7f46f415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f40b20d0 0x7f46f415b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 -- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e00039b0 msgr2=0x7f46f415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e00039b0 0x7f46f415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 -- 172.21.15.99:0/2994186149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40cf650 con 0x7f46f40b69f0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f37fe640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f46f40c9aa0 tx=0x7f46ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01a020 con 0x7f46f40b69f0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec011c30 con 0x7f46f40b69f0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f40b2660 con 0x7f46f40b69f0 2024-09-10T11:31:37.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec016470 con 0x7f46f40b69f0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.096+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40b0700 con 0x7f46f40b69f0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.097+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f40761a0 con 0x7f46f40b69f0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.098+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46ec005d10 con 0x7f46f40b69f0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.098+0000 7f46ebfff640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 0x7f46d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.098+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f46ec08b130 con 0x7f46f40b69f0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.098+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 0x7f46d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.099+0000 7f46f2ffd640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 0x7f46d406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46e4033000 tx=0x7f46e4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.103+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46ec024070 con 0x7f46f40b69f0 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.270+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46f40d1be0 con 0x7f46f40b69f0 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.270+0000 7f46ebfff640 1 -- 172.21.15.99:0/2994186149 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f46ec059ec0 con 0x7f46f40b69f0 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:31:37.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:37.272 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 msgr2=0x7f46d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 0x7f46d406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f46e4033000 tx=0x7f46e4008030 comp rx=0 tx=0).stop 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 msgr2=0x7f46f40b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f46f40c9aa0 tx=0x7f46ec0115a0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 shutdown_connections 2024-09-10T11:31:37.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46d406be30 0x7f46d406e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46f40b69f0 0x7f46f40b6e80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46f40b20d0 0x7f46f415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 --2- 172.21.15.99:0/2994186149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46e00039b0 0x7f46f415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 >> 172.21.15.99:0/2994186149 conn(0x7f46f40b4570 msgr2=0x7f46f40cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 shutdown_connections 2024-09-10T11:31:37.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.272+0000 7f46f94c9640 1 -- 172.21.15.99:0/2994186149 wait complete. 2024-09-10T11:31:37.284 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:31:37.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:31:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.385+0000 7fb536111640 1 Processor -- start 2024-09-10T11:31:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.385+0000 7fb536111640 1 -- start start 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb536111640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb536111640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb536111640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb530152e60 con 0x7fb53015c060 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb536111640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb53006ef50 con 0x7fb53006e5d0 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb536111640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5300d15e0 con 0x7fb5300d1200 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb526ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.386+0000 7fb52f7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:37930/0 (socket says 172.21.15.99:37930) 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb52ffff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb53015c060 0x7fb53015c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37706/0 (socket says 172.21.15.99:37706) 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb52f7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:33026/0 (socket says 172.21.15.99:33026) 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 -- 172.21.15.99:0/4193489739 learned_addr learned my addr 172.21.15.99:0/4193489739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 -- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 msgr2=0x7fb530073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 -- 172.21.15.99:0/4193489739 >> v1:172.21.15.196:6789/0 conn(0x7fb53015c060 legacy=0x7fb53015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb52effd640 1 -- 172.21.15.99:0/4193489739 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3351467157 0 0) 0x7fb530152e60 con 0x7fb53015c060 2024-09-10T11:31:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb526ffd640 1 -- 172.21.15.99:0/4193489739 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530072680 con 0x7fb53006e5d0 2024-09-10T11:31:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.387+0000 7fb52f7fe640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb526ffd640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53006ea10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb518009870 tx=0x7fb51802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb09e9d2c387af1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb52effd640 1 -- 172.21.15.99:0/4193489739 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518037070 con 0x7fb53006e5d0 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb52effd640 1 -- 172.21.15.99:0/4193489739 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb518032440 con 0x7fb53006e5d0 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb52effd640 1 -- 172.21.15.99:0/4193489739 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518032810 con 0x7fb53006e5d0 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 -- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 msgr2=0x7fb53006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53006ea10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb518009870 tx=0x7fb51802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 -- 172.21.15.99:0/4193489739 shutdown_connections 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 --2- 172.21.15.99:0/4193489739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 -- 172.21.15.99:0/4193489739 >> 172.21.15.99:0/4193489739 conn(0x7fb53006bc10 msgr2=0x7fb53006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:37.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 -- 172.21.15.99:0/4193489739 shutdown_connections 2024-09-10T11:31:37.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.388+0000 7fb536111640 1 -- 172.21.15.99:0/4193489739 wait complete. 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 Processor -- start 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 -- start start 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.390+0000 7fb536111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb530072f20 con 0x7fb53015c060 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb536111640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb530072c40 con 0x7fb53006e5d0 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb536111640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb53015cd90 con 0x7fb5300d1200 2024-09-10T11:31:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36642/0 (socket says 172.21.15.99:36642) 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 -- 172.21.15.99:0/722068851 learned_addr learned my addr 172.21.15.99:0/722068851 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb526ffd640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52f7fe640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 -- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 msgr2=0x7fb53015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 -- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 msgr2=0x7fb53014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 -- 172.21.15.99:0/722068851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53014fd80 con 0x7fb53015c060 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52ffff640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c7a0 tx=0x7fb52800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528018070 con 0x7fb53015c060 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52800a640 con 0x7fb53015c060 2024-09-10T11:31:37.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb530150020 con 0x7fb53015c060 2024-09-10T11:31:37.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52800aab0 con 0x7fb53015c060 2024-09-10T11:31:37.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.391+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5301505e0 con 0x7fb53015c060 2024-09-10T11:31:37.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.392+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53006ea10 con 0x7fb53015c060 2024-09-10T11:31:37.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.393+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb52801b070 con 0x7fb53015c060 2024-09-10T11:31:37.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.393+0000 7fb52cff9640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 0x7fb51006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.393+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fb52808a0b0 con 0x7fb53015c060 2024-09-10T11:31:37.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.393+0000 7fb52f7fe640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 0x7fb51006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.394+0000 7fb52f7fe640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 0x7fb51006e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb53014f300 tx=0x7fb520009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.398+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb528058760 con 0x7fb53015c060 2024-09-10T11:31:37.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.565+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb53015ec30 con 0x7fb53015c060 2024-09-10T11:31:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.632+0000 7fb52cff9640 1 -- 172.21.15.99:0/722068851 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7fb52805ca60 con 0x7fb53015c060 2024-09-10T11:31:37.633 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 msgr2=0x7fb51006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 0x7fb51006e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb53014f300 tx=0x7fb520009190 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 msgr2=0x7fb53015d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb52800c7a0 tx=0x7fb52800cc60 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 shutdown_connections 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51006be30 0x7fb51006e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53015c060 0x7fb53015d560 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 --2- 172.21.15.99:0/722068851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53006e5d0 0x7fb53014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 >> 172.21.15.99:0/722068851 conn(0x7fb53006bc10 msgr2=0x7fb530155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:37.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 shutdown_connections 2024-09-10T11:31:37.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.634+0000 7fb536111640 1 -- 172.21.15.99:0/722068851 wait complete. 2024-09-10T11:31:37.648 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:31:37.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.753+0000 7f3cfefa6640 1 Processor -- start 2024-09-10T11:31:37.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.753+0000 7f3cfefa6640 1 -- start start 2024-09-10T11:31:37.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 0x7f3cf80cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3cf815cc30 0x7f3cf80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 0x7f3cf80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf80d1540 con 0x7f3cf815cc30 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf80d1770 con 0x7f3cf80cf7f0 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cfefa6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf80cd810 con 0x7f3cf80cf410 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.754+0000 7f3cf7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3cf815cc30 0x7f3cf80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3cf815cc30 0x7f3cf80d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36646/0 (socket says 172.21.15.99:36646) 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 -- 172.21.15.99:0/703082106 learned_addr learned my addr 172.21.15.99:0/703082106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cfd543640 1 --2- 172.21.15.99:0/703082106 >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 0x7f3cf80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cfcd42640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 0x7f3cf80cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 -- 172.21.15.99:0/703082106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 msgr2=0x7f3cf80cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 0x7f3cf80cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 -- 172.21.15.99:0/703082106 >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 msgr2=0x7f3cf80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:37.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 --2- 172.21.15.99:0/703082106 >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 0x7f3cf80ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 -- 172.21.15.99:0/703082106 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf80cdaf0 con 0x7f3cf815cc30 2024-09-10T11:31:37.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cfd543640 1 --2- 172.21.15.99:0/703082106 >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 0x7f3cf80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:37.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cfcd42640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 0x7f3cf80cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:37.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.755+0000 7f3cf7fff640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf815cc30 0x7f3cf80d1000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80cd210 tx=0x7f3cec02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173932fe7ba1528a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cf77fe640 1 -- 172.21.15.99:0/703082106 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec037070 con 0x7f3cf815cc30 2024-09-10T11:31:37.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cf77fe640 1 -- 172.21.15.99:0/703082106 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cec02ae00 con 0x7f3cf815cc30 2024-09-10T11:31:37.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cf77fe640 1 -- 172.21.15.99:0/703082106 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cec032660 con 0x7f3cf815cc30 2024-09-10T11:31:37.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 -- 172.21.15.99:0/703082106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf815cc30 msgr2=0x7f3cf80d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf815cc30 0x7f3cf80d1000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80cd210 tx=0x7f3cec02a0b0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 -- 172.21.15.99:0/703082106 shutdown_connections 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/703082106 >> v2:172.21.15.196:3300/0 conn(0x7f3cf80cf7f0 0x7f3cf80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf815cc30 0x7f3cf80d1000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/703082106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf80cf410 0x7f3cf80cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 -- 172.21.15.99:0/703082106 >> 172.21.15.99:0/703082106 conn(0x7f3cf80b4570 msgr2=0x7f3cf80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 -- 172.21.15.99:0/703082106 shutdown_connections 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.756+0000 7f3cfefa6640 1 -- 172.21.15.99:0/703082106 wait complete. 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.757+0000 7f3cfefa6640 1 Processor -- start 2024-09-10T11:31:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.757+0000 7f3cfefa6640 1 -- start start 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cf80cf7f0 0x7f3cf8147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf815cc30 0x7f3cf80c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf80cfda0 con 0x7f3cf80cf410 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf80cfc20 con 0x7f3cf80cf7f0 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfefa6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf80ce830 con 0x7f3cf815cc30 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfcd42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfcd42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36658/0 (socket says 172.21.15.99:36658) 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.758+0000 7f3cfcd42640 1 -- 172.21.15.99:0/2076351555 learned_addr learned my addr 172.21.15.99:0/2076351555 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 -- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf815cc30 msgr2=0x7f3cf80c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf815cc30 0x7f3cf80c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 -- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cf80cf7f0 msgr2=0x7f3cf8147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cf80cf7f0 0x7f3cf8147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 -- 172.21.15.99:0/2076351555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf80c8950 con 0x7f3cf80cf410 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfcd42640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e840 tx=0x7f3ce800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800f040 con 0x7f3cf80cf410 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce8004030 con 0x7f3cf80cf410 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8004400 con 0x7f3cf80cf410 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf80c8bf0 con 0x7f3cf80cf410 2024-09-10T11:31:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.759+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf80c91b0 con 0x7f3cf80cf410 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.760+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf80ce2f0 con 0x7f3cf80cf410 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.761+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ce800c6d0 con 0x7f3cf80cf410 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.761+0000 7f3cf57fa640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 0x7f3ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.761+0000 7f3cf7fff640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 0x7f3ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.761+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f3ce808a870 con 0x7f3cf80cf410 2024-09-10T11:31:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.762+0000 7f3cf7fff640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 0x7f3ce406e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cec034000 tx=0x7f3cec02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.766+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3ce8017020 con 0x7f3cf80cf410 2024-09-10T11:31:37.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:37.931+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3cf806fcb0 con 0x7f3cf80cf410 2024-09-10T11:31:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.636+0000 7f3cf57fa640 1 -- 172.21.15.99:0/2076351555 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f3ce8059490 con 0x7f3cf80cf410 2024-09-10T11:31:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:31:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.643+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 msgr2=0x7f3ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.643+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 0x7f3ce406e2f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3cec034000 tx=0x7f3cec02c000 comp rx=0 tx=0).stop 2024-09-10T11:31:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.643+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 msgr2=0x7f3cf80c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.643+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3ce800e840 tx=0x7f3ce800ed00 comp rx=0 tx=0).stop 2024-09-10T11:31:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 shutdown_connections 2024-09-10T11:31:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3ce406be30 0x7f3ce406e2f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cf815cc30 0x7f3cf80c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cf80cf7f0 0x7f3cf8147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 --2- 172.21.15.99:0/2076351555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3cf80cf410 0x7f3cf80c9cb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.648+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 >> 172.21.15.99:0/2076351555 conn(0x7f3cf80b4570 msgr2=0x7f3cf80b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.649+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 shutdown_connections 2024-09-10T11:31:38.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.649+0000 7f3cfefa6640 1 -- 172.21.15.99:0/2076351555 wait complete. 2024-09-10T11:31:38.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:31:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.768+0000 7f7eb1e5e640 1 Processor -- start 2024-09-10T11:31:38.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.768+0000 7f7eb1e5e640 1 -- start start 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eb1e5e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7eac15f1e0 0x7f7eac0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eb1e5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac0b0090 0x7f7eac0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eb1e5e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eac0b40e0 con 0x7f7eac15ee00 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eb1e5e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f7eac0b5ab0 con 0x7f7eac15f1e0 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eb1e5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac073c50 con 0x7f7eac0b0090 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eab7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7eac15ee00 0x7f7eac0b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40586/0 (socket says 172.21.15.99:40586) 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eaaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7eac15f1e0 0x7f7eac0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eab7fe640 1 -- 172.21.15.99:0/90204580 learned_addr learned my addr 172.21.15.99:0/90204580 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eaaffd640 1 -- 172.21.15.99:0/90204580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac0b0090 msgr2=0x7f7eac0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:38.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eaaffd640 1 --2- 172.21.15.99:0/90204580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac0b0090 0x7f7eac0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eaaffd640 1 -- 172.21.15.99:0/90204580 >> v1:172.21.15.99:6789/0 conn(0x7f7eac15ee00 legacy=0x7f7eac0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.769+0000 7f7eaaffd640 1 -- 172.21.15.99:0/90204580 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac0b4380 con 0x7f7eac15f1e0 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.770+0000 7f7eaaffd640 1 --2- 172.21.15.99:0/90204580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15f1e0 0x7f7eac0b5f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00097a0 tx=0x7f7ea002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afde07c5744913 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.770+0000 7f7eaa7fc640 1 -- 172.21.15.99:0/90204580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0037070 con 0x7f7eac15f1e0 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.770+0000 7f7eaa7fc640 1 -- 172.21.15.99:0/90204580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea0032440 con 0x7f7eac15f1e0 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.770+0000 7f7eaa7fc640 1 -- 172.21.15.99:0/90204580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0032810 con 0x7f7eac15f1e0 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/90204580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15f1e0 msgr2=0x7f7eac0b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/90204580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15f1e0 0x7f7eac0b5f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00097a0 tx=0x7f7ea002a180 comp rx=0 tx=0).stop 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/90204580 shutdown_connections 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/90204580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac0b0090 0x7f7eac0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/90204580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15f1e0 0x7f7eac0b5f90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/90204580 >> 172.21.15.99:0/90204580 conn(0x7f7eac0730c0 msgr2=0x7f7eac0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/90204580 shutdown_connections 2024-09-10T11:31:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.771+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/90204580 wait complete. 2024-09-10T11:31:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.772+0000 7f7eb1e5e640 1 Processor -- start 2024-09-10T11:31:38.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.772+0000 7f7eb1e5e640 1 -- start start 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 0x7f7eac156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac15f1e0 0x7f7eac1eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0b21b0 con 0x7f7eac15ee00 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac0afd60 con 0x7f7eac0b0090 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eb1e5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac0b0600 con 0x7f7eac15f1e0 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36678/0 (socket says 172.21.15.99:36678) 2024-09-10T11:31:38.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 -- 172.21.15.99:0/2489265020 learned_addr learned my addr 172.21.15.99:0/2489265020 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eab7fe640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 0x7f7eac156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 -- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac15f1e0 msgr2=0x7f7eac1eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eabfff640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac15f1e0 0x7f7eac1eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac15f1e0 0x7f7eac1eebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 -- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 msgr2=0x7f7eac156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 0x7f7eac156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eaaffd640 1 -- 172.21.15.99:0/2489265020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac156630 con 0x7f7eac15ee00 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.773+0000 7f7eab7fe640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 0x7f7eac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7eaaffd640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0035040 tx=0x7f7ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0040020 con 0x7f7eac15ee00 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea0038040 con 0x7f7eac15ee00 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac156910 con 0x7f7eac15ee00 2024-09-10T11:31:38.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0037070 con 0x7f7eac15ee00 2024-09-10T11:31:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.774+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac156ed0 con 0x7f7eac15ee00 2024-09-10T11:31:38.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.775+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac0b5f90 con 0x7f7eac15ee00 2024-09-10T11:31:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.775+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ea0043020 con 0x7f7eac15ee00 2024-09-10T11:31:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.776+0000 7f7e93fff640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 0x7f7e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.776+0000 7f7eab7fe640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 0x7f7e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.776+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7f7ea00ab550 con 0x7f7eac15ee00 2024-09-10T11:31:38.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.776+0000 7f7eab7fe640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 0x7f7e9806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7eac159350 tx=0x7f7e9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:38.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.780+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ea004a030 con 0x7f7eac15ee00 2024-09-10T11:31:38.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:38.981+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7eac15e1a0 con 0x7f7eac15ee00 2024-09-10T11:31:39.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.650+0000 7f7e93fff640 1 -- 172.21.15.99:0/2489265020 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) ==== 172+0+0 (secure 0 0 0) 0x7f7ea0032e20 con 0x7f7eac15ee00 2024-09-10T11:31:39.652 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T11:31:39.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 msgr2=0x7f7e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 0x7f7e9806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7eac159350 tx=0x7f7e9c009040 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 msgr2=0x7f7eac154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0035040 tx=0x7f7ea00046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 shutdown_connections 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7e9806be30 0x7f7e9806e2f0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7eac159350 tx=0x7f7e9c009040 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7eac15f1e0 0x7f7eac1eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7eac15ee00 0x7f7eac154710 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0035040 tx=0x7f7ea00046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 --2- 172.21.15.99:0/2489265020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7eac0b0090 0x7f7eac156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.656+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 >> 172.21.15.99:0/2489265020 conn(0x7f7eac0730c0 msgr2=0x7f7eac0af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:39.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.657+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 shutdown_connections 2024-09-10T11:31:39.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.657+0000 7f7eb1e5e640 1 -- 172.21.15.99:0/2489265020 wait complete. 2024-09-10T11:31:39.674 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.772+0000 7fbaed92c640 1 Processor -- start 2024-09-10T11:31:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.772+0000 7fbaed92c640 1 -- start start 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbaed92c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815f020 0x7fbae806e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbaed92c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbae806f7e0 0x7fbae806fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbaed92c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae806d8d0 con 0x7fbae80b2010 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbaed92c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbae815f400 con 0x7fbae806f7e0 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbaed92c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbae806e200 con 0x7fbae815f020 2024-09-10T11:31:39.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbade7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbae806f7e0 0x7fbae806fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbade7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbae806f7e0 0x7fbae806fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36686/0 (socket says 172.21.15.99:36686) 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbae77fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbae80b2010 0x7fbae80af8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40592/0 (socket says 172.21.15.99:40592) 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbade7fc640 1 -- 172.21.15.99:0/2996628521 learned_addr learned my addr 172.21.15.99:0/2996628521 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.773+0000 7fbae6ffd640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815f020 0x7fbae806e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbade7fc640 1 -- 172.21.15.99:0/2996628521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815f020 msgr2=0x7fbae806e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbade7fc640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815f020 0x7fbae806e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbade7fc640 1 -- 172.21.15.99:0/2996628521 >> v1:172.21.15.99:6789/0 conn(0x7fbae80b2010 legacy=0x7fbae80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbade7fc640 1 -- 172.21.15.99:0/2996628521 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae806dbb0 con 0x7fbae806f7e0 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbade7fc640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae806fea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009870 tx=0x7fbad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4337b7a97117dd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbae67fc640 1 -- 172.21.15.99:0/2996628521 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0037070 con 0x7fbae806f7e0 2024-09-10T11:31:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbae67fc640 1 -- 172.21.15.99:0/2996628521 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad0032440 con 0x7fbae806f7e0 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.774+0000 7fbae67fc640 1 -- 172.21.15.99:0/2996628521 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0032810 con 0x7fbae806f7e0 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 -- 172.21.15.99:0/2996628521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 msgr2=0x7fbae806fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae806fea0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009870 tx=0x7fbad002a270 comp rx=0 tx=0).stop 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 -- 172.21.15.99:0/2996628521 shutdown_connections 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae806fea0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 --2- 172.21.15.99:0/2996628521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815f020 0x7fbae806e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 -- 172.21.15.99:0/2996628521 >> 172.21.15.99:0/2996628521 conn(0x7fbae80b45c0 msgr2=0x7fbae80b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 -- 172.21.15.99:0/2996628521 shutdown_connections 2024-09-10T11:31:39.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.775+0000 7fbaed92c640 1 -- 172.21.15.99:0/2996628521 wait complete. 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.776+0000 7fbaed92c640 1 Processor -- start 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.776+0000 7fbaed92c640 1 -- start start 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 0x7fbae80c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 0x7fbae81f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbae80d2150 con 0x7fbae806f7e0 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbae80d1fd0 con 0x7fbae815f020 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbaed92c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbae806ebd0 con 0x7fbae80b2010 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbae6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbae6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36692/0 (socket says 172.21.15.99:36692) 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.777+0000 7fbae6ffd640 1 -- 172.21.15.99:0/1664194736 learned_addr learned my addr 172.21.15.99:0/1664194736 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae77fe640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 0x7fbae81f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 -- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 msgr2=0x7fbae80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbade7fc640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 0x7fbae80c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 0x7fbae80c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 -- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 msgr2=0x7fbae81f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 0x7fbae81f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 -- 172.21.15.99:0/1664194736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae81f4c90 con 0x7fbae806f7e0 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae77fe640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 0x7fbae81f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbade7fc640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 0x7fbae80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:39.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae6ffd640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbad800e870 tx=0x7fbad800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8017070 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad800c940 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae80c8660 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad800cd30 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.778+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae81f4f50 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.779+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae80761a0 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.779+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbad801a070 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.780+0000 7fbae4ff9640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 0x7fbac806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.780+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fbad808ac80 con 0x7fbae806f7e0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.780+0000 7fbade7fc640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 0x7fbac806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:39.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.780+0000 7fbade7fc640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 0x7fbac806e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbad0034000 tx=0x7fbad002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:39.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.784+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbad80596c0 con 0x7fbae806f7e0 2024-09-10T11:31:39.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.950+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbae80d2830 con 0x7fbae806f7e0 2024-09-10T11:31:39.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.951+0000 7fbae4ff9640 1 -- 172.21.15.99:0/1664194736 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16682 (secure 0 0 0) 0x7fbad8058fc0 con 0x7fbae806f7e0 2024-09-10T11:31:39.952 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:39.952 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":44,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:38.981893+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:37.584024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:37.949922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:39.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 msgr2=0x7fbac806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 0x7fbac806e320 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbad0034000 tx=0x7fbad002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 msgr2=0x7fbae81f5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbad800e870 tx=0x7fbad800ed30 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 shutdown_connections 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbac806be60 0x7fbac806e320 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815f020 0x7fbae81f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80b2010 0x7fbae80c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 --2- 172.21.15.99:0/1664194736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbae806f7e0 0x7fbae81f5750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 >> 172.21.15.99:0/1664194736 conn(0x7fbae80b45c0 msgr2=0x7fbae80d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 shutdown_connections 2024-09-10T11:31:39.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:39.953+0000 7fbaed92c640 1 -- 172.21.15.99:0/1664194736 wait complete. 2024-09-10T11:31:39.965 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.063+0000 7fb05e0a6640 1 Processor -- start 2024-09-10T11:31:40.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.063+0000 7fb05e0a6640 1 -- start start 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb05e0a6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb058073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb05e0a6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb05815c060 0x7fb05815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb05e0a6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb058152e60 con 0x7fb0580d1200 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb05e0a6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb0580d15e0 con 0x7fb05815c060 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb05e0a6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb058072680 con 0x7fb05806ece0 2024-09-10T11:31:40.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb057fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb05815c060 0x7fb05815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb057fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb05815c060 0x7fb05815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36704/0 (socket says 172.21.15.99:36704) 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb0577fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb0580d1200 0x7fb05806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40602/0 (socket says 172.21.15.99:40602) 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.064+0000 7fb057fff640 1 -- 172.21.15.99:0/2320910502 learned_addr learned my addr 172.21.15.99:0/2320910502 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb056ffd640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb058073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb057fff640 1 -- 172.21.15.99:0/2320910502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 msgr2=0x7fb058073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb057fff640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb058073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb057fff640 1 -- 172.21.15.99:0/2320910502 >> v1:172.21.15.99:6789/0 conn(0x7fb0580d1200 legacy=0x7fb05806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb057fff640 1 -- 172.21.15.99:0/2320910502 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb058072960 con 0x7fb05815c060 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb0567fc640 1 -- 172.21.15.99:0/2320910502 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560965492 0 0) 0x7fb058152e60 con 0x7fb0580d1200 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb057fff640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb05815c4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb050009870 tx=0x7fb05002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6207573f17c62202 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb0567fc640 1 -- 172.21.15.99:0/2320910502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050037070 con 0x7fb05815c060 2024-09-10T11:31:40.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb0567fc640 1 -- 172.21.15.99:0/2320910502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb050032440 con 0x7fb05815c060 2024-09-10T11:31:40.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.065+0000 7fb0567fc640 1 -- 172.21.15.99:0/2320910502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050032810 con 0x7fb05815c060 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2320910502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 msgr2=0x7fb05815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb05815c4a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb050009870 tx=0x7fb05002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2320910502 shutdown_connections 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb05815c4a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2320910502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb058073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2320910502 >> 172.21.15.99:0/2320910502 conn(0x7fb05806bc10 msgr2=0x7fb05806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:40.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2320910502 shutdown_connections 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.066+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2320910502 wait complete. 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.067+0000 7fb05e0a6640 1 Processor -- start 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.067+0000 7fb05e0a6640 1 -- start start 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb05806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0580d1200 0x7fb058156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb05815cd90 con 0x7fb05815c060 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb058072f20 con 0x7fb0580d1200 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb05e0a6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb05815c9e0 con 0x7fb05806ece0 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36716/0 (socket says 172.21.15.99:36716) 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 -- 172.21.15.99:0/2873092100 learned_addr learned my addr 172.21.15.99:0/2873092100 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:40.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 -- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 msgr2=0x7fb05806d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb05806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 -- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0580d1200 msgr2=0x7fb058156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0580d1200 0x7fb058156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.068+0000 7fb057fff640 1 -- 172.21.15.99:0/2873092100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb058157930 con 0x7fb05815c060 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb057fff640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a200 tx=0x7fb0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050037040 con 0x7fb05815c060 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb050042070 con 0x7fb05815c060 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb058157c10 con 0x7fb05815c060 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050030bd0 con 0x7fb05815c060 2024-09-10T11:31:40.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.069+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581581d0 con 0x7fb05815c060 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.070+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb058073440 con 0x7fb05815c060 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.070+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb05002ad10 con 0x7fb05815c060 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.071+0000 7fb03ffff640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 0x7fb04406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.071+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fb0500ab760 con 0x7fb05815c060 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.071+0000 7fb0577fe640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 0x7fb04406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.071+0000 7fb0577fe640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 0x7fb04406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb048007fd0 tx=0x7fb048007ce0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.075+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb05007a110 con 0x7fb05815c060 2024-09-10T11:31:40.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.241+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0580b4480 con 0x7fb05815c060 2024-09-10T11:31:40.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.242+0000 7fb03ffff640 1 -- 172.21.15.99:0/2873092100 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16682 (secure 0 0 0) 0x7fb050079a10 con 0x7fb05815c060 2024-09-10T11:31:40.243 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:40.244 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":44,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:38.981893+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:37.584024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:37.949922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:40.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 msgr2=0x7fb04406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 0x7fb04406e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb048007fd0 tx=0x7fb048007ce0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 msgr2=0x7fb058157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a200 tx=0x7fb0500046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 shutdown_connections 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb04406be30 0x7fb04406e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb05815c060 0x7fb058157290 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0580d1200 0x7fb058156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 --2- 172.21.15.99:0/2873092100 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb05806ece0 0x7fb05806d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.244+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 >> 172.21.15.99:0/2873092100 conn(0x7fb05806bc10 msgr2=0x7fb05815fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.245+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 shutdown_connections 2024-09-10T11:31:40.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.245+0000 7fb05e0a6640 1 -- 172.21.15.99:0/2873092100 wait complete. 2024-09-10T11:31:40.257 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:31:40.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.358+0000 7faf7ab54640 1 Processor -- start 2024-09-10T11:31:40.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.358+0000 7faf7ab54640 1 -- start start 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf7ab54640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf7415c7b0 0x7faf740d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf7ab54640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf740d0fb0 0x7faf7406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf7ab54640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740ceb10 con 0x7faf7406ee30 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf7ab54640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7faf740d1420 con 0x7faf740d0fb0 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf7ab54640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faf740b3840 con 0x7faf7415c7b0 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf740d0fb0 0x7faf7406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf740d0fb0 0x7faf7406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36726/0 (socket says 172.21.15.99:36726) 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 -- 172.21.15.99:0/3178907137 learned_addr learned my addr 172.21.15.99:0/3178907137 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf788f0640 1 --2- 172.21.15.99:0/3178907137 >> v2:172.21.15.196:3300/0 conn(0x7faf7415c7b0 0x7faf740d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 -- 172.21.15.99:0/3178907137 >> v2:172.21.15.196:3300/0 conn(0x7faf7415c7b0 msgr2=0x7faf740d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 --2- 172.21.15.99:0/3178907137 >> v2:172.21.15.196:3300/0 conn(0x7faf7415c7b0 0x7faf740d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 -- 172.21.15.99:0/3178907137 >> v1:172.21.15.196:6789/0 conn(0x7faf7406ee30 legacy=0x7faf741604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.359+0000 7faf6bfff640 1 -- 172.21.15.99:0/3178907137 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740b3a90 con 0x7faf740d0fb0 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.360+0000 7faf6bfff640 1 --2- 172.21.15.99:0/3178907137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf740d0fb0 0x7faf7406e8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faf740cd7c0 tx=0x7faf6402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76da8c633105416c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.360+0000 7faf6b7fe640 1 -- 172.21.15.99:0/3178907137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64037070 con 0x7faf740d0fb0 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.360+0000 7faf6b7fe640 1 -- 172.21.15.99:0/3178907137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf64032440 con 0x7faf740d0fb0 2024-09-10T11:31:40.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.360+0000 7faf6b7fe640 1 -- 172.21.15.99:0/3178907137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64032810 con 0x7faf740d0fb0 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 -- 172.21.15.99:0/3178907137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf740d0fb0 msgr2=0x7faf7406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 --2- 172.21.15.99:0/3178907137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf740d0fb0 0x7faf7406e8f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7faf740cd7c0 tx=0x7faf6402a230 comp rx=0 tx=0).stop 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 -- 172.21.15.99:0/3178907137 shutdown_connections 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 --2- 172.21.15.99:0/3178907137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf740d0fb0 0x7faf7406e8f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 --2- 172.21.15.99:0/3178907137 >> v2:172.21.15.196:3300/0 conn(0x7faf7415c7b0 0x7faf740d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 -- 172.21.15.99:0/3178907137 >> 172.21.15.99:0/3178907137 conn(0x7faf7406bc10 msgr2=0x7faf7406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 -- 172.21.15.99:0/3178907137 shutdown_connections 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.361+0000 7faf7ab54640 1 -- 172.21.15.99:0/3178907137 wait complete. 2024-09-10T11:31:40.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.362+0000 7faf7ab54640 1 Processor -- start 2024-09-10T11:31:40.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.362+0000 7faf7ab54640 1 -- start start 2024-09-10T11:31:40.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf7406ee30 0x7faf74154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf740d0fb0 0x7faf740b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf740b4050 con 0x7faf7415c7b0 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf7406f260 con 0x7faf7406ee30 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faf740b4330 con 0x7faf740d0fb0 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36734/0 (socket says 172.21.15.99:36734) 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 -- 172.21.15.99:0/4181888760 learned_addr learned my addr 172.21.15.99:0/4181888760 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf788f0640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf7406ee30 0x7faf74154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf6bfff640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf740d0fb0 0x7faf740b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 -- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf740d0fb0 msgr2=0x7faf740b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf740d0fb0 0x7faf740b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 -- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf7406ee30 msgr2=0x7faf74154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf7406ee30 0x7faf74154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 -- 172.21.15.99:0/4181888760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf74151b10 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf790f1640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c440 tx=0x7faf6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c011790 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf6c017070 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf74152950 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.363+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6c011b70 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.364+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf74153fe0 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.364+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf740cff90 con 0x7faf7415c7b0 2024-09-10T11:31:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.365+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf6c011d10 con 0x7faf7415c7b0 2024-09-10T11:31:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.365+0000 7faf697fa640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 0x7faf6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.365+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7faf6c08aab0 con 0x7faf7415c7b0 2024-09-10T11:31:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.365+0000 7faf788f0640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 0x7faf6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.366+0000 7faf788f0640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 0x7faf6006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf7000f360 tx=0x7faf70007710 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.370+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf740cff90 con 0x7faf7415c7b0 2024-09-10T11:31:40.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.568+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf7415cb90 con 0x7faf7415c7b0 2024-09-10T11:31:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.657+0000 7faf697fa640 1 -- 172.21.15.99:0/4181888760 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) ==== 118+0+0 (secure 0 0 0) 0x7faf6c058ea0 con 0x7faf7415c7b0 2024-09-10T11:31:40.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 msgr2=0x7faf6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 0x7faf6006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf7000f360 tx=0x7faf70007710 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 msgr2=0x7faf740b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c440 tx=0x7faf6c00c900 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 shutdown_connections 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf6006be30 0x7faf6006e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7415c7b0 0x7faf740b1070 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf740d0fb0 0x7faf740b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 --2- 172.21.15.99:0/4181888760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf7406ee30 0x7faf74154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 >> 172.21.15.99:0/4181888760 conn(0x7faf7406bc10 msgr2=0x7faf7406dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 shutdown_connections 2024-09-10T11:31:40.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.659+0000 7faf7ab54640 1 -- 172.21.15.99:0/4181888760 wait complete. 2024-09-10T11:31:40.671 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:31:40.671 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:31:40.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.772+0000 7fe6729c4640 1 Processor -- start 2024-09-10T11:31:40.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.772+0000 7fe6729c4640 1 -- start start 2024-09-10T11:31:40.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe6729c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe66c160660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe6729c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe66c160ba0 0x7fe66c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe6729c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c06f2c0 con 0x7fe66c146920 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe6729c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe66c06f5a0 con 0x7fe66c160ba0 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe6729c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0ced70 con 0x7fe66c161270 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe66bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe66c146920 0x7fe66c0718f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40614/0 (socket says 172.21.15.99:40614) 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe66bfff640 1 -- 172.21.15.99:0/3601814262 learned_addr learned my addr 172.21.15.99:0/3601814262 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe66c160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe670f61640 1 --2- 172.21.15.99:0/3601814262 >> v2:172.21.15.196:3300/0 conn(0x7fe66c160ba0 0x7fe66c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2381256429 0 0) 0x7fe66c06f2c0 con 0x7fe66c146920 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.773+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe654003520 con 0x7fe66c146920 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2231753891 0 0) 0x7fe654003520 con 0x7fe66c146920 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 msgr2=0x7fe66c160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe66c160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 >> v2:172.21.15.196:3300/0 conn(0x7fe66c160ba0 msgr2=0x7fe66c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 --2- 172.21.15.99:0/3601814262 >> v2:172.21.15.196:3300/0 conn(0x7fe66c160ba0 0x7fe66c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c0cf050 con 0x7fe66c146920 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fe65c001670 con 0x7fe66c146920 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 >> v1:172.21.15.99:6789/0 conn(0x7fe66c146920 legacy=0x7fe66c0718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6540039b0 0x7fe654003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe654004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c06f5a0 con 0x7fe654004d40 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0ced70 con 0x7fe6540039b0 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe65400ba40 con 0x7fe66c161270 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66bfff640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6540039b0 0x7fe654003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe654004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 -- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 msgr2=0x7fe654004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c161270 0x7fe654004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 -- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6540039b0 msgr2=0x7fe654003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6540039b0 0x7fe654003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe670f61640 1 -- 172.21.15.99:0/3601814262 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe654003520 con 0x7fe654004d40 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.774+0000 7fe66bfff640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6540039b0 0x7fe654003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe670f61640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe664009fc0 tx=0x7fe654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54c5eac7c886a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664009c10 con 0x7fe654004d40 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe664036070 con 0x7fe654004d40 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe66affd640 1 -- 172.21.15.99:0/3601814262 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66402d070 con 0x7fe654004d40 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe6729c4640 1 -- 172.21.15.99:0/3601814262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 msgr2=0x7fe6540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe6729c4640 1 --2- 172.21.15.99:0/3601814262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe6540051c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe664009fc0 tx=0x7fe654005830 comp rx=0 tx=0).stop 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe670f61640 1 -- 172.21.15.99:0/3601814262 reap_dead start 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe6729c4640 1 -- 172.21.15.99:0/3601814262 shutdown_connections 2024-09-10T11:31:40.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe6729c4640 1 -- 172.21.15.99:0/3601814262 >> 172.21.15.99:0/3601814262 conn(0x7fe66c0b2a80 msgr2=0x7fe66c0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:40.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.775+0000 7fe6729c4640 1 -- 172.21.15.99:0/3601814262 shutdown_connections 2024-09-10T11:31:40.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.776+0000 7fe6729c4640 1 -- 172.21.15.99:0/3601814262 wait complete. 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.777+0000 7fe6729c4640 1 Processor -- start 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.777+0000 7fe6729c4640 1 -- start start 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.777+0000 7fe6729c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.777+0000 7fe6729c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 0x7fe66c09ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.777+0000 7fe6729c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c14c2d0 0x7fe66c09bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe6729c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1607e0 con 0x7fe654004d40 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe6729c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c160660 con 0x7fe66c14be50 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe6729c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c160960 con 0x7fe66c14c2d0 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36762/0 (socket says 172.21.15.99:36762) 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 -- 172.21.15.99:0/2988317759 learned_addr learned my addr 172.21.15.99:0/2988317759 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe670f61640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c14c2d0 0x7fe66c09bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 0x7fe66c09ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 -- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c14c2d0 msgr2=0x7fe66c09bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c14c2d0 0x7fe66c09bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 -- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 msgr2=0x7fe66c09ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 0x7fe66c09ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 -- 172.21.15.99:0/2988317759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1e7080 con 0x7fe654004d40 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 0x7fe66c09ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:40.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe66bfff640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e780 tx=0x7fe65c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00ee30 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe65c018070 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c1e7360 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c00ca90 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.778+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1e7920 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.779+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c0718f0 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.780+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe65c00cc30 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.780+0000 7fe668ff9640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.780+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe65c08ac60 con 0x7fe654004d40 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.780+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:40.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.781+0000 7fe66b7fe640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe66c15fed0 tx=0x7fe660008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:40.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.784+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe65c059610 con 0x7fe654004d40 2024-09-10T11:31:40.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:40.984+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe66c0d0710 con 0x7fe654004d40 2024-09-10T11:31:41.619 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:31:41.619+0000 7fee99eab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:31:41.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.661+0000 7fe668ff9640 1 -- 172.21.15.99:0/2988317759 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7fe65c058f10 con 0x7fe654004d40 2024-09-10T11:31:41.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.663+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 msgr2=0x7fe65806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.663+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 0x7fe65806e320 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe66c15fed0 tx=0x7fe660008040 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.663+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 msgr2=0x7fe66c1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.663+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00e780 tx=0x7fe65c00ec40 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 shutdown_connections 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe65806be60 0x7fe65806e320 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe66c14c2d0 0x7fe66c09bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe66c14be50 0x7fe66c09ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 --2- 172.21.15.99:0/2988317759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe654004d40 0x7fe66c1401e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 >> 172.21.15.99:0/2988317759 conn(0x7fe66c0b2a80 msgr2=0x7fe66c0732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 shutdown_connections 2024-09-10T11:31:41.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.664+0000 7fe6729c4640 1 -- 172.21.15.99:0/2988317759 wait complete. 2024-09-10T11:31:41.677 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:31:41.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.775+0000 7f0cf1fd8640 1 Processor -- start 2024-09-10T11:31:41.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.775+0000 7f0cf1fd8640 1 -- start start 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0cf1fd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec163580 0x7f0cec0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0cf1fd8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cec0b3960 0x7f0cec0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0cf1fd8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec0b3620 con 0x7f0cec06cd20 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0cf1fd8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec06d960 con 0x7f0cec0b3960 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0cf1fd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec06dc40 con 0x7f0cec163580 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cec0b3960 0x7f0cec0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0cec0b3960 0x7f0cec0d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36766/0 (socket says 172.21.15.99:36766) 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 -- 172.21.15.99:0/925054604 learned_addr learned my addr 172.21.15.99:0/925054604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 -- 172.21.15.99:0/925054604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec163580 msgr2=0x7f0cec0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 --2- 172.21.15.99:0/925054604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec163580 0x7f0cec0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 -- 172.21.15.99:0/925054604 >> v1:172.21.15.196:6789/0 conn(0x7f0cec06cd20 legacy=0x7f0cec06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.776+0000 7f0ceaffd640 1 -- 172.21.15.99:0/925054604 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec06df20 con 0x7f0cec0b3960 2024-09-10T11:31:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.777+0000 7f0ceaffd640 1 --2- 172.21.15.99:0/925054604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec0b3960 0x7f0cec0d1be0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f184f6bcdd9717 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.777+0000 7f0cea7fc640 1 -- 172.21.15.99:0/925054604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0037070 con 0x7f0cec0b3960 2024-09-10T11:31:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.777+0000 7f0cea7fc640 1 -- 172.21.15.99:0/925054604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce0032440 con 0x7f0cec0b3960 2024-09-10T11:31:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.777+0000 7f0cea7fc640 1 -- 172.21.15.99:0/925054604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0032810 con 0x7f0cec0b3960 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/925054604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec0b3960 msgr2=0x7f0cec0d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/925054604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec0b3960 0x7f0cec0d1be0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce002a180 comp rx=0 tx=0).stop 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/925054604 shutdown_connections 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/925054604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec0b3960 0x7f0cec0d1be0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/925054604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec163580 0x7f0cec0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/925054604 >> 172.21.15.99:0/925054604 conn(0x7f0cec0b4570 msgr2=0x7f0cec0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/925054604 shutdown_connections 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.778+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/925054604 wait complete. 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.779+0000 7f0cf1fd8640 1 Processor -- start 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.779+0000 7f0cf1fd8640 1 -- start start 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec0b3960 0x7f0cec0b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec163580 0x7f0cec0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec0b3160 con 0x7f0cec06cd20 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec0d2120 con 0x7f0cec163580 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0b32e0 con 0x7f0cec0b3960 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36772/0 (socket says 172.21.15.99:36772) 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 -- 172.21.15.99:0/3895925360 learned_addr learned my addr 172.21.15.99:0/3895925360 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cebfff640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec163580 0x7f0cec0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceaffd640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec0b3960 0x7f0cec0b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 -- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec0b3960 msgr2=0x7f0cec0b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec0b3960 0x7f0cec0b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 -- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec163580 msgr2=0x7f0cec0b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec163580 0x7f0cec0b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 -- 172.21.15.99:0/3895925360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec0b72e0 con 0x7f0cec06cd20 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0ceb7fe640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b4d0 tx=0x7f0cdc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc014070 con 0x7f0cec06cd20 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cdc010870 con 0x7f0cec06cd20 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc010c60 con 0x7f0cec06cd20 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.780+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec0b75c0 con 0x7f0cec06cd20 2024-09-10T11:31:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.781+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec0b7b80 con 0x7f0cec06cd20 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.782+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cec0761a0 con 0x7f0cec06cd20 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.782+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cdc017070 con 0x7f0cec06cd20 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.783+0000 7f0cd3fff640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.783+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0cdc08ac30 con 0x7f0cec06cd20 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.783+0000 7f0ceaffd640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:41.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.783+0000 7f0ceaffd640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 0x7f0cd806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0034000 tx=0x7f0ce002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.787+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0cdc058f90 con 0x7f0cec06cd20 2024-09-10T11:31:41.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.986+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cec0d1be0 con 0x7f0cec06cd20 2024-09-10T11:31:41.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.986+0000 7f0cd3fff640 1 -- 172.21.15.99:0/3895925360 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7f0cdc05d290 con 0x7f0cec06cd20 2024-09-10T11:31:41.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:41.987 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:31:41.988 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:31:41.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 msgr2=0x7f0cd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 0x7f0cd806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0034000 tx=0x7f0ce002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:41.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 msgr2=0x7f0cec0cfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b4d0 tx=0x7f0cdc00b990 comp rx=0 tx=0).stop 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 shutdown_connections 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cd806be30 0x7f0cd806e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cec163580 0x7f0cec0b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0cec0b3960 0x7f0cec0b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 --2- 172.21.15.99:0/3895925360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0cec06cd20 0x7f0cec0cfc60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 >> 172.21.15.99:0/3895925360 conn(0x7f0cec0b4570 msgr2=0x7f0cec070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 shutdown_connections 2024-09-10T11:31:41.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:41.988+0000 7f0cf1fd8640 1 -- 172.21.15.99:0/3895925360 wait complete. 2024-09-10T11:31:42.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:31:42.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.099+0000 7f5c6a7bd640 1 Processor -- start 2024-09-10T11:31:42.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.099+0000 7f5c6a7bd640 1 -- start start 2024-09-10T11:31:42.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c6a7bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c6a7bd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c64161270 0x7f5c64160660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c6a7bd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c64071e30 con 0x7f5c64160ba0 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c6a7bd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5c640d0580 con 0x7f5c64161270 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c6a7bd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6406f2c0 con 0x7f5c64146920 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c637fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c64161270 0x7f5c64160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c68d5a640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5c64160ba0 0x7f5c6406d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37734/0 (socket says 172.21.15.99:37734) 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:33162/0 (socket says 172.21.15.99:33162) 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 -- 172.21.15.99:0/612058672 learned_addr learned my addr 172.21.15.99:0/612058672 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 -- 172.21.15.99:0/612058672 >> v2:172.21.15.196:3300/0 conn(0x7f5c64161270 msgr2=0x7f5c64160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 --2- 172.21.15.99:0/612058672 >> v2:172.21.15.196:3300/0 conn(0x7f5c64161270 0x7f5c64160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 -- 172.21.15.99:0/612058672 >> v1:172.21.15.196:6789/0 conn(0x7f5c64160ba0 legacy=0x7f5c6406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.100+0000 7f5c63fff640 1 -- 172.21.15.99:0/612058672 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6406f5a0 con 0x7f5c64146920 2024-09-10T11:31:42.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.101+0000 7f5c63fff640 1 --2- 172.21.15.99:0/612058672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009870 tx=0x7f5c5402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6e271ff5de043f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.101+0000 7f5c62ffd640 1 -- 172.21.15.99:0/612058672 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54037070 con 0x7f5c64146920 2024-09-10T11:31:42.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c62ffd640 1 -- 172.21.15.99:0/612058672 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54032440 con 0x7f5c64146920 2024-09-10T11:31:42.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c62ffd640 1 -- 172.21.15.99:0/612058672 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54032810 con 0x7f5c64146920 2024-09-10T11:31:42.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/612058672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 msgr2=0x7f5c640718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/612058672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c54009870 tx=0x7f5c5402a130 comp rx=0 tx=0).stop 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/612058672 shutdown_connections 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/612058672 >> v2:172.21.15.196:3300/0 conn(0x7f5c64161270 0x7f5c64160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/612058672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64146920 0x7f5c640718f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/612058672 >> 172.21.15.99:0/612058672 conn(0x7f5c640b2a80 msgr2=0x7f5c640b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/612058672 shutdown_connections 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.102+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/612058672 wait complete. 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.103+0000 7f5c6a7bd640 1 Processor -- start 2024-09-10T11:31:42.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.103+0000 7f5c6a7bd640 1 -- start start 2024-09-10T11:31:42.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64146920 0x7f5c641f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64160ba0 0x7f5c640c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640add60 con 0x7f5c64161270 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5c640cf050 con 0x7f5c64146920 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c6a7bd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5c640adee0 con 0x7f5c64160ba0 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c68d5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c68d5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36774/0 (socket says 172.21.15.99:36774) 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c68d5a640 1 -- 172.21.15.99:0/1047818775 learned_addr learned my addr 172.21.15.99:0/1047818775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c63fff640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64146920 0x7f5c641f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.104+0000 7f5c637fe640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64160ba0 0x7f5c640c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 -- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64160ba0 msgr2=0x7f5c640c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64160ba0 0x7f5c640c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 -- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64146920 msgr2=0x7f5c641f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64146920 0x7f5c641f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 -- 172.21.15.99:0/1047818775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c640c6fa0 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c68d5a640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c7b0 tx=0x7f5c5c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c017070 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5c00a640 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c640c7280 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c00a9f0 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.105+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c640c7840 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.106+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c640718f0 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.106+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c5c01a070 con 0x7f5c64161270 2024-09-10T11:31:42.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.107+0000 7f5c60ff9640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.107+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5c5c08a3b0 con 0x7f5c64161270 2024-09-10T11:31:42.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.107+0000 7f5c63fff640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.107+0000 7f5c63fff640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 0x7f5c5006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c54034000 tx=0x7f5c5402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.111+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c5c058710 con 0x7f5c64161270 2024-09-10T11:31:42.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.276+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c640d0710 con 0x7f5c64161270 2024-09-10T11:31:42.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.278+0000 7f5c60ff9640 1 -- 172.21.15.99:0/1047818775 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16669 (secure 0 0 0) 0x7f5c5c05ca10 con 0x7f5c64161270 2024-09-10T11:31:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:42.279 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":45,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:31:40.638452+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:37.584024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:37.949922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 msgr2=0x7f5c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 0x7f5c5006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5c54034000 tx=0x7f5c5402e000 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 msgr2=0x7f5c64144f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c7b0 tx=0x7f5c5c00cc70 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 shutdown_connections 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c64161270 0x7f5c64144f40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c64160ba0 0x7f5c640c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 --2- 172.21.15.99:0/1047818775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c64146920 0x7f5c641f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 >> 172.21.15.99:0/1047818775 conn(0x7f5c640b2a80 msgr2=0x7f5c640b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 shutdown_connections 2024-09-10T11:31:42.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.280+0000 7f5c6a7bd640 1 -- 172.21.15.99:0/1047818775 wait complete. 2024-09-10T11:31:42.293 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:31:42.294 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:42.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.391+0000 7fcdefea5640 1 Processor -- start 2024-09-10T11:31:42.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.391+0000 7fcdefea5640 1 -- start start 2024-09-10T11:31:42.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.392+0000 7fcdefea5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.392+0000 7fcdefea5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 0x7fcde80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.392+0000 7fcdefea5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcde80b40e0 con 0x7fcde815ee00 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.392+0000 7fcdefea5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcde80b5ab0 con 0x7fcde80b0090 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.392+0000 7fcdefea5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcde8073c50 con 0x7fcde815f1e0 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcdee442640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 0x7fcde80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcded440640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcdedc41640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcde815ee00 0x7fcde80b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37738/0 (socket says 172.21.15.99:37738) 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcdee442640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 0x7fcde80b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:38086/0 (socket says 172.21.15.99:38086) 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcdedc41640 1 -- 172.21.15.99:0/2566365775 learned_addr learned my addr 172.21.15.99:0/2566365775 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcded440640 1 -- 172.21.15.99:0/2566365775 >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 msgr2=0x7fcde80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcded440640 1 --2- 172.21.15.99:0/2566365775 >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 0x7fcde80b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcded440640 1 -- 172.21.15.99:0/2566365775 >> v1:172.21.15.196:6789/0 conn(0x7fcde815ee00 legacy=0x7fcde80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.393+0000 7fcded440640 1 -- 172.21.15.99:0/2566365775 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80b4380 con 0x7fcde815f1e0 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcded440640 1 --2- 172.21.15.99:0/2566365775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde80b5f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d854cda5edcb8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdecc3f640 1 -- 172.21.15.99:0/2566365775 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8037070 con 0x7fcde815f1e0 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdecc3f640 1 -- 172.21.15.99:0/2566365775 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd8032440 con 0x7fcde815f1e0 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdecc3f640 1 -- 172.21.15.99:0/2566365775 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8032810 con 0x7fcde815f1e0 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 -- 172.21.15.99:0/2566365775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 msgr2=0x7fcde80b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 --2- 172.21.15.99:0/2566365775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde80b5f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80097a0 tx=0x7fcdd802a180 comp rx=0 tx=0).stop 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 -- 172.21.15.99:0/2566365775 shutdown_connections 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 --2- 172.21.15.99:0/2566365775 >> v2:172.21.15.196:3300/0 conn(0x7fcde80b0090 0x7fcde80b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 --2- 172.21.15.99:0/2566365775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde80b5f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.394+0000 7fcdefea5640 1 -- 172.21.15.99:0/2566365775 >> 172.21.15.99:0/2566365775 conn(0x7fcde80730c0 msgr2=0x7fcde80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.395+0000 7fcdefea5640 1 -- 172.21.15.99:0/2566365775 shutdown_connections 2024-09-10T11:31:42.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.395+0000 7fcdefea5640 1 -- 172.21.15.99:0/2566365775 wait complete. 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 Processor -- start 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 -- start start 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 0x7fcde8151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde81622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80b21b0 con 0x7fcde815ee00 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcde80afd60 con 0x7fcde80b0090 2024-09-10T11:31:42.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.396+0000 7fcdefea5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcde80b0600 con 0x7fcde815f1e0 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdedc41640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 0x7fcde8151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36780/0 (socket says 172.21.15.99:36780) 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 -- 172.21.15.99:0/3770013245 learned_addr learned my addr 172.21.15.99:0/3770013245 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdee442640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde81622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 -- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 msgr2=0x7fcde81622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde81622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 -- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 msgr2=0x7fcde8151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 0x7fcde8151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 -- 172.21.15.99:0/3770013245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde80cd840 con 0x7fcde815ee00 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdedc41640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 0x7fcde8151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcded440640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8042020 con 0x7fcde815ee00 2024-09-10T11:31:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdd803d730 con 0x7fcde815ee00 2024-09-10T11:31:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8037040 con 0x7fcde815ee00 2024-09-10T11:31:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde81f8b10 con 0x7fcde815ee00 2024-09-10T11:31:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.397+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde81f8030 con 0x7fcde815ee00 2024-09-10T11:31:42.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.398+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde80b5f90 con 0x7fcde815ee00 2024-09-10T11:31:42.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.399+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdd8045020 con 0x7fcde815ee00 2024-09-10T11:31:42.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.399+0000 7fcdde7fc640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 0x7fcdcc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.399+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcdd80ab820 con 0x7fcde815ee00 2024-09-10T11:31:42.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.399+0000 7fcdedc41640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 0x7fcdcc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.400+0000 7fcdedc41640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 0x7fcdcc06e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcde4007fd0 tx=0x7fcde4007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.403+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcdd804c030 con 0x7fcde815ee00 2024-09-10T11:31:42.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.564+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcde8151a40 con 0x7fcdcc06be60 2024-09-10T11:31:42.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.569+0000 7fcdde7fc640 1 -- 172.21.15.99:0/3770013245 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236288 (secure 0 0 0) 0x7fcde4001830 con 0x7fcdcc06be60 2024-09-10T11:31:42.573 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:42.574 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:42.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 msgr2=0x7fcdcc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 0x7fcdcc06e320 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcde4007fd0 tx=0x7fcde4007e80 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 msgr2=0x7fcde8161db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd80046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 shutdown_connections 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcdcc06be60 0x7fcdcc06e320 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcde815f1e0 0x7fcde81622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcde815ee00 0x7fcde8161db0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 --2- 172.21.15.99:0/3770013245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcde80b0090 0x7fcde8151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 >> 172.21.15.99:0/3770013245 conn(0x7fcde80730c0 msgr2=0x7fcde806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 shutdown_connections 2024-09-10T11:31:42.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.572+0000 7fcdefea5640 1 -- 172.21.15.99:0/3770013245 wait complete. 2024-09-10T11:31:42.584 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T11:31:40.924470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2511,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559988,"statfs":{"total":1600000000,"available":1597428448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000599"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T11:31:33.075014+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:33.075014+0000","last_peered":"2024-09-10T11:31:33.075014+0000","last_clean":"2024-09-10T11:31:33.075014+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:33.075014+0000","last_undegraded":"2024-09-10T11:31:33.075014+0000","last_fullsized":"2024-09-10T11:31:33.075014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194571,"statfs":{"total":200000000,"available":199240876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":184,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195127,"statfs":{"total":200000000,"available":199810860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":184,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195127,"statfs":{"total":200000000,"available":199810860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194571,"statfs":{"total":200000000,"available":199240876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:42.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:42.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.722+0000 7f896c68d640 1 Processor -- start 2024-09-10T11:31:42.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.722+0000 7f896c68d640 1 -- start start 2024-09-10T11:31:42.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.723+0000 7f896c68d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89640d1200 0x7f8964073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.723+0000 7f896c68d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f896406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.723+0000 7f896c68d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8964152e60 con 0x7f896415c060 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.723+0000 7f896c68d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f896406ef50 con 0x7f89640d1200 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.723+0000 7f896c68d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f89640d15e0 con 0x7f896406e5d0 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89640d1200 0x7f8964073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f89640d1200 0x7f8964073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60470/0 (socket says 172.21.15.99:60470) 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 -- 172.21.15.99:0/1099580628 learned_addr learned my addr 172.21.15.99:0/1099580628 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f8969c28640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f896406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 -- 172.21.15.99:0/1099580628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 msgr2=0x7f896406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f896406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 -- 172.21.15.99:0/1099580628 >> v1:172.21.15.196:6789/0 conn(0x7f896415c060 legacy=0x7f896415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.724+0000 7f896a429640 1 -- 172.21.15.99:0/1099580628 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8964072680 con 0x7f89640d1200 2024-09-10T11:31:42.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896a429640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f8964073440 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8958009870 tx=0x7f895802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30e2406de89786a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f8969427640 1 -- 172.21.15.99:0/1099580628 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958037070 con 0x7f89640d1200 2024-09-10T11:31:42.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f8969427640 1 -- 172.21.15.99:0/1099580628 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8958032440 con 0x7f89640d1200 2024-09-10T11:31:42.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f8969427640 1 -- 172.21.15.99:0/1099580628 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958032810 con 0x7f89640d1200 2024-09-10T11:31:42.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 -- 172.21.15.99:0/1099580628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 msgr2=0x7f8964073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f8964073440 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8958009870 tx=0x7f895802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 -- 172.21.15.99:0/1099580628 shutdown_connections 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f896406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 --2- 172.21.15.99:0/1099580628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f8964073440 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 -- 172.21.15.99:0/1099580628 >> 172.21.15.99:0/1099580628 conn(0x7f896406bc10 msgr2=0x7f896406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 -- 172.21.15.99:0/1099580628 shutdown_connections 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.725+0000 7f896c68d640 1 -- 172.21.15.99:0/1099580628 wait complete. 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.726+0000 7f896c68d640 1 Processor -- start 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 -- start start 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f8964159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f896415c060 0x7f89641ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8964072f20 con 0x7f89640d1200 2024-09-10T11:31:42.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8964072c40 con 0x7f896415c060 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896c68d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f896415cd90 con 0x7f896406e5d0 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f8969c28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f8969c28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60472/0 (socket says 172.21.15.99:60472) 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f8969c28640 1 -- 172.21.15.99:0/3643405716 learned_addr learned my addr 172.21.15.99:0/3643405716 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896ac2a640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f896415c060 0x7f89641ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.727+0000 7f896a429640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f8964159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 -- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 msgr2=0x7f8964159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f8964159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 -- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f896415c060 msgr2=0x7f89641ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f896415c060 0x7f89641ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 -- 172.21.15.99:0/3643405716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89641f59e0 con 0x7f89640d1200 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8969c28640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f896000d5f0 tx=0x7f896000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600125f0 con 0x7f89640d1200 2024-09-10T11:31:42.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8960019070 con 0x7f89640d1200 2024-09-10T11:31:42.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8964156770 con 0x7f89640d1200 2024-09-10T11:31:42.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600129d0 con 0x7f89640d1200 2024-09-10T11:31:42.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.728+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8964156d30 con 0x7f89640d1200 2024-09-10T11:31:42.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.729+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8964160320 con 0x7f89640d1200 2024-09-10T11:31:42.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.729+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8960012b70 con 0x7f89640d1200 2024-09-10T11:31:42.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.730+0000 7f8956ffd640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 0x7f894806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:42.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.730+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f896008b0f0 con 0x7f89640d1200 2024-09-10T11:31:42.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.730+0000 7f896a429640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 0x7f894806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:42.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.730+0000 7f896a429640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 0x7f894806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8958034000 tx=0x7f895802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:42.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.734+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8964160320 con 0x7f89640d1200 2024-09-10T11:31:42.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.897+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f896406ea10 con 0x7f894806be30 2024-09-10T11:31:42.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.902+0000 7f8956ffd640 1 -- 172.21.15.99:0/3643405716 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+236288 (secure 0 0 0) 0x7f896406ea10 con 0x7f894806be30 2024-09-10T11:31:42.908 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:42.909 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.904+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 msgr2=0x7f894806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.904+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 0x7f894806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8958034000 tx=0x7f895802e000 comp rx=0 tx=0).stop 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.904+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 msgr2=0x7f89641f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.904+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f896000d5f0 tx=0x7f896000dab0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 shutdown_connections 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f894806be30 0x7f894806e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f896415c060 0x7f89641ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89640d1200 0x7f89641f54a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 --2- 172.21.15.99:0/3643405716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f896406e5d0 0x7f8964159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 >> 172.21.15.99:0/3643405716 conn(0x7f896406bc10 msgr2=0x7f896415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 shutdown_connections 2024-09-10T11:31:42.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:42.905+0000 7f896c68d640 1 -- 172.21.15.99:0/3643405716 wait complete. 2024-09-10T11:31:42.917 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T11:31:40.924470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2511,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559988,"statfs":{"total":1600000000,"available":1597428448,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000599"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:38.632524+0000","last_change":"2024-09-10T11:31:38.632524+0000","last_active":"2024-09-10T11:31:38.632524+0000","last_peered":"2024-09-10T11:31:38.632524+0000","last_clean":"2024-09-10T11:31:38.632524+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:38.632524+0000","last_undegraded":"2024-09-10T11:31:38.632524+0000","last_fullsized":"2024-09-10T11:31:38.632524+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:31:37.625976+0000","last_change":"2024-09-10T11:31:37.625976+0000","last_active":"2024-09-10T11:31:37.625976+0000","last_peered":"2024-09-10T11:31:37.625976+0000","last_clean":"2024-09-10T11:31:37.625976+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:31:37.625976+0000","last_undegraded":"2024-09-10T11:31:37.625976+0000","last_fullsized":"2024-09-10T11:31:37.625976+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":202,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-10T11:31:33.075014+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:33.075014+0000","last_peered":"2024-09-10T11:31:33.075014+0000","last_clean":"2024-09-10T11:31:33.075014+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:33.075014+0000","last_undegraded":"2024-09-10T11:31:33.075014+0000","last_fullsized":"2024-09-10T11:31:33.075014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194571,"statfs":{"total":200000000,"available":199240876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":184,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195127,"statfs":{"total":200000000,"available":199810860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":184,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195127,"statfs":{"total":200000000,"available":199810860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194571,"statfs":{"total":200000000,"available":199240876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195147,"statfs":{"total":200000000,"available":199831244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:42.920 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.055+0000 7fcfb0274640 1 Processor -- start 2024-09-10T11:31:43.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.055+0000 7fcfb0274640 1 -- start start 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfb0274640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcfa815eb80 0x7fcfa80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfb0274640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa815ff70 con 0x7fcfa81626c0 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfb0274640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfa80d1620 con 0x7fcfa80d5010 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfb0274640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcfa8162bc0 con 0x7fcfa815eb80 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfae010640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcfa815eb80 0x7fcfa80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.056+0000 7fcfad80f640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcfa81626c0 0x7fcfa80d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37052/0 (socket says 172.21.15.99:37052) 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.057+0000 7fcfad80f640 1 -- 172.21.15.99:0/1488088836 learned_addr learned my addr 172.21.15.99:0/1488088836 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.057+0000 7fcfae010640 1 -- 172.21.15.99:0/1488088836 >> v1:172.21.15.196:6789/0 conn(0x7fcfa80d5010 legacy=0x7fcfa80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.057+0000 7fcfae010640 1 -- 172.21.15.99:0/1488088836 >> v1:172.21.15.99:6789/0 conn(0x7fcfa81626c0 legacy=0x7fcfa80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.057+0000 7fcfae010640 1 -- 172.21.15.99:0/1488088836 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa8162ea0 con 0x7fcfa815eb80 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.057+0000 7fcfae010640 1 --2- 172.21.15.99:0/1488088836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa815eb80 0x7fcfa80d29b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009d90 tx=0x7fcf9802a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17574e216c3813c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfad00e640 1 -- 172.21.15.99:0/1488088836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98007740 con 0x7fcfa815eb80 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfad00e640 1 -- 172.21.15.99:0/1488088836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf980078e0 con 0x7fcfa815eb80 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfad00e640 1 -- 172.21.15.99:0/1488088836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf98007cb0 con 0x7fcfa815eb80 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 -- 172.21.15.99:0/1488088836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa815eb80 msgr2=0x7fcfa80d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 --2- 172.21.15.99:0/1488088836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa815eb80 0x7fcfa80d29b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009d90 tx=0x7fcf9802a510 comp rx=0 tx=0).stop 2024-09-10T11:31:43.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 -- 172.21.15.99:0/1488088836 shutdown_connections 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 --2- 172.21.15.99:0/1488088836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa815eb80 0x7fcfa80d29b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 -- 172.21.15.99:0/1488088836 >> 172.21.15.99:0/1488088836 conn(0x7fcfa808e680 msgr2=0x7fcfa809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 -- 172.21.15.99:0/1488088836 shutdown_connections 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.058+0000 7fcfb0274640 1 -- 172.21.15.99:0/1488088836 wait complete. 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.059+0000 7fcfb0274640 1 Processor -- start 2024-09-10T11:31:43.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.059+0000 7fcfb0274640 1 -- start start 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfa80d5010 0x7fcfa80cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 0x7fcfa815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa80d2ef0 con 0x7fcfa81626c0 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa8163460 con 0x7fcfa815eb80 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfb0274640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa80d3070 con 0x7fcfa80d5010 2024-09-10T11:31:43.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae811640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae811640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60486/0 (socket says 172.21.15.99:60486) 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae811640 1 -- 172.21.15.99:0/2295559983 learned_addr learned my addr 172.21.15.99:0/2295559983 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae811640 1 -- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfa80d5010 msgr2=0x7fcfa80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae010640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfa80d5010 0x7fcfa80cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfad80f640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 0x7fcfa815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.060+0000 7fcfae811640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfa80d5010 0x7fcfa80cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfae811640 1 -- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 msgr2=0x7fcfa815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfae811640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 0x7fcfa815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfae811640 1 -- 172.21.15.99:0/2295559983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa815a910 con 0x7fcfa81626c0 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfad80f640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 0x7fcfa815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:43.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfae811640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b440 tx=0x7fcfa000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0015020 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa0011830 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa80cd290 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0011c20 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.061+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa80b1aa0 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.062+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa80d29b0 con 0x7fcfa81626c0 2024-09-10T11:31:43.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.062+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcfa0018020 con 0x7fcfa81626c0 2024-09-10T11:31:43.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.063+0000 7fcfa6ffd640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 0x7fcf9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.063+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcfa008abc0 con 0x7fcfa81626c0 2024-09-10T11:31:43.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.063+0000 7fcfae010640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 0x7fcf9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.063+0000 7fcfae010640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 0x7fcf9406e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf98034000 tx=0x7fcf9802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.067+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcfa001f020 con 0x7fcfa81626c0 2024-09-10T11:31:43.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.230+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcfa815e0e0 con 0x7fcf9406be60 2024-09-10T11:31:43.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.236+0000 7fcfa6ffd640 1 -- 172.21.15.99:0/2295559983 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247480 (secure 0 0 0) 0x7fcfa815e0e0 con 0x7fcf9406be60 2024-09-10T11:31:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:43.240 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 msgr2=0x7fcf9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 0x7fcf9406e320 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf98034000 tx=0x7fcf9802f000 comp rx=0 tx=0).stop 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 msgr2=0x7fcfa815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000b440 tx=0x7fcfa000b900 comp rx=0 tx=0).stop 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 shutdown_connections 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf9406be60 0x7fcf9406e320 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfa81626c0 0x7fcfa815c0f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfa815eb80 0x7fcfa815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 --2- 172.21.15.99:0/2295559983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfa80d5010 0x7fcfa80cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 >> 172.21.15.99:0/2295559983 conn(0x7fcfa808e680 msgr2=0x7fcfa80d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 shutdown_connections 2024-09-10T11:31:43.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.239+0000 7fcfb0274640 1 -- 172.21.15.99:0/2295559983 wait complete. 2024-09-10T11:31:43.251 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T11:31:42.924724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2807,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559692,"statfs":{"total":1600000000,"available":1597125188,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.291293"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643782+0000","last_change":"2024-09-10T11:31:39.645497+0000","last_active":"2024-09-10T11:31:40.643782+0000","last_peered":"2024-09-10T11:31:40.643782+0000","last_clean":"2024-09-10T11:31:40.643782+0000","last_became_active":"2024-09-10T11:31:39.645175+0000","last_became_peered":"2024-09-10T11:31:39.645175+0000","last_unstale":"2024-09-10T11:31:40.643782+0000","last_undegraded":"2024-09-10T11:31:40.643782+0000","last_fullsized":"2024-09-10T11:31:40.643782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:57.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646626+0000","last_change":"2024-09-10T11:31:39.653090+0000","last_active":"2024-09-10T11:31:40.646626+0000","last_peered":"2024-09-10T11:31:40.646626+0000","last_clean":"2024-09-10T11:31:40.646626+0000","last_became_active":"2024-09-10T11:31:39.652865+0000","last_became_peered":"2024-09-10T11:31:39.652865+0000","last_unstale":"2024-09-10T11:31:40.646626+0000","last_undegraded":"2024-09-10T11:31:40.646626+0000","last_fullsized":"2024-09-10T11:31:40.646626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:08.493521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655676+0000","last_change":"2024-09-10T11:31:39.655491+0000","last_active":"2024-09-10T11:31:39.655676+0000","last_peered":"2024-09-10T11:31:39.655676+0000","last_clean":"2024-09-10T11:31:39.655676+0000","last_became_active":"2024-09-10T11:31:39.655120+0000","last_became_peered":"2024-09-10T11:31:39.655120+0000","last_unstale":"2024-09-10T11:31:39.655676+0000","last_undegraded":"2024-09-10T11:31:39.655676+0000","last_fullsized":"2024-09-10T11:31:39.655676+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:45.132618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657665+0000","last_change":"2024-09-10T11:31:39.650249+0000","last_active":"2024-09-10T11:31:41.657665+0000","last_peered":"2024-09-10T11:31:41.657665+0000","last_clean":"2024-09-10T11:31:41.657665+0000","last_became_active":"2024-09-10T11:31:39.649905+0000","last_became_peered":"2024-09-10T11:31:39.649905+0000","last_unstale":"2024-09-10T11:31:41.657665+0000","last_undegraded":"2024-09-10T11:31:41.657665+0000","last_fullsized":"2024-09-10T11:31:41.657665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:09.765200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643841+0000","last_change":"2024-09-10T11:31:39.646393+0000","last_active":"2024-09-10T11:31:40.643841+0000","last_peered":"2024-09-10T11:31:40.643841+0000","last_clean":"2024-09-10T11:31:40.643841+0000","last_became_active":"2024-09-10T11:31:39.646154+0000","last_became_peered":"2024-09-10T11:31:39.646154+0000","last_unstale":"2024-09-10T11:31:40.643841+0000","last_undegraded":"2024-09-10T11:31:40.643841+0000","last_fullsized":"2024-09-10T11:31:40.643841+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:38.360639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641243+0000","last_change":"2024-09-10T11:31:39.654253+0000","last_active":"2024-09-10T11:31:40.641243+0000","last_peered":"2024-09-10T11:31:40.641243+0000","last_clean":"2024-09-10T11:31:40.641243+0000","last_became_active":"2024-09-10T11:31:39.653964+0000","last_became_peered":"2024-09-10T11:31:39.653964+0000","last_unstale":"2024-09-10T11:31:40.641243+0000","last_undegraded":"2024-09-10T11:31:40.641243+0000","last_fullsized":"2024-09-10T11:31:40.641243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:46.259879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658993+0000","last_change":"2024-09-10T11:31:39.656014+0000","last_active":"2024-09-10T11:31:41.658993+0000","last_peered":"2024-09-10T11:31:41.658993+0000","last_clean":"2024-09-10T11:31:41.658993+0000","last_became_active":"2024-09-10T11:31:39.655797+0000","last_became_peered":"2024-09-10T11:31:39.655797+0000","last_unstale":"2024-09-10T11:31:41.658993+0000","last_undegraded":"2024-09-10T11:31:41.658993+0000","last_fullsized":"2024-09-10T11:31:41.658993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.186968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657591+0000","last_change":"2024-09-10T11:31:39.654806+0000","last_active":"2024-09-10T11:31:41.657591+0000","last_peered":"2024-09-10T11:31:41.657591+0000","last_clean":"2024-09-10T11:31:41.657591+0000","last_became_active":"2024-09-10T11:31:39.653029+0000","last_became_peered":"2024-09-10T11:31:39.653029+0000","last_unstale":"2024-09-10T11:31:41.657591+0000","last_undegraded":"2024-09-10T11:31:41.657591+0000","last_fullsized":"2024-09-10T11:31:41.657591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.640172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641152+0000","last_change":"2024-09-10T11:31:39.658665+0000","last_active":"2024-09-10T11:31:40.641152+0000","last_peered":"2024-09-10T11:31:40.641152+0000","last_clean":"2024-09-10T11:31:40.641152+0000","last_became_active":"2024-09-10T11:31:39.658365+0000","last_became_peered":"2024-09-10T11:31:39.658365+0000","last_unstale":"2024-09-10T11:31:40.641152+0000","last_undegraded":"2024-09-10T11:31:40.641152+0000","last_fullsized":"2024-09-10T11:31:40.641152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:22.800217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871184+0000","last_change":"2024-09-10T11:31:39.656494+0000","last_active":"2024-09-10T11:31:41.871184+0000","last_peered":"2024-09-10T11:31:41.871184+0000","last_clean":"2024-09-10T11:31:41.871184+0000","last_became_active":"2024-09-10T11:31:39.656264+0000","last_became_peered":"2024-09-10T11:31:39.656264+0000","last_unstale":"2024-09-10T11:31:41.871184+0000","last_undegraded":"2024-09-10T11:31:41.871184+0000","last_fullsized":"2024-09-10T11:31:41.871184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:54.180344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657795+0000","last_change":"2024-09-10T11:31:39.653639+0000","last_active":"2024-09-10T11:31:41.657795+0000","last_peered":"2024-09-10T11:31:41.657795+0000","last_clean":"2024-09-10T11:31:41.657795+0000","last_became_active":"2024-09-10T11:31:39.652232+0000","last_became_peered":"2024-09-10T11:31:39.652232+0000","last_unstale":"2024-09-10T11:31:41.657795+0000","last_undegraded":"2024-09-10T11:31:41.657795+0000","last_fullsized":"2024-09-10T11:31:41.657795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:10.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646762+0000","last_change":"2024-09-10T11:31:39.647797+0000","last_active":"2024-09-10T11:31:40.646762+0000","last_peered":"2024-09-10T11:31:40.646762+0000","last_clean":"2024-09-10T11:31:40.646762+0000","last_became_active":"2024-09-10T11:31:39.647576+0000","last_became_peered":"2024-09-10T11:31:39.647576+0000","last_unstale":"2024-09-10T11:31:40.646762+0000","last_undegraded":"2024-09-10T11:31:40.646762+0000","last_fullsized":"2024-09-10T11:31:40.646762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:42.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641057+0000","last_change":"2024-09-10T11:31:39.648764+0000","last_active":"2024-09-10T11:31:40.641057+0000","last_peered":"2024-09-10T11:31:40.641057+0000","last_clean":"2024-09-10T11:31:40.641057+0000","last_became_active":"2024-09-10T11:31:39.648525+0000","last_became_peered":"2024-09-10T11:31:39.648525+0000","last_unstale":"2024-09-10T11:31:40.641057+0000","last_undegraded":"2024-09-10T11:31:40.641057+0000","last_fullsized":"2024-09-10T11:31:40.641057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:15.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658043+0000","last_change":"2024-09-10T11:31:39.658849+0000","last_active":"2024-09-10T11:31:41.658043+0000","last_peered":"2024-09-10T11:31:41.658043+0000","last_clean":"2024-09-10T11:31:41.658043+0000","last_became_active":"2024-09-10T11:31:39.658315+0000","last_became_peered":"2024-09-10T11:31:39.658315+0000","last_unstale":"2024-09-10T11:31:41.658043+0000","last_undegraded":"2024-09-10T11:31:41.658043+0000","last_fullsized":"2024-09-10T11:31:41.658043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:18.791072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657667+0000","last_change":"2024-09-10T11:31:39.655320+0000","last_active":"2024-09-10T11:31:41.657667+0000","last_peered":"2024-09-10T11:31:41.657667+0000","last_clean":"2024-09-10T11:31:41.657667+0000","last_became_active":"2024-09-10T11:31:39.654324+0000","last_became_peered":"2024-09-10T11:31:39.654324+0000","last_unstale":"2024-09-10T11:31:41.657667+0000","last_undegraded":"2024-09-10T11:31:41.657667+0000","last_fullsized":"2024-09-10T11:31:41.657667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:39.168999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643984+0000","last_change":"2024-09-10T11:31:39.656422+0000","last_active":"2024-09-10T11:31:40.643984+0000","last_peered":"2024-09-10T11:31:40.643984+0000","last_clean":"2024-09-10T11:31:40.643984+0000","last_became_active":"2024-09-10T11:31:39.656184+0000","last_became_peered":"2024-09-10T11:31:39.656184+0000","last_unstale":"2024-09-10T11:31:40.643984+0000","last_undegraded":"2024-09-10T11:31:40.643984+0000","last_fullsized":"2024-09-10T11:31:40.643984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:15.186401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646872+0000","last_change":"2024-09-10T11:31:39.647034+0000","last_active":"2024-09-10T11:31:40.646872+0000","last_peered":"2024-09-10T11:31:40.646872+0000","last_clean":"2024-09-10T11:31:40.646872+0000","last_became_active":"2024-09-10T11:31:39.646566+0000","last_became_peered":"2024-09-10T11:31:39.646566+0000","last_unstale":"2024-09-10T11:31:40.646872+0000","last_undegraded":"2024-09-10T11:31:40.646872+0000","last_fullsized":"2024-09-10T11:31:40.646872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:44.052632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655420+0000","last_change":"2024-09-10T11:31:39.655310+0000","last_active":"2024-09-10T11:31:39.655420+0000","last_peered":"2024-09-10T11:31:39.655420+0000","last_clean":"2024-09-10T11:31:39.655420+0000","last_became_active":"2024-09-10T11:31:39.654844+0000","last_became_peered":"2024-09-10T11:31:39.654844+0000","last_unstale":"2024-09-10T11:31:39.655420+0000","last_undegraded":"2024-09-10T11:31:39.655420+0000","last_fullsized":"2024-09-10T11:31:39.655420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:44.558181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643835+0000","last_change":"2024-09-10T11:31:39.645579+0000","last_active":"2024-09-10T11:31:40.643835+0000","last_peered":"2024-09-10T11:31:40.643835+0000","last_clean":"2024-09-10T11:31:40.643835+0000","last_became_active":"2024-09-10T11:31:39.645354+0000","last_became_peered":"2024-09-10T11:31:39.645354+0000","last_unstale":"2024-09-10T11:31:40.643835+0000","last_undegraded":"2024-09-10T11:31:40.643835+0000","last_fullsized":"2024-09-10T11:31:40.643835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:49.467444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871633+0000","last_change":"2024-09-10T11:31:39.654118+0000","last_active":"2024-09-10T11:31:41.871633+0000","last_peered":"2024-09-10T11:31:41.871633+0000","last_clean":"2024-09-10T11:31:41.871633+0000","last_became_active":"2024-09-10T11:31:39.652435+0000","last_became_peered":"2024-09-10T11:31:39.652435+0000","last_unstale":"2024-09-10T11:31:41.871633+0000","last_undegraded":"2024-09-10T11:31:41.871633+0000","last_fullsized":"2024-09-10T11:31:41.871633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:31.098768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658543+0000","last_change":"2024-09-10T11:31:39.652924+0000","last_active":"2024-09-10T11:31:41.658543+0000","last_peered":"2024-09-10T11:31:41.658543+0000","last_clean":"2024-09-10T11:31:41.658543+0000","last_became_active":"2024-09-10T11:31:39.650464+0000","last_became_peered":"2024-09-10T11:31:39.650464+0000","last_unstale":"2024-09-10T11:31:41.658543+0000","last_undegraded":"2024-09-10T11:31:41.658543+0000","last_fullsized":"2024-09-10T11:31:41.658543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:17.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470652+0000","last_change":"2024-09-10T11:31:39.650521+0000","last_active":"2024-09-10T11:31:41.470652+0000","last_peered":"2024-09-10T11:31:41.470652+0000","last_clean":"2024-09-10T11:31:41.470652+0000","last_became_active":"2024-09-10T11:31:39.650297+0000","last_became_peered":"2024-09-10T11:31:39.650297+0000","last_unstale":"2024-09-10T11:31:41.470652+0000","last_undegraded":"2024-09-10T11:31:41.470652+0000","last_fullsized":"2024-09-10T11:31:41.470652+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:39.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659573+0000","last_change":"2024-09-10T11:31:39.650038+0000","last_active":"2024-09-10T11:31:41.659573+0000","last_peered":"2024-09-10T11:31:41.659573+0000","last_clean":"2024-09-10T11:31:41.659573+0000","last_became_active":"2024-09-10T11:31:39.649807+0000","last_became_peered":"2024-09-10T11:31:39.649807+0000","last_unstale":"2024-09-10T11:31:41.659573+0000","last_undegraded":"2024-09-10T11:31:41.659573+0000","last_fullsized":"2024-09-10T11:31:41.659573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:04.977562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646605+0000","last_change":"2024-09-10T11:31:39.651240+0000","last_active":"2024-09-10T11:31:40.646605+0000","last_peered":"2024-09-10T11:31:40.646605+0000","last_clean":"2024-09-10T11:31:40.646605+0000","last_became_active":"2024-09-10T11:31:39.651015+0000","last_became_peered":"2024-09-10T11:31:39.651015+0000","last_unstale":"2024-09-10T11:31:40.646605+0000","last_undegraded":"2024-09-10T11:31:40.646605+0000","last_fullsized":"2024-09-10T11:31:40.646605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:50.935894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643988+0000","last_change":"2024-09-10T11:31:39.645360+0000","last_active":"2024-09-10T11:31:40.643988+0000","last_peered":"2024-09-10T11:31:40.643988+0000","last_clean":"2024-09-10T11:31:40.643988+0000","last_became_active":"2024-09-10T11:31:39.645089+0000","last_became_peered":"2024-09-10T11:31:39.645089+0000","last_unstale":"2024-09-10T11:31:40.643988+0000","last_undegraded":"2024-09-10T11:31:40.643988+0000","last_fullsized":"2024-09-10T11:31:40.643988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:05.904582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657992+0000","last_change":"2024-09-10T11:31:39.653442+0000","last_active":"2024-09-10T11:31:41.657992+0000","last_peered":"2024-09-10T11:31:41.657992+0000","last_clean":"2024-09-10T11:31:41.657992+0000","last_became_active":"2024-09-10T11:31:39.651982+0000","last_became_peered":"2024-09-10T11:31:39.651982+0000","last_unstale":"2024-09-10T11:31:41.657992+0000","last_undegraded":"2024-09-10T11:31:41.657992+0000","last_fullsized":"2024-09-10T11:31:41.657992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:17.090501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659162+0000","last_change":"2024-09-10T11:31:39.650720+0000","last_active":"2024-09-10T11:31:41.659162+0000","last_peered":"2024-09-10T11:31:41.659162+0000","last_clean":"2024-09-10T11:31:41.659162+0000","last_became_active":"2024-09-10T11:31:39.650477+0000","last_became_peered":"2024-09-10T11:31:39.650477+0000","last_unstale":"2024-09-10T11:31:41.659162+0000","last_undegraded":"2024-09-10T11:31:41.659162+0000","last_fullsized":"2024-09-10T11:31:41.659162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:54.152529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659073+0000","last_change":"2024-09-10T11:31:39.651780+0000","last_active":"2024-09-10T11:31:41.659073+0000","last_peered":"2024-09-10T11:31:41.659073+0000","last_clean":"2024-09-10T11:31:41.659073+0000","last_became_active":"2024-09-10T11:31:39.651388+0000","last_became_peered":"2024-09-10T11:31:39.651388+0000","last_unstale":"2024-09-10T11:31:41.659073+0000","last_undegraded":"2024-09-10T11:31:41.659073+0000","last_fullsized":"2024-09-10T11:31:41.659073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:16.395096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643642+0000","last_change":"2024-09-10T11:31:39.658219+0000","last_active":"2024-09-10T11:31:40.643642+0000","last_peered":"2024-09-10T11:31:40.643642+0000","last_clean":"2024-09-10T11:31:40.643642+0000","last_became_active":"2024-09-10T11:31:39.657959+0000","last_became_peered":"2024-09-10T11:31:39.657959+0000","last_unstale":"2024-09-10T11:31:40.643642+0000","last_undegraded":"2024-09-10T11:31:40.643642+0000","last_fullsized":"2024-09-10T11:31:40.643642+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:38.467359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646828+0000","last_change":"2024-09-10T11:31:39.646856+0000","last_active":"2024-09-10T11:31:40.646828+0000","last_peered":"2024-09-10T11:31:40.646828+0000","last_clean":"2024-09-10T11:31:40.646828+0000","last_became_active":"2024-09-10T11:31:39.646630+0000","last_became_peered":"2024-09-10T11:31:39.646630+0000","last_unstale":"2024-09-10T11:31:40.646828+0000","last_undegraded":"2024-09-10T11:31:40.646828+0000","last_fullsized":"2024-09-10T11:31:40.646828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:27.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641394+0000","last_change":"2024-09-10T11:31:39.644641+0000","last_active":"2024-09-10T11:31:40.641394+0000","last_peered":"2024-09-10T11:31:40.641394+0000","last_clean":"2024-09-10T11:31:40.641394+0000","last_became_active":"2024-09-10T11:31:39.644390+0000","last_became_peered":"2024-09-10T11:31:39.644390+0000","last_unstale":"2024-09-10T11:31:40.641394+0000","last_undegraded":"2024-09-10T11:31:40.641394+0000","last_fullsized":"2024-09-10T11:31:40.641394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:55.588242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657857+0000","last_change":"2024-09-10T11:31:39.654918+0000","last_active":"2024-09-10T11:31:41.657857+0000","last_peered":"2024-09-10T11:31:41.657857+0000","last_clean":"2024-09-10T11:31:41.657857+0000","last_became_active":"2024-09-10T11:31:39.654587+0000","last_became_peered":"2024-09-10T11:31:39.654587+0000","last_unstale":"2024-09-10T11:31:41.657857+0000","last_undegraded":"2024-09-10T11:31:41.657857+0000","last_fullsized":"2024-09-10T11:31:41.657857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:40.331083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470394+0000","last_change":"2024-09-10T11:31:39.659286+0000","last_active":"2024-09-10T11:31:41.470394+0000","last_peered":"2024-09-10T11:31:41.470394+0000","last_clean":"2024-09-10T11:31:41.470394+0000","last_became_active":"2024-09-10T11:31:39.658774+0000","last_became_peered":"2024-09-10T11:31:39.658774+0000","last_unstale":"2024-09-10T11:31:41.470394+0000","last_undegraded":"2024-09-10T11:31:41.470394+0000","last_fullsized":"2024-09-10T11:31:41.470394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:30.202847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641353+0000","last_change":"2024-09-10T11:31:38.648561+0000","last_active":"2024-09-10T11:31:40.641353+0000","last_peered":"2024-09-10T11:31:40.641353+0000","last_clean":"2024-09-10T11:31:40.641353+0000","last_became_active":"2024-09-10T11:31:38.647403+0000","last_became_peered":"2024-09-10T11:31:38.647403+0000","last_unstale":"2024-09-10T11:31:40.641353+0000","last_undegraded":"2024-09-10T11:31:40.641353+0000","last_fullsized":"2024-09-10T11:31:40.641353+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:33.760518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644028+0000","last_change":"2024-09-10T11:31:39.645767+0000","last_active":"2024-09-10T11:31:40.644028+0000","last_peered":"2024-09-10T11:31:40.644028+0000","last_clean":"2024-09-10T11:31:40.644028+0000","last_became_active":"2024-09-10T11:31:39.645548+0000","last_became_peered":"2024-09-10T11:31:39.645548+0000","last_unstale":"2024-09-10T11:31:40.644028+0000","last_undegraded":"2024-09-10T11:31:40.644028+0000","last_fullsized":"2024-09-10T11:31:40.644028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.922464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659303+0000","last_change":"2024-09-10T11:31:38.644985+0000","last_active":"2024-09-10T11:31:41.659303+0000","last_peered":"2024-09-10T11:31:41.659303+0000","last_clean":"2024-09-10T11:31:41.659303+0000","last_became_active":"2024-09-10T11:31:38.644751+0000","last_became_peered":"2024-09-10T11:31:38.644751+0000","last_unstale":"2024-09-10T11:31:41.659303+0000","last_undegraded":"2024-09-10T11:31:41.659303+0000","last_fullsized":"2024-09-10T11:31:41.659303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:39.728019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646869+0000","last_change":"2024-09-10T11:31:39.649105+0000","last_active":"2024-09-10T11:31:40.646869+0000","last_peered":"2024-09-10T11:31:40.646869+0000","last_clean":"2024-09-10T11:31:40.646869+0000","last_became_active":"2024-09-10T11:31:39.648513+0000","last_became_peered":"2024-09-10T11:31:39.648513+0000","last_unstale":"2024-09-10T11:31:40.646869+0000","last_undegraded":"2024-09-10T11:31:40.646869+0000","last_fullsized":"2024-09-10T11:31:40.646869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:30.397629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641324+0000","last_change":"2024-09-10T11:31:38.647068+0000","last_active":"2024-09-10T11:31:40.641324+0000","last_peered":"2024-09-10T11:31:40.641324+0000","last_clean":"2024-09-10T11:31:40.641324+0000","last_became_active":"2024-09-10T11:31:38.646806+0000","last_became_peered":"2024-09-10T11:31:38.646806+0000","last_unstale":"2024-09-10T11:31:40.641324+0000","last_undegraded":"2024-09-10T11:31:40.641324+0000","last_fullsized":"2024-09-10T11:31:40.641324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:41.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657848+0000","last_change":"2024-09-10T11:31:39.655753+0000","last_active":"2024-09-10T11:31:41.657848+0000","last_peered":"2024-09-10T11:31:41.657848+0000","last_clean":"2024-09-10T11:31:41.657848+0000","last_became_active":"2024-09-10T11:31:39.654854+0000","last_became_peered":"2024-09-10T11:31:39.654854+0000","last_unstale":"2024-09-10T11:31:41.657848+0000","last_undegraded":"2024-09-10T11:31:41.657848+0000","last_fullsized":"2024-09-10T11:31:41.657848+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:02.970230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658983+0000","last_change":"2024-09-10T11:31:38.645523+0000","last_active":"2024-09-10T11:31:41.658983+0000","last_peered":"2024-09-10T11:31:41.658983+0000","last_clean":"2024-09-10T11:31:41.658983+0000","last_became_active":"2024-09-10T11:31:38.644841+0000","last_became_peered":"2024-09-10T11:31:38.644841+0000","last_unstale":"2024-09-10T11:31:41.658983+0000","last_undegraded":"2024-09-10T11:31:41.658983+0000","last_fullsized":"2024-09-10T11:31:41.658983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:44.618873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659386+0000","last_change":"2024-09-10T11:31:39.651607+0000","last_active":"2024-09-10T11:31:41.659386+0000","last_peered":"2024-09-10T11:31:41.659386+0000","last_clean":"2024-09-10T11:31:41.659386+0000","last_became_active":"2024-09-10T11:31:39.651070+0000","last_became_peered":"2024-09-10T11:31:39.651070+0000","last_unstale":"2024-09-10T11:31:41.659386+0000","last_undegraded":"2024-09-10T11:31:41.659386+0000","last_fullsized":"2024-09-10T11:31:41.659386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:03.428955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641087+0000","last_change":"2024-09-10T11:31:38.650460+0000","last_active":"2024-09-10T11:31:40.641087+0000","last_peered":"2024-09-10T11:31:40.641087+0000","last_clean":"2024-09-10T11:31:40.641087+0000","last_became_active":"2024-09-10T11:31:38.649203+0000","last_became_peered":"2024-09-10T11:31:38.649203+0000","last_unstale":"2024-09-10T11:31:40.641087+0000","last_undegraded":"2024-09-10T11:31:40.641087+0000","last_fullsized":"2024-09-10T11:31:40.641087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:22.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871112+0000","last_change":"2024-09-10T11:31:39.655468+0000","last_active":"2024-09-10T11:31:41.871112+0000","last_peered":"2024-09-10T11:31:41.871112+0000","last_clean":"2024-09-10T11:31:41.871112+0000","last_became_active":"2024-09-10T11:31:39.655169+0000","last_became_peered":"2024-09-10T11:31:39.655169+0000","last_unstale":"2024-09-10T11:31:41.871112+0000","last_undegraded":"2024-09-10T11:31:41.871112+0000","last_fullsized":"2024-09-10T11:31:41.871112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:14.378123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659663+0000","last_change":"2024-09-10T11:31:38.647076+0000","last_active":"2024-09-10T11:31:41.659663+0000","last_peered":"2024-09-10T11:31:41.659663+0000","last_clean":"2024-09-10T11:31:41.659663+0000","last_became_active":"2024-09-10T11:31:38.644372+0000","last_became_peered":"2024-09-10T11:31:38.644372+0000","last_unstale":"2024-09-10T11:31:41.659663+0000","last_undegraded":"2024-09-10T11:31:41.659663+0000","last_fullsized":"2024-09-10T11:31:41.659663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:24.834804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871333+0000","last_change":"2024-09-10T11:31:39.653224+0000","last_active":"2024-09-10T11:31:41.871333+0000","last_peered":"2024-09-10T11:31:41.871333+0000","last_clean":"2024-09-10T11:31:41.871333+0000","last_became_active":"2024-09-10T11:31:39.652149+0000","last_became_peered":"2024-09-10T11:31:39.652149+0000","last_unstale":"2024-09-10T11:31:41.871333+0000","last_undegraded":"2024-09-10T11:31:41.871333+0000","last_fullsized":"2024-09-10T11:31:41.871333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:41.564425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643711+0000","last_change":"2024-09-10T11:31:38.645179+0000","last_active":"2024-09-10T11:31:40.643711+0000","last_peered":"2024-09-10T11:31:40.643711+0000","last_clean":"2024-09-10T11:31:40.643711+0000","last_became_active":"2024-09-10T11:31:38.644585+0000","last_became_peered":"2024-09-10T11:31:38.644585+0000","last_unstale":"2024-09-10T11:31:40.643711+0000","last_undegraded":"2024-09-10T11:31:40.643711+0000","last_fullsized":"2024-09-10T11:31:40.643711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:41.481696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659066+0000","last_change":"2024-09-10T11:31:39.648380+0000","last_active":"2024-09-10T11:31:41.659066+0000","last_peered":"2024-09-10T11:31:41.659066+0000","last_clean":"2024-09-10T11:31:41.659066+0000","last_became_active":"2024-09-10T11:31:39.647706+0000","last_became_peered":"2024-09-10T11:31:39.647706+0000","last_unstale":"2024-09-10T11:31:41.659066+0000","last_undegraded":"2024-09-10T11:31:41.659066+0000","last_fullsized":"2024-09-10T11:31:41.659066+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:01.152814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871691+0000","last_change":"2024-09-10T11:31:38.646450+0000","last_active":"2024-09-10T11:31:41.871691+0000","last_peered":"2024-09-10T11:31:41.871691+0000","last_clean":"2024-09-10T11:31:41.871691+0000","last_became_active":"2024-09-10T11:31:38.645570+0000","last_became_peered":"2024-09-10T11:31:38.645570+0000","last_unstale":"2024-09-10T11:31:41.871691+0000","last_undegraded":"2024-09-10T11:31:41.871691+0000","last_fullsized":"2024-09-10T11:31:41.871691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:18.608550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.651523+0000","last_change":"2024-09-10T11:31:39.651334+0000","last_active":"2024-09-10T11:31:39.651523+0000","last_peered":"2024-09-10T11:31:39.651523+0000","last_clean":"2024-09-10T11:31:39.651523+0000","last_became_active":"2024-09-10T11:31:39.651115+0000","last_became_peered":"2024-09-10T11:31:39.651115+0000","last_unstale":"2024-09-10T11:31:39.651523+0000","last_undegraded":"2024-09-10T11:31:39.651523+0000","last_fullsized":"2024-09-10T11:31:39.651523+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:46.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.649943+0000","last_change":"2024-09-10T11:31:38.644876+0000","last_active":"2024-09-10T11:31:39.649943+0000","last_peered":"2024-09-10T11:31:39.649943+0000","last_clean":"2024-09-10T11:31:39.649943+0000","last_became_active":"2024-09-10T11:31:38.644638+0000","last_became_peered":"2024-09-10T11:31:38.644638+0000","last_unstale":"2024-09-10T11:31:39.649943+0000","last_undegraded":"2024-09-10T11:31:39.649943+0000","last_fullsized":"2024-09-10T11:31:39.649943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:06.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.651506+0000","last_change":"2024-09-10T11:31:39.651235+0000","last_active":"2024-09-10T11:31:39.651506+0000","last_peered":"2024-09-10T11:31:39.651506+0000","last_clean":"2024-09-10T11:31:39.651506+0000","last_became_active":"2024-09-10T11:31:39.650992+0000","last_became_peered":"2024-09-10T11:31:39.650992+0000","last_unstale":"2024-09-10T11:31:39.651506+0000","last_undegraded":"2024-09-10T11:31:39.651506+0000","last_fullsized":"2024-09-10T11:31:39.651506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:55.584918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871445+0000","last_change":"2024-09-10T11:31:38.644477+0000","last_active":"2024-09-10T11:31:41.871445+0000","last_peered":"2024-09-10T11:31:41.871445+0000","last_clean":"2024-09-10T11:31:41.871445+0000","last_became_active":"2024-09-10T11:31:38.644004+0000","last_became_peered":"2024-09-10T11:31:38.644004+0000","last_unstale":"2024-09-10T11:31:41.871445+0000","last_undegraded":"2024-09-10T11:31:41.871445+0000","last_fullsized":"2024-09-10T11:31:41.871445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:30.845972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657805+0000","last_change":"2024-09-10T11:31:39.659038+0000","last_active":"2024-09-10T11:31:41.657805+0000","last_peered":"2024-09-10T11:31:41.657805+0000","last_clean":"2024-09-10T11:31:41.657805+0000","last_became_active":"2024-09-10T11:31:39.658627+0000","last_became_peered":"2024-09-10T11:31:39.658627+0000","last_unstale":"2024-09-10T11:31:41.657805+0000","last_undegraded":"2024-09-10T11:31:41.657805+0000","last_fullsized":"2024-09-10T11:31:41.657805+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:39.568361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646809+0000","last_change":"2024-09-10T11:31:38.641412+0000","last_active":"2024-09-10T11:31:40.646809+0000","last_peered":"2024-09-10T11:31:40.646809+0000","last_clean":"2024-09-10T11:31:40.646809+0000","last_became_active":"2024-09-10T11:31:38.641198+0000","last_became_peered":"2024-09-10T11:31:38.641198+0000","last_unstale":"2024-09-10T11:31:40.646809+0000","last_undegraded":"2024-09-10T11:31:40.646809+0000","last_fullsized":"2024-09-10T11:31:40.646809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:51.831719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659238+0000","last_change":"2024-09-10T11:31:39.654093+0000","last_active":"2024-09-10T11:31:41.659238+0000","last_peered":"2024-09-10T11:31:41.659238+0000","last_clean":"2024-09-10T11:31:41.659238+0000","last_became_active":"2024-09-10T11:31:39.653847+0000","last_became_peered":"2024-09-10T11:31:39.653847+0000","last_unstale":"2024-09-10T11:31:41.659238+0000","last_undegraded":"2024-09-10T11:31:41.659238+0000","last_fullsized":"2024-09-10T11:31:41.659238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:44.950654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871592+0000","last_change":"2024-09-10T11:31:38.650626+0000","last_active":"2024-09-10T11:31:41.871592+0000","last_peered":"2024-09-10T11:31:41.871592+0000","last_clean":"2024-09-10T11:31:41.871592+0000","last_became_active":"2024-09-10T11:31:38.649886+0000","last_became_peered":"2024-09-10T11:31:38.649886+0000","last_unstale":"2024-09-10T11:31:41.871592+0000","last_undegraded":"2024-09-10T11:31:41.871592+0000","last_fullsized":"2024-09-10T11:31:41.871592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:13.577324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655003+0000","last_change":"2024-09-10T11:31:39.654867+0000","last_active":"2024-09-10T11:31:39.655003+0000","last_peered":"2024-09-10T11:31:39.655003+0000","last_clean":"2024-09-10T11:31:39.655003+0000","last_became_active":"2024-09-10T11:31:39.654510+0000","last_became_peered":"2024-09-10T11:31:39.654510+0000","last_unstale":"2024-09-10T11:31:39.655003+0000","last_undegraded":"2024-09-10T11:31:39.655003+0000","last_fullsized":"2024-09-10T11:31:39.655003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:34.064896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470705+0000","last_change":"2024-09-10T11:31:38.645340+0000","last_active":"2024-09-10T11:31:41.470705+0000","last_peered":"2024-09-10T11:31:41.470705+0000","last_clean":"2024-09-10T11:31:41.470705+0000","last_became_active":"2024-09-10T11:31:38.644397+0000","last_became_peered":"2024-09-10T11:31:38.644397+0000","last_unstale":"2024-09-10T11:31:41.470705+0000","last_undegraded":"2024-09-10T11:31:41.470705+0000","last_fullsized":"2024-09-10T11:31:41.470705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:31.391443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470508+0000","last_change":"2024-09-10T11:31:39.644876+0000","last_active":"2024-09-10T11:31:41.470508+0000","last_peered":"2024-09-10T11:31:41.470508+0000","last_clean":"2024-09-10T11:31:41.470508+0000","last_became_active":"2024-09-10T11:31:39.644621+0000","last_became_peered":"2024-09-10T11:31:39.644621+0000","last_unstale":"2024-09-10T11:31:41.470508+0000","last_undegraded":"2024-09-10T11:31:41.470508+0000","last_fullsized":"2024-09-10T11:31:41.470508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:47.080676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644059+0000","last_change":"2024-09-10T11:31:38.644155+0000","last_active":"2024-09-10T11:31:40.644059+0000","last_peered":"2024-09-10T11:31:40.644059+0000","last_clean":"2024-09-10T11:31:40.644059+0000","last_became_active":"2024-09-10T11:31:38.643945+0000","last_became_peered":"2024-09-10T11:31:38.643945+0000","last_unstale":"2024-09-10T11:31:40.644059+0000","last_undegraded":"2024-09-10T11:31:40.644059+0000","last_fullsized":"2024-09-10T11:31:40.644059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:21.520519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470393+0000","last_change":"2024-09-10T11:31:39.647987+0000","last_active":"2024-09-10T11:31:41.470393+0000","last_peered":"2024-09-10T11:31:41.470393+0000","last_clean":"2024-09-10T11:31:41.470393+0000","last_became_active":"2024-09-10T11:31:39.647749+0000","last_became_peered":"2024-09-10T11:31:39.647749+0000","last_unstale":"2024-09-10T11:31:41.470393+0000","last_undegraded":"2024-09-10T11:31:41.470393+0000","last_fullsized":"2024-09-10T11:31:41.470393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:26.433379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641168+0000","last_change":"2024-09-10T11:31:38.650733+0000","last_active":"2024-09-10T11:31:40.641168+0000","last_peered":"2024-09-10T11:31:40.641168+0000","last_clean":"2024-09-10T11:31:40.641168+0000","last_became_active":"2024-09-10T11:31:38.649956+0000","last_became_peered":"2024-09-10T11:31:38.649956+0000","last_unstale":"2024-09-10T11:31:40.641168+0000","last_undegraded":"2024-09-10T11:31:40.641168+0000","last_fullsized":"2024-09-10T11:31:40.641168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:57.508017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646938+0000","last_change":"2024-09-10T11:31:39.646806+0000","last_active":"2024-09-10T11:31:40.646938+0000","last_peered":"2024-09-10T11:31:40.646938+0000","last_clean":"2024-09-10T11:31:40.646938+0000","last_became_active":"2024-09-10T11:31:39.646247+0000","last_became_peered":"2024-09-10T11:31:39.646247+0000","last_unstale":"2024-09-10T11:31:40.646938+0000","last_undegraded":"2024-09-10T11:31:40.646938+0000","last_fullsized":"2024-09-10T11:31:40.646938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:15.258005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871032+0000","last_change":"2024-09-10T11:31:38.649645+0000","last_active":"2024-09-10T11:31:41.871032+0000","last_peered":"2024-09-10T11:31:41.871032+0000","last_clean":"2024-09-10T11:31:41.871032+0000","last_became_active":"2024-09-10T11:31:38.649397+0000","last_became_peered":"2024-09-10T11:31:38.649397+0000","last_unstale":"2024-09-10T11:31:41.871032+0000","last_undegraded":"2024-09-10T11:31:41.871032+0000","last_fullsized":"2024-09-10T11:31:41.871032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:06.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470641+0000","last_change":"2024-09-10T11:31:39.651245+0000","last_active":"2024-09-10T11:31:41.470641+0000","last_peered":"2024-09-10T11:31:41.470641+0000","last_clean":"2024-09-10T11:31:41.470641+0000","last_became_active":"2024-09-10T11:31:39.651006+0000","last_became_peered":"2024-09-10T11:31:39.651006+0000","last_unstale":"2024-09-10T11:31:41.470641+0000","last_undegraded":"2024-09-10T11:31:41.470641+0000","last_fullsized":"2024-09-10T11:31:41.470641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:47.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657907+0000","last_change":"2024-09-10T11:31:38.648863+0000","last_active":"2024-09-10T11:31:41.657907+0000","last_peered":"2024-09-10T11:31:41.657907+0000","last_clean":"2024-09-10T11:31:41.657907+0000","last_became_active":"2024-09-10T11:31:38.648599+0000","last_became_peered":"2024-09-10T11:31:38.648599+0000","last_unstale":"2024-09-10T11:31:41.657907+0000","last_undegraded":"2024-09-10T11:31:41.657907+0000","last_fullsized":"2024-09-10T11:31:41.657907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:33.989466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470512+0000","last_change":"2024-09-10T11:31:39.658976+0000","last_active":"2024-09-10T11:31:41.470512+0000","last_peered":"2024-09-10T11:31:41.470512+0000","last_clean":"2024-09-10T11:31:41.470512+0000","last_became_active":"2024-09-10T11:31:39.658429+0000","last_became_peered":"2024-09-10T11:31:39.658429+0000","last_unstale":"2024-09-10T11:31:41.470512+0000","last_undegraded":"2024-09-10T11:31:41.470512+0000","last_fullsized":"2024-09-10T11:31:41.470512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:44.516767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.661286+0000","last_change":"2024-09-10T11:31:38.647793+0000","last_active":"2024-09-10T11:31:39.661286+0000","last_peered":"2024-09-10T11:31:39.661286+0000","last_clean":"2024-09-10T11:31:39.661286+0000","last_became_active":"2024-09-10T11:31:38.647565+0000","last_became_peered":"2024-09-10T11:31:38.647565+0000","last_unstale":"2024-09-10T11:31:39.661286+0000","last_undegraded":"2024-09-10T11:31:39.661286+0000","last_fullsized":"2024-09-10T11:31:39.661286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:36.572000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.647006+0000","last_change":"2024-09-10T11:31:39.654846+0000","last_active":"2024-09-10T11:31:40.647006+0000","last_peered":"2024-09-10T11:31:40.647006+0000","last_clean":"2024-09-10T11:31:40.647006+0000","last_became_active":"2024-09-10T11:31:39.654329+0000","last_became_peered":"2024-09-10T11:31:39.654329+0000","last_unstale":"2024-09-10T11:31:40.647006+0000","last_undegraded":"2024-09-10T11:31:40.647006+0000","last_fullsized":"2024-09-10T11:31:40.647006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:42.607436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.661693+0000","last_change":"2024-09-10T11:31:38.646374+0000","last_active":"2024-09-10T11:31:39.661693+0000","last_peered":"2024-09-10T11:31:39.661693+0000","last_clean":"2024-09-10T11:31:39.661693+0000","last_became_active":"2024-09-10T11:31:38.646154+0000","last_became_peered":"2024-09-10T11:31:38.646154+0000","last_unstale":"2024-09-10T11:31:39.661693+0000","last_undegraded":"2024-09-10T11:31:39.661693+0000","last_fullsized":"2024-09-10T11:31:39.661693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:27.682580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655591+0000","last_change":"2024-09-10T11:31:39.655470+0000","last_active":"2024-09-10T11:31:39.655591+0000","last_peered":"2024-09-10T11:31:39.655591+0000","last_clean":"2024-09-10T11:31:39.655591+0000","last_became_active":"2024-09-10T11:31:39.655200+0000","last_became_peered":"2024-09-10T11:31:39.655200+0000","last_unstale":"2024-09-10T11:31:39.655591+0000","last_undegraded":"2024-09-10T11:31:39.655591+0000","last_fullsized":"2024-09-10T11:31:39.655591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:00.308890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641246+0000","last_change":"2024-09-10T11:31:38.650913+0000","last_active":"2024-09-10T11:31:40.641246+0000","last_peered":"2024-09-10T11:31:40.641246+0000","last_clean":"2024-09-10T11:31:40.641246+0000","last_became_active":"2024-09-10T11:31:38.650275+0000","last_became_peered":"2024-09-10T11:31:38.650275+0000","last_unstale":"2024-09-10T11:31:40.641246+0000","last_undegraded":"2024-09-10T11:31:40.641246+0000","last_fullsized":"2024-09-10T11:31:40.641246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:24.669282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871491+0000","last_change":"2024-09-10T11:31:39.657948+0000","last_active":"2024-09-10T11:31:41.871491+0000","last_peered":"2024-09-10T11:31:41.871491+0000","last_clean":"2024-09-10T11:31:41.871491+0000","last_became_active":"2024-09-10T11:31:39.657439+0000","last_became_peered":"2024-09-10T11:31:39.657439+0000","last_unstale":"2024-09-10T11:31:41.871491+0000","last_undegraded":"2024-09-10T11:31:41.871491+0000","last_fullsized":"2024-09-10T11:31:41.871491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:58.280079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871110+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871110+0000","last_peered":"2024-09-10T11:31:41.871110+0000","last_clean":"2024-09-10T11:31:41.871110+0000","last_became_active":"2024-09-10T11:31:38.647865+0000","last_became_peered":"2024-09-10T11:31:38.647865+0000","last_unstale":"2024-09-10T11:31:41.871110+0000","last_undegraded":"2024-09-10T11:31:41.871110+0000","last_fullsized":"2024-09-10T11:31:41.871110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:31.961548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641242+0000","last_change":"2024-09-10T11:31:39.644873+0000","last_active":"2024-09-10T11:31:40.641242+0000","last_peered":"2024-09-10T11:31:40.641242+0000","last_clean":"2024-09-10T11:31:40.641242+0000","last_became_active":"2024-09-10T11:31:39.644154+0000","last_became_peered":"2024-09-10T11:31:39.644154+0000","last_unstale":"2024-09-10T11:31:40.641242+0000","last_undegraded":"2024-09-10T11:31:40.641242+0000","last_fullsized":"2024-09-10T11:31:40.641242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:11.485938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643759+0000","last_change":"2024-09-10T11:31:38.648391+0000","last_active":"2024-09-10T11:31:40.643759+0000","last_peered":"2024-09-10T11:31:40.643759+0000","last_clean":"2024-09-10T11:31:40.643759+0000","last_became_active":"2024-09-10T11:31:38.648144+0000","last_became_peered":"2024-09-10T11:31:38.648144+0000","last_unstale":"2024-09-10T11:31:40.643759+0000","last_undegraded":"2024-09-10T11:31:40.643759+0000","last_fullsized":"2024-09-10T11:31:40.643759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:03.992675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658135+0000","last_change":"2024-09-10T11:31:39.654383+0000","last_active":"2024-09-10T11:31:41.658135+0000","last_peered":"2024-09-10T11:31:41.658135+0000","last_clean":"2024-09-10T11:31:41.658135+0000","last_became_active":"2024-09-10T11:31:39.653946+0000","last_became_peered":"2024-09-10T11:31:39.653946+0000","last_unstale":"2024-09-10T11:31:41.658135+0000","last_undegraded":"2024-09-10T11:31:41.658135+0000","last_fullsized":"2024-09-10T11:31:41.658135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:27.993903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871742+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871742+0000","last_peered":"2024-09-10T11:31:41.871742+0000","last_clean":"2024-09-10T11:31:41.871742+0000","last_became_active":"2024-09-10T11:31:38.647836+0000","last_became_peered":"2024-09-10T11:31:38.647836+0000","last_unstale":"2024-09-10T11:31:41.871742+0000","last_undegraded":"2024-09-10T11:31:41.871742+0000","last_fullsized":"2024-09-10T11:31:41.871742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:39.211943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657981+0000","last_change":"2024-09-10T11:31:39.652538+0000","last_active":"2024-09-10T11:31:41.657981+0000","last_peered":"2024-09-10T11:31:41.657981+0000","last_clean":"2024-09-10T11:31:41.657981+0000","last_became_active":"2024-09-10T11:31:39.652254+0000","last_became_peered":"2024-09-10T11:31:39.652254+0000","last_unstale":"2024-09-10T11:31:41.657981+0000","last_undegraded":"2024-09-10T11:31:41.657981+0000","last_fullsized":"2024-09-10T11:31:41.657981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:39.583627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659103+0000","last_change":"2024-09-10T11:31:38.645087+0000","last_active":"2024-09-10T11:31:41.659103+0000","last_peered":"2024-09-10T11:31:41.659103+0000","last_clean":"2024-09-10T11:31:41.659103+0000","last_became_active":"2024-09-10T11:31:38.644513+0000","last_became_peered":"2024-09-10T11:31:38.644513+0000","last_unstale":"2024-09-10T11:31:41.659103+0000","last_undegraded":"2024-09-10T11:31:41.659103+0000","last_fullsized":"2024-09-10T11:31:41.659103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:41.110510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659259+0000","last_change":"2024-09-10T11:31:39.653575+0000","last_active":"2024-09-10T11:31:41.659259+0000","last_peered":"2024-09-10T11:31:41.659259+0000","last_clean":"2024-09-10T11:31:41.659259+0000","last_became_active":"2024-09-10T11:31:39.653326+0000","last_became_peered":"2024-09-10T11:31:39.653326+0000","last_unstale":"2024-09-10T11:31:41.659259+0000","last_undegraded":"2024-09-10T11:31:41.659259+0000","last_fullsized":"2024-09-10T11:31:41.659259+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:18.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659164+0000","last_change":"2024-09-10T11:31:38.646127+0000","last_active":"2024-09-10T11:31:41.659164+0000","last_peered":"2024-09-10T11:31:41.659164+0000","last_clean":"2024-09-10T11:31:41.659164+0000","last_became_active":"2024-09-10T11:31:38.645283+0000","last_became_peered":"2024-09-10T11:31:38.645283+0000","last_unstale":"2024-09-10T11:31:41.659164+0000","last_undegraded":"2024-09-10T11:31:41.659164+0000","last_fullsized":"2024-09-10T11:31:41.659164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:35.070668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871485+0000","last_change":"2024-09-10T11:31:39.652663+0000","last_active":"2024-09-10T11:31:41.871485+0000","last_peered":"2024-09-10T11:31:41.871485+0000","last_clean":"2024-09-10T11:31:41.871485+0000","last_became_active":"2024-09-10T11:31:39.651824+0000","last_became_peered":"2024-09-10T11:31:39.651824+0000","last_unstale":"2024-09-10T11:31:41.871485+0000","last_undegraded":"2024-09-10T11:31:41.871485+0000","last_fullsized":"2024-09-10T11:31:41.871485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:12.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643885+0000","last_change":"2024-09-10T11:31:38.647875+0000","last_active":"2024-09-10T11:31:40.643885+0000","last_peered":"2024-09-10T11:31:40.643885+0000","last_clean":"2024-09-10T11:31:40.643885+0000","last_became_active":"2024-09-10T11:31:38.647001+0000","last_became_peered":"2024-09-10T11:31:38.647001+0000","last_unstale":"2024-09-10T11:31:40.643885+0000","last_undegraded":"2024-09-10T11:31:40.643885+0000","last_fullsized":"2024-09-10T11:31:40.643885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:46.157562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659210+0000","last_change":"2024-09-10T11:31:39.653859+0000","last_active":"2024-09-10T11:31:41.659210+0000","last_peered":"2024-09-10T11:31:41.659210+0000","last_clean":"2024-09-10T11:31:41.659210+0000","last_became_active":"2024-09-10T11:31:39.653630+0000","last_became_peered":"2024-09-10T11:31:39.653630+0000","last_unstale":"2024-09-10T11:31:41.659210+0000","last_undegraded":"2024-09-10T11:31:41.659210+0000","last_fullsized":"2024-09-10T11:31:41.659210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:06.312549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470757+0000","last_change":"2024-09-10T11:31:38.645848+0000","last_active":"2024-09-10T11:31:41.470757+0000","last_peered":"2024-09-10T11:31:41.470757+0000","last_clean":"2024-09-10T11:31:41.470757+0000","last_became_active":"2024-09-10T11:31:38.645636+0000","last_became_peered":"2024-09-10T11:31:38.645636+0000","last_unstale":"2024-09-10T11:31:41.470757+0000","last_undegraded":"2024-09-10T11:31:41.470757+0000","last_fullsized":"2024-09-10T11:31:41.470757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:37.623350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871341+0000","last_change":"2024-09-10T11:31:39.656607+0000","last_active":"2024-09-10T11:31:41.871341+0000","last_peered":"2024-09-10T11:31:41.871341+0000","last_clean":"2024-09-10T11:31:41.871341+0000","last_became_active":"2024-09-10T11:31:39.656322+0000","last_became_peered":"2024-09-10T11:31:39.656322+0000","last_unstale":"2024-09-10T11:31:41.871341+0000","last_undegraded":"2024-09-10T11:31:41.871341+0000","last_fullsized":"2024-09-10T11:31:41.871341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:44.579593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644106+0000","last_change":"2024-09-10T11:31:38.646845+0000","last_active":"2024-09-10T11:31:40.644106+0000","last_peered":"2024-09-10T11:31:40.644106+0000","last_clean":"2024-09-10T11:31:40.644106+0000","last_became_active":"2024-09-10T11:31:38.646550+0000","last_became_peered":"2024-09-10T11:31:38.646550+0000","last_unstale":"2024-09-10T11:31:40.644106+0000","last_undegraded":"2024-09-10T11:31:40.644106+0000","last_fullsized":"2024-09-10T11:31:40.644106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:48.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658289+0000","last_change":"2024-09-10T11:31:39.653111+0000","last_active":"2024-09-10T11:31:41.658289+0000","last_peered":"2024-09-10T11:31:41.658289+0000","last_clean":"2024-09-10T11:31:41.658289+0000","last_became_active":"2024-09-10T11:31:39.651634+0000","last_became_peered":"2024-09-10T11:31:39.651634+0000","last_unstale":"2024-09-10T11:31:41.658289+0000","last_undegraded":"2024-09-10T11:31:41.658289+0000","last_fullsized":"2024-09-10T11:31:41.658289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:16:12.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659399+0000","last_change":"2024-09-10T11:31:38.649507+0000","last_active":"2024-09-10T11:31:41.659399+0000","last_peered":"2024-09-10T11:31:41.659399+0000","last_clean":"2024-09-10T11:31:41.659399+0000","last_became_active":"2024-09-10T11:31:38.649245+0000","last_became_peered":"2024-09-10T11:31:38.649245+0000","last_unstale":"2024-09-10T11:31:41.659399+0000","last_undegraded":"2024-09-10T11:31:41.659399+0000","last_fullsized":"2024-09-10T11:31:41.659399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:28.249420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.653218+0000","last_change":"2024-09-10T11:31:39.653066+0000","last_active":"2024-09-10T11:31:39.653218+0000","last_peered":"2024-09-10T11:31:39.653218+0000","last_clean":"2024-09-10T11:31:39.653218+0000","last_became_active":"2024-09-10T11:31:39.652856+0000","last_became_peered":"2024-09-10T11:31:39.652856+0000","last_unstale":"2024-09-10T11:31:39.653218+0000","last_undegraded":"2024-09-10T11:31:39.653218+0000","last_fullsized":"2024-09-10T11:31:39.653218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:06.455112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641445+0000","last_change":"2024-09-10T11:31:38.649054+0000","last_active":"2024-09-10T11:31:40.641445+0000","last_peered":"2024-09-10T11:31:40.641445+0000","last_clean":"2024-09-10T11:31:40.641445+0000","last_became_active":"2024-09-10T11:31:38.648838+0000","last_became_peered":"2024-09-10T11:31:38.648838+0000","last_unstale":"2024-09-10T11:31:40.641445+0000","last_undegraded":"2024-09-10T11:31:40.641445+0000","last_fullsized":"2024-09-10T11:31:40.641445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:47.015802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646718+0000","last_change":"2024-09-10T11:31:39.649938+0000","last_active":"2024-09-10T11:31:40.646718+0000","last_peered":"2024-09-10T11:31:40.646718+0000","last_clean":"2024-09-10T11:31:40.646718+0000","last_became_active":"2024-09-10T11:31:39.649709+0000","last_became_peered":"2024-09-10T11:31:39.649709+0000","last_unstale":"2024-09-10T11:31:40.646718+0000","last_undegraded":"2024-09-10T11:31:40.646718+0000","last_fullsized":"2024-09-10T11:31:40.646718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:33.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470585+0000","last_change":"2024-09-10T11:31:38.649265+0000","last_active":"2024-09-10T11:31:41.470585+0000","last_peered":"2024-09-10T11:31:41.470585+0000","last_clean":"2024-09-10T11:31:41.470585+0000","last_became_active":"2024-09-10T11:31:38.648733+0000","last_became_peered":"2024-09-10T11:31:38.648733+0000","last_unstale":"2024-09-10T11:31:41.470585+0000","last_undegraded":"2024-09-10T11:31:41.470585+0000","last_fullsized":"2024-09-10T11:31:41.470585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:13.293892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657783+0000","last_change":"2024-09-10T11:31:39.654260+0000","last_active":"2024-09-10T11:31:41.657783+0000","last_peered":"2024-09-10T11:31:41.657783+0000","last_clean":"2024-09-10T11:31:41.657783+0000","last_became_active":"2024-09-10T11:31:39.652267+0000","last_became_peered":"2024-09-10T11:31:39.652267+0000","last_unstale":"2024-09-10T11:31:41.657783+0000","last_undegraded":"2024-09-10T11:31:41.657783+0000","last_fullsized":"2024-09-10T11:31:41.657783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:27.642082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659343+0000","last_change":"2024-09-10T11:31:38.648727+0000","last_active":"2024-09-10T11:31:41.659343+0000","last_peered":"2024-09-10T11:31:41.659343+0000","last_clean":"2024-09-10T11:31:41.659343+0000","last_became_active":"2024-09-10T11:31:38.648279+0000","last_became_peered":"2024-09-10T11:31:38.648279+0000","last_unstale":"2024-09-10T11:31:41.659343+0000","last_undegraded":"2024-09-10T11:31:41.659343+0000","last_fullsized":"2024-09-10T11:31:41.659343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:06.205973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.650272+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:39.650272+0000","last_peered":"2024-09-10T11:31:39.650272+0000","last_clean":"2024-09-10T11:31:39.650272+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:39.650272+0000","last_undegraded":"2024-09-10T11:31:39.650272+0000","last_fullsized":"2024-09-10T11:31:39.650272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199791959,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199201603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":204,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195107,"statfs":{"total":200000000,"available":199790289,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195101,"statfs":{"total":200000000,"available":199784081,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199201555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:43.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T11:31:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.388+0000 7f491895f640 1 Processor -- start 2024-09-10T11:31:43.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.388+0000 7f491895f640 1 -- start start 2024-09-10T11:31:43.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491895f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49140b02a0 0x7f49140aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491895f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49140af3d0 0x7f49140b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491895f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4914073d70 con 0x7f491415ba90 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491895f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f49140b5360 con 0x7f49140af3d0 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491895f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f49140b55f0 con 0x7f49140b02a0 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491259c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f491415ba90 0x7f49140ae780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37058/0 (socket says 172.21.15.99:37058) 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f491259c640 1 -- 172.21.15.99:0/439846794 learned_addr learned my addr 172.21.15.99:0/439846794 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3205720419 0 0) 0x7f4914073d70 con 0x7f491415ba90 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4900003520 con 0x7f491415ba90 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49140b02a0 0x7f49140aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4912d9d640 1 --2- 172.21.15.99:0/439846794 >> v2:172.21.15.196:3300/0 conn(0x7f49140af3d0 0x7f49140b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 523328746 0 0) 0x7f4900003520 con 0x7f491415ba90 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49140b02a0 msgr2=0x7f49140aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49140b02a0 0x7f49140aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 >> v2:172.21.15.196:3300/0 conn(0x7f49140af3d0 msgr2=0x7f49140b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 --2- 172.21.15.99:0/439846794 >> v2:172.21.15.196:3300/0 conn(0x7f49140af3d0 0x7f49140b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49140b0970 con 0x7f491415ba90 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4904001670 con 0x7f491415ba90 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 >> v1:172.21.15.99:6789/0 conn(0x7f491415ba90 legacy=0x7f49140ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49000039b0 0x7f4900003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f4900004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49000051c0 0x7f490000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49140b5360 con 0x7f4900004800 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49140b55f0 con 0x7f49000039b0 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f490000c000 con 0x7f49000051c0 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f4912d9d640 1 --2- 172.21.15.99:0/439846794 >> v2:172.21.15.196:3300/0 conn(0x7f49140af3d0 0x7f49140b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.389+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49140b02a0 0x7f49140aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f491259c640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49000039b0 0x7f4900003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f4900004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f4912d9d640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49000051c0 0x7f490000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 -- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49000051c0 msgr2=0x7f490000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49000051c0 0x7f490000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 -- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49000039b0 msgr2=0x7f4900003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49000039b0 0x7f4900003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f490bfff640 1 -- 172.21.15.99:0/439846794 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4900003520 con 0x7f4900004800 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.390+0000 7f4912d9d640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49000051c0 0x7f490000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f490bfff640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f4900004c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f491415f620 tx=0x7f4900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6e2582890c0375 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc037070 con 0x7f4900004800 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48fc02ab10 con 0x7f4900004800 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f4911d9b640 1 -- 172.21.15.99:0/439846794 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00f040 con 0x7f4900004800 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 -- 172.21.15.99:0/439846794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 msgr2=0x7f4900004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 --2- 172.21.15.99:0/439846794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f4900004c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f491415f620 tx=0x7f4900005830 comp rx=0 tx=0).stop 2024-09-10T11:31:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f4912d9d640 1 -- 172.21.15.99:0/439846794 reap_dead start 2024-09-10T11:31:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 -- 172.21.15.99:0/439846794 shutdown_connections 2024-09-10T11:31:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 -- 172.21.15.99:0/439846794 >> 172.21.15.99:0/439846794 conn(0x7f49140730c0 msgr2=0x7f49140734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 -- 172.21.15.99:0/439846794 shutdown_connections 2024-09-10T11:31:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.391+0000 7f491895f640 1 -- 172.21.15.99:0/439846794 wait complete. 2024-09-10T11:31:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 Processor -- start 2024-09-10T11:31:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 -- start start 2024-09-10T11:31:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f491414a550 0x7f491409e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4914152180 0x7f4914152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f491415be70 con 0x7f4900004800 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f49140b0f30 con 0x7f4914152180 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491895f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f491415bff0 con 0x7f491414a550 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60502/0 (socket says 172.21.15.99:60502) 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.393+0000 7f491259c640 1 -- 172.21.15.99:0/4167770092 learned_addr learned my addr 172.21.15.99:0/4167770092 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 -- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f491414a550 msgr2=0x7f491409e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f491414a550 0x7f491409e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 -- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4914152180 msgr2=0x7f4914152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4914152180 0x7f4914152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 -- 172.21.15.99:0/4167770092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491409b3f0 con 0x7f4900004800 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491259c640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4904007340 tx=0x7f4904007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490401a070 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490400f8c0 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491409b6d0 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490400fc90 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.394+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491409bc90 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.395+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49140ae780 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.395+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f490401d070 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.396+0000 7f490affd640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 0x7f48f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.396+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f490408a770 con 0x7f4900004800 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.396+0000 7f490bfff640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 0x7f48f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.396+0000 7f490bfff640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 0x7f48f406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48fc034000 tx=0x7f48fc02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.400+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49040591b0 con 0x7f4900004800 2024-09-10T11:31:43.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.629+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f491406ea60 con 0x7f4900004800 2024-09-10T11:31:43.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.629+0000 7f490affd640 1 -- 172.21.15.99:0/4167770092 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1606 (secure 0 0 0) 0x7f4904058ab0 con 0x7f4900004800 2024-09-10T11:31:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:43.630 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":92,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1725967815,"num_in_osds":8,"osd_in_since":1725967812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2571552,"bytes_avail":1597428448,"bytes_total":1600000000,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5211}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T11:31:16.920785+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 msgr2=0x7f48f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 0x7f48f406e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48fc034000 tx=0x7f48fc02c000 comp rx=0 tx=0).stop 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 msgr2=0x7f491406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4904007340 tx=0x7f4904007800 comp rx=0 tx=0).stop 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 shutdown_connections 2024-09-10T11:31:43.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48f406be30 0x7f48f406e2f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4914152180 0x7f4914152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f491414a550 0x7f491409e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 --2- 172.21.15.99:0/4167770092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4900004800 0x7f491406f110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 >> 172.21.15.99:0/4167770092 conn(0x7f49140730c0 msgr2=0x7f49140ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 shutdown_connections 2024-09-10T11:31:43.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.631+0000 7f491895f640 1 -- 172.21.15.99:0/4167770092 wait complete. 2024-09-10T11:31:43.643 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T11:31:43.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:43.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.741+0000 7f0141c8d640 1 Processor -- start 2024-09-10T11:31:43.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.741+0000 7f0141c8d640 1 -- start start 2024-09-10T11:31:43.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f0141c8d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 0x7f013c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f0141c8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f013c06e5d0 0x7f013c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f0141c8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013c152e60 con 0x7f013c15c060 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f0141c8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f013c06ef80 con 0x7f013c06e5d0 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f0141c8d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f013c0d15e0 con 0x7f013c0d1200 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f013affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f013c06e5d0 0x7f013c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f013bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f013c15c060 0x7f013c15c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37066/0 (socket says 172.21.15.99:37066) 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.742+0000 7f013bfff640 1 -- 172.21.15.99:0/3225598961 learned_addr learned my addr 172.21.15.99:0/3225598961 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013affd640 1 -- 172.21.15.99:0/3225598961 >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 msgr2=0x7f013c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013b7fe640 1 --2- 172.21.15.99:0/3225598961 >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 0x7f013c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013affd640 1 --2- 172.21.15.99:0/3225598961 >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 0x7f013c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013affd640 1 -- 172.21.15.99:0/3225598961 >> v1:172.21.15.99:6789/0 conn(0x7f013c15c060 legacy=0x7f013c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013a7fc640 1 -- 172.21.15.99:0/3225598961 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414972417 0 0) 0x7f013c152e60 con 0x7f013c15c060 2024-09-10T11:31:43.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013affd640 1 -- 172.21.15.99:0/3225598961 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c072680 con 0x7f013c06e5d0 2024-09-10T11:31:43.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013b7fe640 1 --2- 172.21.15.99:0/3225598961 >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 0x7f013c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:43.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013affd640 1 --2- 172.21.15.99:0/3225598961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c06e5d0 0x7f013c06ea40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f013c0d1850 tx=0x7f013002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6110a0cf911d6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013a7fc640 1 -- 172.21.15.99:0/3225598961 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130037070 con 0x7f013c06e5d0 2024-09-10T11:31:43.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.743+0000 7f013a7fc640 1 -- 172.21.15.99:0/3225598961 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130032440 con 0x7f013c06e5d0 2024-09-10T11:31:43.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f013a7fc640 1 -- 172.21.15.99:0/3225598961 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013002ae60 con 0x7f013c06e5d0 2024-09-10T11:31:43.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 -- 172.21.15.99:0/3225598961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c06e5d0 msgr2=0x7f013c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 --2- 172.21.15.99:0/3225598961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c06e5d0 0x7f013c06ea40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f013c0d1850 tx=0x7f013002a110 comp rx=0 tx=0).stop 2024-09-10T11:31:43.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 -- 172.21.15.99:0/3225598961 shutdown_connections 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 --2- 172.21.15.99:0/3225598961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c06e5d0 0x7f013c06ea40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 --2- 172.21.15.99:0/3225598961 >> v2:172.21.15.196:3300/0 conn(0x7f013c0d1200 0x7f013c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 -- 172.21.15.99:0/3225598961 >> 172.21.15.99:0/3225598961 conn(0x7f013c06bc10 msgr2=0x7f013c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 -- 172.21.15.99:0/3225598961 shutdown_connections 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.744+0000 7f0141c8d640 1 -- 172.21.15.99:0/3225598961 wait complete. 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.745+0000 7f0141c8d640 1 Processor -- start 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.745+0000 7f0141c8d640 1 -- start start 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f013c06e5d0 0x7f013c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f013c15c060 0x7f013c0cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f013c072f20 con 0x7f013c0d1200 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f013c072c40 con 0x7f013c15c060 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f0141c8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f013c15cd90 con 0x7f013c06e5d0 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f013affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f013affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60520/0 (socket says 172.21.15.99:60520) 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f013affd640 1 -- 172.21.15.99:0/2429030688 learned_addr learned my addr 172.21.15.99:0/2429030688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f013b7fe640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f013c06e5d0 0x7f013c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.746+0000 7f013bfff640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f013c15c060 0x7f013c0cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 -- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f013c06e5d0 msgr2=0x7f013c14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f013c06e5d0 0x7f013c14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 -- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f013c15c060 msgr2=0x7f013c0cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f013c15c060 0x7f013c0cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 -- 172.21.15.99:0/2429030688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1f2010 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f013affd640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f01300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130037040 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130042070 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c14e730 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130030a50 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.747+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c156f30 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.748+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c06ea40 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.748+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0130030bf0 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.749+0000 7f0123fff640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 0x7f012806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.749+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f01300ab910 con 0x7f013c0d1200 2024-09-10T11:31:43.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.749+0000 7f013b7fe640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 0x7f012806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:43.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.749+0000 7f013b7fe640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 0x7f012806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f012c007510 tx=0x7f012c0074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:43.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.753+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f013007a2c0 con 0x7f013c0d1200 2024-09-10T11:31:43.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.915+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f013c154340 con 0x7f012806be30 2024-09-10T11:31:43.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.921+0000 7f0123fff640 1 -- 172.21.15.99:0/2429030688 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247480 (secure 0 0 0) 0x7f013c154340 con 0x7f012806be30 2024-09-10T11:31:43.924 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:43.925 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 msgr2=0x7f012806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 0x7f012806e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f012c007510 tx=0x7f012c0074a0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 msgr2=0x7f013c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f01300046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 shutdown_connections 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f012806be30 0x7f012806e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f013c15c060 0x7f013c0cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f013c0d1200 0x7f013c1565d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 --2- 172.21.15.99:0/2429030688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f013c06e5d0 0x7f013c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 >> 172.21.15.99:0/2429030688 conn(0x7f013c06bc10 msgr2=0x7f013c151ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:43.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 shutdown_connections 2024-09-10T11:31:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:43.924+0000 7f0141c8d640 1 -- 172.21.15.99:0/2429030688 wait complete. 2024-09-10T11:31:43.935 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-10T11:31:42.924724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":102,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2807,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559692,"statfs":{"total":1600000000,"available":1597125188,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.291293"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643782+0000","last_change":"2024-09-10T11:31:39.645497+0000","last_active":"2024-09-10T11:31:40.643782+0000","last_peered":"2024-09-10T11:31:40.643782+0000","last_clean":"2024-09-10T11:31:40.643782+0000","last_became_active":"2024-09-10T11:31:39.645175+0000","last_became_peered":"2024-09-10T11:31:39.645175+0000","last_unstale":"2024-09-10T11:31:40.643782+0000","last_undegraded":"2024-09-10T11:31:40.643782+0000","last_fullsized":"2024-09-10T11:31:40.643782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:57.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646626+0000","last_change":"2024-09-10T11:31:39.653090+0000","last_active":"2024-09-10T11:31:40.646626+0000","last_peered":"2024-09-10T11:31:40.646626+0000","last_clean":"2024-09-10T11:31:40.646626+0000","last_became_active":"2024-09-10T11:31:39.652865+0000","last_became_peered":"2024-09-10T11:31:39.652865+0000","last_unstale":"2024-09-10T11:31:40.646626+0000","last_undegraded":"2024-09-10T11:31:40.646626+0000","last_fullsized":"2024-09-10T11:31:40.646626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:08.493521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655676+0000","last_change":"2024-09-10T11:31:39.655491+0000","last_active":"2024-09-10T11:31:39.655676+0000","last_peered":"2024-09-10T11:31:39.655676+0000","last_clean":"2024-09-10T11:31:39.655676+0000","last_became_active":"2024-09-10T11:31:39.655120+0000","last_became_peered":"2024-09-10T11:31:39.655120+0000","last_unstale":"2024-09-10T11:31:39.655676+0000","last_undegraded":"2024-09-10T11:31:39.655676+0000","last_fullsized":"2024-09-10T11:31:39.655676+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:45.132618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657665+0000","last_change":"2024-09-10T11:31:39.650249+0000","last_active":"2024-09-10T11:31:41.657665+0000","last_peered":"2024-09-10T11:31:41.657665+0000","last_clean":"2024-09-10T11:31:41.657665+0000","last_became_active":"2024-09-10T11:31:39.649905+0000","last_became_peered":"2024-09-10T11:31:39.649905+0000","last_unstale":"2024-09-10T11:31:41.657665+0000","last_undegraded":"2024-09-10T11:31:41.657665+0000","last_fullsized":"2024-09-10T11:31:41.657665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:09.765200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643841+0000","last_change":"2024-09-10T11:31:39.646393+0000","last_active":"2024-09-10T11:31:40.643841+0000","last_peered":"2024-09-10T11:31:40.643841+0000","last_clean":"2024-09-10T11:31:40.643841+0000","last_became_active":"2024-09-10T11:31:39.646154+0000","last_became_peered":"2024-09-10T11:31:39.646154+0000","last_unstale":"2024-09-10T11:31:40.643841+0000","last_undegraded":"2024-09-10T11:31:40.643841+0000","last_fullsized":"2024-09-10T11:31:40.643841+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:38.360639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641243+0000","last_change":"2024-09-10T11:31:39.654253+0000","last_active":"2024-09-10T11:31:40.641243+0000","last_peered":"2024-09-10T11:31:40.641243+0000","last_clean":"2024-09-10T11:31:40.641243+0000","last_became_active":"2024-09-10T11:31:39.653964+0000","last_became_peered":"2024-09-10T11:31:39.653964+0000","last_unstale":"2024-09-10T11:31:40.641243+0000","last_undegraded":"2024-09-10T11:31:40.641243+0000","last_fullsized":"2024-09-10T11:31:40.641243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:46.259879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658993+0000","last_change":"2024-09-10T11:31:39.656014+0000","last_active":"2024-09-10T11:31:41.658993+0000","last_peered":"2024-09-10T11:31:41.658993+0000","last_clean":"2024-09-10T11:31:41.658993+0000","last_became_active":"2024-09-10T11:31:39.655797+0000","last_became_peered":"2024-09-10T11:31:39.655797+0000","last_unstale":"2024-09-10T11:31:41.658993+0000","last_undegraded":"2024-09-10T11:31:41.658993+0000","last_fullsized":"2024-09-10T11:31:41.658993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.186968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657591+0000","last_change":"2024-09-10T11:31:39.654806+0000","last_active":"2024-09-10T11:31:41.657591+0000","last_peered":"2024-09-10T11:31:41.657591+0000","last_clean":"2024-09-10T11:31:41.657591+0000","last_became_active":"2024-09-10T11:31:39.653029+0000","last_became_peered":"2024-09-10T11:31:39.653029+0000","last_unstale":"2024-09-10T11:31:41.657591+0000","last_undegraded":"2024-09-10T11:31:41.657591+0000","last_fullsized":"2024-09-10T11:31:41.657591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.640172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641152+0000","last_change":"2024-09-10T11:31:39.658665+0000","last_active":"2024-09-10T11:31:40.641152+0000","last_peered":"2024-09-10T11:31:40.641152+0000","last_clean":"2024-09-10T11:31:40.641152+0000","last_became_active":"2024-09-10T11:31:39.658365+0000","last_became_peered":"2024-09-10T11:31:39.658365+0000","last_unstale":"2024-09-10T11:31:40.641152+0000","last_undegraded":"2024-09-10T11:31:40.641152+0000","last_fullsized":"2024-09-10T11:31:40.641152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:22.800217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871184+0000","last_change":"2024-09-10T11:31:39.656494+0000","last_active":"2024-09-10T11:31:41.871184+0000","last_peered":"2024-09-10T11:31:41.871184+0000","last_clean":"2024-09-10T11:31:41.871184+0000","last_became_active":"2024-09-10T11:31:39.656264+0000","last_became_peered":"2024-09-10T11:31:39.656264+0000","last_unstale":"2024-09-10T11:31:41.871184+0000","last_undegraded":"2024-09-10T11:31:41.871184+0000","last_fullsized":"2024-09-10T11:31:41.871184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:54.180344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657795+0000","last_change":"2024-09-10T11:31:39.653639+0000","last_active":"2024-09-10T11:31:41.657795+0000","last_peered":"2024-09-10T11:31:41.657795+0000","last_clean":"2024-09-10T11:31:41.657795+0000","last_became_active":"2024-09-10T11:31:39.652232+0000","last_became_peered":"2024-09-10T11:31:39.652232+0000","last_unstale":"2024-09-10T11:31:41.657795+0000","last_undegraded":"2024-09-10T11:31:41.657795+0000","last_fullsized":"2024-09-10T11:31:41.657795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:10.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646762+0000","last_change":"2024-09-10T11:31:39.647797+0000","last_active":"2024-09-10T11:31:40.646762+0000","last_peered":"2024-09-10T11:31:40.646762+0000","last_clean":"2024-09-10T11:31:40.646762+0000","last_became_active":"2024-09-10T11:31:39.647576+0000","last_became_peered":"2024-09-10T11:31:39.647576+0000","last_unstale":"2024-09-10T11:31:40.646762+0000","last_undegraded":"2024-09-10T11:31:40.646762+0000","last_fullsized":"2024-09-10T11:31:40.646762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:42.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641057+0000","last_change":"2024-09-10T11:31:39.648764+0000","last_active":"2024-09-10T11:31:40.641057+0000","last_peered":"2024-09-10T11:31:40.641057+0000","last_clean":"2024-09-10T11:31:40.641057+0000","last_became_active":"2024-09-10T11:31:39.648525+0000","last_became_peered":"2024-09-10T11:31:39.648525+0000","last_unstale":"2024-09-10T11:31:40.641057+0000","last_undegraded":"2024-09-10T11:31:40.641057+0000","last_fullsized":"2024-09-10T11:31:40.641057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:15.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658043+0000","last_change":"2024-09-10T11:31:39.658849+0000","last_active":"2024-09-10T11:31:41.658043+0000","last_peered":"2024-09-10T11:31:41.658043+0000","last_clean":"2024-09-10T11:31:41.658043+0000","last_became_active":"2024-09-10T11:31:39.658315+0000","last_became_peered":"2024-09-10T11:31:39.658315+0000","last_unstale":"2024-09-10T11:31:41.658043+0000","last_undegraded":"2024-09-10T11:31:41.658043+0000","last_fullsized":"2024-09-10T11:31:41.658043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:18.791072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657667+0000","last_change":"2024-09-10T11:31:39.655320+0000","last_active":"2024-09-10T11:31:41.657667+0000","last_peered":"2024-09-10T11:31:41.657667+0000","last_clean":"2024-09-10T11:31:41.657667+0000","last_became_active":"2024-09-10T11:31:39.654324+0000","last_became_peered":"2024-09-10T11:31:39.654324+0000","last_unstale":"2024-09-10T11:31:41.657667+0000","last_undegraded":"2024-09-10T11:31:41.657667+0000","last_fullsized":"2024-09-10T11:31:41.657667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:39.168999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643984+0000","last_change":"2024-09-10T11:31:39.656422+0000","last_active":"2024-09-10T11:31:40.643984+0000","last_peered":"2024-09-10T11:31:40.643984+0000","last_clean":"2024-09-10T11:31:40.643984+0000","last_became_active":"2024-09-10T11:31:39.656184+0000","last_became_peered":"2024-09-10T11:31:39.656184+0000","last_unstale":"2024-09-10T11:31:40.643984+0000","last_undegraded":"2024-09-10T11:31:40.643984+0000","last_fullsized":"2024-09-10T11:31:40.643984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:15.186401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646872+0000","last_change":"2024-09-10T11:31:39.647034+0000","last_active":"2024-09-10T11:31:40.646872+0000","last_peered":"2024-09-10T11:31:40.646872+0000","last_clean":"2024-09-10T11:31:40.646872+0000","last_became_active":"2024-09-10T11:31:39.646566+0000","last_became_peered":"2024-09-10T11:31:39.646566+0000","last_unstale":"2024-09-10T11:31:40.646872+0000","last_undegraded":"2024-09-10T11:31:40.646872+0000","last_fullsized":"2024-09-10T11:31:40.646872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:44.052632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655420+0000","last_change":"2024-09-10T11:31:39.655310+0000","last_active":"2024-09-10T11:31:39.655420+0000","last_peered":"2024-09-10T11:31:39.655420+0000","last_clean":"2024-09-10T11:31:39.655420+0000","last_became_active":"2024-09-10T11:31:39.654844+0000","last_became_peered":"2024-09-10T11:31:39.654844+0000","last_unstale":"2024-09-10T11:31:39.655420+0000","last_undegraded":"2024-09-10T11:31:39.655420+0000","last_fullsized":"2024-09-10T11:31:39.655420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:44.558181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643835+0000","last_change":"2024-09-10T11:31:39.645579+0000","last_active":"2024-09-10T11:31:40.643835+0000","last_peered":"2024-09-10T11:31:40.643835+0000","last_clean":"2024-09-10T11:31:40.643835+0000","last_became_active":"2024-09-10T11:31:39.645354+0000","last_became_peered":"2024-09-10T11:31:39.645354+0000","last_unstale":"2024-09-10T11:31:40.643835+0000","last_undegraded":"2024-09-10T11:31:40.643835+0000","last_fullsized":"2024-09-10T11:31:40.643835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:49.467444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871633+0000","last_change":"2024-09-10T11:31:39.654118+0000","last_active":"2024-09-10T11:31:41.871633+0000","last_peered":"2024-09-10T11:31:41.871633+0000","last_clean":"2024-09-10T11:31:41.871633+0000","last_became_active":"2024-09-10T11:31:39.652435+0000","last_became_peered":"2024-09-10T11:31:39.652435+0000","last_unstale":"2024-09-10T11:31:41.871633+0000","last_undegraded":"2024-09-10T11:31:41.871633+0000","last_fullsized":"2024-09-10T11:31:41.871633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:31.098768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658543+0000","last_change":"2024-09-10T11:31:39.652924+0000","last_active":"2024-09-10T11:31:41.658543+0000","last_peered":"2024-09-10T11:31:41.658543+0000","last_clean":"2024-09-10T11:31:41.658543+0000","last_became_active":"2024-09-10T11:31:39.650464+0000","last_became_peered":"2024-09-10T11:31:39.650464+0000","last_unstale":"2024-09-10T11:31:41.658543+0000","last_undegraded":"2024-09-10T11:31:41.658543+0000","last_fullsized":"2024-09-10T11:31:41.658543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:17.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470652+0000","last_change":"2024-09-10T11:31:39.650521+0000","last_active":"2024-09-10T11:31:41.470652+0000","last_peered":"2024-09-10T11:31:41.470652+0000","last_clean":"2024-09-10T11:31:41.470652+0000","last_became_active":"2024-09-10T11:31:39.650297+0000","last_became_peered":"2024-09-10T11:31:39.650297+0000","last_unstale":"2024-09-10T11:31:41.470652+0000","last_undegraded":"2024-09-10T11:31:41.470652+0000","last_fullsized":"2024-09-10T11:31:41.470652+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:39.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659573+0000","last_change":"2024-09-10T11:31:39.650038+0000","last_active":"2024-09-10T11:31:41.659573+0000","last_peered":"2024-09-10T11:31:41.659573+0000","last_clean":"2024-09-10T11:31:41.659573+0000","last_became_active":"2024-09-10T11:31:39.649807+0000","last_became_peered":"2024-09-10T11:31:39.649807+0000","last_unstale":"2024-09-10T11:31:41.659573+0000","last_undegraded":"2024-09-10T11:31:41.659573+0000","last_fullsized":"2024-09-10T11:31:41.659573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:04.977562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646605+0000","last_change":"2024-09-10T11:31:39.651240+0000","last_active":"2024-09-10T11:31:40.646605+0000","last_peered":"2024-09-10T11:31:40.646605+0000","last_clean":"2024-09-10T11:31:40.646605+0000","last_became_active":"2024-09-10T11:31:39.651015+0000","last_became_peered":"2024-09-10T11:31:39.651015+0000","last_unstale":"2024-09-10T11:31:40.646605+0000","last_undegraded":"2024-09-10T11:31:40.646605+0000","last_fullsized":"2024-09-10T11:31:40.646605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:50.935894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643988+0000","last_change":"2024-09-10T11:31:39.645360+0000","last_active":"2024-09-10T11:31:40.643988+0000","last_peered":"2024-09-10T11:31:40.643988+0000","last_clean":"2024-09-10T11:31:40.643988+0000","last_became_active":"2024-09-10T11:31:39.645089+0000","last_became_peered":"2024-09-10T11:31:39.645089+0000","last_unstale":"2024-09-10T11:31:40.643988+0000","last_undegraded":"2024-09-10T11:31:40.643988+0000","last_fullsized":"2024-09-10T11:31:40.643988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:05.904582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657992+0000","last_change":"2024-09-10T11:31:39.653442+0000","last_active":"2024-09-10T11:31:41.657992+0000","last_peered":"2024-09-10T11:31:41.657992+0000","last_clean":"2024-09-10T11:31:41.657992+0000","last_became_active":"2024-09-10T11:31:39.651982+0000","last_became_peered":"2024-09-10T11:31:39.651982+0000","last_unstale":"2024-09-10T11:31:41.657992+0000","last_undegraded":"2024-09-10T11:31:41.657992+0000","last_fullsized":"2024-09-10T11:31:41.657992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:17.090501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659162+0000","last_change":"2024-09-10T11:31:39.650720+0000","last_active":"2024-09-10T11:31:41.659162+0000","last_peered":"2024-09-10T11:31:41.659162+0000","last_clean":"2024-09-10T11:31:41.659162+0000","last_became_active":"2024-09-10T11:31:39.650477+0000","last_became_peered":"2024-09-10T11:31:39.650477+0000","last_unstale":"2024-09-10T11:31:41.659162+0000","last_undegraded":"2024-09-10T11:31:41.659162+0000","last_fullsized":"2024-09-10T11:31:41.659162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:54.152529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659073+0000","last_change":"2024-09-10T11:31:39.651780+0000","last_active":"2024-09-10T11:31:41.659073+0000","last_peered":"2024-09-10T11:31:41.659073+0000","last_clean":"2024-09-10T11:31:41.659073+0000","last_became_active":"2024-09-10T11:31:39.651388+0000","last_became_peered":"2024-09-10T11:31:39.651388+0000","last_unstale":"2024-09-10T11:31:41.659073+0000","last_undegraded":"2024-09-10T11:31:41.659073+0000","last_fullsized":"2024-09-10T11:31:41.659073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:16.395096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643642+0000","last_change":"2024-09-10T11:31:39.658219+0000","last_active":"2024-09-10T11:31:40.643642+0000","last_peered":"2024-09-10T11:31:40.643642+0000","last_clean":"2024-09-10T11:31:40.643642+0000","last_became_active":"2024-09-10T11:31:39.657959+0000","last_became_peered":"2024-09-10T11:31:39.657959+0000","last_unstale":"2024-09-10T11:31:40.643642+0000","last_undegraded":"2024-09-10T11:31:40.643642+0000","last_fullsized":"2024-09-10T11:31:40.643642+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:38.467359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646828+0000","last_change":"2024-09-10T11:31:39.646856+0000","last_active":"2024-09-10T11:31:40.646828+0000","last_peered":"2024-09-10T11:31:40.646828+0000","last_clean":"2024-09-10T11:31:40.646828+0000","last_became_active":"2024-09-10T11:31:39.646630+0000","last_became_peered":"2024-09-10T11:31:39.646630+0000","last_unstale":"2024-09-10T11:31:40.646828+0000","last_undegraded":"2024-09-10T11:31:40.646828+0000","last_fullsized":"2024-09-10T11:31:40.646828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:27.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641394+0000","last_change":"2024-09-10T11:31:39.644641+0000","last_active":"2024-09-10T11:31:40.641394+0000","last_peered":"2024-09-10T11:31:40.641394+0000","last_clean":"2024-09-10T11:31:40.641394+0000","last_became_active":"2024-09-10T11:31:39.644390+0000","last_became_peered":"2024-09-10T11:31:39.644390+0000","last_unstale":"2024-09-10T11:31:40.641394+0000","last_undegraded":"2024-09-10T11:31:40.641394+0000","last_fullsized":"2024-09-10T11:31:40.641394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:55.588242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657857+0000","last_change":"2024-09-10T11:31:39.654918+0000","last_active":"2024-09-10T11:31:41.657857+0000","last_peered":"2024-09-10T11:31:41.657857+0000","last_clean":"2024-09-10T11:31:41.657857+0000","last_became_active":"2024-09-10T11:31:39.654587+0000","last_became_peered":"2024-09-10T11:31:39.654587+0000","last_unstale":"2024-09-10T11:31:41.657857+0000","last_undegraded":"2024-09-10T11:31:41.657857+0000","last_fullsized":"2024-09-10T11:31:41.657857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:40.331083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470394+0000","last_change":"2024-09-10T11:31:39.659286+0000","last_active":"2024-09-10T11:31:41.470394+0000","last_peered":"2024-09-10T11:31:41.470394+0000","last_clean":"2024-09-10T11:31:41.470394+0000","last_became_active":"2024-09-10T11:31:39.658774+0000","last_became_peered":"2024-09-10T11:31:39.658774+0000","last_unstale":"2024-09-10T11:31:41.470394+0000","last_undegraded":"2024-09-10T11:31:41.470394+0000","last_fullsized":"2024-09-10T11:31:41.470394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:30.202847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641353+0000","last_change":"2024-09-10T11:31:38.648561+0000","last_active":"2024-09-10T11:31:40.641353+0000","last_peered":"2024-09-10T11:31:40.641353+0000","last_clean":"2024-09-10T11:31:40.641353+0000","last_became_active":"2024-09-10T11:31:38.647403+0000","last_became_peered":"2024-09-10T11:31:38.647403+0000","last_unstale":"2024-09-10T11:31:40.641353+0000","last_undegraded":"2024-09-10T11:31:40.641353+0000","last_fullsized":"2024-09-10T11:31:40.641353+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:33.760518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644028+0000","last_change":"2024-09-10T11:31:39.645767+0000","last_active":"2024-09-10T11:31:40.644028+0000","last_peered":"2024-09-10T11:31:40.644028+0000","last_clean":"2024-09-10T11:31:40.644028+0000","last_became_active":"2024-09-10T11:31:39.645548+0000","last_became_peered":"2024-09-10T11:31:39.645548+0000","last_unstale":"2024-09-10T11:31:40.644028+0000","last_undegraded":"2024-09-10T11:31:40.644028+0000","last_fullsized":"2024-09-10T11:31:40.644028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.922464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659303+0000","last_change":"2024-09-10T11:31:38.644985+0000","last_active":"2024-09-10T11:31:41.659303+0000","last_peered":"2024-09-10T11:31:41.659303+0000","last_clean":"2024-09-10T11:31:41.659303+0000","last_became_active":"2024-09-10T11:31:38.644751+0000","last_became_peered":"2024-09-10T11:31:38.644751+0000","last_unstale":"2024-09-10T11:31:41.659303+0000","last_undegraded":"2024-09-10T11:31:41.659303+0000","last_fullsized":"2024-09-10T11:31:41.659303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:39.728019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646869+0000","last_change":"2024-09-10T11:31:39.649105+0000","last_active":"2024-09-10T11:31:40.646869+0000","last_peered":"2024-09-10T11:31:40.646869+0000","last_clean":"2024-09-10T11:31:40.646869+0000","last_became_active":"2024-09-10T11:31:39.648513+0000","last_became_peered":"2024-09-10T11:31:39.648513+0000","last_unstale":"2024-09-10T11:31:40.646869+0000","last_undegraded":"2024-09-10T11:31:40.646869+0000","last_fullsized":"2024-09-10T11:31:40.646869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:30.397629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641324+0000","last_change":"2024-09-10T11:31:38.647068+0000","last_active":"2024-09-10T11:31:40.641324+0000","last_peered":"2024-09-10T11:31:40.641324+0000","last_clean":"2024-09-10T11:31:40.641324+0000","last_became_active":"2024-09-10T11:31:38.646806+0000","last_became_peered":"2024-09-10T11:31:38.646806+0000","last_unstale":"2024-09-10T11:31:40.641324+0000","last_undegraded":"2024-09-10T11:31:40.641324+0000","last_fullsized":"2024-09-10T11:31:40.641324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:41.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657848+0000","last_change":"2024-09-10T11:31:39.655753+0000","last_active":"2024-09-10T11:31:41.657848+0000","last_peered":"2024-09-10T11:31:41.657848+0000","last_clean":"2024-09-10T11:31:41.657848+0000","last_became_active":"2024-09-10T11:31:39.654854+0000","last_became_peered":"2024-09-10T11:31:39.654854+0000","last_unstale":"2024-09-10T11:31:41.657848+0000","last_undegraded":"2024-09-10T11:31:41.657848+0000","last_fullsized":"2024-09-10T11:31:41.657848+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:02.970230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658983+0000","last_change":"2024-09-10T11:31:38.645523+0000","last_active":"2024-09-10T11:31:41.658983+0000","last_peered":"2024-09-10T11:31:41.658983+0000","last_clean":"2024-09-10T11:31:41.658983+0000","last_became_active":"2024-09-10T11:31:38.644841+0000","last_became_peered":"2024-09-10T11:31:38.644841+0000","last_unstale":"2024-09-10T11:31:41.658983+0000","last_undegraded":"2024-09-10T11:31:41.658983+0000","last_fullsized":"2024-09-10T11:31:41.658983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:44.618873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659386+0000","last_change":"2024-09-10T11:31:39.651607+0000","last_active":"2024-09-10T11:31:41.659386+0000","last_peered":"2024-09-10T11:31:41.659386+0000","last_clean":"2024-09-10T11:31:41.659386+0000","last_became_active":"2024-09-10T11:31:39.651070+0000","last_became_peered":"2024-09-10T11:31:39.651070+0000","last_unstale":"2024-09-10T11:31:41.659386+0000","last_undegraded":"2024-09-10T11:31:41.659386+0000","last_fullsized":"2024-09-10T11:31:41.659386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:03.428955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641087+0000","last_change":"2024-09-10T11:31:38.650460+0000","last_active":"2024-09-10T11:31:40.641087+0000","last_peered":"2024-09-10T11:31:40.641087+0000","last_clean":"2024-09-10T11:31:40.641087+0000","last_became_active":"2024-09-10T11:31:38.649203+0000","last_became_peered":"2024-09-10T11:31:38.649203+0000","last_unstale":"2024-09-10T11:31:40.641087+0000","last_undegraded":"2024-09-10T11:31:40.641087+0000","last_fullsized":"2024-09-10T11:31:40.641087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:22.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871112+0000","last_change":"2024-09-10T11:31:39.655468+0000","last_active":"2024-09-10T11:31:41.871112+0000","last_peered":"2024-09-10T11:31:41.871112+0000","last_clean":"2024-09-10T11:31:41.871112+0000","last_became_active":"2024-09-10T11:31:39.655169+0000","last_became_peered":"2024-09-10T11:31:39.655169+0000","last_unstale":"2024-09-10T11:31:41.871112+0000","last_undegraded":"2024-09-10T11:31:41.871112+0000","last_fullsized":"2024-09-10T11:31:41.871112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:14.378123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659663+0000","last_change":"2024-09-10T11:31:38.647076+0000","last_active":"2024-09-10T11:31:41.659663+0000","last_peered":"2024-09-10T11:31:41.659663+0000","last_clean":"2024-09-10T11:31:41.659663+0000","last_became_active":"2024-09-10T11:31:38.644372+0000","last_became_peered":"2024-09-10T11:31:38.644372+0000","last_unstale":"2024-09-10T11:31:41.659663+0000","last_undegraded":"2024-09-10T11:31:41.659663+0000","last_fullsized":"2024-09-10T11:31:41.659663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:24.834804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871333+0000","last_change":"2024-09-10T11:31:39.653224+0000","last_active":"2024-09-10T11:31:41.871333+0000","last_peered":"2024-09-10T11:31:41.871333+0000","last_clean":"2024-09-10T11:31:41.871333+0000","last_became_active":"2024-09-10T11:31:39.652149+0000","last_became_peered":"2024-09-10T11:31:39.652149+0000","last_unstale":"2024-09-10T11:31:41.871333+0000","last_undegraded":"2024-09-10T11:31:41.871333+0000","last_fullsized":"2024-09-10T11:31:41.871333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:41.564425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643711+0000","last_change":"2024-09-10T11:31:38.645179+0000","last_active":"2024-09-10T11:31:40.643711+0000","last_peered":"2024-09-10T11:31:40.643711+0000","last_clean":"2024-09-10T11:31:40.643711+0000","last_became_active":"2024-09-10T11:31:38.644585+0000","last_became_peered":"2024-09-10T11:31:38.644585+0000","last_unstale":"2024-09-10T11:31:40.643711+0000","last_undegraded":"2024-09-10T11:31:40.643711+0000","last_fullsized":"2024-09-10T11:31:40.643711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:41.481696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659066+0000","last_change":"2024-09-10T11:31:39.648380+0000","last_active":"2024-09-10T11:31:41.659066+0000","last_peered":"2024-09-10T11:31:41.659066+0000","last_clean":"2024-09-10T11:31:41.659066+0000","last_became_active":"2024-09-10T11:31:39.647706+0000","last_became_peered":"2024-09-10T11:31:39.647706+0000","last_unstale":"2024-09-10T11:31:41.659066+0000","last_undegraded":"2024-09-10T11:31:41.659066+0000","last_fullsized":"2024-09-10T11:31:41.659066+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:01.152814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871691+0000","last_change":"2024-09-10T11:31:38.646450+0000","last_active":"2024-09-10T11:31:41.871691+0000","last_peered":"2024-09-10T11:31:41.871691+0000","last_clean":"2024-09-10T11:31:41.871691+0000","last_became_active":"2024-09-10T11:31:38.645570+0000","last_became_peered":"2024-09-10T11:31:38.645570+0000","last_unstale":"2024-09-10T11:31:41.871691+0000","last_undegraded":"2024-09-10T11:31:41.871691+0000","last_fullsized":"2024-09-10T11:31:41.871691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:18.608550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.651523+0000","last_change":"2024-09-10T11:31:39.651334+0000","last_active":"2024-09-10T11:31:39.651523+0000","last_peered":"2024-09-10T11:31:39.651523+0000","last_clean":"2024-09-10T11:31:39.651523+0000","last_became_active":"2024-09-10T11:31:39.651115+0000","last_became_peered":"2024-09-10T11:31:39.651115+0000","last_unstale":"2024-09-10T11:31:39.651523+0000","last_undegraded":"2024-09-10T11:31:39.651523+0000","last_fullsized":"2024-09-10T11:31:39.651523+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:46.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.649943+0000","last_change":"2024-09-10T11:31:38.644876+0000","last_active":"2024-09-10T11:31:39.649943+0000","last_peered":"2024-09-10T11:31:39.649943+0000","last_clean":"2024-09-10T11:31:39.649943+0000","last_became_active":"2024-09-10T11:31:38.644638+0000","last_became_peered":"2024-09-10T11:31:38.644638+0000","last_unstale":"2024-09-10T11:31:39.649943+0000","last_undegraded":"2024-09-10T11:31:39.649943+0000","last_fullsized":"2024-09-10T11:31:39.649943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:06.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.651506+0000","last_change":"2024-09-10T11:31:39.651235+0000","last_active":"2024-09-10T11:31:39.651506+0000","last_peered":"2024-09-10T11:31:39.651506+0000","last_clean":"2024-09-10T11:31:39.651506+0000","last_became_active":"2024-09-10T11:31:39.650992+0000","last_became_peered":"2024-09-10T11:31:39.650992+0000","last_unstale":"2024-09-10T11:31:39.651506+0000","last_undegraded":"2024-09-10T11:31:39.651506+0000","last_fullsized":"2024-09-10T11:31:39.651506+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:03:55.584918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871445+0000","last_change":"2024-09-10T11:31:38.644477+0000","last_active":"2024-09-10T11:31:41.871445+0000","last_peered":"2024-09-10T11:31:41.871445+0000","last_clean":"2024-09-10T11:31:41.871445+0000","last_became_active":"2024-09-10T11:31:38.644004+0000","last_became_peered":"2024-09-10T11:31:38.644004+0000","last_unstale":"2024-09-10T11:31:41.871445+0000","last_undegraded":"2024-09-10T11:31:41.871445+0000","last_fullsized":"2024-09-10T11:31:41.871445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:30.845972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657805+0000","last_change":"2024-09-10T11:31:39.659038+0000","last_active":"2024-09-10T11:31:41.657805+0000","last_peered":"2024-09-10T11:31:41.657805+0000","last_clean":"2024-09-10T11:31:41.657805+0000","last_became_active":"2024-09-10T11:31:39.658627+0000","last_became_peered":"2024-09-10T11:31:39.658627+0000","last_unstale":"2024-09-10T11:31:41.657805+0000","last_undegraded":"2024-09-10T11:31:41.657805+0000","last_fullsized":"2024-09-10T11:31:41.657805+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:39.568361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646809+0000","last_change":"2024-09-10T11:31:38.641412+0000","last_active":"2024-09-10T11:31:40.646809+0000","last_peered":"2024-09-10T11:31:40.646809+0000","last_clean":"2024-09-10T11:31:40.646809+0000","last_became_active":"2024-09-10T11:31:38.641198+0000","last_became_peered":"2024-09-10T11:31:38.641198+0000","last_unstale":"2024-09-10T11:31:40.646809+0000","last_undegraded":"2024-09-10T11:31:40.646809+0000","last_fullsized":"2024-09-10T11:31:40.646809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:51.831719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659238+0000","last_change":"2024-09-10T11:31:39.654093+0000","last_active":"2024-09-10T11:31:41.659238+0000","last_peered":"2024-09-10T11:31:41.659238+0000","last_clean":"2024-09-10T11:31:41.659238+0000","last_became_active":"2024-09-10T11:31:39.653847+0000","last_became_peered":"2024-09-10T11:31:39.653847+0000","last_unstale":"2024-09-10T11:31:41.659238+0000","last_undegraded":"2024-09-10T11:31:41.659238+0000","last_fullsized":"2024-09-10T11:31:41.659238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:44.950654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871592+0000","last_change":"2024-09-10T11:31:38.650626+0000","last_active":"2024-09-10T11:31:41.871592+0000","last_peered":"2024-09-10T11:31:41.871592+0000","last_clean":"2024-09-10T11:31:41.871592+0000","last_became_active":"2024-09-10T11:31:38.649886+0000","last_became_peered":"2024-09-10T11:31:38.649886+0000","last_unstale":"2024-09-10T11:31:41.871592+0000","last_undegraded":"2024-09-10T11:31:41.871592+0000","last_fullsized":"2024-09-10T11:31:41.871592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:13.577324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655003+0000","last_change":"2024-09-10T11:31:39.654867+0000","last_active":"2024-09-10T11:31:39.655003+0000","last_peered":"2024-09-10T11:31:39.655003+0000","last_clean":"2024-09-10T11:31:39.655003+0000","last_became_active":"2024-09-10T11:31:39.654510+0000","last_became_peered":"2024-09-10T11:31:39.654510+0000","last_unstale":"2024-09-10T11:31:39.655003+0000","last_undegraded":"2024-09-10T11:31:39.655003+0000","last_fullsized":"2024-09-10T11:31:39.655003+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:34.064896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470705+0000","last_change":"2024-09-10T11:31:38.645340+0000","last_active":"2024-09-10T11:31:41.470705+0000","last_peered":"2024-09-10T11:31:41.470705+0000","last_clean":"2024-09-10T11:31:41.470705+0000","last_became_active":"2024-09-10T11:31:38.644397+0000","last_became_peered":"2024-09-10T11:31:38.644397+0000","last_unstale":"2024-09-10T11:31:41.470705+0000","last_undegraded":"2024-09-10T11:31:41.470705+0000","last_fullsized":"2024-09-10T11:31:41.470705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:31.391443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470508+0000","last_change":"2024-09-10T11:31:39.644876+0000","last_active":"2024-09-10T11:31:41.470508+0000","last_peered":"2024-09-10T11:31:41.470508+0000","last_clean":"2024-09-10T11:31:41.470508+0000","last_became_active":"2024-09-10T11:31:39.644621+0000","last_became_peered":"2024-09-10T11:31:39.644621+0000","last_unstale":"2024-09-10T11:31:41.470508+0000","last_undegraded":"2024-09-10T11:31:41.470508+0000","last_fullsized":"2024-09-10T11:31:41.470508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:47.080676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644059+0000","last_change":"2024-09-10T11:31:38.644155+0000","last_active":"2024-09-10T11:31:40.644059+0000","last_peered":"2024-09-10T11:31:40.644059+0000","last_clean":"2024-09-10T11:31:40.644059+0000","last_became_active":"2024-09-10T11:31:38.643945+0000","last_became_peered":"2024-09-10T11:31:38.643945+0000","last_unstale":"2024-09-10T11:31:40.644059+0000","last_undegraded":"2024-09-10T11:31:40.644059+0000","last_fullsized":"2024-09-10T11:31:40.644059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:21.520519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470393+0000","last_change":"2024-09-10T11:31:39.647987+0000","last_active":"2024-09-10T11:31:41.470393+0000","last_peered":"2024-09-10T11:31:41.470393+0000","last_clean":"2024-09-10T11:31:41.470393+0000","last_became_active":"2024-09-10T11:31:39.647749+0000","last_became_peered":"2024-09-10T11:31:39.647749+0000","last_unstale":"2024-09-10T11:31:41.470393+0000","last_undegraded":"2024-09-10T11:31:41.470393+0000","last_fullsized":"2024-09-10T11:31:41.470393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:26.433379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641168+0000","last_change":"2024-09-10T11:31:38.650733+0000","last_active":"2024-09-10T11:31:40.641168+0000","last_peered":"2024-09-10T11:31:40.641168+0000","last_clean":"2024-09-10T11:31:40.641168+0000","last_became_active":"2024-09-10T11:31:38.649956+0000","last_became_peered":"2024-09-10T11:31:38.649956+0000","last_unstale":"2024-09-10T11:31:40.641168+0000","last_undegraded":"2024-09-10T11:31:40.641168+0000","last_fullsized":"2024-09-10T11:31:40.641168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:57.508017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646938+0000","last_change":"2024-09-10T11:31:39.646806+0000","last_active":"2024-09-10T11:31:40.646938+0000","last_peered":"2024-09-10T11:31:40.646938+0000","last_clean":"2024-09-10T11:31:40.646938+0000","last_became_active":"2024-09-10T11:31:39.646247+0000","last_became_peered":"2024-09-10T11:31:39.646247+0000","last_unstale":"2024-09-10T11:31:40.646938+0000","last_undegraded":"2024-09-10T11:31:40.646938+0000","last_fullsized":"2024-09-10T11:31:40.646938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:15.258005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871032+0000","last_change":"2024-09-10T11:31:38.649645+0000","last_active":"2024-09-10T11:31:41.871032+0000","last_peered":"2024-09-10T11:31:41.871032+0000","last_clean":"2024-09-10T11:31:41.871032+0000","last_became_active":"2024-09-10T11:31:38.649397+0000","last_became_peered":"2024-09-10T11:31:38.649397+0000","last_unstale":"2024-09-10T11:31:41.871032+0000","last_undegraded":"2024-09-10T11:31:41.871032+0000","last_fullsized":"2024-09-10T11:31:41.871032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:06.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470641+0000","last_change":"2024-09-10T11:31:39.651245+0000","last_active":"2024-09-10T11:31:41.470641+0000","last_peered":"2024-09-10T11:31:41.470641+0000","last_clean":"2024-09-10T11:31:41.470641+0000","last_became_active":"2024-09-10T11:31:39.651006+0000","last_became_peered":"2024-09-10T11:31:39.651006+0000","last_unstale":"2024-09-10T11:31:41.470641+0000","last_undegraded":"2024-09-10T11:31:41.470641+0000","last_fullsized":"2024-09-10T11:31:41.470641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:47.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657907+0000","last_change":"2024-09-10T11:31:38.648863+0000","last_active":"2024-09-10T11:31:41.657907+0000","last_peered":"2024-09-10T11:31:41.657907+0000","last_clean":"2024-09-10T11:31:41.657907+0000","last_became_active":"2024-09-10T11:31:38.648599+0000","last_became_peered":"2024-09-10T11:31:38.648599+0000","last_unstale":"2024-09-10T11:31:41.657907+0000","last_undegraded":"2024-09-10T11:31:41.657907+0000","last_fullsized":"2024-09-10T11:31:41.657907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:33.989466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470512+0000","last_change":"2024-09-10T11:31:39.658976+0000","last_active":"2024-09-10T11:31:41.470512+0000","last_peered":"2024-09-10T11:31:41.470512+0000","last_clean":"2024-09-10T11:31:41.470512+0000","last_became_active":"2024-09-10T11:31:39.658429+0000","last_became_peered":"2024-09-10T11:31:39.658429+0000","last_unstale":"2024-09-10T11:31:41.470512+0000","last_undegraded":"2024-09-10T11:31:41.470512+0000","last_fullsized":"2024-09-10T11:31:41.470512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:44.516767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.661286+0000","last_change":"2024-09-10T11:31:38.647793+0000","last_active":"2024-09-10T11:31:39.661286+0000","last_peered":"2024-09-10T11:31:39.661286+0000","last_clean":"2024-09-10T11:31:39.661286+0000","last_became_active":"2024-09-10T11:31:38.647565+0000","last_became_peered":"2024-09-10T11:31:38.647565+0000","last_unstale":"2024-09-10T11:31:39.661286+0000","last_undegraded":"2024-09-10T11:31:39.661286+0000","last_fullsized":"2024-09-10T11:31:39.661286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:36.572000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.647006+0000","last_change":"2024-09-10T11:31:39.654846+0000","last_active":"2024-09-10T11:31:40.647006+0000","last_peered":"2024-09-10T11:31:40.647006+0000","last_clean":"2024-09-10T11:31:40.647006+0000","last_became_active":"2024-09-10T11:31:39.654329+0000","last_became_peered":"2024-09-10T11:31:39.654329+0000","last_unstale":"2024-09-10T11:31:40.647006+0000","last_undegraded":"2024-09-10T11:31:40.647006+0000","last_fullsized":"2024-09-10T11:31:40.647006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:42.607436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.661693+0000","last_change":"2024-09-10T11:31:38.646374+0000","last_active":"2024-09-10T11:31:39.661693+0000","last_peered":"2024-09-10T11:31:39.661693+0000","last_clean":"2024-09-10T11:31:39.661693+0000","last_became_active":"2024-09-10T11:31:38.646154+0000","last_became_peered":"2024-09-10T11:31:38.646154+0000","last_unstale":"2024-09-10T11:31:39.661693+0000","last_undegraded":"2024-09-10T11:31:39.661693+0000","last_fullsized":"2024-09-10T11:31:39.661693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:27.682580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.655591+0000","last_change":"2024-09-10T11:31:39.655470+0000","last_active":"2024-09-10T11:31:39.655591+0000","last_peered":"2024-09-10T11:31:39.655591+0000","last_clean":"2024-09-10T11:31:39.655591+0000","last_became_active":"2024-09-10T11:31:39.655200+0000","last_became_peered":"2024-09-10T11:31:39.655200+0000","last_unstale":"2024-09-10T11:31:39.655591+0000","last_undegraded":"2024-09-10T11:31:39.655591+0000","last_fullsized":"2024-09-10T11:31:39.655591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:03:00.308890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641246+0000","last_change":"2024-09-10T11:31:38.650913+0000","last_active":"2024-09-10T11:31:40.641246+0000","last_peered":"2024-09-10T11:31:40.641246+0000","last_clean":"2024-09-10T11:31:40.641246+0000","last_became_active":"2024-09-10T11:31:38.650275+0000","last_became_peered":"2024-09-10T11:31:38.650275+0000","last_unstale":"2024-09-10T11:31:40.641246+0000","last_undegraded":"2024-09-10T11:31:40.641246+0000","last_fullsized":"2024-09-10T11:31:40.641246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:24.669282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871491+0000","last_change":"2024-09-10T11:31:39.657948+0000","last_active":"2024-09-10T11:31:41.871491+0000","last_peered":"2024-09-10T11:31:41.871491+0000","last_clean":"2024-09-10T11:31:41.871491+0000","last_became_active":"2024-09-10T11:31:39.657439+0000","last_became_peered":"2024-09-10T11:31:39.657439+0000","last_unstale":"2024-09-10T11:31:41.871491+0000","last_undegraded":"2024-09-10T11:31:41.871491+0000","last_fullsized":"2024-09-10T11:31:41.871491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:58.280079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871110+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871110+0000","last_peered":"2024-09-10T11:31:41.871110+0000","last_clean":"2024-09-10T11:31:41.871110+0000","last_became_active":"2024-09-10T11:31:38.647865+0000","last_became_peered":"2024-09-10T11:31:38.647865+0000","last_unstale":"2024-09-10T11:31:41.871110+0000","last_undegraded":"2024-09-10T11:31:41.871110+0000","last_fullsized":"2024-09-10T11:31:41.871110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:31.961548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641242+0000","last_change":"2024-09-10T11:31:39.644873+0000","last_active":"2024-09-10T11:31:40.641242+0000","last_peered":"2024-09-10T11:31:40.641242+0000","last_clean":"2024-09-10T11:31:40.641242+0000","last_became_active":"2024-09-10T11:31:39.644154+0000","last_became_peered":"2024-09-10T11:31:39.644154+0000","last_unstale":"2024-09-10T11:31:40.641242+0000","last_undegraded":"2024-09-10T11:31:40.641242+0000","last_fullsized":"2024-09-10T11:31:40.641242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:11.485938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643759+0000","last_change":"2024-09-10T11:31:38.648391+0000","last_active":"2024-09-10T11:31:40.643759+0000","last_peered":"2024-09-10T11:31:40.643759+0000","last_clean":"2024-09-10T11:31:40.643759+0000","last_became_active":"2024-09-10T11:31:38.648144+0000","last_became_peered":"2024-09-10T11:31:38.648144+0000","last_unstale":"2024-09-10T11:31:40.643759+0000","last_undegraded":"2024-09-10T11:31:40.643759+0000","last_fullsized":"2024-09-10T11:31:40.643759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:03.992675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658135+0000","last_change":"2024-09-10T11:31:39.654383+0000","last_active":"2024-09-10T11:31:41.658135+0000","last_peered":"2024-09-10T11:31:41.658135+0000","last_clean":"2024-09-10T11:31:41.658135+0000","last_became_active":"2024-09-10T11:31:39.653946+0000","last_became_peered":"2024-09-10T11:31:39.653946+0000","last_unstale":"2024-09-10T11:31:41.658135+0000","last_undegraded":"2024-09-10T11:31:41.658135+0000","last_fullsized":"2024-09-10T11:31:41.658135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:27.993903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871742+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871742+0000","last_peered":"2024-09-10T11:31:41.871742+0000","last_clean":"2024-09-10T11:31:41.871742+0000","last_became_active":"2024-09-10T11:31:38.647836+0000","last_became_peered":"2024-09-10T11:31:38.647836+0000","last_unstale":"2024-09-10T11:31:41.871742+0000","last_undegraded":"2024-09-10T11:31:41.871742+0000","last_fullsized":"2024-09-10T11:31:41.871742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:39.211943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657981+0000","last_change":"2024-09-10T11:31:39.652538+0000","last_active":"2024-09-10T11:31:41.657981+0000","last_peered":"2024-09-10T11:31:41.657981+0000","last_clean":"2024-09-10T11:31:41.657981+0000","last_became_active":"2024-09-10T11:31:39.652254+0000","last_became_peered":"2024-09-10T11:31:39.652254+0000","last_unstale":"2024-09-10T11:31:41.657981+0000","last_undegraded":"2024-09-10T11:31:41.657981+0000","last_fullsized":"2024-09-10T11:31:41.657981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:39.583627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659103+0000","last_change":"2024-09-10T11:31:38.645087+0000","last_active":"2024-09-10T11:31:41.659103+0000","last_peered":"2024-09-10T11:31:41.659103+0000","last_clean":"2024-09-10T11:31:41.659103+0000","last_became_active":"2024-09-10T11:31:38.644513+0000","last_became_peered":"2024-09-10T11:31:38.644513+0000","last_unstale":"2024-09-10T11:31:41.659103+0000","last_undegraded":"2024-09-10T11:31:41.659103+0000","last_fullsized":"2024-09-10T11:31:41.659103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:41.110510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659259+0000","last_change":"2024-09-10T11:31:39.653575+0000","last_active":"2024-09-10T11:31:41.659259+0000","last_peered":"2024-09-10T11:31:41.659259+0000","last_clean":"2024-09-10T11:31:41.659259+0000","last_became_active":"2024-09-10T11:31:39.653326+0000","last_became_peered":"2024-09-10T11:31:39.653326+0000","last_unstale":"2024-09-10T11:31:41.659259+0000","last_undegraded":"2024-09-10T11:31:41.659259+0000","last_fullsized":"2024-09-10T11:31:41.659259+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:18.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659164+0000","last_change":"2024-09-10T11:31:38.646127+0000","last_active":"2024-09-10T11:31:41.659164+0000","last_peered":"2024-09-10T11:31:41.659164+0000","last_clean":"2024-09-10T11:31:41.659164+0000","last_became_active":"2024-09-10T11:31:38.645283+0000","last_became_peered":"2024-09-10T11:31:38.645283+0000","last_unstale":"2024-09-10T11:31:41.659164+0000","last_undegraded":"2024-09-10T11:31:41.659164+0000","last_fullsized":"2024-09-10T11:31:41.659164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:35.070668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871485+0000","last_change":"2024-09-10T11:31:39.652663+0000","last_active":"2024-09-10T11:31:41.871485+0000","last_peered":"2024-09-10T11:31:41.871485+0000","last_clean":"2024-09-10T11:31:41.871485+0000","last_became_active":"2024-09-10T11:31:39.651824+0000","last_became_peered":"2024-09-10T11:31:39.651824+0000","last_unstale":"2024-09-10T11:31:41.871485+0000","last_undegraded":"2024-09-10T11:31:41.871485+0000","last_fullsized":"2024-09-10T11:31:41.871485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:12.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643885+0000","last_change":"2024-09-10T11:31:38.647875+0000","last_active":"2024-09-10T11:31:40.643885+0000","last_peered":"2024-09-10T11:31:40.643885+0000","last_clean":"2024-09-10T11:31:40.643885+0000","last_became_active":"2024-09-10T11:31:38.647001+0000","last_became_peered":"2024-09-10T11:31:38.647001+0000","last_unstale":"2024-09-10T11:31:40.643885+0000","last_undegraded":"2024-09-10T11:31:40.643885+0000","last_fullsized":"2024-09-10T11:31:40.643885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:46.157562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659210+0000","last_change":"2024-09-10T11:31:39.653859+0000","last_active":"2024-09-10T11:31:41.659210+0000","last_peered":"2024-09-10T11:31:41.659210+0000","last_clean":"2024-09-10T11:31:41.659210+0000","last_became_active":"2024-09-10T11:31:39.653630+0000","last_became_peered":"2024-09-10T11:31:39.653630+0000","last_unstale":"2024-09-10T11:31:41.659210+0000","last_undegraded":"2024-09-10T11:31:41.659210+0000","last_fullsized":"2024-09-10T11:31:41.659210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:06.312549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470757+0000","last_change":"2024-09-10T11:31:38.645848+0000","last_active":"2024-09-10T11:31:41.470757+0000","last_peered":"2024-09-10T11:31:41.470757+0000","last_clean":"2024-09-10T11:31:41.470757+0000","last_became_active":"2024-09-10T11:31:38.645636+0000","last_became_peered":"2024-09-10T11:31:38.645636+0000","last_unstale":"2024-09-10T11:31:41.470757+0000","last_undegraded":"2024-09-10T11:31:41.470757+0000","last_fullsized":"2024-09-10T11:31:41.470757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:37.623350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871341+0000","last_change":"2024-09-10T11:31:39.656607+0000","last_active":"2024-09-10T11:31:41.871341+0000","last_peered":"2024-09-10T11:31:41.871341+0000","last_clean":"2024-09-10T11:31:41.871341+0000","last_became_active":"2024-09-10T11:31:39.656322+0000","last_became_peered":"2024-09-10T11:31:39.656322+0000","last_unstale":"2024-09-10T11:31:41.871341+0000","last_undegraded":"2024-09-10T11:31:41.871341+0000","last_fullsized":"2024-09-10T11:31:41.871341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:44.579593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644106+0000","last_change":"2024-09-10T11:31:38.646845+0000","last_active":"2024-09-10T11:31:40.644106+0000","last_peered":"2024-09-10T11:31:40.644106+0000","last_clean":"2024-09-10T11:31:40.644106+0000","last_became_active":"2024-09-10T11:31:38.646550+0000","last_became_peered":"2024-09-10T11:31:38.646550+0000","last_unstale":"2024-09-10T11:31:40.644106+0000","last_undegraded":"2024-09-10T11:31:40.644106+0000","last_fullsized":"2024-09-10T11:31:40.644106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:48.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658289+0000","last_change":"2024-09-10T11:31:39.653111+0000","last_active":"2024-09-10T11:31:41.658289+0000","last_peered":"2024-09-10T11:31:41.658289+0000","last_clean":"2024-09-10T11:31:41.658289+0000","last_became_active":"2024-09-10T11:31:39.651634+0000","last_became_peered":"2024-09-10T11:31:39.651634+0000","last_unstale":"2024-09-10T11:31:41.658289+0000","last_undegraded":"2024-09-10T11:31:41.658289+0000","last_fullsized":"2024-09-10T11:31:41.658289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:16:12.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659399+0000","last_change":"2024-09-10T11:31:38.649507+0000","last_active":"2024-09-10T11:31:41.659399+0000","last_peered":"2024-09-10T11:31:41.659399+0000","last_clean":"2024-09-10T11:31:41.659399+0000","last_became_active":"2024-09-10T11:31:38.649245+0000","last_became_peered":"2024-09-10T11:31:38.649245+0000","last_unstale":"2024-09-10T11:31:41.659399+0000","last_undegraded":"2024-09-10T11:31:41.659399+0000","last_fullsized":"2024-09-10T11:31:41.659399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:28.249420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.653218+0000","last_change":"2024-09-10T11:31:39.653066+0000","last_active":"2024-09-10T11:31:39.653218+0000","last_peered":"2024-09-10T11:31:39.653218+0000","last_clean":"2024-09-10T11:31:39.653218+0000","last_became_active":"2024-09-10T11:31:39.652856+0000","last_became_peered":"2024-09-10T11:31:39.652856+0000","last_unstale":"2024-09-10T11:31:39.653218+0000","last_undegraded":"2024-09-10T11:31:39.653218+0000","last_fullsized":"2024-09-10T11:31:39.653218+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:06.455112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641445+0000","last_change":"2024-09-10T11:31:38.649054+0000","last_active":"2024-09-10T11:31:40.641445+0000","last_peered":"2024-09-10T11:31:40.641445+0000","last_clean":"2024-09-10T11:31:40.641445+0000","last_became_active":"2024-09-10T11:31:38.648838+0000","last_became_peered":"2024-09-10T11:31:38.648838+0000","last_unstale":"2024-09-10T11:31:40.641445+0000","last_undegraded":"2024-09-10T11:31:40.641445+0000","last_fullsized":"2024-09-10T11:31:40.641445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:47.015802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646718+0000","last_change":"2024-09-10T11:31:39.649938+0000","last_active":"2024-09-10T11:31:40.646718+0000","last_peered":"2024-09-10T11:31:40.646718+0000","last_clean":"2024-09-10T11:31:40.646718+0000","last_became_active":"2024-09-10T11:31:39.649709+0000","last_became_peered":"2024-09-10T11:31:39.649709+0000","last_unstale":"2024-09-10T11:31:40.646718+0000","last_undegraded":"2024-09-10T11:31:40.646718+0000","last_fullsized":"2024-09-10T11:31:40.646718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:33.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470585+0000","last_change":"2024-09-10T11:31:38.649265+0000","last_active":"2024-09-10T11:31:41.470585+0000","last_peered":"2024-09-10T11:31:41.470585+0000","last_clean":"2024-09-10T11:31:41.470585+0000","last_became_active":"2024-09-10T11:31:38.648733+0000","last_became_peered":"2024-09-10T11:31:38.648733+0000","last_unstale":"2024-09-10T11:31:41.470585+0000","last_undegraded":"2024-09-10T11:31:41.470585+0000","last_fullsized":"2024-09-10T11:31:41.470585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:13.293892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657783+0000","last_change":"2024-09-10T11:31:39.654260+0000","last_active":"2024-09-10T11:31:41.657783+0000","last_peered":"2024-09-10T11:31:41.657783+0000","last_clean":"2024-09-10T11:31:41.657783+0000","last_became_active":"2024-09-10T11:31:39.652267+0000","last_became_peered":"2024-09-10T11:31:39.652267+0000","last_unstale":"2024-09-10T11:31:41.657783+0000","last_undegraded":"2024-09-10T11:31:41.657783+0000","last_fullsized":"2024-09-10T11:31:41.657783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:27.642082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659343+0000","last_change":"2024-09-10T11:31:38.648727+0000","last_active":"2024-09-10T11:31:41.659343+0000","last_peered":"2024-09-10T11:31:41.659343+0000","last_clean":"2024-09-10T11:31:41.659343+0000","last_became_active":"2024-09-10T11:31:38.648279+0000","last_became_peered":"2024-09-10T11:31:38.648279+0000","last_unstale":"2024-09-10T11:31:41.659343+0000","last_undegraded":"2024-09-10T11:31:41.659343+0000","last_fullsized":"2024-09-10T11:31:41.659343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:06.205973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":214,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T11:31:39.650272+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:39.650272+0000","last_peered":"2024-09-10T11:31:39.650272+0000","last_clean":"2024-09-10T11:31:39.650272+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:39.650272+0000","last_undegraded":"2024-09-10T11:31:39.650272+0000","last_fullsized":"2024-09-10T11:31:39.650272+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":203,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195109,"statfs":{"total":200000000,"available":199791959,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199201603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":204,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195107,"statfs":{"total":200000000,"available":199790289,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195101,"statfs":{"total":200000000,"available":199784081,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":779,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194532,"statfs":{"total":200000000,"available":199201555,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:46.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:31:47.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.049+0000 7f37bf65e640 1 Processor -- start 2024-09-10T11:31:47.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.050+0000 7f37bf65e640 1 -- start start 2024-09-10T11:31:47.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.050+0000 7f37bf65e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37b8071bb0 0x7f37b80b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bf65e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b806ee10 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bf65e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b80cd9a0 con 0x7f37b815aef0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bf65e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f37b80b0290 con 0x7f37b8071bb0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bcbf9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f37b815b2d0 0x7f37b80714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37076/0 (socket says 172.21.15.99:37076) 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bcbf9640 1 -- 172.21.15.99:0/2727968160 learned_addr learned my addr 172.21.15.99:0/2727968160 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/2727968160 >> v2:172.21.15.196:3300/0 conn(0x7f37b8071bb0 0x7f37b80b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292614794 0 0) 0x7f37b806ee10 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a4003520 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4198015576 0 0) 0x7f37a4003520 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 >> v2:172.21.15.196:3300/0 conn(0x7f37b8071bb0 msgr2=0x7f37b80b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 --2- 172.21.15.99:0/2727968160 >> v2:172.21.15.196:3300/0 conn(0x7f37b8071bb0 0x7f37b80b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 >> v1:172.21.15.196:6789/0 conn(0x7f37b815aef0 legacy=0x7f37b80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b806e570 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f37ac001670 con 0x7f37b815b2d0 2024-09-10T11:31:47.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 >> v1:172.21.15.99:6789/0 conn(0x7f37b815b2d0 legacy=0x7f37b80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a40039b0 0x7f37a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a4004800 0x7f37a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 0x7f37a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37b80b0290 con 0x7f37a40051c0 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400c060 con 0x7f37a40039b0 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f37a400c260 con 0x7f37a4004800 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.051+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 0x7f37a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bd3fa640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a40039b0 0x7f37a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bcbf9640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a4004800 0x7f37a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 -- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a4004800 msgr2=0x7f37a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a4004800 0x7f37a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 -- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a40039b0 msgr2=0x7f37a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:47.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37a40039b0 0x7f37a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 -- 172.21.15.99:0/2727968160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a4003520 con 0x7f37a40051c0 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 -- 172.21.15.99:0/2727968160 reap_dead start 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.052+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 0x7f37a400b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009fc0 tx=0x7f37a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6250b06fb91504 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0035020 con 0x7f37a40051c0 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b0003bd0 con 0x7f37a40051c0 2024-09-10T11:31:47.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37abfff640 1 -- 172.21.15.99:0/2727968160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0009c10 con 0x7f37a40051c0 2024-09-10T11:31:47.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 -- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 msgr2=0x7f37a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 0x7f37a400b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009fc0 tx=0x7f37a4005830 comp rx=0 tx=0).stop 2024-09-10T11:31:47.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 -- 172.21.15.99:0/2727968160 shutdown_connections 2024-09-10T11:31:47.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 --2- 172.21.15.99:0/2727968160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37a40051c0 0x7f37a400b9c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 -- 172.21.15.99:0/2727968160 >> 172.21.15.99:0/2727968160 conn(0x7f37b806bc10 msgr2=0x7f37b806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 -- 172.21.15.99:0/2727968160 shutdown_connections 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.053+0000 7f37bf65e640 1 -- 172.21.15.99:0/2727968160 wait complete. 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 Processor -- start 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 -- start start 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a40051c0 0x7f37b81427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b8152430 0x7f37b81528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37b80b4510 con 0x7f37b80ae910 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37b806eb30 con 0x7f37b8152430 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bf65e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f37b80b4690 con 0x7f37a40051c0 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bdbfb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bdbfb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60534/0 (socket says 172.21.15.99:60534) 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bdbfb640 1 -- 172.21.15.99:0/332763347 learned_addr learned my addr 172.21.15.99:0/332763347 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.055+0000 7f37bcbf9640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b8152430 0x7f37b81528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bd3fa640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a40051c0 0x7f37b81427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 -- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a40051c0 msgr2=0x7f37b81427e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a40051c0 0x7f37b81427e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 -- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b8152430 msgr2=0x7f37b81528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b8152430 0x7f37b81528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 -- 172.21.15.99:0/332763347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b80af2d0 con 0x7f37b80ae910 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bdbfb640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37b000e230 tx=0x7f37b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0035030 con 0x7f37b80ae910 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b0043070 con 0x7f37b80ae910 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b8145330 con 0x7f37b80ae910 2024-09-10T11:31:47.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b002d070 con 0x7f37b80ae910 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.056+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81458d0 con 0x7f37b80ae910 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.057+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b80afbb0 con 0x7f37b80ae910 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.058+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37b0005d10 con 0x7f37b80ae910 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.058+0000 7f37a9ffb640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 0x7f379c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.058+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f37b00abb20 con 0x7f37b80ae910 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.058+0000 7f37bd3fa640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 0x7f379c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.059+0000 7f37bd3fa640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 0x7f379c06e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.063+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37b0079fa0 con 0x7f37b80ae910 2024-09-10T11:31:47.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.224+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37b8158230 con 0x7f379c06be30 2024-09-10T11:31:47.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.229+0000 7f37a9ffb640 1 -- 172.21.15.99:0/332763347 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247491 (secure 0 0 0) 0x7f37b8158230 con 0x7f379c06be30 2024-09-10T11:31:47.233 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:47.234 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:31:47.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 msgr2=0x7f379c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 0x7f379c06e2f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37b4000c00 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 msgr2=0x7f37b80aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f37b000e230 tx=0x7f37b0009750 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 shutdown_connections 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f379c06be30 0x7f379c06e2f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b80ae910 0x7f37b80aed90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37b8152430 0x7f37b81528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 --2- 172.21.15.99:0/332763347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37a40051c0 0x7f37b81427e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 >> 172.21.15.99:0/332763347 conn(0x7f37b806bc10 msgr2=0x7f37b80714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 shutdown_connections 2024-09-10T11:31:47.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.232+0000 7f37bf65e640 1 -- 172.21.15.99:0/332763347 wait complete. 2024-09-10T11:31:47.244 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-10T11:31:46.925479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559667,"statfs":{"total":1600000000,"available":1597099988,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001311"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643782+0000","last_change":"2024-09-10T11:31:39.645497+0000","last_active":"2024-09-10T11:31:40.643782+0000","last_peered":"2024-09-10T11:31:40.643782+0000","last_clean":"2024-09-10T11:31:40.643782+0000","last_became_active":"2024-09-10T11:31:39.645175+0000","last_became_peered":"2024-09-10T11:31:39.645175+0000","last_unstale":"2024-09-10T11:31:40.643782+0000","last_undegraded":"2024-09-10T11:31:40.643782+0000","last_fullsized":"2024-09-10T11:31:40.643782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:13:57.319459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646626+0000","last_change":"2024-09-10T11:31:39.653090+0000","last_active":"2024-09-10T11:31:40.646626+0000","last_peered":"2024-09-10T11:31:40.646626+0000","last_clean":"2024-09-10T11:31:40.646626+0000","last_became_active":"2024-09-10T11:31:39.652865+0000","last_became_peered":"2024-09-10T11:31:39.652865+0000","last_unstale":"2024-09-10T11:31:40.646626+0000","last_undegraded":"2024-09-10T11:31:40.646626+0000","last_fullsized":"2024-09-10T11:31:40.646626+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:08.493521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081248+0000","last_change":"2024-09-10T11:31:39.655491+0000","last_active":"2024-09-10T11:31:42.081248+0000","last_peered":"2024-09-10T11:31:42.081248+0000","last_clean":"2024-09-10T11:31:42.081248+0000","last_became_active":"2024-09-10T11:31:39.655120+0000","last_became_peered":"2024-09-10T11:31:39.655120+0000","last_unstale":"2024-09-10T11:31:42.081248+0000","last_undegraded":"2024-09-10T11:31:42.081248+0000","last_fullsized":"2024-09-10T11:31:42.081248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:12.504871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657665+0000","last_change":"2024-09-10T11:31:39.650249+0000","last_active":"2024-09-10T11:31:41.657665+0000","last_peered":"2024-09-10T11:31:41.657665+0000","last_clean":"2024-09-10T11:31:41.657665+0000","last_became_active":"2024-09-10T11:31:39.649905+0000","last_became_peered":"2024-09-10T11:31:39.649905+0000","last_unstale":"2024-09-10T11:31:41.657665+0000","last_undegraded":"2024-09-10T11:31:41.657665+0000","last_fullsized":"2024-09-10T11:31:41.657665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:09.765200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643841+0000","last_change":"2024-09-10T11:31:39.646393+0000","last_active":"2024-09-10T11:31:40.643841+0000","last_peered":"2024-09-10T11:31:40.643841+0000","last_clean":"2024-09-10T11:31:40.643841+0000","last_became_active":"2024-09-10T11:31:39.646154+0000","last_became_peered":"2024-09-10T11:31:39.646154+0000","last_unstale":"2024-09-10T11:31:40.643841+0000","last_undegraded":"2024-09-10T11:31:40.643841+0000","last_fullsized":"2024-09-10T11:31:40.643841+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:38.360639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641243+0000","last_change":"2024-09-10T11:31:39.654253+0000","last_active":"2024-09-10T11:31:40.641243+0000","last_peered":"2024-09-10T11:31:40.641243+0000","last_clean":"2024-09-10T11:31:40.641243+0000","last_became_active":"2024-09-10T11:31:39.653964+0000","last_became_peered":"2024-09-10T11:31:39.653964+0000","last_unstale":"2024-09-10T11:31:40.641243+0000","last_undegraded":"2024-09-10T11:31:40.641243+0000","last_fullsized":"2024-09-10T11:31:40.641243+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:46.259879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658993+0000","last_change":"2024-09-10T11:31:39.656014+0000","last_active":"2024-09-10T11:31:41.658993+0000","last_peered":"2024-09-10T11:31:41.658993+0000","last_clean":"2024-09-10T11:31:41.658993+0000","last_became_active":"2024-09-10T11:31:39.655797+0000","last_became_peered":"2024-09-10T11:31:39.655797+0000","last_unstale":"2024-09-10T11:31:41.658993+0000","last_undegraded":"2024-09-10T11:31:41.658993+0000","last_fullsized":"2024-09-10T11:31:41.658993+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:59.186968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657591+0000","last_change":"2024-09-10T11:31:39.654806+0000","last_active":"2024-09-10T11:31:41.657591+0000","last_peered":"2024-09-10T11:31:41.657591+0000","last_clean":"2024-09-10T11:31:41.657591+0000","last_became_active":"2024-09-10T11:31:39.653029+0000","last_became_peered":"2024-09-10T11:31:39.653029+0000","last_unstale":"2024-09-10T11:31:41.657591+0000","last_undegraded":"2024-09-10T11:31:41.657591+0000","last_fullsized":"2024-09-10T11:31:41.657591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.640172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641152+0000","last_change":"2024-09-10T11:31:39.658665+0000","last_active":"2024-09-10T11:31:40.641152+0000","last_peered":"2024-09-10T11:31:40.641152+0000","last_clean":"2024-09-10T11:31:40.641152+0000","last_became_active":"2024-09-10T11:31:39.658365+0000","last_became_peered":"2024-09-10T11:31:39.658365+0000","last_unstale":"2024-09-10T11:31:40.641152+0000","last_undegraded":"2024-09-10T11:31:40.641152+0000","last_fullsized":"2024-09-10T11:31:40.641152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:22.800217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871184+0000","last_change":"2024-09-10T11:31:39.656494+0000","last_active":"2024-09-10T11:31:41.871184+0000","last_peered":"2024-09-10T11:31:41.871184+0000","last_clean":"2024-09-10T11:31:41.871184+0000","last_became_active":"2024-09-10T11:31:39.656264+0000","last_became_peered":"2024-09-10T11:31:39.656264+0000","last_unstale":"2024-09-10T11:31:41.871184+0000","last_undegraded":"2024-09-10T11:31:41.871184+0000","last_fullsized":"2024-09-10T11:31:41.871184+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:54.180344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657795+0000","last_change":"2024-09-10T11:31:39.653639+0000","last_active":"2024-09-10T11:31:41.657795+0000","last_peered":"2024-09-10T11:31:41.657795+0000","last_clean":"2024-09-10T11:31:41.657795+0000","last_became_active":"2024-09-10T11:31:39.652232+0000","last_became_peered":"2024-09-10T11:31:39.652232+0000","last_unstale":"2024-09-10T11:31:41.657795+0000","last_undegraded":"2024-09-10T11:31:41.657795+0000","last_fullsized":"2024-09-10T11:31:41.657795+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:10.021178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646762+0000","last_change":"2024-09-10T11:31:39.647797+0000","last_active":"2024-09-10T11:31:40.646762+0000","last_peered":"2024-09-10T11:31:40.646762+0000","last_clean":"2024-09-10T11:31:40.646762+0000","last_became_active":"2024-09-10T11:31:39.647576+0000","last_became_peered":"2024-09-10T11:31:39.647576+0000","last_unstale":"2024-09-10T11:31:40.646762+0000","last_undegraded":"2024-09-10T11:31:40.646762+0000","last_fullsized":"2024-09-10T11:31:40.646762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:42.829472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641057+0000","last_change":"2024-09-10T11:31:39.648764+0000","last_active":"2024-09-10T11:31:40.641057+0000","last_peered":"2024-09-10T11:31:40.641057+0000","last_clean":"2024-09-10T11:31:40.641057+0000","last_became_active":"2024-09-10T11:31:39.648525+0000","last_became_peered":"2024-09-10T11:31:39.648525+0000","last_unstale":"2024-09-10T11:31:40.641057+0000","last_undegraded":"2024-09-10T11:31:40.641057+0000","last_fullsized":"2024-09-10T11:31:40.641057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:15.895136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658043+0000","last_change":"2024-09-10T11:31:39.658849+0000","last_active":"2024-09-10T11:31:41.658043+0000","last_peered":"2024-09-10T11:31:41.658043+0000","last_clean":"2024-09-10T11:31:41.658043+0000","last_became_active":"2024-09-10T11:31:39.658315+0000","last_became_peered":"2024-09-10T11:31:39.658315+0000","last_unstale":"2024-09-10T11:31:41.658043+0000","last_undegraded":"2024-09-10T11:31:41.658043+0000","last_fullsized":"2024-09-10T11:31:41.658043+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:18.791072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657667+0000","last_change":"2024-09-10T11:31:39.655320+0000","last_active":"2024-09-10T11:31:41.657667+0000","last_peered":"2024-09-10T11:31:41.657667+0000","last_clean":"2024-09-10T11:31:41.657667+0000","last_became_active":"2024-09-10T11:31:39.654324+0000","last_became_peered":"2024-09-10T11:31:39.654324+0000","last_unstale":"2024-09-10T11:31:41.657667+0000","last_undegraded":"2024-09-10T11:31:41.657667+0000","last_fullsized":"2024-09-10T11:31:41.657667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:39.168999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643984+0000","last_change":"2024-09-10T11:31:39.656422+0000","last_active":"2024-09-10T11:31:40.643984+0000","last_peered":"2024-09-10T11:31:40.643984+0000","last_clean":"2024-09-10T11:31:40.643984+0000","last_became_active":"2024-09-10T11:31:39.656184+0000","last_became_peered":"2024-09-10T11:31:39.656184+0000","last_unstale":"2024-09-10T11:31:40.643984+0000","last_undegraded":"2024-09-10T11:31:40.643984+0000","last_fullsized":"2024-09-10T11:31:40.643984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:15.186401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646872+0000","last_change":"2024-09-10T11:31:39.647034+0000","last_active":"2024-09-10T11:31:40.646872+0000","last_peered":"2024-09-10T11:31:40.646872+0000","last_clean":"2024-09-10T11:31:40.646872+0000","last_became_active":"2024-09-10T11:31:39.646566+0000","last_became_peered":"2024-09-10T11:31:39.646566+0000","last_unstale":"2024-09-10T11:31:40.646872+0000","last_undegraded":"2024-09-10T11:31:40.646872+0000","last_fullsized":"2024-09-10T11:31:40.646872+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:44.052632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081396+0000","last_change":"2024-09-10T11:31:39.655310+0000","last_active":"2024-09-10T11:31:42.081396+0000","last_peered":"2024-09-10T11:31:42.081396+0000","last_clean":"2024-09-10T11:31:42.081396+0000","last_became_active":"2024-09-10T11:31:39.654844+0000","last_became_peered":"2024-09-10T11:31:39.654844+0000","last_unstale":"2024-09-10T11:31:42.081396+0000","last_undegraded":"2024-09-10T11:31:42.081396+0000","last_fullsized":"2024-09-10T11:31:42.081396+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:57.254796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643835+0000","last_change":"2024-09-10T11:31:39.645579+0000","last_active":"2024-09-10T11:31:40.643835+0000","last_peered":"2024-09-10T11:31:40.643835+0000","last_clean":"2024-09-10T11:31:40.643835+0000","last_became_active":"2024-09-10T11:31:39.645354+0000","last_became_peered":"2024-09-10T11:31:39.645354+0000","last_unstale":"2024-09-10T11:31:40.643835+0000","last_undegraded":"2024-09-10T11:31:40.643835+0000","last_fullsized":"2024-09-10T11:31:40.643835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:49.467444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871633+0000","last_change":"2024-09-10T11:31:39.654118+0000","last_active":"2024-09-10T11:31:41.871633+0000","last_peered":"2024-09-10T11:31:41.871633+0000","last_clean":"2024-09-10T11:31:41.871633+0000","last_became_active":"2024-09-10T11:31:39.652435+0000","last_became_peered":"2024-09-10T11:31:39.652435+0000","last_unstale":"2024-09-10T11:31:41.871633+0000","last_undegraded":"2024-09-10T11:31:41.871633+0000","last_fullsized":"2024-09-10T11:31:41.871633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:02:31.098768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658543+0000","last_change":"2024-09-10T11:31:39.652924+0000","last_active":"2024-09-10T11:31:41.658543+0000","last_peered":"2024-09-10T11:31:41.658543+0000","last_clean":"2024-09-10T11:31:41.658543+0000","last_became_active":"2024-09-10T11:31:39.650464+0000","last_became_peered":"2024-09-10T11:31:39.650464+0000","last_unstale":"2024-09-10T11:31:41.658543+0000","last_undegraded":"2024-09-10T11:31:41.658543+0000","last_fullsized":"2024-09-10T11:31:41.658543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:17.704454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470652+0000","last_change":"2024-09-10T11:31:39.650521+0000","last_active":"2024-09-10T11:31:41.470652+0000","last_peered":"2024-09-10T11:31:41.470652+0000","last_clean":"2024-09-10T11:31:41.470652+0000","last_became_active":"2024-09-10T11:31:39.650297+0000","last_became_peered":"2024-09-10T11:31:39.650297+0000","last_unstale":"2024-09-10T11:31:41.470652+0000","last_undegraded":"2024-09-10T11:31:41.470652+0000","last_fullsized":"2024-09-10T11:31:41.470652+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:39.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659573+0000","last_change":"2024-09-10T11:31:39.650038+0000","last_active":"2024-09-10T11:31:41.659573+0000","last_peered":"2024-09-10T11:31:41.659573+0000","last_clean":"2024-09-10T11:31:41.659573+0000","last_became_active":"2024-09-10T11:31:39.649807+0000","last_became_peered":"2024-09-10T11:31:39.649807+0000","last_unstale":"2024-09-10T11:31:41.659573+0000","last_undegraded":"2024-09-10T11:31:41.659573+0000","last_fullsized":"2024-09-10T11:31:41.659573+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:04.977562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646605+0000","last_change":"2024-09-10T11:31:39.651240+0000","last_active":"2024-09-10T11:31:40.646605+0000","last_peered":"2024-09-10T11:31:40.646605+0000","last_clean":"2024-09-10T11:31:40.646605+0000","last_became_active":"2024-09-10T11:31:39.651015+0000","last_became_peered":"2024-09-10T11:31:39.651015+0000","last_unstale":"2024-09-10T11:31:40.646605+0000","last_undegraded":"2024-09-10T11:31:40.646605+0000","last_fullsized":"2024-09-10T11:31:40.646605+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:50.935894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643988+0000","last_change":"2024-09-10T11:31:39.645360+0000","last_active":"2024-09-10T11:31:40.643988+0000","last_peered":"2024-09-10T11:31:40.643988+0000","last_clean":"2024-09-10T11:31:40.643988+0000","last_became_active":"2024-09-10T11:31:39.645089+0000","last_became_peered":"2024-09-10T11:31:39.645089+0000","last_unstale":"2024-09-10T11:31:40.643988+0000","last_undegraded":"2024-09-10T11:31:40.643988+0000","last_fullsized":"2024-09-10T11:31:40.643988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:05.904582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657992+0000","last_change":"2024-09-10T11:31:39.653442+0000","last_active":"2024-09-10T11:31:41.657992+0000","last_peered":"2024-09-10T11:31:41.657992+0000","last_clean":"2024-09-10T11:31:41.657992+0000","last_became_active":"2024-09-10T11:31:39.651982+0000","last_became_peered":"2024-09-10T11:31:39.651982+0000","last_unstale":"2024-09-10T11:31:41.657992+0000","last_undegraded":"2024-09-10T11:31:41.657992+0000","last_fullsized":"2024-09-10T11:31:41.657992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:17.090501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659162+0000","last_change":"2024-09-10T11:31:39.650720+0000","last_active":"2024-09-10T11:31:41.659162+0000","last_peered":"2024-09-10T11:31:41.659162+0000","last_clean":"2024-09-10T11:31:41.659162+0000","last_became_active":"2024-09-10T11:31:39.650477+0000","last_became_peered":"2024-09-10T11:31:39.650477+0000","last_unstale":"2024-09-10T11:31:41.659162+0000","last_undegraded":"2024-09-10T11:31:41.659162+0000","last_fullsized":"2024-09-10T11:31:41.659162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:54.152529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659073+0000","last_change":"2024-09-10T11:31:39.651780+0000","last_active":"2024-09-10T11:31:41.659073+0000","last_peered":"2024-09-10T11:31:41.659073+0000","last_clean":"2024-09-10T11:31:41.659073+0000","last_became_active":"2024-09-10T11:31:39.651388+0000","last_became_peered":"2024-09-10T11:31:39.651388+0000","last_unstale":"2024-09-10T11:31:41.659073+0000","last_undegraded":"2024-09-10T11:31:41.659073+0000","last_fullsized":"2024-09-10T11:31:41.659073+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:16.395096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643642+0000","last_change":"2024-09-10T11:31:39.658219+0000","last_active":"2024-09-10T11:31:40.643642+0000","last_peered":"2024-09-10T11:31:40.643642+0000","last_clean":"2024-09-10T11:31:40.643642+0000","last_became_active":"2024-09-10T11:31:39.657959+0000","last_became_peered":"2024-09-10T11:31:39.657959+0000","last_unstale":"2024-09-10T11:31:40.643642+0000","last_undegraded":"2024-09-10T11:31:40.643642+0000","last_fullsized":"2024-09-10T11:31:40.643642+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:38.467359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646828+0000","last_change":"2024-09-10T11:31:39.646856+0000","last_active":"2024-09-10T11:31:40.646828+0000","last_peered":"2024-09-10T11:31:40.646828+0000","last_clean":"2024-09-10T11:31:40.646828+0000","last_became_active":"2024-09-10T11:31:39.646630+0000","last_became_peered":"2024-09-10T11:31:39.646630+0000","last_unstale":"2024-09-10T11:31:40.646828+0000","last_undegraded":"2024-09-10T11:31:40.646828+0000","last_fullsized":"2024-09-10T11:31:40.646828+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:27.829818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641394+0000","last_change":"2024-09-10T11:31:39.644641+0000","last_active":"2024-09-10T11:31:40.641394+0000","last_peered":"2024-09-10T11:31:40.641394+0000","last_clean":"2024-09-10T11:31:40.641394+0000","last_became_active":"2024-09-10T11:31:39.644390+0000","last_became_peered":"2024-09-10T11:31:39.644390+0000","last_unstale":"2024-09-10T11:31:40.641394+0000","last_undegraded":"2024-09-10T11:31:40.641394+0000","last_fullsized":"2024-09-10T11:31:40.641394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:55.588242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657857+0000","last_change":"2024-09-10T11:31:39.654918+0000","last_active":"2024-09-10T11:31:41.657857+0000","last_peered":"2024-09-10T11:31:41.657857+0000","last_clean":"2024-09-10T11:31:41.657857+0000","last_became_active":"2024-09-10T11:31:39.654587+0000","last_became_peered":"2024-09-10T11:31:39.654587+0000","last_unstale":"2024-09-10T11:31:41.657857+0000","last_undegraded":"2024-09-10T11:31:41.657857+0000","last_fullsized":"2024-09-10T11:31:41.657857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:40.331083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470394+0000","last_change":"2024-09-10T11:31:39.659286+0000","last_active":"2024-09-10T11:31:41.470394+0000","last_peered":"2024-09-10T11:31:41.470394+0000","last_clean":"2024-09-10T11:31:41.470394+0000","last_became_active":"2024-09-10T11:31:39.658774+0000","last_became_peered":"2024-09-10T11:31:39.658774+0000","last_unstale":"2024-09-10T11:31:41.470394+0000","last_undegraded":"2024-09-10T11:31:41.470394+0000","last_fullsized":"2024-09-10T11:31:41.470394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:30.202847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641353+0000","last_change":"2024-09-10T11:31:38.648561+0000","last_active":"2024-09-10T11:31:40.641353+0000","last_peered":"2024-09-10T11:31:40.641353+0000","last_clean":"2024-09-10T11:31:40.641353+0000","last_became_active":"2024-09-10T11:31:38.647403+0000","last_became_peered":"2024-09-10T11:31:38.647403+0000","last_unstale":"2024-09-10T11:31:40.641353+0000","last_undegraded":"2024-09-10T11:31:40.641353+0000","last_fullsized":"2024-09-10T11:31:40.641353+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:33.760518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644028+0000","last_change":"2024-09-10T11:31:39.645767+0000","last_active":"2024-09-10T11:31:40.644028+0000","last_peered":"2024-09-10T11:31:40.644028+0000","last_clean":"2024-09-10T11:31:40.644028+0000","last_became_active":"2024-09-10T11:31:39.645548+0000","last_became_peered":"2024-09-10T11:31:39.645548+0000","last_unstale":"2024-09-10T11:31:40.644028+0000","last_undegraded":"2024-09-10T11:31:40.644028+0000","last_fullsized":"2024-09-10T11:31:40.644028+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:41.922464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659303+0000","last_change":"2024-09-10T11:31:38.644985+0000","last_active":"2024-09-10T11:31:41.659303+0000","last_peered":"2024-09-10T11:31:41.659303+0000","last_clean":"2024-09-10T11:31:41.659303+0000","last_became_active":"2024-09-10T11:31:38.644751+0000","last_became_peered":"2024-09-10T11:31:38.644751+0000","last_unstale":"2024-09-10T11:31:41.659303+0000","last_undegraded":"2024-09-10T11:31:41.659303+0000","last_fullsized":"2024-09-10T11:31:41.659303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:39.728019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646869+0000","last_change":"2024-09-10T11:31:39.649105+0000","last_active":"2024-09-10T11:31:40.646869+0000","last_peered":"2024-09-10T11:31:40.646869+0000","last_clean":"2024-09-10T11:31:40.646869+0000","last_became_active":"2024-09-10T11:31:39.648513+0000","last_became_peered":"2024-09-10T11:31:39.648513+0000","last_unstale":"2024-09-10T11:31:40.646869+0000","last_undegraded":"2024-09-10T11:31:40.646869+0000","last_fullsized":"2024-09-10T11:31:40.646869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:30.397629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641324+0000","last_change":"2024-09-10T11:31:38.647068+0000","last_active":"2024-09-10T11:31:40.641324+0000","last_peered":"2024-09-10T11:31:40.641324+0000","last_clean":"2024-09-10T11:31:40.641324+0000","last_became_active":"2024-09-10T11:31:38.646806+0000","last_became_peered":"2024-09-10T11:31:38.646806+0000","last_unstale":"2024-09-10T11:31:40.641324+0000","last_undegraded":"2024-09-10T11:31:40.641324+0000","last_fullsized":"2024-09-10T11:31:40.641324+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:41.318026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657848+0000","last_change":"2024-09-10T11:31:39.655753+0000","last_active":"2024-09-10T11:31:41.657848+0000","last_peered":"2024-09-10T11:31:41.657848+0000","last_clean":"2024-09-10T11:31:41.657848+0000","last_became_active":"2024-09-10T11:31:39.654854+0000","last_became_peered":"2024-09-10T11:31:39.654854+0000","last_unstale":"2024-09-10T11:31:41.657848+0000","last_undegraded":"2024-09-10T11:31:41.657848+0000","last_fullsized":"2024-09-10T11:31:41.657848+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:02.970230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658983+0000","last_change":"2024-09-10T11:31:38.645523+0000","last_active":"2024-09-10T11:31:41.658983+0000","last_peered":"2024-09-10T11:31:41.658983+0000","last_clean":"2024-09-10T11:31:41.658983+0000","last_became_active":"2024-09-10T11:31:38.644841+0000","last_became_peered":"2024-09-10T11:31:38.644841+0000","last_unstale":"2024-09-10T11:31:41.658983+0000","last_undegraded":"2024-09-10T11:31:41.658983+0000","last_fullsized":"2024-09-10T11:31:41.658983+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:44.618873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659386+0000","last_change":"2024-09-10T11:31:39.651607+0000","last_active":"2024-09-10T11:31:41.659386+0000","last_peered":"2024-09-10T11:31:41.659386+0000","last_clean":"2024-09-10T11:31:41.659386+0000","last_became_active":"2024-09-10T11:31:39.651070+0000","last_became_peered":"2024-09-10T11:31:39.651070+0000","last_unstale":"2024-09-10T11:31:41.659386+0000","last_undegraded":"2024-09-10T11:31:41.659386+0000","last_fullsized":"2024-09-10T11:31:41.659386+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:03.428955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641087+0000","last_change":"2024-09-10T11:31:38.650460+0000","last_active":"2024-09-10T11:31:40.641087+0000","last_peered":"2024-09-10T11:31:40.641087+0000","last_clean":"2024-09-10T11:31:40.641087+0000","last_became_active":"2024-09-10T11:31:38.649203+0000","last_became_peered":"2024-09-10T11:31:38.649203+0000","last_unstale":"2024-09-10T11:31:40.641087+0000","last_undegraded":"2024-09-10T11:31:40.641087+0000","last_fullsized":"2024-09-10T11:31:40.641087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:22.043506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871112+0000","last_change":"2024-09-10T11:31:39.655468+0000","last_active":"2024-09-10T11:31:41.871112+0000","last_peered":"2024-09-10T11:31:41.871112+0000","last_clean":"2024-09-10T11:31:41.871112+0000","last_became_active":"2024-09-10T11:31:39.655169+0000","last_became_peered":"2024-09-10T11:31:39.655169+0000","last_unstale":"2024-09-10T11:31:41.871112+0000","last_undegraded":"2024-09-10T11:31:41.871112+0000","last_fullsized":"2024-09-10T11:31:41.871112+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:14.378123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659663+0000","last_change":"2024-09-10T11:31:38.647076+0000","last_active":"2024-09-10T11:31:41.659663+0000","last_peered":"2024-09-10T11:31:41.659663+0000","last_clean":"2024-09-10T11:31:41.659663+0000","last_became_active":"2024-09-10T11:31:38.644372+0000","last_became_peered":"2024-09-10T11:31:38.644372+0000","last_unstale":"2024-09-10T11:31:41.659663+0000","last_undegraded":"2024-09-10T11:31:41.659663+0000","last_fullsized":"2024-09-10T11:31:41.659663+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:24.834804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871333+0000","last_change":"2024-09-10T11:31:39.653224+0000","last_active":"2024-09-10T11:31:41.871333+0000","last_peered":"2024-09-10T11:31:41.871333+0000","last_clean":"2024-09-10T11:31:41.871333+0000","last_became_active":"2024-09-10T11:31:39.652149+0000","last_became_peered":"2024-09-10T11:31:39.652149+0000","last_unstale":"2024-09-10T11:31:41.871333+0000","last_undegraded":"2024-09-10T11:31:41.871333+0000","last_fullsized":"2024-09-10T11:31:41.871333+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:41.564425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643711+0000","last_change":"2024-09-10T11:31:38.645179+0000","last_active":"2024-09-10T11:31:40.643711+0000","last_peered":"2024-09-10T11:31:40.643711+0000","last_clean":"2024-09-10T11:31:40.643711+0000","last_became_active":"2024-09-10T11:31:38.644585+0000","last_became_peered":"2024-09-10T11:31:38.644585+0000","last_unstale":"2024-09-10T11:31:40.643711+0000","last_undegraded":"2024-09-10T11:31:40.643711+0000","last_fullsized":"2024-09-10T11:31:40.643711+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:41.481696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659066+0000","last_change":"2024-09-10T11:31:39.648380+0000","last_active":"2024-09-10T11:31:41.659066+0000","last_peered":"2024-09-10T11:31:41.659066+0000","last_clean":"2024-09-10T11:31:41.659066+0000","last_became_active":"2024-09-10T11:31:39.647706+0000","last_became_peered":"2024-09-10T11:31:39.647706+0000","last_unstale":"2024-09-10T11:31:41.659066+0000","last_undegraded":"2024-09-10T11:31:41.659066+0000","last_fullsized":"2024-09-10T11:31:41.659066+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:01.152814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871691+0000","last_change":"2024-09-10T11:31:38.646450+0000","last_active":"2024-09-10T11:31:41.871691+0000","last_peered":"2024-09-10T11:31:41.871691+0000","last_clean":"2024-09-10T11:31:41.871691+0000","last_became_active":"2024-09-10T11:31:38.645570+0000","last_became_peered":"2024-09-10T11:31:38.645570+0000","last_unstale":"2024-09-10T11:31:41.871691+0000","last_undegraded":"2024-09-10T11:31:41.871691+0000","last_fullsized":"2024-09-10T11:31:41.871691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:35:18.608550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081490+0000","last_change":"2024-09-10T11:31:39.651334+0000","last_active":"2024-09-10T11:31:42.081490+0000","last_peered":"2024-09-10T11:31:42.081490+0000","last_clean":"2024-09-10T11:31:42.081490+0000","last_became_active":"2024-09-10T11:31:39.651115+0000","last_became_peered":"2024-09-10T11:31:39.651115+0000","last_unstale":"2024-09-10T11:31:42.081490+0000","last_undegraded":"2024-09-10T11:31:42.081490+0000","last_fullsized":"2024-09-10T11:31:42.081490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:19.977270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081633+0000","last_change":"2024-09-10T11:31:38.644876+0000","last_active":"2024-09-10T11:31:42.081633+0000","last_peered":"2024-09-10T11:31:42.081633+0000","last_clean":"2024-09-10T11:31:42.081633+0000","last_became_active":"2024-09-10T11:31:38.644638+0000","last_became_peered":"2024-09-10T11:31:38.644638+0000","last_unstale":"2024-09-10T11:31:42.081633+0000","last_undegraded":"2024-09-10T11:31:42.081633+0000","last_fullsized":"2024-09-10T11:31:42.081633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:06.837861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081274+0000","last_change":"2024-09-10T11:31:39.651235+0000","last_active":"2024-09-10T11:31:42.081274+0000","last_peered":"2024-09-10T11:31:42.081274+0000","last_clean":"2024-09-10T11:31:42.081274+0000","last_became_active":"2024-09-10T11:31:39.650992+0000","last_became_peered":"2024-09-10T11:31:39.650992+0000","last_unstale":"2024-09-10T11:31:42.081274+0000","last_undegraded":"2024-09-10T11:31:42.081274+0000","last_fullsized":"2024-09-10T11:31:42.081274+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:09.775618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871445+0000","last_change":"2024-09-10T11:31:38.644477+0000","last_active":"2024-09-10T11:31:41.871445+0000","last_peered":"2024-09-10T11:31:41.871445+0000","last_clean":"2024-09-10T11:31:41.871445+0000","last_became_active":"2024-09-10T11:31:38.644004+0000","last_became_peered":"2024-09-10T11:31:38.644004+0000","last_unstale":"2024-09-10T11:31:41.871445+0000","last_undegraded":"2024-09-10T11:31:41.871445+0000","last_fullsized":"2024-09-10T11:31:41.871445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:30.845972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657805+0000","last_change":"2024-09-10T11:31:39.659038+0000","last_active":"2024-09-10T11:31:41.657805+0000","last_peered":"2024-09-10T11:31:41.657805+0000","last_clean":"2024-09-10T11:31:41.657805+0000","last_became_active":"2024-09-10T11:31:39.658627+0000","last_became_peered":"2024-09-10T11:31:39.658627+0000","last_unstale":"2024-09-10T11:31:41.657805+0000","last_undegraded":"2024-09-10T11:31:41.657805+0000","last_fullsized":"2024-09-10T11:31:41.657805+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:39.568361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646809+0000","last_change":"2024-09-10T11:31:38.641412+0000","last_active":"2024-09-10T11:31:40.646809+0000","last_peered":"2024-09-10T11:31:40.646809+0000","last_clean":"2024-09-10T11:31:40.646809+0000","last_became_active":"2024-09-10T11:31:38.641198+0000","last_became_peered":"2024-09-10T11:31:38.641198+0000","last_unstale":"2024-09-10T11:31:40.646809+0000","last_undegraded":"2024-09-10T11:31:40.646809+0000","last_fullsized":"2024-09-10T11:31:40.646809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:51.831719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659238+0000","last_change":"2024-09-10T11:31:39.654093+0000","last_active":"2024-09-10T11:31:41.659238+0000","last_peered":"2024-09-10T11:31:41.659238+0000","last_clean":"2024-09-10T11:31:41.659238+0000","last_became_active":"2024-09-10T11:31:39.653847+0000","last_became_peered":"2024-09-10T11:31:39.653847+0000","last_unstale":"2024-09-10T11:31:41.659238+0000","last_undegraded":"2024-09-10T11:31:41.659238+0000","last_fullsized":"2024-09-10T11:31:41.659238+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:44.950654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871592+0000","last_change":"2024-09-10T11:31:38.650626+0000","last_active":"2024-09-10T11:31:41.871592+0000","last_peered":"2024-09-10T11:31:41.871592+0000","last_clean":"2024-09-10T11:31:41.871592+0000","last_became_active":"2024-09-10T11:31:38.649886+0000","last_became_peered":"2024-09-10T11:31:38.649886+0000","last_unstale":"2024-09-10T11:31:41.871592+0000","last_undegraded":"2024-09-10T11:31:41.871592+0000","last_fullsized":"2024-09-10T11:31:41.871592+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:13.577324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081420+0000","last_change":"2024-09-10T11:31:39.654867+0000","last_active":"2024-09-10T11:31:42.081420+0000","last_peered":"2024-09-10T11:31:42.081420+0000","last_clean":"2024-09-10T11:31:42.081420+0000","last_became_active":"2024-09-10T11:31:39.654510+0000","last_became_peered":"2024-09-10T11:31:39.654510+0000","last_unstale":"2024-09-10T11:31:42.081420+0000","last_undegraded":"2024-09-10T11:31:42.081420+0000","last_fullsized":"2024-09-10T11:31:42.081420+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:19.770214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470705+0000","last_change":"2024-09-10T11:31:38.645340+0000","last_active":"2024-09-10T11:31:41.470705+0000","last_peered":"2024-09-10T11:31:41.470705+0000","last_clean":"2024-09-10T11:31:41.470705+0000","last_became_active":"2024-09-10T11:31:38.644397+0000","last_became_peered":"2024-09-10T11:31:38.644397+0000","last_unstale":"2024-09-10T11:31:41.470705+0000","last_undegraded":"2024-09-10T11:31:41.470705+0000","last_fullsized":"2024-09-10T11:31:41.470705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:31.391443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470508+0000","last_change":"2024-09-10T11:31:39.644876+0000","last_active":"2024-09-10T11:31:41.470508+0000","last_peered":"2024-09-10T11:31:41.470508+0000","last_clean":"2024-09-10T11:31:41.470508+0000","last_became_active":"2024-09-10T11:31:39.644621+0000","last_became_peered":"2024-09-10T11:31:39.644621+0000","last_unstale":"2024-09-10T11:31:41.470508+0000","last_undegraded":"2024-09-10T11:31:41.470508+0000","last_fullsized":"2024-09-10T11:31:41.470508+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:47.080676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644059+0000","last_change":"2024-09-10T11:31:38.644155+0000","last_active":"2024-09-10T11:31:40.644059+0000","last_peered":"2024-09-10T11:31:40.644059+0000","last_clean":"2024-09-10T11:31:40.644059+0000","last_became_active":"2024-09-10T11:31:38.643945+0000","last_became_peered":"2024-09-10T11:31:38.643945+0000","last_unstale":"2024-09-10T11:31:40.644059+0000","last_undegraded":"2024-09-10T11:31:40.644059+0000","last_fullsized":"2024-09-10T11:31:40.644059+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:21.520519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470393+0000","last_change":"2024-09-10T11:31:39.647987+0000","last_active":"2024-09-10T11:31:41.470393+0000","last_peered":"2024-09-10T11:31:41.470393+0000","last_clean":"2024-09-10T11:31:41.470393+0000","last_became_active":"2024-09-10T11:31:39.647749+0000","last_became_peered":"2024-09-10T11:31:39.647749+0000","last_unstale":"2024-09-10T11:31:41.470393+0000","last_undegraded":"2024-09-10T11:31:41.470393+0000","last_fullsized":"2024-09-10T11:31:41.470393+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:26.433379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641168+0000","last_change":"2024-09-10T11:31:38.650733+0000","last_active":"2024-09-10T11:31:40.641168+0000","last_peered":"2024-09-10T11:31:40.641168+0000","last_clean":"2024-09-10T11:31:40.641168+0000","last_became_active":"2024-09-10T11:31:38.649956+0000","last_became_peered":"2024-09-10T11:31:38.649956+0000","last_unstale":"2024-09-10T11:31:40.641168+0000","last_undegraded":"2024-09-10T11:31:40.641168+0000","last_fullsized":"2024-09-10T11:31:40.641168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:57.508017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646938+0000","last_change":"2024-09-10T11:31:39.646806+0000","last_active":"2024-09-10T11:31:40.646938+0000","last_peered":"2024-09-10T11:31:40.646938+0000","last_clean":"2024-09-10T11:31:40.646938+0000","last_became_active":"2024-09-10T11:31:39.646247+0000","last_became_peered":"2024-09-10T11:31:39.646247+0000","last_unstale":"2024-09-10T11:31:40.646938+0000","last_undegraded":"2024-09-10T11:31:40.646938+0000","last_fullsized":"2024-09-10T11:31:40.646938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:15.258005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871032+0000","last_change":"2024-09-10T11:31:38.649645+0000","last_active":"2024-09-10T11:31:41.871032+0000","last_peered":"2024-09-10T11:31:41.871032+0000","last_clean":"2024-09-10T11:31:41.871032+0000","last_became_active":"2024-09-10T11:31:38.649397+0000","last_became_peered":"2024-09-10T11:31:38.649397+0000","last_unstale":"2024-09-10T11:31:41.871032+0000","last_undegraded":"2024-09-10T11:31:41.871032+0000","last_fullsized":"2024-09-10T11:31:41.871032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:06.616794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470641+0000","last_change":"2024-09-10T11:31:39.651245+0000","last_active":"2024-09-10T11:31:41.470641+0000","last_peered":"2024-09-10T11:31:41.470641+0000","last_clean":"2024-09-10T11:31:41.470641+0000","last_became_active":"2024-09-10T11:31:39.651006+0000","last_became_peered":"2024-09-10T11:31:39.651006+0000","last_unstale":"2024-09-10T11:31:41.470641+0000","last_undegraded":"2024-09-10T11:31:41.470641+0000","last_fullsized":"2024-09-10T11:31:41.470641+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:37:47.083732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657907+0000","last_change":"2024-09-10T11:31:38.648863+0000","last_active":"2024-09-10T11:31:41.657907+0000","last_peered":"2024-09-10T11:31:41.657907+0000","last_clean":"2024-09-10T11:31:41.657907+0000","last_became_active":"2024-09-10T11:31:38.648599+0000","last_became_peered":"2024-09-10T11:31:38.648599+0000","last_unstale":"2024-09-10T11:31:41.657907+0000","last_undegraded":"2024-09-10T11:31:41.657907+0000","last_fullsized":"2024-09-10T11:31:41.657907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:42:33.989466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470512+0000","last_change":"2024-09-10T11:31:39.658976+0000","last_active":"2024-09-10T11:31:41.470512+0000","last_peered":"2024-09-10T11:31:41.470512+0000","last_clean":"2024-09-10T11:31:41.470512+0000","last_became_active":"2024-09-10T11:31:39.658429+0000","last_became_peered":"2024-09-10T11:31:39.658429+0000","last_unstale":"2024-09-10T11:31:41.470512+0000","last_undegraded":"2024-09-10T11:31:41.470512+0000","last_fullsized":"2024-09-10T11:31:41.470512+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:44.516767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081429+0000","last_change":"2024-09-10T11:31:38.647793+0000","last_active":"2024-09-10T11:31:42.081429+0000","last_peered":"2024-09-10T11:31:42.081429+0000","last_clean":"2024-09-10T11:31:42.081429+0000","last_became_active":"2024-09-10T11:31:38.647565+0000","last_became_peered":"2024-09-10T11:31:38.647565+0000","last_unstale":"2024-09-10T11:31:42.081429+0000","last_undegraded":"2024-09-10T11:31:42.081429+0000","last_fullsized":"2024-09-10T11:31:42.081429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:36.572000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.647006+0000","last_change":"2024-09-10T11:31:39.654846+0000","last_active":"2024-09-10T11:31:40.647006+0000","last_peered":"2024-09-10T11:31:40.647006+0000","last_clean":"2024-09-10T11:31:40.647006+0000","last_became_active":"2024-09-10T11:31:39.654329+0000","last_became_peered":"2024-09-10T11:31:39.654329+0000","last_unstale":"2024-09-10T11:31:40.647006+0000","last_undegraded":"2024-09-10T11:31:40.647006+0000","last_fullsized":"2024-09-10T11:31:40.647006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:42.607436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081715+0000","last_change":"2024-09-10T11:31:38.646374+0000","last_active":"2024-09-10T11:31:42.081715+0000","last_peered":"2024-09-10T11:31:42.081715+0000","last_clean":"2024-09-10T11:31:42.081715+0000","last_became_active":"2024-09-10T11:31:38.646154+0000","last_became_peered":"2024-09-10T11:31:38.646154+0000","last_unstale":"2024-09-10T11:31:42.081715+0000","last_undegraded":"2024-09-10T11:31:42.081715+0000","last_fullsized":"2024-09-10T11:31:42.081715+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:08:27.682580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081558+0000","last_change":"2024-09-10T11:31:39.655470+0000","last_active":"2024-09-10T11:31:42.081558+0000","last_peered":"2024-09-10T11:31:42.081558+0000","last_clean":"2024-09-10T11:31:42.081558+0000","last_became_active":"2024-09-10T11:31:39.655200+0000","last_became_peered":"2024-09-10T11:31:39.655200+0000","last_unstale":"2024-09-10T11:31:42.081558+0000","last_undegraded":"2024-09-10T11:31:42.081558+0000","last_fullsized":"2024-09-10T11:31:42.081558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:53.807550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641246+0000","last_change":"2024-09-10T11:31:38.650913+0000","last_active":"2024-09-10T11:31:40.641246+0000","last_peered":"2024-09-10T11:31:40.641246+0000","last_clean":"2024-09-10T11:31:40.641246+0000","last_became_active":"2024-09-10T11:31:38.650275+0000","last_became_peered":"2024-09-10T11:31:38.650275+0000","last_unstale":"2024-09-10T11:31:40.641246+0000","last_undegraded":"2024-09-10T11:31:40.641246+0000","last_fullsized":"2024-09-10T11:31:40.641246+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:24.669282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871491+0000","last_change":"2024-09-10T11:31:39.657948+0000","last_active":"2024-09-10T11:31:41.871491+0000","last_peered":"2024-09-10T11:31:41.871491+0000","last_clean":"2024-09-10T11:31:41.871491+0000","last_became_active":"2024-09-10T11:31:39.657439+0000","last_became_peered":"2024-09-10T11:31:39.657439+0000","last_unstale":"2024-09-10T11:31:41.871491+0000","last_undegraded":"2024-09-10T11:31:41.871491+0000","last_fullsized":"2024-09-10T11:31:41.871491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:58.280079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871110+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871110+0000","last_peered":"2024-09-10T11:31:41.871110+0000","last_clean":"2024-09-10T11:31:41.871110+0000","last_became_active":"2024-09-10T11:31:38.647865+0000","last_became_peered":"2024-09-10T11:31:38.647865+0000","last_unstale":"2024-09-10T11:31:41.871110+0000","last_undegraded":"2024-09-10T11:31:41.871110+0000","last_fullsized":"2024-09-10T11:31:41.871110+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:31.961548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641242+0000","last_change":"2024-09-10T11:31:39.644873+0000","last_active":"2024-09-10T11:31:40.641242+0000","last_peered":"2024-09-10T11:31:40.641242+0000","last_clean":"2024-09-10T11:31:40.641242+0000","last_became_active":"2024-09-10T11:31:39.644154+0000","last_became_peered":"2024-09-10T11:31:39.644154+0000","last_unstale":"2024-09-10T11:31:40.641242+0000","last_undegraded":"2024-09-10T11:31:40.641242+0000","last_fullsized":"2024-09-10T11:31:40.641242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:11.485938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643759+0000","last_change":"2024-09-10T11:31:38.648391+0000","last_active":"2024-09-10T11:31:40.643759+0000","last_peered":"2024-09-10T11:31:40.643759+0000","last_clean":"2024-09-10T11:31:40.643759+0000","last_became_active":"2024-09-10T11:31:38.648144+0000","last_became_peered":"2024-09-10T11:31:38.648144+0000","last_unstale":"2024-09-10T11:31:40.643759+0000","last_undegraded":"2024-09-10T11:31:40.643759+0000","last_fullsized":"2024-09-10T11:31:40.643759+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:03.992675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658135+0000","last_change":"2024-09-10T11:31:39.654383+0000","last_active":"2024-09-10T11:31:41.658135+0000","last_peered":"2024-09-10T11:31:41.658135+0000","last_clean":"2024-09-10T11:31:41.658135+0000","last_became_active":"2024-09-10T11:31:39.653946+0000","last_became_peered":"2024-09-10T11:31:39.653946+0000","last_unstale":"2024-09-10T11:31:41.658135+0000","last_undegraded":"2024-09-10T11:31:41.658135+0000","last_fullsized":"2024-09-10T11:31:41.658135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:27.993903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871742+0000","last_change":"2024-09-10T11:31:38.648063+0000","last_active":"2024-09-10T11:31:41.871742+0000","last_peered":"2024-09-10T11:31:41.871742+0000","last_clean":"2024-09-10T11:31:41.871742+0000","last_became_active":"2024-09-10T11:31:38.647836+0000","last_became_peered":"2024-09-10T11:31:38.647836+0000","last_unstale":"2024-09-10T11:31:41.871742+0000","last_undegraded":"2024-09-10T11:31:41.871742+0000","last_fullsized":"2024-09-10T11:31:41.871742+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:39.211943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657981+0000","last_change":"2024-09-10T11:31:39.652538+0000","last_active":"2024-09-10T11:31:41.657981+0000","last_peered":"2024-09-10T11:31:41.657981+0000","last_clean":"2024-09-10T11:31:41.657981+0000","last_became_active":"2024-09-10T11:31:39.652254+0000","last_became_peered":"2024-09-10T11:31:39.652254+0000","last_unstale":"2024-09-10T11:31:41.657981+0000","last_undegraded":"2024-09-10T11:31:41.657981+0000","last_fullsized":"2024-09-10T11:31:41.657981+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:39.583627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659103+0000","last_change":"2024-09-10T11:31:38.645087+0000","last_active":"2024-09-10T11:31:41.659103+0000","last_peered":"2024-09-10T11:31:41.659103+0000","last_clean":"2024-09-10T11:31:41.659103+0000","last_became_active":"2024-09-10T11:31:38.644513+0000","last_became_peered":"2024-09-10T11:31:38.644513+0000","last_unstale":"2024-09-10T11:31:41.659103+0000","last_undegraded":"2024-09-10T11:31:41.659103+0000","last_fullsized":"2024-09-10T11:31:41.659103+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:41.110510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659259+0000","last_change":"2024-09-10T11:31:39.653575+0000","last_active":"2024-09-10T11:31:41.659259+0000","last_peered":"2024-09-10T11:31:41.659259+0000","last_clean":"2024-09-10T11:31:41.659259+0000","last_became_active":"2024-09-10T11:31:39.653326+0000","last_became_peered":"2024-09-10T11:31:39.653326+0000","last_unstale":"2024-09-10T11:31:41.659259+0000","last_undegraded":"2024-09-10T11:31:41.659259+0000","last_fullsized":"2024-09-10T11:31:41.659259+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:18.047600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659164+0000","last_change":"2024-09-10T11:31:38.646127+0000","last_active":"2024-09-10T11:31:41.659164+0000","last_peered":"2024-09-10T11:31:41.659164+0000","last_clean":"2024-09-10T11:31:41.659164+0000","last_became_active":"2024-09-10T11:31:38.645283+0000","last_became_peered":"2024-09-10T11:31:38.645283+0000","last_unstale":"2024-09-10T11:31:41.659164+0000","last_undegraded":"2024-09-10T11:31:41.659164+0000","last_fullsized":"2024-09-10T11:31:41.659164+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:35.070668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871485+0000","last_change":"2024-09-10T11:31:39.652663+0000","last_active":"2024-09-10T11:31:41.871485+0000","last_peered":"2024-09-10T11:31:41.871485+0000","last_clean":"2024-09-10T11:31:41.871485+0000","last_became_active":"2024-09-10T11:31:39.651824+0000","last_became_peered":"2024-09-10T11:31:39.651824+0000","last_unstale":"2024-09-10T11:31:41.871485+0000","last_undegraded":"2024-09-10T11:31:41.871485+0000","last_fullsized":"2024-09-10T11:31:41.871485+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:12.395549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.643885+0000","last_change":"2024-09-10T11:31:38.647875+0000","last_active":"2024-09-10T11:31:40.643885+0000","last_peered":"2024-09-10T11:31:40.643885+0000","last_clean":"2024-09-10T11:31:40.643885+0000","last_became_active":"2024-09-10T11:31:38.647001+0000","last_became_peered":"2024-09-10T11:31:38.647001+0000","last_unstale":"2024-09-10T11:31:40.643885+0000","last_undegraded":"2024-09-10T11:31:40.643885+0000","last_fullsized":"2024-09-10T11:31:40.643885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:46.157562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659210+0000","last_change":"2024-09-10T11:31:39.653859+0000","last_active":"2024-09-10T11:31:41.659210+0000","last_peered":"2024-09-10T11:31:41.659210+0000","last_clean":"2024-09-10T11:31:41.659210+0000","last_became_active":"2024-09-10T11:31:39.653630+0000","last_became_peered":"2024-09-10T11:31:39.653630+0000","last_unstale":"2024-09-10T11:31:41.659210+0000","last_undegraded":"2024-09-10T11:31:41.659210+0000","last_fullsized":"2024-09-10T11:31:41.659210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:06.312549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470757+0000","last_change":"2024-09-10T11:31:38.645848+0000","last_active":"2024-09-10T11:31:41.470757+0000","last_peered":"2024-09-10T11:31:41.470757+0000","last_clean":"2024-09-10T11:31:41.470757+0000","last_became_active":"2024-09-10T11:31:38.645636+0000","last_became_peered":"2024-09-10T11:31:38.645636+0000","last_unstale":"2024-09-10T11:31:41.470757+0000","last_undegraded":"2024-09-10T11:31:41.470757+0000","last_fullsized":"2024-09-10T11:31:41.470757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:37.623350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.871341+0000","last_change":"2024-09-10T11:31:39.656607+0000","last_active":"2024-09-10T11:31:41.871341+0000","last_peered":"2024-09-10T11:31:41.871341+0000","last_clean":"2024-09-10T11:31:41.871341+0000","last_became_active":"2024-09-10T11:31:39.656322+0000","last_became_peered":"2024-09-10T11:31:39.656322+0000","last_unstale":"2024-09-10T11:31:41.871341+0000","last_undegraded":"2024-09-10T11:31:41.871341+0000","last_fullsized":"2024-09-10T11:31:41.871341+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:44.579593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.644106+0000","last_change":"2024-09-10T11:31:38.646845+0000","last_active":"2024-09-10T11:31:40.644106+0000","last_peered":"2024-09-10T11:31:40.644106+0000","last_clean":"2024-09-10T11:31:40.644106+0000","last_became_active":"2024-09-10T11:31:38.646550+0000","last_became_peered":"2024-09-10T11:31:38.646550+0000","last_unstale":"2024-09-10T11:31:40.644106+0000","last_undegraded":"2024-09-10T11:31:40.644106+0000","last_fullsized":"2024-09-10T11:31:40.644106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:48.907368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.658289+0000","last_change":"2024-09-10T11:31:39.653111+0000","last_active":"2024-09-10T11:31:41.658289+0000","last_peered":"2024-09-10T11:31:41.658289+0000","last_clean":"2024-09-10T11:31:41.658289+0000","last_became_active":"2024-09-10T11:31:39.651634+0000","last_became_peered":"2024-09-10T11:31:39.651634+0000","last_unstale":"2024-09-10T11:31:41.658289+0000","last_undegraded":"2024-09-10T11:31:41.658289+0000","last_fullsized":"2024-09-10T11:31:41.658289+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:16:12.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659399+0000","last_change":"2024-09-10T11:31:38.649507+0000","last_active":"2024-09-10T11:31:41.659399+0000","last_peered":"2024-09-10T11:31:41.659399+0000","last_clean":"2024-09-10T11:31:41.659399+0000","last_became_active":"2024-09-10T11:31:38.649245+0000","last_became_peered":"2024-09-10T11:31:38.649245+0000","last_unstale":"2024-09-10T11:31:41.659399+0000","last_undegraded":"2024-09-10T11:31:41.659399+0000","last_fullsized":"2024-09-10T11:31:41.659399+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:28.249420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081086+0000","last_change":"2024-09-10T11:31:39.653066+0000","last_active":"2024-09-10T11:31:42.081086+0000","last_peered":"2024-09-10T11:31:42.081086+0000","last_clean":"2024-09-10T11:31:42.081086+0000","last_became_active":"2024-09-10T11:31:39.652856+0000","last_became_peered":"2024-09-10T11:31:39.652856+0000","last_unstale":"2024-09-10T11:31:42.081086+0000","last_undegraded":"2024-09-10T11:31:42.081086+0000","last_fullsized":"2024-09-10T11:31:42.081086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:11.114591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.641445+0000","last_change":"2024-09-10T11:31:38.649054+0000","last_active":"2024-09-10T11:31:40.641445+0000","last_peered":"2024-09-10T11:31:40.641445+0000","last_clean":"2024-09-10T11:31:40.641445+0000","last_became_active":"2024-09-10T11:31:38.648838+0000","last_became_peered":"2024-09-10T11:31:38.648838+0000","last_unstale":"2024-09-10T11:31:40.641445+0000","last_undegraded":"2024-09-10T11:31:40.641445+0000","last_fullsized":"2024-09-10T11:31:40.641445+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:47.015802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:40.646718+0000","last_change":"2024-09-10T11:31:39.649938+0000","last_active":"2024-09-10T11:31:40.646718+0000","last_peered":"2024-09-10T11:31:40.646718+0000","last_clean":"2024-09-10T11:31:40.646718+0000","last_became_active":"2024-09-10T11:31:39.649709+0000","last_became_peered":"2024-09-10T11:31:39.649709+0000","last_unstale":"2024-09-10T11:31:40.646718+0000","last_undegraded":"2024-09-10T11:31:40.646718+0000","last_fullsized":"2024-09-10T11:31:40.646718+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:33.341943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.470585+0000","last_change":"2024-09-10T11:31:38.649265+0000","last_active":"2024-09-10T11:31:41.470585+0000","last_peered":"2024-09-10T11:31:41.470585+0000","last_clean":"2024-09-10T11:31:41.470585+0000","last_became_active":"2024-09-10T11:31:38.648733+0000","last_became_peered":"2024-09-10T11:31:38.648733+0000","last_unstale":"2024-09-10T11:31:41.470585+0000","last_undegraded":"2024-09-10T11:31:41.470585+0000","last_fullsized":"2024-09-10T11:31:41.470585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:06:13.293892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.657783+0000","last_change":"2024-09-10T11:31:39.654260+0000","last_active":"2024-09-10T11:31:41.657783+0000","last_peered":"2024-09-10T11:31:41.657783+0000","last_clean":"2024-09-10T11:31:41.657783+0000","last_became_active":"2024-09-10T11:31:39.652267+0000","last_became_peered":"2024-09-10T11:31:39.652267+0000","last_unstale":"2024-09-10T11:31:41.657783+0000","last_undegraded":"2024-09-10T11:31:41.657783+0000","last_fullsized":"2024-09-10T11:31:41.657783+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:38.632524+0000","last_clean_scrub_stamp":"2024-09-10T11:31:38.632524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:27.642082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:41.659343+0000","last_change":"2024-09-10T11:31:38.648727+0000","last_active":"2024-09-10T11:31:41.659343+0000","last_peered":"2024-09-10T11:31:41.659343+0000","last_clean":"2024-09-10T11:31:41.659343+0000","last_became_active":"2024-09-10T11:31:38.648279+0000","last_became_peered":"2024-09-10T11:31:38.648279+0000","last_unstale":"2024-09-10T11:31:41.659343+0000","last_undegraded":"2024-09-10T11:31:41.659343+0000","last_fullsized":"2024-09-10T11:31:41.659343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:31:37.625976+0000","last_clean_scrub_stamp":"2024-09-10T11:31:37.625976+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:06.205973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-10T11:31:42.081282+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:31:42.081282+0000","last_peered":"2024-09-10T11:31:42.081282+0000","last_clean":"2024-09-10T11:31:42.081282+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:31:42.081282+0000","last_undegraded":"2024-09-10T11:31:42.081282+0000","last_fullsized":"2024-09-10T11:31:42.081282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:20.522739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785659,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785077,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195303,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195101,"statfs":{"total":200000000,"available":199783989,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195101,"statfs":{"total":200000000,"available":199784081,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195103,"statfs":{"total":200000000,"available":199785603,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":785,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194526,"statfs":{"total":200000000,"available":199195255,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":209,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195102,"statfs":{"total":200000000,"available":199785021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:31:47.248 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:31:47.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:31:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.390+0000 7f0812a9b640 1 Processor -- start 2024-09-10T11:31:47.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.390+0000 7f0812a9b640 1 -- start start 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0812a9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f080c15c7b0 0x7f080c0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0812a9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0812a9b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0ceb10 con 0x7f080c06ee30 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0812a9b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f080c0d1420 con 0x7f080c15c7b0 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0812a9b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f080c06f2a0 con 0x7f080c0d0fb0 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0810837640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f080c15c7b0 0x7f080c0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0811038640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f080c06ee30 0x7f080c1604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37090/0 (socket says 172.21.15.99:37090) 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0811038640 1 -- 172.21.15.99:0/340156436 learned_addr learned my addr 172.21.15.99:0/340156436 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:47.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0810837640 1 -- 172.21.15.99:0/340156436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 msgr2=0x7f080c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0803fff640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f08037fe640 1 -- 172.21.15.99:0/340156436 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166515995 0 0) 0x7f0804001490 con 0x7f080c06ee30 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0810837640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0810837640 1 -- 172.21.15.99:0/340156436 >> v1:172.21.15.99:6789/0 conn(0x7f080c06ee30 legacy=0x7f080c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0810837640 1 -- 172.21.15.99:0/340156436 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0b3a90 con 0x7f080c15c7b0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.391+0000 7f0803fff640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.392+0000 7f0810837640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c15c7b0 0x7f080c0d0a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08080097a0 tx=0x7f080802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cab38e7e7b7f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.392+0000 7f08037fe640 1 -- 172.21.15.99:0/340156436 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808037070 con 0x7f080c15c7b0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.392+0000 7f08037fe640 1 -- 172.21.15.99:0/340156436 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0808032440 con 0x7f080c15c7b0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.392+0000 7f08037fe640 1 -- 172.21.15.99:0/340156436 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808032810 con 0x7f080c15c7b0 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 -- 172.21.15.99:0/340156436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c15c7b0 msgr2=0x7f080c0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c15c7b0 0x7f080c0d0a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f08080097a0 tx=0x7f080802a180 comp rx=0 tx=0).stop 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 -- 172.21.15.99:0/340156436 shutdown_connections 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 --2- 172.21.15.99:0/340156436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c15c7b0 0x7f080c0d0a70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 -- 172.21.15.99:0/340156436 >> 172.21.15.99:0/340156436 conn(0x7f080c06bc10 msgr2=0x7f080c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 -- 172.21.15.99:0/340156436 shutdown_connections 2024-09-10T11:31:47.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.393+0000 7f0812a9b640 1 -- 172.21.15.99:0/340156436 wait complete. 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.394+0000 7f0812a9b640 1 Processor -- start 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.394+0000 7f0812a9b640 1 -- start start 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 0x7f080c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0b4050 con 0x7f080c06ee30 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0b3840 con 0x7f080c15c7b0 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0812a9b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f080c0b4330 con 0x7f080c0d0fb0 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60558/0 (socket says 172.21.15.99:60558) 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 -- 172.21.15.99:0/3430873596 learned_addr learned my addr 172.21.15.99:0/3430873596 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0811038640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 0x7f080c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0803fff640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 -- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 msgr2=0x7f080c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 -- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 msgr2=0x7f080c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 0x7f080c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 -- 172.21.15.99:0/3430873596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c072650 con 0x7f080c06ee30 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0811038640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 0x7f080c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0803fff640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.395+0000 7f0810837640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0808035040 tx=0x7f08080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.396+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808037040 con 0x7f080c06ee30 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.396+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0808030770 con 0x7f080c06ee30 2024-09-10T11:31:47.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.396+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808030b80 con 0x7f080c06ee30 2024-09-10T11:31:47.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.396+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c072930 con 0x7f080c06ee30 2024-09-10T11:31:47.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.396+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c072ef0 con 0x7f080c06ee30 2024-09-10T11:31:47.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.397+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c15cb90 con 0x7f080c06ee30 2024-09-10T11:31:47.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.397+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0808044020 con 0x7f080c06ee30 2024-09-10T11:31:47.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.398+0000 7f08017fa640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 0x7f07f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.398+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f08080ab4d0 con 0x7f080c06ee30 2024-09-10T11:31:47.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.398+0000 7f0803fff640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 0x7f07f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.398+0000 7f0803fff640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 0x7f07f806e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07fc006fd0 tx=0x7f07fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.402+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0808032e20 con 0x7f080c06ee30 2024-09-10T11:31:47.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.633+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f080c06d840 con 0x7f080c06ee30 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.635+0000 7f08017fa640 1 -- 172.21.15.99:0/3430873596 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f08080a9e20 con 0x7f080c06ee30 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:31:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:31:47.637 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:31:47.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.637+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 msgr2=0x7f07f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.637+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 0x7f07f806e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f07fc006fd0 tx=0x7f07fc008040 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.637+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 msgr2=0x7f080c14de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.637+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0808035040 tx=0x7f08080046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 shutdown_connections 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07f806be30 0x7f07f806e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f080c15c7b0 0x7f080c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f080c0d0fb0 0x7f080c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 --2- 172.21.15.99:0/3430873596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f080c06ee30 0x7f080c14de40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 >> 172.21.15.99:0/3430873596 conn(0x7f080c06bc10 msgr2=0x7f080c0d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 shutdown_connections 2024-09-10T11:31:47.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.638+0000 7f0812a9b640 1 -- 172.21.15.99:0/3430873596 wait complete. 2024-09-10T11:31:47.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:47.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.749+0000 7f0342e82640 1 Processor -- start 2024-09-10T11:31:47.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.749+0000 7f0342e82640 1 -- start start 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0342e82640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f033c073440 0x7f033c15c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0342e82640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033c152e60 con 0x7f033c0d1200 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0342e82640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033c0d15e0 con 0x7f033c06ece0 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0342e82640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f033c072680 con 0x7f033c073440 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0340c1e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f033c0d1200 0x7f033c06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37106/0 (socket says 172.21.15.99:37106) 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f0340c1e640 1 -- 172.21.15.99:0/2986956320 learned_addr learned my addr 172.21.15.99:0/2986956320 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f034141f640 1 --2- 172.21.15.99:0/2986956320 >> v2:172.21.15.99:3300/0 conn(0x7f033c073440 0x7f033c15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f034141f640 1 -- 172.21.15.99:0/2986956320 >> v1:172.21.15.196:6789/0 conn(0x7f033c06ece0 legacy=0x7f033c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f034141f640 1 -- 172.21.15.99:0/2986956320 >> v1:172.21.15.99:6789/0 conn(0x7f033c0d1200 legacy=0x7f033c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f034141f640 1 -- 172.21.15.99:0/2986956320 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c072960 con 0x7f033c073440 2024-09-10T11:31:47.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.750+0000 7f033b7fe640 1 -- 172.21.15.99:0/2986956320 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562007593 0 0) 0x7f033c152e60 con 0x7f033c0d1200 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f034141f640 1 --2- 172.21.15.99:0/2986956320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c15c770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03240097a0 tx=0x7f032402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b590054eae5bbd44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f033b7fe640 1 -- 172.21.15.99:0/2986956320 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324037070 con 0x7f033c073440 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f033b7fe640 1 -- 172.21.15.99:0/2986956320 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032402ae50 con 0x7f033c073440 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f033b7fe640 1 -- 172.21.15.99:0/2986956320 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324032660 con 0x7f033c073440 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 -- 172.21.15.99:0/2986956320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 msgr2=0x7f033c15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 --2- 172.21.15.99:0/2986956320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c15c770 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f03240097a0 tx=0x7f032402a140 comp rx=0 tx=0).stop 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 -- 172.21.15.99:0/2986956320 shutdown_connections 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 --2- 172.21.15.99:0/2986956320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c15c770 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 -- 172.21.15.99:0/2986956320 >> 172.21.15.99:0/2986956320 conn(0x7f033c06bc10 msgr2=0x7f033c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.751+0000 7f0342e82640 1 -- 172.21.15.99:0/2986956320 shutdown_connections 2024-09-10T11:31:47.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.752+0000 7f0342e82640 1 -- 172.21.15.99:0/2986956320 wait complete. 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.753+0000 7f0342e82640 1 Processor -- start 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.753+0000 7f0342e82640 1 -- start start 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.753+0000 7f0342e82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 0x7f033c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 0x7f033c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f033c15cd90 con 0x7f033c073440 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f033c072f20 con 0x7f033c0d1200 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f033c159b50 con 0x7f033c06ece0 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60580/0 (socket says 172.21.15.99:60580) 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 -- 172.21.15.99:0/2720547213 learned_addr learned my addr 172.21.15.99:0/2720547213 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0340c1e640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 0x7f033c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f034141f640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 0x7f033c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 -- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 msgr2=0x7f033c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 0x7f033c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 -- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 msgr2=0x7f033c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 0x7f033c15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 -- 172.21.15.99:0/2720547213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c0aeff0 con 0x7f033c073440 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0340c1e640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 0x7f033c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f034141f640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 0x7f033c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f033bfff640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c400 tx=0x7f032c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c011790 con 0x7f033c073440 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c017070 con 0x7f033c073440 2024-09-10T11:31:47.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c0af2d0 con 0x7f033c073440 2024-09-10T11:31:47.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c011bb0 con 0x7f033c073440 2024-09-10T11:31:47.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.754+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c0af890 con 0x7f033c073440 2024-09-10T11:31:47.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.755+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033c06e5d0 con 0x7f033c073440 2024-09-10T11:31:47.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.756+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f032c01c460 con 0x7f033c073440 2024-09-10T11:31:47.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.756+0000 7f03397fa640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:47.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.756+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f032c08acc0 con 0x7f033c073440 2024-09-10T11:31:47.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.756+0000 7f0340c1e640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:47.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.756+0000 7f0340c1e640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 0x7f031c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03340012e0 tx=0x7f0334005b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:47.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.760+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f032c059670 con 0x7f033c073440 2024-09-10T11:31:47.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.959+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f033c071ce0 con 0x7f033c073440 2024-09-10T11:31:47.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.959+0000 7f03397fa640 1 -- 172.21.15.99:0/2720547213 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7f032c058f70 con 0x7f033c073440 2024-09-10T11:31:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:47.960 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:31:47.961 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 msgr2=0x7f031c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 0x7f031c06e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03340012e0 tx=0x7f0334005b90 comp rx=0 tx=0).stop 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 msgr2=0x7f033c151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c400 tx=0x7f032c00c8c0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 shutdown_connections 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f031c06be30 0x7f031c06e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f033c0d1200 0x7f033c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f033c073440 0x7f033c151df0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 --2- 172.21.15.99:0/2720547213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f033c06ece0 0x7f033c0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:47.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 >> 172.21.15.99:0/2720547213 conn(0x7f033c06bc10 msgr2=0x7f033c15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:47.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 shutdown_connections 2024-09-10T11:31:47.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:47.961+0000 7f0342e82640 1 -- 172.21.15.99:0/2720547213 wait complete. 2024-09-10T11:31:48.375 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:31:38.982189+0000', 'modified': '2024-09-10T11:31:40.985291+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5211}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5211': {'gid': 5211, 'name': 'a', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.99:6836/3824950902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 3824950902}, {'type': 'v1', 'addr': '172.21.15.99:6836', 'nonce': 3824950902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5211, 'qdb_cluster': [5211]} 2024-09-10T11:31:48.376 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:31:48.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:31:48.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.475+0000 7fa4d3911640 1 Processor -- start 2024-09-10T11:31:48.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.475+0000 7fa4d3911640 1 -- start start 2024-09-10T11:31:48.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d3911640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4cc15b2d0 0x7fa4cc0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d3911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc0afbb0 0x7fa4cc0714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d3911640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4cc0b0020 con 0x7fa4cc15aef0 2024-09-10T11:31:48.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d3911640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc06ee10 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d3911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0cd9a0 con 0x7fa4cc0afbb0 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d16ad640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa4cc15aef0 0x7fa4cc0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37116/0 (socket says 172.21.15.99:37116) 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d0eac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa4cc15b2d0 0x7fa4cc0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d16ad640 1 -- 172.21.15.99:0/2112509880 learned_addr learned my addr 172.21.15.99:0/2112509880 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc0afbb0 0x7fa4cc0714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2112509880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc0afbb0 msgr2=0x7fa4cc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc0afbb0 0x7fa4cc0714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2112509880 >> v1:172.21.15.99:6789/0 conn(0x7fa4cc15aef0 legacy=0x7fa4cc0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4c3fff640 1 -- 172.21.15.99:0/2112509880 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367861083 0 0) 0x7fa4cc0b0020 con 0x7fa4cc15aef0 2024-09-10T11:31:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.476+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2112509880 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc0b0290 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.477+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15b2d0 0x7fa4cc0b4510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0097a0 tx=0x7fa4bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df483d22824eddb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.477+0000 7fa4c3fff640 1 -- 172.21.15.99:0/2112509880 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037070 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.477+0000 7fa4c3fff640 1 -- 172.21.15.99:0/2112509880 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc032440 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.477+0000 7fa4c3fff640 1 -- 172.21.15.99:0/2112509880 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc032810 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 -- 172.21.15.99:0/2112509880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15b2d0 msgr2=0x7fa4cc0b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15b2d0 0x7fa4cc0b4510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0097a0 tx=0x7fa4bc02a180 comp rx=0 tx=0).stop 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 -- 172.21.15.99:0/2112509880 shutdown_connections 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc0afbb0 0x7fa4cc0714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2112509880 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15b2d0 0x7fa4cc0b4510 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 -- 172.21.15.99:0/2112509880 >> 172.21.15.99:0/2112509880 conn(0x7fa4cc06bc10 msgr2=0x7fa4cc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 -- 172.21.15.99:0/2112509880 shutdown_connections 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.478+0000 7fa4d3911640 1 -- 172.21.15.99:0/2112509880 wait complete. 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.479+0000 7fa4d3911640 1 Processor -- start 2024-09-10T11:31:48.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.479+0000 7fa4d3911640 1 -- start start 2024-09-10T11:31:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc0afbb0 0x7fa4cc14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc15b2d0 0x7fa4cc0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc06eb30 con 0x7fa4cc15aef0 2024-09-10T11:31:48.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc06e850 con 0x7fa4cc0afbb0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d3911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0719e0 con 0x7fa4cc15b2d0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60608/0 (socket says 172.21.15.99:60608) 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2064886613 learned_addr learned my addr 172.21.15.99:0/2064886613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc15b2d0 0x7fa4cc0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc0afbb0 0x7fa4cc14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc15b2d0 msgr2=0x7fa4cc0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc15b2d0 0x7fa4cc0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc0afbb0 msgr2=0x7fa4cc14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc0afbb0 0x7fa4cc14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 -- 172.21.15.99:0/2064886613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc15c9b0 con 0x7fa4cc15aef0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.480+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc035040 tx=0x7fa4bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.481+0000 7fa4c1ffb640 1 -- 172.21.15.99:0/2064886613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc037040 con 0x7fa4cc15aef0 2024-09-10T11:31:48.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.481+0000 7fa4c1ffb640 1 -- 172.21.15.99:0/2064886613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4bc030770 con 0x7fa4cc15aef0 2024-09-10T11:31:48.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.481+0000 7fa4c1ffb640 1 -- 172.21.15.99:0/2064886613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc030b80 con 0x7fa4cc15aef0 2024-09-10T11:31:48.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.481+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc15cc90 con 0x7fa4cc15aef0 2024-09-10T11:31:48.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.481+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc15d530 con 0x7fa4cc15aef0 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.482+0000 7fa4c1ffb640 1 -- 172.21.15.99:0/2064886613 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4bc044020 con 0x7fa4cc15aef0 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.483+0000 7fa4c1ffb640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 0x7fa4b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.483+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 0x7fa4b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.483+0000 7fa4c1ffb640 1 -- 172.21.15.99:0/2064886613 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa4bc0ab280 con 0x7fa4cc15aef0 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.484+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 0x7fa4b006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000f60 tx=0x7fa4c8007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.484+0000 7fa4d66ad740 1 Processor -- start 2024-09-10T11:31:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.484+0000 7fa4d66ad740 1 -- start start 2024-09-10T11:31:48.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 0x55974f8ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 0x55974f8c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55974f830520 con 0x55974f8ce2c0 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55974f7d83e0 con 0x55974f8cba80 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55974f82c4e0 con 0x55974f8cfa50 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60620/0 (socket says 172.21.15.99:60620) 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 -- 172.21.15.99:0/617529457 learned_addr learned my addr 172.21.15.99:0/617529457 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 0x55974f8ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 0x55974f8c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 -- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 msgr2=0x55974f8c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 0x55974f8c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 -- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 msgr2=0x55974f8ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 0x55974f8ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 -- 172.21.15.99:0/617529457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974f7f32f0 con 0x55974f8ce2c0 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 0x55974f8ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 0x55974f8c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8001920 tx=0x7fa4c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6faae186b9191e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4aeffd640 1 -- 172.21.15.99:0/617529457 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8026070 con 0x55974f8ce2c0 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4aeffd640 1 -- 172.21.15.99:0/617529457 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c8013b80 con 0x55974f8ce2c0 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.485+0000 7fa4aeffd640 1 -- 172.21.15.99:0/617529457 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8012350 con 0x55974f8ce2c0 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 -- 172.21.15.99:0/617529457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 msgr2=0x55974f913a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8001920 tx=0x7fa4c8013000 comp rx=0 tx=0).stop 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 -- 172.21.15.99:0/617529457 shutdown_connections 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8cfa50 0x55974f8c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cba80 0x55974f8ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/617529457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f913a60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 -- 172.21.15.99:0/617529457 >> 172.21.15.99:0/617529457 conn(0x55974f8caa80 msgr2=0x55974f912f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 -- 172.21.15.99:0/617529457 shutdown_connections 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.486+0000 7fa4d66ad740 1 -- 172.21.15.99:0/617529457 wait complete. 2024-09-10T11:31:48.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.487+0000 7fa4d66ad740 1 -- start start 2024-09-10T11:31:48.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.487+0000 7fa4d66ad740 1 Processor -- start 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f927600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f8ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55974f8c9f20 con 0x55974f8ccb80 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55974f7b9450 con 0x55974f8cfa50 2024-09-10T11:31:48.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55974f914290 con 0x55974f8ce2c0 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 10 client.0 _ll_register_callbacks cb 0x55974f542ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60628/0 (socket says 172.21.15.99:60628) 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 -- 172.21.15.99:0/117237470 learned_addr learned my addr 172.21.15.99:0/117237470 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 -- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 msgr2=0x55974f927600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f927600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f8ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f927600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 -- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 msgr2=0x55974f8ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f8ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 -- 172.21.15.99:0/117237470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974f5432d0 con 0x55974f8ccb80 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f927600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f8ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4008a90 tx=0x7fa4c4008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8742ce3e4910403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4007c40 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa4c4007de0 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c8013b80 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa4c4023050 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 10 client.5364 fetch_fsmap learned FSMap version 21 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.488+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55974f60d6b0 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c8013b80 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 3502+0+0 (secure 0 0 0) 0x7fa4c4026020 con 0x55974f8ccb80 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 10 client.5364 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 10 client.5364 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 20 client.5364 populate_metadata read hostname 'smithi099' 2024-09-10T11:31:48.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ed6d0 0x55974f8edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 4 client.5364 mds_command: new command op to 5211 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 --> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55974f83dc50 con 0x55974f8ed6d0 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ed6d0 0x55974f8edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4a7fff640 10 client.5364 ms_handle_connect on v2:172.21.15.99:6834/3824950902 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.489+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ed6d0 0x55974f8edab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2cb98007dd9155 server_cookie=78215a0dcabaf584 in_seq=0 out_seq=0 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4a7fff640 1 -- 172.21.15.99:0/117237470 <== mds.0 v2:172.21.15.99:6834/3824950902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55974f83dc50 con 0x55974f8ed6d0 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4a7fff640 10 client.5364 handle_command_reply: tid=0 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 client.5364 shutdown 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 msgr2=0x55974f8cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4008a90 tx=0x7fa4c4008f50 comp rx=0 tx=0).stop 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 shutdown_connections 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ed6d0 0x55974f8edab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cd000 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4a7fff640 0 client.5364 ms_handle_reset on v2:172.21.15.99:6834/3824950902 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f8ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/117237470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f927600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 >> 172.21.15.99:0/117237470 conn(0x55974f8cba80 msgr2=0x55974f910fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 shutdown_connections 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 1 -- 172.21.15.99:0/117237470 wait complete. 2024-09-10T11:31:48.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.490+0000 7fa4d66ad740 20 client.5364 trim_cache size 0 max 16384 2024-09-10T11:31:48.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.508+0000 7fa4d66ad740 1 Processor -- start 2024-09-10T11:31:48.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.508+0000 7fa4d66ad740 1 -- start start 2024-09-10T11:31:48.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f9111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f937090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc1ffec0 con 0x55974f8ccb80 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0ce7b0 con 0x55974f8cfa50 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0cee00 con 0x55974f8ce2c0 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60630/0 (socket says 172.21.15.99:60630) 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 -- 172.21.15.99:0/3825677476 learned_addr learned my addr 172.21.15.99:0/3825677476 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f937090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f9111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 -- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 msgr2=0x55974f9111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f9111a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 -- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 msgr2=0x55974f937090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f937090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 -- 172.21.15.99:0/3825677476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974f8c4a30 con 0x55974f8ccb80 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f937090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f9111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa4c801f000 tx=0x7fa4c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3fc4e70032d91b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4ae7fc640 1 -- 172.21.15.99:0/3825677476 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8026040 con 0x55974f8ccb80 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4ae7fc640 1 -- 172.21.15.99:0/3825677476 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c8027040 con 0x55974f8ccb80 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.509+0000 7fa4ae7fc640 1 -- 172.21.15.99:0/3825677476 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8037650 con 0x55974f8ccb80 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- 172.21.15.99:0/3825677476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 msgr2=0x55974f8cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa4c801f000 tx=0x7fa4c8014000 comp rx=0 tx=0).stop 2024-09-10T11:31:48.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- 172.21.15.99:0/3825677476 shutdown_connections 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f937090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ce2c0 0x55974f9111a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/3825677476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ccb80 0x55974f8cdad0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- 172.21.15.99:0/3825677476 >> 172.21.15.99:0/3825677476 conn(0x55974f8cba80 msgr2=0x55974f8c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- 172.21.15.99:0/3825677476 shutdown_connections 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- 172.21.15.99:0/3825677476 wait complete. 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.510+0000 7fa4d66ad740 1 -- start start 2024-09-10T11:31:48.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 Processor -- start 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ccb80 0x55974f936770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f9273e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55974f8ed510 con 0x55974f8ce2c0 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55974f8f7b60 con 0x55974f8cfa50 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55974f8f79a0 con 0x55974f8ccb80 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d66ad740 10 client.0 _ll_register_callbacks cb 0x7fa4c400a050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.511+0000 7fa4d0eac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60636/0 (socket says 172.21.15.99:60636) 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 -- 172.21.15.99:0/1289446962 learned_addr learned my addr 172.21.15.99:0/1289446962 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ccb80 0x55974f936770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f9273e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 -- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ccb80 msgr2=0x55974f936770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ccb80 0x55974f936770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 -- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 msgr2=0x55974f9273e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f9273e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 -- 172.21.15.99:0/1289446962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974f8c8b50 con 0x55974f8ce2c0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d1eae640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f9273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d0eac640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc006ce0 tx=0x7fa4bc078ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948251249d3385ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc082100 con 0x55974f8ce2c0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa4bc079830 con 0x55974f8ce2c0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c8027040 con 0x55974f8ce2c0 2024-09-10T11:31:48.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa4bc079a30 con 0x55974f8ce2c0 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 10 client.5370 fetch_fsmap learned FSMap version 21 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55974f8cdad0 con 0x55974f8ce2c0 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4c8027040 con 0x55974f8ce2c0 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 21) ==== 3502+0+0 (secure 0 0 0) 0x7fa4bc03a030 con 0x55974f8ce2c0 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 10 client.5370 fetch_fsmap finished waiting for FSMap version 21 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 10 client.5370 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 20 client.5370 populate_metadata read hostname 'smithi099' 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ee170 0x55974f90f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.512+0000 7fa4d66ad740 4 client.5370 mds_command: new command op to 5211 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 --> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] -- command(tid 0: {"prefix": "status"}) -- 0x55974f8ea580 con 0x55974f8ee170 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ee170 0x55974f90f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4a7fff640 10 client.5370 ms_handle_connect on v2:172.21.15.99:6834/3824950902 2024-09-10T11:31:48.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d16ad640 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ee170 0x55974f90f570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab0fec9498d2cf7 server_cookie=2959536fa984da71 in_seq=0 out_seq=0 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4a7fff640 1 -- 172.21.15.99:0/1289446962 <== mds.0 v2:172.21.15.99:6834/3824950902 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55974f8ea580 con 0x55974f8ee170 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4a7fff640 10 client.5370 handle_command_reply: tid=0 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 client.5370 shutdown 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 msgr2=0x55974f993df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc006ce0 tx=0x7fa4bc078ff0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 shutdown_connections 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:6834/3824950902,v1:172.21.15.99:6836/3824950902] conn(0x55974f8ee170 0x55974f90f570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55974f8cfa50 0x55974f9273e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4a7fff640 0 client.5370 ms_handle_reset on v2:172.21.15.99:6834/3824950902 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55974f8ce2c0 0x55974f993df0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 --2- 172.21.15.99:0/1289446962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55974f8ccb80 0x55974f936770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 >> 172.21.15.99:0/1289446962 conn(0x55974f8cba80 msgr2=0x55974f8eca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 shutdown_connections 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.513+0000 7fa4d66ad740 1 -- 172.21.15.99:0/1289446962 wait complete. 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d66ad740 20 client.5370 trim_cache size 0 max 16384 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5211, 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:31:48.523 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 8.8616323870000002, 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 21, 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 45, 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 45, 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 12.838832559 2024-09-10T11:31:48.524 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:48.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 msgr2=0x7fa4b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 0x7fa4b006e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000f60 tx=0x7fa4c8007d50 comp rx=0 tx=0).stop 2024-09-10T11:31:48.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 msgr2=0x7fa4cc15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc035040 tx=0x7fa4bc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 shutdown_connections 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa4b006be30 0x7fa4b006e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4cc15b2d0 0x7fa4cc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4cc15aef0 0x7fa4cc15e080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 --2- 172.21.15.99:0/2064886613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4cc0afbb0 0x7fa4cc14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 >> 172.21.15.99:0/2064886613 conn(0x7fa4cc06bc10 msgr2=0x7fa4cc06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 shutdown_connections 2024-09-10T11:31:48.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.514+0000 7fa4d3911640 1 -- 172.21.15.99:0/2064886613 wait complete. 2024-09-10T11:31:48.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:31:48.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.624+0000 7fc4ff6af640 1 Processor -- start 2024-09-10T11:31:48.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.624+0000 7fc4ff6af640 1 -- start start 2024-09-10T11:31:48.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 0x7fc4f80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f81626c0 0x7fc4f80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc4f80d4e40 0x7fc4f80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f80d52f0 con 0x7fc4f80d4e40 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f80d1620 con 0x7fc4f815eb80 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4ff6af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f8162bc0 con 0x7fc4f81626c0 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc4f80d4e40 0x7fc4f80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc4f80d4e40 0x7fc4f80d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60646/0 (socket says 172.21.15.99:60646) 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/1649868761 learned_addr learned my addr 172.21.15.99:0/1649868761 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/1649868761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f81626c0 msgr2=0x7fc4f80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fcc4a640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f81626c0 0x7fc4f80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fd44b640 1 --2- 172.21.15.99:0/1649868761 >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 0x7fc4f80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f81626c0 0x7fc4f80d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/1649868761 >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 msgr2=0x7fc4f80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/1649868761 >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 0x7fc4f80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/1649868761 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f8162ea0 con 0x7fc4f80d4e40 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.625+0000 7fc4fd44b640 1 --2- 172.21.15.99:0/1649868761 >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 0x7fc4f80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f80d4e40 0x7fc4f80d1b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80d27e0 tx=0x7fc4f402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259ed2e3a5d40a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ebfff640 1 -- 172.21.15.99:0/1649868761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4036070 con 0x7fc4f80d4e40 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ebfff640 1 -- 172.21.15.99:0/1649868761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f4031600 con 0x7fc4f80d4e40 2024-09-10T11:31:48.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ebfff640 1 -- 172.21.15.99:0/1649868761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40319d0 con 0x7fc4f80d4e40 2024-09-10T11:31:48.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 -- 172.21.15.99:0/1649868761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f80d4e40 msgr2=0x7fc4f80d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f80d4e40 0x7fc4f80d1b20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80d27e0 tx=0x7fc4f402a610 comp rx=0 tx=0).stop 2024-09-10T11:31:48.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 -- 172.21.15.99:0/1649868761 shutdown_connections 2024-09-10T11:31:48.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f80d4e40 0x7fc4f80d1b20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/1649868761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f81626c0 0x7fc4f80d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/1649868761 >> v2:172.21.15.196:3300/0 conn(0x7fc4f815eb80 0x7fc4f80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.626+0000 7fc4ff6af640 1 -- 172.21.15.99:0/1649868761 >> 172.21.15.99:0/1649868761 conn(0x7fc4f808e680 msgr2=0x7fc4f809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.627+0000 7fc4ff6af640 1 -- 172.21.15.99:0/1649868761 shutdown_connections 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.627+0000 7fc4ff6af640 1 -- 172.21.15.99:0/1649868761 wait complete. 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 Processor -- start 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 -- start start 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f80d4e40 0x7fc4f81f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f815eb80 0x7fc4f815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f80d2ef0 con 0x7fc4f81626c0 2024-09-10T11:31:48.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f8163460 con 0x7fc4f815eb80 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.628+0000 7fc4ff6af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f80d3070 con 0x7fc4f80d4e40 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60648/0 (socket says 172.21.15.99:60648) 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/2579757804 learned_addr learned my addr 172.21.15.99:0/2579757804 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fcc4a640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f815eb80 0x7fc4f815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fd44b640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f80d4e40 0x7fc4f81f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f80d4e40 msgr2=0x7fc4f81f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f80d4e40 0x7fc4f81f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f815eb80 msgr2=0x7fc4f815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f815eb80 0x7fc4f815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 -- 172.21.15.99:0/2579757804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f815e070 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4fdc4c640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4034040 tx=0x7fc4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4041020 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4f40077a0 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f81f8b80 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4036040 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.629+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f81f85c0 con 0x7fc4f81626c0 2024-09-10T11:31:48.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.630+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80d29b0 con 0x7fc4f81626c0 2024-09-10T11:31:48.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.631+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4f4044020 con 0x7fc4f81626c0 2024-09-10T11:31:48.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.631+0000 7fc4e9ffb640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 0x7fc4dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:48.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.631+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc4f40abc10 con 0x7fc4f81626c0 2024-09-10T11:31:48.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.631+0000 7fc4fd44b640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 0x7fc4dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:48.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.632+0000 7fc4fd44b640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 0x7fc4dc06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80b6340 tx=0x7fc4f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:48.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.635+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc4f404b030 con 0x7fc4f81626c0 2024-09-10T11:31:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.839+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4f806c4d0 con 0x7fc4f81626c0 2024-09-10T11:31:48.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.840+0000 7fc4e9ffb640 1 -- 172.21.15.99:0/2579757804 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7fc4f407a5c0 con 0x7fc4f81626c0 2024-09-10T11:31:48.841 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 msgr2=0x7fc4dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 0x7fc4dc06e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80b6340 tx=0x7fc4f0008040 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 msgr2=0x7fc4f815ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4034040 tx=0x7fc4f4004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 shutdown_connections 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc4dc06be30 0x7fc4dc06e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc4f81626c0 0x7fc4f815ef60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4f815eb80 0x7fc4f815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 --2- 172.21.15.99:0/2579757804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4f80d4e40 0x7fc4f81f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 >> 172.21.15.99:0/2579757804 conn(0x7fc4f808e680 msgr2=0x7fc4f80b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 shutdown_connections 2024-09-10T11:31:48.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:48.842+0000 7fc4ff6af640 1 -- 172.21.15.99:0/2579757804 wait complete. 2024-09-10T11:31:48.853 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:31:48.854 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:48.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:48.854 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:31:48.871 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:31:48.871 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:31:48.871 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:31:48.872 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42740sec preferred_lft 42740sec 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:31:48.874 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:48.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:48.875 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:31:48.936 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:31:48.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:48.938 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:31:48.993 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:31:48.993 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:31:48.993 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:31:48.993 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:31:48.994 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:31:48.994 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:31:49.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:31:49.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:49.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.119 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:31:49.134 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:31:49.135 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:31:49.135 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:49.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:31:49.200 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:31:49.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:49.200 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:49.255 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:31:49.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:49.255 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:49.310 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:31:49.375 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:49.432 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:31:49.432 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:31:49.432 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:31:49.432 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:31:49.433 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:31:49.434 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:49.435 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:49.435 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:49.435 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:49.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.435 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:31:49.488 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:31:49.488 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:31:49.530 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:31:49.554 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:31:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:31:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:31:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:31:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:31:49.612 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:31:49.612 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:31:49.612 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:31:49.612 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:31:49.612 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:31:49.613 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:31:49.613 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:31:49.613 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:49.613 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:31:49.613 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:31:49.614 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:31:49.615 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:31:49.616 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:49.617 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:31:49.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.617 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:31:49.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:31:49.610+0000 7f1f5603d480 -1 init, newargv = 0x5626ab3773f0 newargc=15 2024-09-10T11:31:49.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[41425]: starting ceph client 2024-09-10T11:31:49.619 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[41425]: starting fuse 2024-09-10T11:31:49.634 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:31:49.634 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:31:49.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:49.635 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:31:49.635 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:31:49.635 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:31:49.635 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:31:49.636 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:49.637 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:31:49.638 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:31:49.639 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:49.639 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:31:49.639 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:31:49.639 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:31:49.751 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.41425.asok 2024-09-10T11:31:49.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:49.762 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41425.asok 2024-09-10T11:31:49.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41425.asok status 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "41425", 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:31:49.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5382, 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5382 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2063738275 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:49.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2063738275 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5382 192.168.0.1:0/2063738275", 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2063738275", 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 21, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 45, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:31:49.897 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:31:49.898 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:49.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.903 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:31:49.903 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:49.920 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:31:49.920 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:49.921 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:31:49.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:31:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.123+0000 7fee2f386640 1 Processor -- start 2024-09-10T11:31:50.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.123+0000 7fee2f386640 1 -- start start 2024-09-10T11:31:50.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2f386640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee280d1200 0x7fee28073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2f386640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fee2815c770 0x7fee2806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2f386640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee28152e60 con 0x7fee2806e5d0 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2f386640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fee2806ef80 con 0x7fee280d1200 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2f386640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fee280d15e0 con 0x7fee2815c770 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d122640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fee280d1200 0x7fee28073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2c921640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fee2806e5d0 0x7fee2815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37132/0 (socket says 172.21.15.99:37132) 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2c921640 1 -- 172.21.15.99:0/718748067 learned_addr learned my addr 172.21.15.99:0/718748067 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d923640 1 --2- 172.21.15.99:0/718748067 >> v2:172.21.15.196:3300/0 conn(0x7fee2815c770 0x7fee2806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d122640 1 -- 172.21.15.99:0/718748067 >> v2:172.21.15.196:3300/0 conn(0x7fee2815c770 msgr2=0x7fee2806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d122640 1 --2- 172.21.15.99:0/718748067 >> v2:172.21.15.196:3300/0 conn(0x7fee2815c770 0x7fee2806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d122640 1 -- 172.21.15.99:0/718748067 >> v1:172.21.15.99:6789/0 conn(0x7fee2806e5d0 legacy=0x7fee2815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee27fff640 1 -- 172.21.15.99:0/718748067 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666630590 0 0) 0x7fee28152e60 con 0x7fee2806e5d0 2024-09-10T11:31:50.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.124+0000 7fee2d122640 1 -- 172.21.15.99:0/718748067 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee28072680 con 0x7fee280d1200 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.125+0000 7fee2d122640 1 --2- 172.21.15.99:0/718748067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee280d1200 0x7fee28073440 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee280720c0 tx=0x7fee1802c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88b3b6017a4143c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.125+0000 7fee27fff640 1 -- 172.21.15.99:0/718748067 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18009540 con 0x7fee280d1200 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.125+0000 7fee27fff640 1 -- 172.21.15.99:0/718748067 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee180096e0 con 0x7fee280d1200 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.125+0000 7fee27fff640 1 -- 172.21.15.99:0/718748067 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18009ab0 con 0x7fee280d1200 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 -- 172.21.15.99:0/718748067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee280d1200 msgr2=0x7fee28073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 --2- 172.21.15.99:0/718748067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee280d1200 0x7fee28073440 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fee280720c0 tx=0x7fee1802c7f0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 -- 172.21.15.99:0/718748067 shutdown_connections 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 --2- 172.21.15.99:0/718748067 >> v2:172.21.15.196:3300/0 conn(0x7fee2815c770 0x7fee2806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 --2- 172.21.15.99:0/718748067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee280d1200 0x7fee28073440 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 -- 172.21.15.99:0/718748067 >> 172.21.15.99:0/718748067 conn(0x7fee2806bc10 msgr2=0x7fee2806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:50.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 -- 172.21.15.99:0/718748067 shutdown_connections 2024-09-10T11:31:50.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.126+0000 7fee2f386640 1 -- 172.21.15.99:0/718748067 wait complete. 2024-09-10T11:31:50.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.127+0000 7fee2f386640 1 Processor -- start 2024-09-10T11:31:50.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.127+0000 7fee2f386640 1 -- start start 2024-09-10T11:31:50.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee280d1200 0x7fee2815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 0x7fee2815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fee28072f20 con 0x7fee2806e5d0 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fee28072c40 con 0x7fee280d1200 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2f386640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fee2815cd90 con 0x7fee2815c770 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60656/0 (socket says 172.21.15.99:60656) 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 -- 172.21.15.99:0/973572613 learned_addr learned my addr 172.21.15.99:0/973572613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2c921640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee280d1200 0x7fee2815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d923640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 0x7fee2815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 -- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 msgr2=0x7fee2815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 0x7fee2815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 -- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee280d1200 msgr2=0x7fee2815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee280d1200 0x7fee2815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 -- 172.21.15.99:0/973572613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281528a0 con 0x7fee2806e5d0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d923640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 0x7fee2815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.128+0000 7fee2d122640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee18037040 tx=0x7fee18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.129+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18044070 con 0x7fee2806e5d0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.129+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee18040930 con 0x7fee2806e5d0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.129+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281591e0 con 0x7fee2806e5d0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.129+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18040d20 con 0x7fee2806e5d0 2024-09-10T11:31:50.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.129+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee28159a40 con 0x7fee2806e5d0 2024-09-10T11:31:50.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.130+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee28073440 con 0x7fee2806e5d0 2024-09-10T11:31:50.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.130+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee18035430 con 0x7fee2806e5d0 2024-09-10T11:31:50.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.131+0000 7fee25ffb640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 0x7fee1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:50.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.131+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fee180adcd0 con 0x7fee2806e5d0 2024-09-10T11:31:50.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.131+0000 7fee2c921640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 0x7fee1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:50.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.131+0000 7fee2c921640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 0x7fee1406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee280b1820 tx=0x7fee1c007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:50.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.135+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee281528a0 con 0x7fee2806e5d0 2024-09-10T11:31:50.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.333+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fee280b4440 con 0x7fee2806e5d0 2024-09-10T11:31:50.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.334+0000 7fee25ffb640 1 -- 172.21.15.99:0/973572613 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7fee1807bfa0 con 0x7fee2806e5d0 2024-09-10T11:31:50.335 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:31:50.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 msgr2=0x7fee1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 0x7fee1406e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fee280b1820 tx=0x7fee1c007e40 comp rx=0 tx=0).stop 2024-09-10T11:31:50.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 msgr2=0x7fee28152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:50.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fee18037040 tx=0x7fee18004a40 comp rx=0 tx=0).stop 2024-09-10T11:31:50.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 shutdown_connections 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fee1406be30 0x7fee1406e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee2815c770 0x7fee2815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee280d1200 0x7fee2815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 --2- 172.21.15.99:0/973572613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fee2806e5d0 0x7fee28152360 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 >> 172.21.15.99:0/973572613 conn(0x7fee2806bc10 msgr2=0x7fee2815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 shutdown_connections 2024-09-10T11:31:50.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:50.336+0000 7fee2f386640 1 -- 172.21.15.99:0/973572613 wait complete. 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:31:50.348 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:50.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.349 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:50.369 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42754sec preferred_lft 42754sec 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.370 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:31:50.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.372 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:31:50.437 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:31:50.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.439 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:31:50.498 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:31:50.498 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:31:50.499 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:31:50.499 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:31:50.499 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:31:50.499 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:31:50.499 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:31:50.499 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:31:50.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:31:50.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:31:50.500 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:31:50.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:31:50.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:50.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.633 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:31:50.649 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:31:50.649 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:31:50.649 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:50.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.649 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:31:50.714 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:31:50.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:50.714 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:50.770 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:31:50.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:31:50.771 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:50.826 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:31:50.890 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:31:50.947 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:31:50.948 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:50.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:50.949 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:31:51.002 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:31:51.002 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:31:51.045 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:31:51.069 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:31:51.126 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:31:51.127 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:31:51.128 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:51.129 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:31:51.129 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:51.129 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:31:51.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:51.129 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:31:51.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:31:51.125+0000 7fcb7c520480 -1 init, newargv = 0x55c34b24c1a0 newargc=15 2024-09-10T11:31:51.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34814]: starting ceph client 2024-09-10T11:31:51.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34814]: starting fuse 2024-09-10T11:31:51.145 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:31:51.145 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:31:51.145 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:31:51.146 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:31:51.147 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:31:51.147 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:31:51.147 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:31:51.147 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:31:51.262 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.34814.asok 2024-09-10T11:31:51.265 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:31:51.272 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34814.asok 2024-09-10T11:31:51.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:51.273 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34814.asok status 2024-09-10T11:31:51.405 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:31:51.405 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:31:51.405 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:31:51.405 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "34814", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5191, 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 5191 2024-09-10T11:31:51.406 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 385775615 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 385775615 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.5191 192.168.0.1:0/385775615", 2024-09-10T11:31:51.407 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/385775615", 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 21, 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 45, 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:31:51.408 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:31:51.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:51.414 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:31:51.414 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:51.429 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:31:51.429 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:51.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:51.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:31:51.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:31:51.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.602+0000 7f58e8113640 1 Processor -- start 2024-09-10T11:31:51.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.603+0000 7f58e8113640 1 -- start start 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e8113640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58e015eb80 0x7f58e00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e8113640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f58e00d5010 0x7f58e00d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e8113640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e015ff70 con 0x7f58e01626c0 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e8113640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f58e00d1620 con 0x7f58e015eb80 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e8113640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f58e0162bc0 con 0x7f58e00d5010 2024-09-10T11:31:51.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58e015eb80 0x7f58e00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58e015eb80 0x7f58e00d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60672/0 (socket says 172.21.15.99:60672) 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 -- 172.21.15.99:0/3332540216 learned_addr learned my addr 172.21.15.99:0/3332540216 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e66b0640 1 --2- 172.21.15.99:0/3332540216 >> v2:172.21.15.196:3300/0 conn(0x7f58e00d5010 0x7f58e00d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 -- 172.21.15.99:0/3332540216 >> v2:172.21.15.196:3300/0 conn(0x7f58e00d5010 msgr2=0x7f58e00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 --2- 172.21.15.99:0/3332540216 >> v2:172.21.15.196:3300/0 conn(0x7f58e00d5010 0x7f58e00d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 -- 172.21.15.99:0/3332540216 >> v1:172.21.15.196:6789/0 conn(0x7f58e01626c0 legacy=0x7f58e00d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.604+0000 7f58e5eaf640 1 -- 172.21.15.99:0/3332540216 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e0162ea0 con 0x7f58e015eb80 2024-09-10T11:31:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.605+0000 7f58e5eaf640 1 --2- 172.21.15.99:0/3332540216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e015eb80 0x7f58e00d29b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449d116ab551d543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.605+0000 7f58e4ead640 1 -- 172.21.15.99:0/3332540216 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037070 con 0x7f58e015eb80 2024-09-10T11:31:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.605+0000 7f58e4ead640 1 -- 172.21.15.99:0/3332540216 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4032440 con 0x7f58e015eb80 2024-09-10T11:31:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.605+0000 7f58e4ead640 1 -- 172.21.15.99:0/3332540216 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4032810 con 0x7f58e015eb80 2024-09-10T11:31:51.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 -- 172.21.15.99:0/3332540216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e015eb80 msgr2=0x7f58e00d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 --2- 172.21.15.99:0/3332540216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e015eb80 0x7f58e00d29b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009870 tx=0x7f58d402a210 comp rx=0 tx=0).stop 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 -- 172.21.15.99:0/3332540216 shutdown_connections 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 --2- 172.21.15.99:0/3332540216 >> v2:172.21.15.196:3300/0 conn(0x7f58e00d5010 0x7f58e00d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 --2- 172.21.15.99:0/3332540216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e015eb80 0x7f58e00d29b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 -- 172.21.15.99:0/3332540216 >> 172.21.15.99:0/3332540216 conn(0x7f58e008e680 msgr2=0x7f58e009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 -- 172.21.15.99:0/3332540216 shutdown_connections 2024-09-10T11:31:51.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.606+0000 7f58e8113640 1 -- 172.21.15.99:0/3332540216 wait complete. 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.607+0000 7f58e8113640 1 Processor -- start 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.607+0000 7f58e8113640 1 -- start start 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e015eb80 0x7f58e01609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e01626c0 0x7f58e0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58e00d2ef0 con 0x7f58e00d5010 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0163460 con 0x7f58e015eb80 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e8113640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58e00d3070 con 0x7f58e01626c0 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60676/0 (socket says 172.21.15.99:60676) 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 -- 172.21.15.99:0/2324720643 learned_addr learned my addr 172.21.15.99:0/2324720643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e56ae640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e015eb80 0x7f58e01609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e66b0640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e01626c0 0x7f58e0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 -- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e01626c0 msgr2=0x7f58e0157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e01626c0 0x7f58e0157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 -- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e015eb80 msgr2=0x7f58e01609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e015eb80 0x7f58e01609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 -- 172.21.15.99:0/2324720643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e0160f30 con 0x7f58e00d5010 2024-09-10T11:31:51.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.608+0000 7f58e5eaf640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58d402a250 tx=0x7f58d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.609+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4041020 con 0x7f58e00d5010 2024-09-10T11:31:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.609+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d40306c0 con 0x7f58e00d5010 2024-09-10T11:31:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.609+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037040 con 0x7f58e00d5010 2024-09-10T11:31:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.609+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e0157650 con 0x7f58e00d5010 2024-09-10T11:31:51.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.609+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e015b480 con 0x7f58e00d5010 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.610+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e00d29b0 con 0x7f58e00d5010 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.610+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58d4044020 con 0x7f58e00d5010 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.611+0000 7f58d27fc640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 0x7f58c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.611+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f58d40ab480 con 0x7f58e00d5010 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.611+0000 7f58e56ae640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 0x7f58c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:51.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.611+0000 7f58e56ae640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 0x7f58c406e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f58e00d0ef0 tx=0x7f58dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:51.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.615+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58d404b030 con 0x7f58e00d5010 2024-09-10T11:31:51.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.814+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e015af70 con 0x7f58e00d5010 2024-09-10T11:31:51.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.815+0000 7f58d27fc640 1 -- 172.21.15.99:0/2324720643 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7f58d4079870 con 0x7f58e00d5010 2024-09-10T11:31:51.817 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:51.817 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:31:51.817 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 msgr2=0x7f58c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 0x7f58c406e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f58e00d0ef0 tx=0x7f58dc009040 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 msgr2=0x7f58e00b6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f58d402a250 tx=0x7f58d40046c0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 shutdown_connections 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58c406be30 0x7f58c406e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e01626c0 0x7f58e0157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e015eb80 0x7f58e01609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 --2- 172.21.15.99:0/2324720643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e00d5010 0x7f58e00b6540 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:51.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 >> 172.21.15.99:0/2324720643 conn(0x7f58e008e680 msgr2=0x7f58e006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 shutdown_connections 2024-09-10T11:31:51.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:51.817+0000 7f58e8113640 1 -- 172.21.15.99:0/2324720643 wait complete. 2024-09-10T11:31:51.830 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:31:51.946 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":5211,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.28491833,"mdsmap_epoch":21,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":16.262120701000001} 2024-09-10T11:31:51.947 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 5211, "mdsmap_epoch": 21, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 12.28491833, "state": "up:active", "uptime": 16.262120701, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:31:51.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:31:52.055 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:31:52.055 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:31:52.056 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-09-10T11:31:52.056 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:52.056 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:52.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:52.073 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-09-10T11:31:53.198 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records in 2024-09-10T11:31:53.198 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records out 2024-09-10T11:31:53.199 INFO:teuthology.orchestra.run.smithi099.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.06285 s, 261 MB/s 2024-09-10T11:31:53.199 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:31:53.199 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:31:53.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:31:53.214 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-09-10T11:31:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records in 2024-09-10T11:31:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records out 2024-09-10T11:31:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.09368 s, 280 MB/s 2024-09-10T11:31:54.367 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-09-10T11:31:54.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:31:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.464+0000 7f860834b640 1 Processor -- start 2024-09-10T11:31:54.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 -- start start 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f860015eb80 0x7f86000d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f86000d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f860015ff70 con 0x7f86001626c0 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86000d1620 con 0x7f860015eb80 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.465+0000 7f860834b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8600162bc0 con 0x7f86000d5010 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86058e6640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f86001626c0 0x7f86000d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58272/0 (socket says 172.21.15.99:58272) 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86058e6640 1 -- 172.21.15.99:0/760388836 learned_addr learned my addr 172.21.15.99:0/760388836 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:54.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86060e7640 1 --2- 172.21.15.99:0/760388836 >> v2:172.21.15.196:3300/0 conn(0x7f860015eb80 0x7f86000d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86068e8640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f86000d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358137187 0 0) 0x7f860015ff70 con 0x7f86001626c0 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85ec003520 con 0x7f86001626c0 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86060e7640 1 -- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 msgr2=0x7f86000d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86060e7640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f86000d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86060e7640 1 -- 172.21.15.99:0/760388836 >> v1:172.21.15.99:6789/0 conn(0x7f86001626c0 legacy=0x7f86000d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 662144191 0 0) 0x7f85ec003520 con 0x7f86001626c0 2024-09-10T11:31:54.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.466+0000 7f86060e7640 1 -- 172.21.15.99:0/760388836 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600162ea0 con 0x7f860015eb80 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.467+0000 7f86060e7640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000d29b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f002a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15fc92969a063e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.467+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0037070 con 0x7f860015eb80 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.467+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f002ab40 con 0x7f860015eb80 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.467+0000 7f86050e5640 1 -- 172.21.15.99:0/760388836 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc001670 con 0x7f860015eb80 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 -- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 msgr2=0x7f86000d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000d29b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f002a0b0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 -- 172.21.15.99:0/760388836 shutdown_connections 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f86000d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 --2- 172.21.15.99:0/760388836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000d29b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 -- 172.21.15.99:0/760388836 >> 172.21.15.99:0/760388836 conn(0x7f860008e680 msgr2=0x7f860009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 -- 172.21.15.99:0/760388836 shutdown_connections 2024-09-10T11:31:54.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.468+0000 7f860834b640 1 -- 172.21.15.99:0/760388836 wait complete. 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.469+0000 7f860834b640 1 Processor -- start 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.469+0000 7f860834b640 1 -- start start 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f8600153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86000d2ef0 con 0x7f86001626c0 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8600163460 con 0x7f860015eb80 2024-09-10T11:31:54.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86000d3070 con 0x7f86000d5010 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86060e7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f8600153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53544/0 (socket says 172.21.15.99:53544) 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 -- 172.21.15.99:0/2378612461 learned_addr learned my addr 172.21.15.99:0/2378612461 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86058e6640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 -- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 msgr2=0x7f8600153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f8600153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 -- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 msgr2=0x7f86000b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 -- 172.21.15.99:0/2378612461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86000cc270 con 0x7f86001626c0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86058e6640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f86068e8640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f85f800efc0 tx=0x7f85f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8018020 con 0x7f86001626c0 2024-09-10T11:31:54.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f8014480 con 0x7f86001626c0 2024-09-10T11:31:54.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86000cc550 con 0x7f86001626c0 2024-09-10T11:31:54.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.470+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8009e60 con 0x7f86001626c0 2024-09-10T11:31:54.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.471+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86000ccb10 con 0x7f86001626c0 2024-09-10T11:31:54.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.471+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86000d29b0 con 0x7f86001626c0 2024-09-10T11:31:54.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.472+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85f8005d10 con 0x7f86001626c0 2024-09-10T11:31:54.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.472+0000 7f85f6ffd640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 0x7f85e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.473+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f85f808bb30 con 0x7f86001626c0 2024-09-10T11:31:54.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.473+0000 7f86060e7640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 0x7f85e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.473+0000 7f86060e7640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 0x7f85e406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85f0034000 tx=0x7f85f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.477+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f85f8022030 con 0x7f86001626c0 2024-09-10T11:31:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.641+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f8600161870 con 0x7f86001626c0 2024-09-10T11:31:54.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.642+0000 7f85f6ffd640 1 -- 172.21.15.99:0/2378612461 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v45) ==== 81+0+34200 (secure 0 0 0) 0x7f85f8010040 con 0x7f86001626c0 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 45, 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:31:40.638452+0000", 2024-09-10T11:31:54.645 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:31:54.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:31:54.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:31:54.648 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "16", 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:31:54.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:31:54.650 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:31:54.651 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:31:54.652 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:31:54.653 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:31:54.654 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:31:54.655 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:31:54.656 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "44", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:31:54.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:31:54.658 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:31:54.659 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:31:54.660 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:31:54.661 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:31:54.662 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "45", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:31:54.663 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:31:54.664 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:31:54.665 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:31:54.666 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:31:54.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.668 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.673 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.674 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.682 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.683 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.685 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.686 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.692 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.693 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:31:54.696 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.697 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.701 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:31:54.703 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:31:54.708 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:31:54.709 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:31:54.710 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.711 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.712 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.713 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:31:54.714 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.715 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:31:54.716 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.717 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.718 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:31:54.719 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:31:54.720 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:31:54.721 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:31:54.722 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:31:54.723 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 msgr2=0x7f85e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 0x7f85e406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85f0034000 tx=0x7f85f002e000 comp rx=0 tx=0).stop 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 msgr2=0x7f86000b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f85f800efc0 tx=0x7f85f800c490 comp rx=0 tx=0).stop 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 shutdown_connections 2024-09-10T11:31:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f85e406be30 0x7f85e406e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86001626c0 0x7f86000b2f00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f860015eb80 0x7f86000b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 --2- 172.21.15.99:0/2378612461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86000d5010 0x7f8600153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.644+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 >> 172.21.15.99:0/2378612461 conn(0x7f860008e680 msgr2=0x7f86000d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.645+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 shutdown_connections 2024-09-10T11:31:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:54.645+0000 7f860834b640 1 -- 172.21.15.99:0/2378612461 wait complete. 2024-09-10T11:31:54.727 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:31:55.693 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:31:55.692+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-09-10T11:31:57.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:31:57.963 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:31:59.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:31:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.837+0000 7fca2cb28640 1 Processor -- start 2024-09-10T11:31:59.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.837+0000 7fca2cb28640 1 -- start start 2024-09-10T11:31:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.838+0000 7fca2cb28640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca2806bc10 0x7fca28163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.838+0000 7fca2cb28640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 0x7fca2806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.838+0000 7fca2cb28640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2806c530 con 0x7fca280b4930 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.838+0000 7fca2cb28640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fca280b5420 con 0x7fca2806bc10 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.838+0000 7fca2cb28640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca280b5140 con 0x7fca28160030 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca25d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 0x7fca2806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2659c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca2806bc10 0x7fca28163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca26d9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fca280b4930 0x7fca2806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:33330/0 (socket says 172.21.15.99:33330) 2024-09-10T11:31:59.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca26d9d640 1 -- 172.21.15.99:0/3217577373 learned_addr learned my addr 172.21.15.99:0/3217577373 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2659c640 1 -- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 msgr2=0x7fca2806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2659c640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 0x7fca2806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2659c640 1 -- 172.21.15.99:0/3217577373 >> v1:172.21.15.196:6789/0 conn(0x7fca280b4930 legacy=0x7fca2806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2659c640 1 -- 172.21.15.99:0/3217577373 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca28162b40 con 0x7fca2806bc10 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca2559a640 1 -- 172.21.15.99:0/3217577373 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733767551 0 0) 0x7fca2806c530 con 0x7fca280b4930 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.839+0000 7fca25d9b640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 0x7fca2806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.840+0000 7fca2659c640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2806bc10 0x7fca28163620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca180097a0 tx=0x7fca1802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb3bfb8111448bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.840+0000 7fca2559a640 1 -- 172.21.15.99:0/3217577373 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18037070 con 0x7fca2806bc10 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.840+0000 7fca2559a640 1 -- 172.21.15.99:0/3217577373 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1802ae50 con 0x7fca2806bc10 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.840+0000 7fca2559a640 1 -- 172.21.15.99:0/3217577373 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18032660 con 0x7fca2806bc10 2024-09-10T11:31:59.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 -- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2806bc10 msgr2=0x7fca28163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2806bc10 0x7fca28163620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca180097a0 tx=0x7fca1802a100 comp rx=0 tx=0).stop 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 -- 172.21.15.99:0/3217577373 shutdown_connections 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca28160030 0x7fca2806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3217577373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca2806bc10 0x7fca28163620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 -- 172.21.15.99:0/3217577373 >> 172.21.15.99:0/3217577373 conn(0x7fca280b3630 msgr2=0x7fca280b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 -- 172.21.15.99:0/3217577373 shutdown_connections 2024-09-10T11:31:59.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.841+0000 7fca2cb28640 1 -- 172.21.15.99:0/3217577373 wait complete. 2024-09-10T11:31:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.842+0000 7fca2cb28640 1 Processor -- start 2024-09-10T11:31:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.842+0000 7fca2cb28640 1 -- start start 2024-09-10T11:31:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2806bc10 0x7fca2815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca28160030 0x7fca2813ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca280b4d90 con 0x7fca280b4930 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca28163100 con 0x7fca28160030 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2cb28640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fca2815ed30 con 0x7fca2806bc10 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53548/0 (socket says 172.21.15.99:53548) 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 -- 172.21.15.99:0/3319687045 learned_addr learned my addr 172.21.15.99:0/3319687045 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca26d9d640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca28160030 0x7fca2813ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca2659c640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2806bc10 0x7fca2815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 -- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2806bc10 msgr2=0x7fca2815e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2806bc10 0x7fca2815e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 -- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca28160030 msgr2=0x7fca2813ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:59.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca28160030 0x7fca2813ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.843+0000 7fca25d9b640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca28135120 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca25d9b640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e990 tx=0x7fca1c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c017070 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c00c9e0 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca281ea390 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00cdb0 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.844+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281ea950 con 0x7fca280b4930 2024-09-10T11:31:59.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.845+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca280d1560 con 0x7fca280b4930 2024-09-10T11:31:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.845+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca1c01a070 con 0x7fca280b4930 2024-09-10T11:31:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.846+0000 7fca16ffd640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 0x7fca0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:59.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.846+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fca1c08b020 con 0x7fca280b4930 2024-09-10T11:31:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.846+0000 7fca2659c640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 0x7fca0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.846+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7fca08073960 con 0x7fca280b4930 2024-09-10T11:31:59.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.846+0000 7fca2659c640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 0x7fca0806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca18034000 tx=0x7fca1802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:59.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:31:59.850+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca1c059360 con 0x7fca280b4930 2024-09-10T11:32:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.016+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fca28073d40 con 0x7fca280b4930 2024-09-10T11:32:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.017+0000 7fca16ffd640 1 -- 172.21.15.99:0/3319687045 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) ==== 81+0+34415 (secure 0 0 0) 0x7fca1c05d660 con 0x7fca280b4930 2024-09-10T11:32:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:00.019 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 49, 2024-09-10T11:32:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:31:59.758630+0000", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:32:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:32:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:32:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:32:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:32:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:32:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:32:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:32:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:32:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:32:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:32:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:32:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:32:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:32:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:32:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:32:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.103 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 msgr2=0x7fca0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 0x7fca0806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fca18034000 tx=0x7fca1802e000 comp rx=0 tx=0).stop 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 msgr2=0x7fca2813a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00e990 tx=0x7fca1c00ee50 comp rx=0 tx=0).stop 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 shutdown_connections 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fca0806be30 0x7fca0806e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca28160030 0x7fca2813ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca280b4930 0x7fca2813a920 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 --2- 172.21.15.99:0/3319687045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca2806bc10 0x7fca2815e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 >> 172.21.15.99:0/3319687045 conn(0x7fca280b3630 msgr2=0x7fca28160a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:00.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 shutdown_connections 2024-09-10T11:32:00.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.019+0000 7fca2cb28640 1 -- 172.21.15.99:0/3319687045 wait complete. 2024-09-10T11:32:00.106 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:32:00.107 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:00.107 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:00.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:00.123 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-09-10T11:32:00.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:32:00.240 INFO:teuthology.orchestra.run.smithi099.stderr:dd: error writing 'large_file_b': No space left on device 2024-09-10T11:32:00.240 INFO:teuthology.orchestra.run.smithi099.stderr:dd: closing output file 'large_file_b': No space left on device 2024-09-10T11:32:00.240 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:32:00.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.336+0000 7f5a0867b640 1 Processor -- start 2024-09-10T11:32:00.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 -- start start 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 0x7f5a000d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a000d4e40 0x7f5a000d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0015ff70 con 0x7f5a0015eb80 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000d30c0 con 0x7f5a000d4e40 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.337+0000 7f5a0867b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000d1620 con 0x7f5a001626c0 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a000d4e40 0x7f5a000d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a000d4e40 0x7f5a000d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53554/0 (socket says 172.21.15.99:53554) 2024-09-10T11:32:00.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 -- 172.21.15.99:0/2479801011 learned_addr learned my addr 172.21.15.99:0/2479801011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 -- 172.21.15.99:0/2479801011 >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 msgr2=0x7f5a000d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a05c16640 1 --2- 172.21.15.99:0/2479801011 >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 0x7f5a000d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 --2- 172.21.15.99:0/2479801011 >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 0x7f5a000d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 -- 172.21.15.99:0/2479801011 >> v1:172.21.15.196:6789/0 conn(0x7f5a0015eb80 legacy=0x7f5a000d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a06c18640 1 -- 172.21.15.99:0/2479801011 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a00162bc0 con 0x7f5a000d4e40 2024-09-10T11:32:00.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.338+0000 7f5a05c16640 1 --2- 172.21.15.99:0/2479801011 >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 0x7f5a000d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:00.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a06c18640 1 --2- 172.21.15.99:0/2479801011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a000d4e40 0x7f5a000d1b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e50 tx=0x7f59f802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b93d80780499cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a05415640 1 -- 172.21.15.99:0/2479801011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8036070 con 0x7f5a000d4e40 2024-09-10T11:32:00.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a05415640 1 -- 172.21.15.99:0/2479801011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f8031600 con 0x7f5a000d4e40 2024-09-10T11:32:00.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a05415640 1 -- 172.21.15.99:0/2479801011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f802ae60 con 0x7f5a000d4e40 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 -- 172.21.15.99:0/2479801011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a000d4e40 msgr2=0x7f5a000d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 --2- 172.21.15.99:0/2479801011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a000d4e40 0x7f5a000d1b20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e50 tx=0x7f59f802a680 comp rx=0 tx=0).stop 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 -- 172.21.15.99:0/2479801011 shutdown_connections 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 --2- 172.21.15.99:0/2479801011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a000d4e40 0x7f5a000d1b20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 --2- 172.21.15.99:0/2479801011 >> v2:172.21.15.196:3300/0 conn(0x7f5a001626c0 0x7f5a000d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 -- 172.21.15.99:0/2479801011 >> 172.21.15.99:0/2479801011 conn(0x7f5a0008e680 msgr2=0x7f5a0009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 -- 172.21.15.99:0/2479801011 shutdown_connections 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.339+0000 7f5a0867b640 1 -- 172.21.15.99:0/2479801011 wait complete. 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 Processor -- start 2024-09-10T11:32:00.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 -- start start 2024-09-10T11:32:00.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a000d4e40 0x7f5a000b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a0015eb80 0x7f5a0014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00163460 con 0x7f5a001626c0 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00163180 con 0x7f5a000d4e40 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a0867b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5a000d5290 con 0x7f5a0015eb80 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a06c18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.341+0000 7f5a06c18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53562/0 (socket says 172.21.15.99:53562) 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 -- 172.21.15.99:0/826810178 learned_addr learned my addr 172.21.15.99:0/826810178 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 -- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a0015eb80 msgr2=0x7f5a0014d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a0015eb80 0x7f5a0014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 -- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a000d4e40 msgr2=0x7f5a000b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06417640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a000d4e40 0x7f5a000b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a000d4e40 0x7f5a000b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a00161340 con 0x7f5a001626c0 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a06c18640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e50 tx=0x7f59f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8036050 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f8042070 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8009510 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a00161620 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.342+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a00162100 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.343+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a000d4900 con 0x7f5a001626c0 2024-09-10T11:32:00.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.343+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59f8007990 con 0x7f5a001626c0 2024-09-10T11:32:00.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.344+0000 7f59f2ffd640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 0x7f59e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.344+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f59f80abc50 con 0x7f5a001626c0 2024-09-10T11:32:00.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.344+0000 7f5a06417640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 0x7f59e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.344+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7f59e4073990 con 0x7f5a001626c0 2024-09-10T11:32:00.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.344+0000 7f5a06417640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 0x7f59e406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00a750 tx=0x7f59fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.348+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59f80b1020 con 0x7f5a001626c0 2024-09-10T11:32:00.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.515+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f5a000cfce0 con 0x7f5a001626c0 2024-09-10T11:32:00.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.516+0000 7f59f2ffd640 1 -- 172.21.15.99:0/826810178 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) ==== 81+0+34415 (secure 0 0 0) 0x7f59f807a600 con 0x7f5a001626c0 2024-09-10T11:32:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:00.518 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 49, 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:31:59.758630+0000", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:32:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:32:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:32:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:32:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:32:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:32:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:32:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:32:00.594 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:32:00.595 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:00.596 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 msgr2=0x7f59e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 0x7f59e406e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00a750 tx=0x7f59fc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 msgr2=0x7f5a001e8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e50 tx=0x7f59f8004a40 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 shutdown_connections 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f59e406be60 0x7f59e406e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a001626c0 0x7f5a001e8550 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a0015eb80 0x7f5a0014d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 --2- 172.21.15.99:0/826810178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a000d4e40 0x7f5a000b6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 >> 172.21.15.99:0/826810178 conn(0x7f5a0008e680 msgr2=0x7f5a0006ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:00.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 shutdown_connections 2024-09-10T11:32:00.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.518+0000 7f5a0867b640 1 -- 172.21.15.99:0/826810178 wait complete. 2024-09-10T11:32:00.599 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:00.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.697+0000 7fbfcc566640 1 Processor -- start 2024-09-10T11:32:00.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.697+0000 7fbfcc566640 1 -- start start 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfcc566640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbfc41605c0 0x7fbfc41609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfcc566640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc40ae020 0x7fbfc4160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfcc566640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc40d1bd0 con 0x7fbfc415cfd0 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfcc566640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc4075620 con 0x7fbfc40ae020 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfcc566640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc40752b0 con 0x7fbfc41605c0 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfc9b01640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc40ae020 0x7fbfc4160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfc9b01640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbfc40ae020 0x7fbfc4160ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53574/0 (socket says 172.21.15.99:53574) 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.698+0000 7fbfc9b01640 1 -- 172.21.15.99:0/2120080002 learned_addr learned my addr 172.21.15.99:0/2120080002 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfca302640 1 --2- 172.21.15.99:0/2120080002 >> v2:172.21.15.196:3300/0 conn(0x7fbfc41605c0 0x7fbfc41609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9b01640 1 -- 172.21.15.99:0/2120080002 >> v2:172.21.15.196:3300/0 conn(0x7fbfc41605c0 msgr2=0x7fbfc41609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9b01640 1 --2- 172.21.15.99:0/2120080002 >> v2:172.21.15.196:3300/0 conn(0x7fbfc41605c0 0x7fbfc41609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9b01640 1 -- 172.21.15.99:0/2120080002 >> v1:172.21.15.196:6789/0 conn(0x7fbfc415cfd0 legacy=0x7fbfc415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:00.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9b01640 1 -- 172.21.15.99:0/2120080002 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc40d1eb0 con 0x7fbfc40ae020 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9b01640 1 --2- 172.21.15.99:0/2120080002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ae020 0x7fbfc4160ee0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4070af0 tx=0x7fbfb402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f277e0ff6ed64b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9300640 1 -- 172.21.15.99:0/2120080002 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4037070 con 0x7fbfc40ae020 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9300640 1 -- 172.21.15.99:0/2120080002 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4032440 con 0x7fbfc40ae020 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.699+0000 7fbfc9300640 1 -- 172.21.15.99:0/2120080002 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4032810 con 0x7fbfc40ae020 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 -- 172.21.15.99:0/2120080002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ae020 msgr2=0x7fbfc4160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 --2- 172.21.15.99:0/2120080002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ae020 0x7fbfc4160ee0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4070af0 tx=0x7fbfb402a230 comp rx=0 tx=0).stop 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 -- 172.21.15.99:0/2120080002 shutdown_connections 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 --2- 172.21.15.99:0/2120080002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc40ae020 0x7fbfc4160ee0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 --2- 172.21.15.99:0/2120080002 >> v2:172.21.15.196:3300/0 conn(0x7fbfc41605c0 0x7fbfc41609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 -- 172.21.15.99:0/2120080002 >> 172.21.15.99:0/2120080002 conn(0x7fbfc40730c0 msgr2=0x7fbfc40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 -- 172.21.15.99:0/2120080002 shutdown_connections 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.700+0000 7fbfcc566640 1 -- 172.21.15.99:0/2120080002 wait complete. 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.701+0000 7fbfcc566640 1 Processor -- start 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.701+0000 7fbfcc566640 1 -- start start 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40ae020 0x7fbfc41f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc415cfd0 0x7fbfc40b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc4073d00 con 0x7fbfc41605c0 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc415a9e0 con 0x7fbfc415cfd0 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcc566640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc4073e80 con 0x7fbfc40ae020 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53580/0 (socket says 172.21.15.99:53580) 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 -- 172.21.15.99:0/1457707928 learned_addr learned my addr 172.21.15.99:0/1457707928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:00.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfc9b01640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc415cfd0 0x7fbfc40b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfca302640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40ae020 0x7fbfc41f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 -- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40ae020 msgr2=0x7fbfc41f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40ae020 0x7fbfc41f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 -- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc415cfd0 msgr2=0x7fbfc40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc415cfd0 0x7fbfc40b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.702+0000 7fbfcab03640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc4153850 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfcab03640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b9c0 tx=0x7fbfbc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc019070 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00cba0 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc4153b30 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc014440 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.703+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc41540f0 con 0x7fbfc41605c0 2024-09-10T11:32:00.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.704+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc40d0b80 con 0x7fbfc41605c0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.704+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfbc01c070 con 0x7fbfc41605c0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.705+0000 7fbfbaffd640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 0x7fbfa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.705+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fbfbc08abf0 con 0x7fbfc41605c0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.705+0000 7fbfca302640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 0x7fbfa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.705+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7fbfa80739c0 con 0x7fbfc41605c0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.705+0000 7fbfca302640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 0x7fbfa806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b7c0 tx=0x7fbfc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:00.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.709+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbfc40d0b80 con 0x7fbfc41605c0 2024-09-10T11:32:00.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.911+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfc41609a0 con 0x7fbfc41605c0 2024-09-10T11:32:00.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.912+0000 7fbfbaffd640 1 -- 172.21.15.99:0/1457707928 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7fbfbc0595a0 con 0x7fbfc41605c0 2024-09-10T11:32:00.914 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:00.914 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:00.914 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 msgr2=0x7fbfa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 0x7fbfa806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000b7c0 tx=0x7fbfc0008040 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 msgr2=0x7fbfc40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc00b9c0 tx=0x7fbfbc00be80 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 shutdown_connections 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbfa806be30 0x7fbfa806e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfc41605c0 0x7fbfc40b1640 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfc415cfd0 0x7fbfc40b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 --2- 172.21.15.99:0/1457707928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbfc40ae020 0x7fbfc41f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 >> 172.21.15.99:0/1457707928 conn(0x7fbfc40730c0 msgr2=0x7fbfc415e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:00.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 shutdown_connections 2024-09-10T11:32:00.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:00.914+0000 7fbfcc566640 1 -- 172.21.15.99:0/1457707928 wait complete. 2024-09-10T11:32:00.926 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:32:01.042 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":5211,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.380774621,"mdsmap_epoch":21,"osdmap_epoch":49,"osdmap_epoch_barrier":45,"uptime":25.357976658999998} 2024-09-10T11:32:01.042 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 5211, "mdsmap_epoch": 21, "osdmap_epoch": 49, "osdmap_epoch_barrier": 45, "rank_uptime": 21.380774621, "state": "up:active", "uptime": 25.357976659, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:32:01.042 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:32:01.042 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:01.042 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:01.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:01.059 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-09-10T11:32:01.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:32:01.113+0000 7f1f437fe640 -1 client.5382 handle_client_reply no pending request on tid 14 safe is:1 2024-09-10T11:32:01.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:32:01.116 INFO:teuthology.orchestra.run.smithi099.stderr:dd: failed to open 'small_file_1': Permission denied 2024-09-10T11:32:01.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:01.116 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-09-10T11:32:01.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:01.189 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-09-10T11:32:01.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:32:01.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.386+0000 7f109b4ba640 1 Processor -- start 2024-09-10T11:32:01.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.387+0000 7f109b4ba640 1 -- start start 2024-09-10T11:32:01.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.387+0000 7f109b4ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.387+0000 7f109b4ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.387+0000 7f109b4ba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10940d52f0 con 0x7f10940d4e40 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f109b4ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f109415ff70 con 0x7f109415eb80 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f109b4ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f10940d1620 con 0x7f10941626c0 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1099256640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1099256640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:52532/0 (socket says 172.21.15.99:52532) 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1099256640 1 -- 172.21.15.99:0/4166573962 learned_addr learned my addr 172.21.15.99:0/4166573962 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1099a57640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f10940d4e40 0x7f10940d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:33346/0 (socket says 172.21.15.99:33346) 2024-09-10T11:32:01.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 --2- 172.21.15.99:0/4166573962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:40186/0 (socket says 172.21.15.99:40186) 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 -- 172.21.15.99:0/4166573962 >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 msgr2=0x7f10940d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 --2- 172.21.15.99:0/4166573962 >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 -- 172.21.15.99:0/4166573962 >> v1:172.21.15.196:6789/0 conn(0x7f10940d4e40 legacy=0x7f10940d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1098a55640 1 -- 172.21.15.99:0/4166573962 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094162bc0 con 0x7f10941626c0 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.388+0000 7f1099256640 1 --2- 172.21.15.99:0/4166573962 >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.389+0000 7f1098a55640 1 --2- 172.21.15.99:0/4166573962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10900097a0 tx=0x7f109002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20971f3940a9d634 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:01.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.389+0000 7f1087fff640 1 -- 172.21.15.99:0/4166573962 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090037070 con 0x7f10941626c0 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.389+0000 7f1087fff640 1 -- 172.21.15.99:0/4166573962 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109002ae50 con 0x7f10941626c0 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.389+0000 7f1087fff640 1 -- 172.21.15.99:0/4166573962 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090032660 con 0x7f10941626c0 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 -- 172.21.15.99:0/4166573962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 msgr2=0x7f10940d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 --2- 172.21.15.99:0/4166573962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10900097a0 tx=0x7f109002a100 comp rx=0 tx=0).stop 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 -- 172.21.15.99:0/4166573962 shutdown_connections 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 --2- 172.21.15.99:0/4166573962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10941626c0 0x7f10940d4900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 --2- 172.21.15.99:0/4166573962 >> v2:172.21.15.196:3300/0 conn(0x7f109415eb80 0x7f10940d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 -- 172.21.15.99:0/4166573962 >> 172.21.15.99:0/4166573962 conn(0x7f109408e680 msgr2=0x7f109409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 -- 172.21.15.99:0/4166573962 shutdown_connections 2024-09-10T11:32:01.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.390+0000 7f109b4ba640 1 -- 172.21.15.99:0/4166573962 wait complete. 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.391+0000 7f109b4ba640 1 Processor -- start 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.391+0000 7f109b4ba640 1 -- start start 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f109415eb80 0x7f109406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 0x7f109406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1094163460 con 0x7f10940d4e40 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1094163180 con 0x7f10941626c0 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f109b4ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f10940d2ef0 con 0x7f109415eb80 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53582/0 (socket says 172.21.15.99:53582) 2024-09-10T11:32:01.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 -- 172.21.15.99:0/3735827340 learned_addr learned my addr 172.21.15.99:0/3735827340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099a57640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 0x7f109406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1098a55640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f109415eb80 0x7f109406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 -- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f109415eb80 msgr2=0x7f109406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f109415eb80 0x7f109406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 -- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 msgr2=0x7f109406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 0x7f109406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1094154330 con 0x7f10940d4e40 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099a57640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 0x7f109406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.392+0000 7f1099256640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f108800e9b0 tx=0x7f108800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.393+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088017070 con 0x7f10940d4e40 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.393+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f108800c9a0 con 0x7f10940d4e40 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.393+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1094154610 con 0x7f10940d4e40 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.393+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800cd50 con 0x7f10940d4e40 2024-09-10T11:32:01.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.393+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1094154bd0 con 0x7f10940d4e40 2024-09-10T11:32:01.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.394+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10940d29b0 con 0x7f10940d4e40 2024-09-10T11:32:01.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.394+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f108801a070 con 0x7f10940d4e40 2024-09-10T11:32:01.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.395+0000 7f1085ffb640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:01.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.395+0000 7f1098a55640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.395+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f108808b090 con 0x7f10940d4e40 2024-09-10T11:32:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.395+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7f1078073960 con 0x7f10940d4e40 2024-09-10T11:32:01.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.395+0000 7f1098a55640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 0x7f107806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:01.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.399+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1088059a40 con 0x7f10940d4e40 2024-09-10T11:32:01.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.565+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f10940d0ad0 con 0x7f10940d4e40 2024-09-10T11:32:01.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.566+0000 7f1085ffb640 1 -- 172.21.15.99:0/3735827340 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v49) ==== 81+0+34415 (secure 0 0 0) 0x7f1088059340 con 0x7f10940d4e40 2024-09-10T11:32:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 49, 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:31:59.758630+0000", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:32:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:32:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:32:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:32:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:32:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:32:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:32:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:32:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "46", 2024-09-10T11:32:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:32:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:32:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:32:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:32:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.618 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.619 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.620 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.621 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:32:01.622 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.623 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:32:01.624 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.625 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.626 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:32:01.628 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.629 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:32:01.630 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:32:01.634 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:01.635 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:32:01.637 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:32:01.638 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:32:01.639 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:32:01.640 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.641 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.643 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.644 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:01.646 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:32:01.647 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:32:01.648 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:01.649 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:01.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 msgr2=0x7f107806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 0x7f107806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1090034000 tx=0x7f109002e000 comp rx=0 tx=0).stop 2024-09-10T11:32:01.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 msgr2=0x7f1094152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:01.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f108800e9b0 tx=0x7f108800ee70 comp rx=0 tx=0).stop 2024-09-10T11:32:01.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 shutdown_connections 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f107806be30 0x7f107806e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10941626c0 0x7f109406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f109415eb80 0x7f109406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 --2- 172.21.15.99:0/3735827340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10940d4e40 0x7f1094152cf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 >> 172.21.15.99:0/3735827340 conn(0x7f109408e680 msgr2=0x7f1094160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 shutdown_connections 2024-09-10T11:32:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:01.569+0000 7f109b4ba640 1 -- 172.21.15.99:0/3735827340 wait complete. 2024-09-10T11:32:01.653 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:32:04.787 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:32:04.785+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2024-09-10T11:32:06.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:32:06.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.762+0000 7f17cbdfd640 1 Processor -- start 2024-09-10T11:32:06.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.762+0000 7f17cbdfd640 1 -- start start 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17cbdfd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f17c40d1200 0x7f17c4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17cbdfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17c406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17cbdfd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c4152e60 con 0x7f17c406e5d0 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17cbdfd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f17c406ef80 con 0x7f17c40d1200 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17cbdfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f17c40d15e0 con 0x7f17c415c770 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17c9398640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f17c406e5d0 0x7f17c415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51924/0 (socket says 172.21.15.99:51924) 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17c9398640 1 -- 172.21.15.99:0/2095142591 learned_addr learned my addr 172.21.15.99:0/2095142591 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2095142591 >> v2:172.21.15.196:3300/0 conn(0x7f17c40d1200 0x7f17c4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17c406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.763+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604097316 0 0) 0x7f17c4152e60 con 0x7f17c406e5d0 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17b0003520 con 0x7f17c406e5d0 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1408005762 0 0) 0x7f17b0003520 con 0x7f17c406e5d0 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 msgr2=0x7f17c406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17c406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 >> v2:172.21.15.196:3300/0 conn(0x7f17c40d1200 msgr2=0x7f17c4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 --2- 172.21.15.99:0/2095142591 >> v2:172.21.15.196:3300/0 conn(0x7f17c40d1200 0x7f17c4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c4072680 con 0x7f17c406e5d0 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f17b4001670 con 0x7f17c406e5d0 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 >> v1:172.21.15.99:6789/0 conn(0x7f17c406e5d0 legacy=0x7f17c415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0003f30 0x7f17b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17b0004d40 0x7f17b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17c406ef80 con 0x7f17b0004d40 2024-09-10T11:32:06.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17c40d15e0 con 0x7f17b0003f30 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f17b000ba10 con 0x7f17c415c770 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17b0004d40 0x7f17b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c9398640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0003f30 0x7f17b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 -- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 msgr2=0x7f17b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 -- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0003f30 msgr2=0x7f17b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0003f30 0x7f17b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17ca39a640 1 -- 172.21.15.99:0/2095142591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b0003520 con 0x7f17b0004d40 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.764+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c415c770 0x7f17b00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.765+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17b0004d40 0x7f17b00051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e170 tx=0x7f17b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6575605e647be8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:06.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.765+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc034020 con 0x7f17b0004d40 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.765+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc0305f0 con 0x7f17b0004d40 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.765+0000 7f17c8b97640 1 -- 172.21.15.99:0/2095142591 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc009c10 con 0x7f17b0004d40 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2095142591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17b0004d40 msgr2=0x7f17b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2095142591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17b0004d40 0x7f17b00051c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00e170 tx=0x7f17b0005830 comp rx=0 tx=0).stop 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17ca39a640 1 -- 172.21.15.99:0/2095142591 reap_dead start 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2095142591 shutdown_connections 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2095142591 >> 172.21.15.99:0/2095142591 conn(0x7f17c406bc10 msgr2=0x7f17c406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2095142591 shutdown_connections 2024-09-10T11:32:06.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.766+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2095142591 wait complete. 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.767+0000 7f17cbdfd640 1 Processor -- start 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.767+0000 7f17cbdfd640 1 -- start start 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0004d40 0x7f17c413b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c413ea00 0x7f17c413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4072f20 con 0x7f17c413bd80 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4072c40 con 0x7f17b0004d40 2024-09-10T11:32:06.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17cbdfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f17c415cd90 con 0x7f17c413ea00 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33350/0 (socket says 172.21.15.99:33350) 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 -- 172.21.15.99:0/2588648046 learned_addr learned my addr 172.21.15.99:0/2588648046 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0004d40 0x7f17c413b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17c9398640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c413ea00 0x7f17c413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 -- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c413ea00 msgr2=0x7f17c413d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c413ea00 0x7f17c413d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 -- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0004d40 msgr2=0x7f17c413b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0004d40 0x7f17c413b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c413dbb0 con 0x7f17c413bd80 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ca39a640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009f90 tx=0x7f17bc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc034030 con 0x7f17c413bd80 2024-09-10T11:32:06.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.768+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc003990 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.769+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc009c10 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.769+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c413c5d0 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.769+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c413cb90 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.770+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c4073440 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.770+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17bc037020 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.771+0000 7f17ba7fc640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 0x7f17a806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.771+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5709+0+0 (secure 0 0 0) 0x7f17bc0ab6b0 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.771+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 0x7f17a806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.771+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=54}) -- 0x7f17a8073990 con 0x7f17c413bd80 2024-09-10T11:32:06.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.771+0000 7f17c9b99640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 0x7f17a806e2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009b40 tx=0x7f17c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:06.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.775+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f17bc048060 con 0x7f17c413bd80 2024-09-10T11:32:06.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.801+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(54..54 src has 1..54) ==== 1468+0+0 (secure 0 0 0) 0x7f17bc0dfd50 con 0x7f17c413bd80 2024-09-10T11:32:06.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.801+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=55}) -- 0x7f17a8074b10 con 0x7f17c413bd80 2024-09-10T11:32:06.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.945+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f17c415b830 con 0x7f17c413bd80 2024-09-10T11:32:06.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.946+0000 7f17ba7fc640 1 -- 172.21.15.99:0/2588648046 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) ==== 81+0+34200 (secure 0 0 0) 0x7f17bc079980 con 0x7f17c413bd80 2024-09-10T11:32:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:06.948 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 54, 2024-09-10T11:32:06.948 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:32:06.798356+0000", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:06.949 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:32:06.950 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:06.951 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:06.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:06.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:06.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:06.955 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:06.956 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:06.957 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:06.958 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:06.959 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:06.960 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:06.961 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.963 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:06.964 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:06.965 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:06.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:06.967 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:06.968 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:06.969 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.970 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:32:06.971 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:32:06.972 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:32:06.973 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:32:06.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:06.975 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.976 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:32:06.977 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:32:06.978 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:32:06.979 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:32:06.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:06.984 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:06.985 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:32:06.986 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.987 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:06.992 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.993 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.994 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:06.998 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:06.999 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.003 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:32:07.005 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:32:07.006 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:32:07.007 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:32:07.008 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.009 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.010 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.013 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:32:07.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.016 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.019 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.020 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:32:07.021 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:32:07.022 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.023 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 msgr2=0x7f17a806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 0x7f17a806e2c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009b40 tx=0x7f17c0008040 comp rx=0 tx=0).stop 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 msgr2=0x7f17c414eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009f90 tx=0x7f17bc009750 comp rx=0 tx=0).stop 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 shutdown_connections 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f17a806be00 0x7f17a806e2c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17c413bd80 0x7f17c414eba0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17c413ea00 0x7f17c413d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 --2- 172.21.15.99:0/2588648046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b0004d40 0x7f17c413b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 >> 172.21.15.99:0/2588648046 conn(0x7f17c406bc10 msgr2=0x7f17c40b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:07.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 shutdown_connections 2024-09-10T11:32:07.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:06.948+0000 7f17cbdfd640 1 -- 172.21.15.99:0/2588648046 wait complete. 2024-09-10T11:32:07.026 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:32:07.027 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:32:07.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.123+0000 7f349d0e4640 1 Processor -- start 2024-09-10T11:32:07.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.123+0000 7f349d0e4640 1 -- start start 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349d0e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f349815c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349d0e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3498152e60 con 0x7f349806ece0 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349d0e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34980d15e0 con 0x7f34980d1200 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349d0e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3498072680 con 0x7f3498073440 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f348e59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f349806ece0 0x7f349815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51928/0 (socket says 172.21.15.99:51928) 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f348e59c640 1 -- 172.21.15.99:0/3565936317 learned_addr learned my addr 172.21.15.99:0/3565936317 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750333874 0 0) 0x7f3498152e60 con 0x7f349806ece0 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349759e640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f349815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3484003520 con 0x7f349806ece0 2024-09-10T11:32:07.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 803842856 0 0) 0x7f3484003520 con 0x7f349806ece0 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 msgr2=0x7f349815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f349815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 >> v1:172.21.15.196:6789/0 conn(0x7f34980d1200 legacy=0x7f349806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3498072960 con 0x7f349806ece0 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f3480001670 con 0x7f349806ece0 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 >> v1:172.21.15.99:6789/0 conn(0x7f349806ece0 legacy=0x7f349815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f3484003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3484004390 0x7f3484004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f3484005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3498072680 con 0x7f34840039e0 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f348400ba70 con 0x7f3484004390 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f348400bc70 con 0x7f3498073440 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.124+0000 7f3496d9d640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f3484003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f348e59c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3484004390 0x7f3484004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f349759e640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f3484005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 -- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 msgr2=0x7f3484005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f3484005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 -- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3484004390 msgr2=0x7f3484004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3484004390 0x7f3484004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 -- 172.21.15.99:0/3565936317 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3484003520 con 0x7f34840039e0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f348e59c640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3484004390 0x7f3484004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f349759e640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3498073440 0x7f3484005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f3496d9d640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f3484003dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3488001a30 tx=0x7f3484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97de80c9423abe12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.125+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488037070 con 0x7f34840039e0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f348802adf0 con 0x7f34840039e0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349659c640 1 -- 172.21.15.99:0/3565936317 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488001600 con 0x7f34840039e0 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 -- 172.21.15.99:0/3565936317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 msgr2=0x7f3484003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 --2- 172.21.15.99:0/3565936317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f3484003dc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3488001a30 tx=0x7f3484005830 comp rx=0 tx=0).stop 2024-09-10T11:32:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349759e640 1 -- 172.21.15.99:0/3565936317 reap_dead start 2024-09-10T11:32:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 -- 172.21.15.99:0/3565936317 shutdown_connections 2024-09-10T11:32:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 -- 172.21.15.99:0/3565936317 >> 172.21.15.99:0/3565936317 conn(0x7f349806bc10 msgr2=0x7f349806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 -- 172.21.15.99:0/3565936317 shutdown_connections 2024-09-10T11:32:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.126+0000 7f349d0e4640 1 -- 172.21.15.99:0/3565936317 wait complete. 2024-09-10T11:32:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.127+0000 7f349d0e4640 1 Processor -- start 2024-09-10T11:32:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 -- start start 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f349814b0c0 0x7f34980cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34980cb9e0 0x7f349806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f349815cd90 con 0x7f34840039e0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3498072f20 con 0x7f34980cb9e0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f349d0e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f349815c770 con 0x7f349814b0c0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f3496d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f3496d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33374/0 (socket says 172.21.15.99:33374) 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.128+0000 7f3496d9d640 1 -- 172.21.15.99:0/333462630 learned_addr learned my addr 172.21.15.99:0/333462630 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f349759e640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34980cb9e0 0x7f349806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f348e59c640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f349814b0c0 0x7f34980cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 -- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f349814b0c0 msgr2=0x7f34980cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f349814b0c0 0x7f34980cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 -- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34980cb9e0 msgr2=0x7f349806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34980cb9e0 0x7f349806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 -- 172.21.15.99:0/333462630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34980cbed0 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f3496d9d640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3488001b50 tx=0x7f3488001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488040020 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3488005290 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3488037040 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34980c9c60 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.129+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34980ca220 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.130+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f349815c060 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.131+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3488043020 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.131+0000 7f348ffff640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 0x7f347806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.131+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f34880aaad0 con 0x7f34840039e0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.131+0000 7f348e59c640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 0x7f347806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.131+0000 7f348e59c640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 0x7f347806e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3480001200 tx=0x7f3480009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.135+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f348804a070 con 0x7f34840039e0 2024-09-10T11:32:07.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.302+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f34980cf4c0 con 0x7f34840039e0 2024-09-10T11:32:07.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.303+0000 7f348ffff640 1 -- 172.21.15.99:0/333462630 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v54) ==== 81+0+34200 (secure 0 0 0) 0x7f34980d15e0 con 0x7f34840039e0 2024-09-10T11:32:07.305 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:07.305 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 54, 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:32:06.798356+0000", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:32:07.306 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 7, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:32:07.307 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:07.308 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:07.310 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:07.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:07.312 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:07.313 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 6, 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.584024+0000", 2024-09-10T11:32:07.314 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:07.315 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.316 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:07.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:07.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:07.319 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:07.320 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 7, 2024-09-10T11:32:07.321 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:31:37.949922+0000", 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:32:07.322 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:32:07.323 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "54", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:32:07.324 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:32:07.325 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:32:07.326 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:32:07.327 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.328 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.331 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.332 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.334 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.335 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.337 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.338 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:07.340 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.343 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.344 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:32:07.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.346 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:32:07.348 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:32:07.349 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:32:07.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:32:07.351 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.352 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:32:07.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.355 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.358 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.363 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.364 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.367 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:32:07.369 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 43, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:32:07.370 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:32:07.371 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:32:07.374 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.375 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.376 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.378 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.379 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:32:07.381 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:32:07.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:07.383 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 msgr2=0x7f347806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 0x7f347806e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3480001200 tx=0x7f3480009040 comp rx=0 tx=0).stop 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 msgr2=0x7f34980ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3488001b50 tx=0x7f3488001400 comp rx=0 tx=0).stop 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 shutdown_connections 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f347806be30 0x7f347806e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34980cb9e0 0x7f349806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f349814b0c0 0x7f34980cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 --2- 172.21.15.99:0/333462630 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34840039e0 0x7f34980ca740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 >> 172.21.15.99:0/333462630 conn(0x7f349806bc10 msgr2=0x7f34980b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:07.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 shutdown_connections 2024-09-10T11:32:07.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.305+0000 7f349d0e4640 1 -- 172.21.15.99:0/333462630 wait complete. 2024-09-10T11:32:07.386 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.483+0000 7f1d4122e640 1 Processor -- start 2024-09-10T11:32:07.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.483+0000 7f1d4122e640 1 -- start start 2024-09-10T11:32:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.483+0000 7f1d4122e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d3c15cb70 0x7f1d3c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d4122e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d3c0b4150 0x7f1d3c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d4122e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3c072650 con 0x7f1d3c0b0df0 2024-09-10T11:32:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d4122e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c0b1240 con 0x7f1d3c15cb70 2024-09-10T11:32:07.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d4122e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c15db70 con 0x7f1d3c0b4150 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d3c15cb70 0x7f1d3c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d3c15cb70 0x7f1d3c072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33388/0 (socket says 172.21.15.99:33388) 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 -- 172.21.15.99:0/1675752341 learned_addr learned my addr 172.21.15.99:0/1675752341 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3259c640 1 --2- 172.21.15.99:0/1675752341 >> v2:172.21.15.196:3300/0 conn(0x7f1d3c0b4150 0x7f1d3c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 -- 172.21.15.99:0/1675752341 >> v2:172.21.15.196:3300/0 conn(0x7f1d3c0b4150 msgr2=0x7f1d3c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 --2- 172.21.15.99:0/1675752341 >> v2:172.21.15.196:3300/0 conn(0x7f1d3c0b4150 0x7f1d3c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 -- 172.21.15.99:0/1675752341 >> v1:172.21.15.196:6789/0 conn(0x7f1d3c0b0df0 legacy=0x7f1d3c15cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.484+0000 7f1d3ad9d640 1 -- 172.21.15.99:0/1675752341 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c15de50 con 0x7f1d3c15cb70 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.485+0000 7f1d3ad9d640 1 --2- 172.21.15.99:0/1675752341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c072de0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c06e080 tx=0x7f1d2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9414a6d75f8575e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.485+0000 7f1d3a59c640 1 -- 172.21.15.99:0/1675752341 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c037070 con 0x7f1d3c15cb70 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.485+0000 7f1d3a59c640 1 -- 172.21.15.99:0/1675752341 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c032440 con 0x7f1d3c15cb70 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.485+0000 7f1d3a59c640 1 -- 172.21.15.99:0/1675752341 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c032810 con 0x7f1d3c15cb70 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 -- 172.21.15.99:0/1675752341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 msgr2=0x7f1d3c072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 --2- 172.21.15.99:0/1675752341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c072de0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c06e080 tx=0x7f1d2c02a270 comp rx=0 tx=0).stop 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 -- 172.21.15.99:0/1675752341 shutdown_connections 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 --2- 172.21.15.99:0/1675752341 >> v2:172.21.15.196:3300/0 conn(0x7f1d3c0b4150 0x7f1d3c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 --2- 172.21.15.99:0/1675752341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c072de0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 -- 172.21.15.99:0/1675752341 >> 172.21.15.99:0/1675752341 conn(0x7f1d3c0b3630 msgr2=0x7f1d3c0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:07.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 -- 172.21.15.99:0/1675752341 shutdown_connections 2024-09-10T11:32:07.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.486+0000 7f1d4122e640 1 -- 172.21.15.99:0/1675752341 wait complete. 2024-09-10T11:32:07.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.487+0000 7f1d4122e640 1 Processor -- start 2024-09-10T11:32:07.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.487+0000 7f1d4122e640 1 -- start start 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0b0df0 0x7f1d3c150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d3c0b4150 0x7f1d3c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c073320 con 0x7f1d3c15cb70 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c15e130 con 0x7f1d3c0b0df0 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d4122e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c0734a0 con 0x7f1d3c0b4150 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33396/0 (socket says 172.21.15.99:33396) 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 -- 172.21.15.99:0/2065689602 learned_addr learned my addr 172.21.15.99:0/2065689602 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3ad9d640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0b0df0 0x7f1d3c150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3259c640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d3c0b4150 0x7f1d3c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 -- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d3c0b4150 msgr2=0x7f1d3c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d3c0b4150 0x7f1d3c14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 -- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0b0df0 msgr2=0x7f1d3c150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0b0df0 0x7f1d3c150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.488+0000 7f1d3b59e640 1 -- 172.21.15.99:0/2065689602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c151510 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d3b59e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400c7d0 tx=0x7f1d3400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34003b70 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d34003d30 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c0aea40 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400aa30 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.489+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c0af000 con 0x7f1d3c15cb70 2024-09-10T11:32:07.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.490+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c0af200 con 0x7f1d3c15cb70 2024-09-10T11:32:07.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.490+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d3400abd0 con 0x7f1d3c15cb70 2024-09-10T11:32:07.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.491+0000 7f1d33fff640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:07.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.491+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f1d3408abd0 con 0x7f1d3c15cb70 2024-09-10T11:32:07.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.491+0000 7f1d3ad9d640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:07.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.491+0000 7f1d3ad9d640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c034000 tx=0x7f1d2c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:07.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.495+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d3c0af200 con 0x7f1d3c15cb70 2024-09-10T11:32:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.701+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d3c14d6f0 con 0x7f1d3c15cb70 2024-09-10T11:32:07.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.702+0000 7f1d33fff640 1 -- 172.21.15.99:0/2065689602 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7f1d34058f30 con 0x7f1d3c15cb70 2024-09-10T11:32:07.704 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:07.704 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:07.704 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 msgr2=0x7f1d1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c034000 tx=0x7f1d2c02e000 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 msgr2=0x7f1d3c14dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400c7d0 tx=0x7f1d3400cc90 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 shutdown_connections 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1d1c06be30 0x7f1d1c06e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d3c15cb70 0x7f1d3c14dc70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d3c0b4150 0x7f1d3c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 --2- 172.21.15.99:0/2065689602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d3c0b0df0 0x7f1d3c150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 >> 172.21.15.99:0/2065689602 conn(0x7f1d3c0b3630 msgr2=0x7f1d3c15cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:07.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 shutdown_connections 2024-09-10T11:32:07.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:07.704+0000 7f1d4122e640 1 -- 172.21.15.99:0/2065689602 wait complete. 2024-09-10T11:32:07.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:32:07.833 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":5211,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.172297853,"mdsmap_epoch":21,"osdmap_epoch":54,"osdmap_epoch_barrier":49,"uptime":32.149500873999997} 2024-09-10T11:32:07.834 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 5211, "mdsmap_epoch": 21, "osdmap_epoch": 54, "osdmap_epoch_barrier": 49, "rank_uptime": 28.172297853, "state": "up:active", "uptime": 32.149500874, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:32:07.834 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:32:07.834 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:07.834 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:07.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:07.850 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-09-10T11:32:08.111 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2024-09-10T11:32:08.112 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2024-09-10T11:32:08.112 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.205242 s, 255 MB/s 2024-09-10T11:32:08.113 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:32:08.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:32:08.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:32:08.196 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:32:08.196 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:32:08.196 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:08.196 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:08.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:32:08.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.350+0000 7f031d518640 1 Processor -- start 2024-09-10T11:32:08.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.350+0000 7f031d518640 1 -- start start 2024-09-10T11:32:08.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f031d518640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f031815eb80 0x7f03180d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f031d518640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 0x7f03180d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f031d518640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03180d52f0 con 0x7f03180d4e40 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f031d518640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f031815ff70 con 0x7f031815eb80 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f031d518640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03180d1620 con 0x7f03181626c0 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f0316ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f031815eb80 0x7f03180d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f0316ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f031815eb80 0x7f03180d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:56276/0 (socket says 172.21.15.99:56276) 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f0316ffd640 1 -- 172.21.15.99:0/1077186317 learned_addr learned my addr 172.21.15.99:0/1077186317 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f03177fe640 1 --1- 172.21.15.99:0/1077186317 >> v1:172.21.15.196:6789/0 conn(0x7f03180d4e40 0x7f03180d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:47434/0 (socket says 172.21.15.99:47434) 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.351+0000 7f030ffff640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 0x7f03180d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f0316ffd640 1 -- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 msgr2=0x7f03180d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:08.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f0316ffd640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 0x7f03180d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f0316ffd640 1 -- 172.21.15.99:0/1077186317 >> v1:172.21.15.196:6789/0 conn(0x7f03180d4e40 legacy=0x7f03180d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f03167fc640 1 -- 172.21.15.99:0/1077186317 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950591093 0 0) 0x7f0310001490 con 0x7f03180d4e40 2024-09-10T11:32:08.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f0316ffd640 1 -- 172.21.15.99:0/1077186317 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0318162bc0 con 0x7f031815eb80 2024-09-10T11:32:08.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f030ffff640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 0x7f03180d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:08.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f0316ffd640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f03180d29b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0308009870 tx=0x7f030802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29a660322d88f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f03167fc640 1 -- 172.21.15.99:0/1077186317 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308037070 con 0x7f031815eb80 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f03167fc640 1 -- 172.21.15.99:0/1077186317 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0308032440 con 0x7f031815eb80 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.352+0000 7f03167fc640 1 -- 172.21.15.99:0/1077186317 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0308032810 con 0x7f031815eb80 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 -- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 msgr2=0x7f03180d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f03180d29b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0308009870 tx=0x7f030802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 -- 172.21.15.99:0/1077186317 shutdown_connections 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03181626c0 0x7f03180d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 --2- 172.21.15.99:0/1077186317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f03180d29b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 -- 172.21.15.99:0/1077186317 >> 172.21.15.99:0/1077186317 conn(0x7f031808e680 msgr2=0x7f031809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 -- 172.21.15.99:0/1077186317 shutdown_connections 2024-09-10T11:32:08.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.353+0000 7f031d518640 1 -- 172.21.15.99:0/1077186317 wait complete. 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.354+0000 7f031d518640 1 Processor -- start 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.354+0000 7f031d518640 1 -- start start 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03180d4e40 0x7f03181f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f0318158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0318163460 con 0x7f03181626c0 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0318163180 con 0x7f031815eb80 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f031d518640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03180d2ef0 con 0x7f03180d4e40 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33406/0 (socket says 172.21.15.99:33406) 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 -- 172.21.15.99:0/3933393205 learned_addr learned my addr 172.21.15.99:0/3933393205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f030ffff640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f0318158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f0316ffd640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03180d4e40 0x7f03181f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 -- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03180d4e40 msgr2=0x7f03181f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03180d4e40 0x7f03181f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 -- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 msgr2=0x7f0318158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f0318158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.355+0000 7f03177fe640 1 -- 172.21.15.99:0/3933393205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031815ce80 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f03177fe640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f031000c7f0 tx=0x7f031000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310018070 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031000a5e0 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0318161f70 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031000aa10 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.356+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181f4fb0 con 0x7f03181626c0 2024-09-10T11:32:08.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.357+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03180d29b0 con 0x7f03181626c0 2024-09-10T11:32:08.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.357+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f031001b070 con 0x7f03181626c0 2024-09-10T11:32:08.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.358+0000 7f030f7fe640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 0x7f02f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.358+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f031008a380 con 0x7f03181626c0 2024-09-10T11:32:08.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.358+0000 7f0316ffd640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 0x7f02f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.358+0000 7f0316ffd640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 0x7f02f806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0308034000 tx=0x7f030802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.362+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03180d52f0 con 0x7f03181626c0 2024-09-10T11:32:08.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.537+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f03180b3dc0 con 0x7f03181626c0 2024-09-10T11:32:08.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.537+0000 7f030f7fe640 1 -- 172.21.15.99:0/3933393205 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f03100586e0 con 0x7f03181626c0 2024-09-10T11:32:08.539 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:32:08.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 msgr2=0x7f02f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 0x7f02f806e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0308034000 tx=0x7f030802e000 comp rx=0 tx=0).stop 2024-09-10T11:32:08.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 msgr2=0x7f031815c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f031000c7f0 tx=0x7f031000ccb0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 shutdown_connections 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f02f806be30 0x7f02f806e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03181626c0 0x7f031815c940 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f031815eb80 0x7f0318158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 --2- 172.21.15.99:0/3933393205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03180d4e40 0x7f03181f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 >> 172.21.15.99:0/3933393205 conn(0x7f031808e680 msgr2=0x7f0318159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 shutdown_connections 2024-09-10T11:32:08.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.539+0000 7f031d518640 1 -- 172.21.15.99:0/3933393205 wait complete. 2024-09-10T11:32:08.551 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:32:08.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:08.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:08.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[41425]: fuse finished with error 0 and tester_r 0 2024-09-10T11:32:08.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:32:08.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:08.581 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:32:08.636 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:32:08.636 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:32:08.636 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:08.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:32:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.773+0000 7f3886566640 1 Processor -- start 2024-09-10T11:32:08.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.773+0000 7f3886566640 1 -- start start 2024-09-10T11:32:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 0x7f3880073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f388015c060 0x7f388015c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f388006ef50 con 0x7f388015c060 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3880152e60 con 0x7f38800d1200 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3886566640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f38800d15e0 con 0x7f388006e5d0 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3884b03640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f388015c060 0x7f388015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3884b03640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f388015c060 0x7f388015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33414/0 (socket says 172.21.15.99:33414) 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f3884b03640 1 -- 172.21.15.99:0/702056769 learned_addr learned my addr 172.21.15.99:0/702056769 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f387ffff640 1 --2- 172.21.15.99:0/702056769 >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 0x7f3880073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.774+0000 7f387f7fe640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 -- 172.21.15.99:0/702056769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 msgr2=0x7f388006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 -- 172.21.15.99:0/702056769 >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 msgr2=0x7f3880073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 --2- 172.21.15.99:0/702056769 >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 0x7f3880073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 -- 172.21.15.99:0/702056769 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3880072680 con 0x7f388015c060 2024-09-10T11:32:08.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f387ffff640 1 --2- 172.21.15.99:0/702056769 >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 0x7f3880073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f3884b03640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f388015c060 0x7f388015c480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38800d1850 tx=0x7f387802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dddc4809da93094 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f387effd640 1 -- 172.21.15.99:0/702056769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878036070 con 0x7f388015c060 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f387effd640 1 -- 172.21.15.99:0/702056769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3878031600 con 0x7f388015c060 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.775+0000 7f387effd640 1 -- 172.21.15.99:0/702056769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38780319d0 con 0x7f388015c060 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 -- 172.21.15.99:0/702056769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f388015c060 msgr2=0x7f388015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f388015c060 0x7f388015c480 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38800d1850 tx=0x7f387802a610 comp rx=0 tx=0).stop 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 -- 172.21.15.99:0/702056769 shutdown_connections 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f388015c060 0x7f388015c480 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 --2- 172.21.15.99:0/702056769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 --2- 172.21.15.99:0/702056769 >> v2:172.21.15.196:3300/0 conn(0x7f38800d1200 0x7f3880073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 -- 172.21.15.99:0/702056769 >> 172.21.15.99:0/702056769 conn(0x7f388006bc10 msgr2=0x7f388006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 -- 172.21.15.99:0/702056769 shutdown_connections 2024-09-10T11:32:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.776+0000 7f3886566640 1 -- 172.21.15.99:0/702056769 wait complete. 2024-09-10T11:32:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.777+0000 7f3886566640 1 Processor -- start 2024-09-10T11:32:08.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.777+0000 7f3886566640 1 -- start start 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388014c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f388015c060 0x7f3880143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3880072f20 con 0x7f38800d1200 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3880072c40 con 0x7f388015c060 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3886566640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f388015cd90 con 0x7f388006e5d0 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f387f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33418/0 (socket says 172.21.15.99:33418) 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f387f7fe640 1 -- 172.21.15.99:0/1863805625 learned_addr learned my addr 172.21.15.99:0/1863805625 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:08.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f387ffff640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388014c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.778+0000 7f3884b03640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f388015c060 0x7f3880143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 -- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 msgr2=0x7f388014c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388014c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 -- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f388015c060 msgr2=0x7f3880143170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f388015c060 0x7f3880143170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 -- 172.21.15.99:0/1863805625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800cce80 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387f7fe640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f387400d620 tx=0x7f387400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400b620 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3874019070 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388014abe0 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400ba00 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.779+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388014b1a0 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.780+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388006ea10 con 0x7f38800d1200 2024-09-10T11:32:08.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.780+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f387400bba0 con 0x7f38800d1200 2024-09-10T11:32:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.781+0000 7f387cff9640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 0x7f386c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.781+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f387408b010 con 0x7f38800d1200 2024-09-10T11:32:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.781+0000 7f387ffff640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 0x7f386c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:08.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.781+0000 7f387ffff640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 0x7f386c06e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3870006e80 tx=0x7f3870008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:08.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.785+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3874059400 con 0x7f38800d1200 2024-09-10T11:32:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.964+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3880073590 con 0x7f38800d1200 2024-09-10T11:32:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.964+0000 7f387cff9640 1 -- 172.21.15.99:0/1863805625 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) ==== 80+0+0 (secure 0 0 0) 0x7f387405d700 con 0x7f38800d1200 2024-09-10T11:32:08.966 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 msgr2=0x7f386c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 0x7f386c06e320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3870006e80 tx=0x7f3870008040 comp rx=0 tx=0).stop 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 msgr2=0x7f38800c9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f387400d620 tx=0x7f387400dae0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 shutdown_connections 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386c06be60 0x7f386c06e320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f388015c060 0x7f3880143170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38800d1200 0x7f38800c9ca0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 --2- 172.21.15.99:0/1863805625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f388006e5d0 0x7f388014c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:08.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 >> 172.21.15.99:0/1863805625 conn(0x7f388006bc10 msgr2=0x7f388006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:08.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 shutdown_connections 2024-09-10T11:32:08.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:08.966+0000 7f3886566640 1 -- 172.21.15.99:0/1863805625 wait complete. 2024-09-10T11:32:08.979 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:32:08.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:08.979 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:09.004 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[34814]: fuse finished with error 0 and tester_r 0 2024-09-10T11:32:09.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:32:09.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:09.007 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:32:09.139 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:32:09.139 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:09.139 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:09.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:32:09.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:32:09.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:09.156 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:32:09.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:32:09.538 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:32:09.538 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:32:09.538 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:32:09.539 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:09.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:32:09.557 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:32:09.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:09.558 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:32:09.615 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:32:09.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:32:09.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:09.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.715+0000 7fd52f8a5640 1 Processor -- start 2024-09-10T11:32:09.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.715+0000 7fd52f8a5640 1 -- start start 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52f8a5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd52815c7b0 0x7fd5280d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52f8a5640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5280ceb10 con 0x7fd52806f000 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52f8a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5280b3a90 con 0x7fd5280d0fb0 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52f8a5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd5280d1420 con 0x7fd52815c7b0 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52d641640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd52815c7b0 0x7fd5280d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:09.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52de42640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd52806f000 0x7fd52806d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51938/0 (socket says 172.21.15.99:51938) 2024-09-10T11:32:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.716+0000 7fd52de42640 1 -- 172.21.15.99:0/4079510981 learned_addr learned my addr 172.21.15.99:0/4079510981 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd52d641640 1 -- 172.21.15.99:0/4079510981 >> v1:172.21.15.196:6789/0 conn(0x7fd5280d0fb0 legacy=0x7fd52806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd52d641640 1 -- 172.21.15.99:0/4079510981 >> v1:172.21.15.99:6789/0 conn(0x7fd52806f000 legacy=0x7fd52806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd52d641640 1 -- 172.21.15.99:0/4079510981 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5280b3d70 con 0x7fd52815c7b0 2024-09-10T11:32:09.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd51ffff640 1 -- 172.21.15.99:0/4079510981 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 241407254 0 0) 0x7fd520001490 con 0x7fd52806f000 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd52d641640 1 --2- 172.21.15.99:0/4079510981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52815c7b0 0x7fd5280d0a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd52402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c83e8439dfd0318 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd51ffff640 1 -- 172.21.15.99:0/4079510981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037070 con 0x7fd52815c7b0 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd51ffff640 1 -- 172.21.15.99:0/4079510981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524032440 con 0x7fd52815c7b0 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.717+0000 7fd51ffff640 1 -- 172.21.15.99:0/4079510981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524032810 con 0x7fd52815c7b0 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 -- 172.21.15.99:0/4079510981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52815c7b0 msgr2=0x7fd5280d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/4079510981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52815c7b0 0x7fd5280d0a70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd52402a1e0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 -- 172.21.15.99:0/4079510981 shutdown_connections 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/4079510981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52815c7b0 0x7fd5280d0a70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 -- 172.21.15.99:0/4079510981 >> 172.21.15.99:0/4079510981 conn(0x7fd52806bc10 msgr2=0x7fd52806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 -- 172.21.15.99:0/4079510981 shutdown_connections 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.718+0000 7fd52f8a5640 1 -- 172.21.15.99:0/4079510981 wait complete. 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.719+0000 7fd52f8a5640 1 Processor -- start 2024-09-10T11:32:09.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.719+0000 7fd52f8a5640 1 -- start start 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5280d0fb0 0x7fd5281e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52815c7b0 0x7fd52814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4330 con 0x7fd52806f000 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b3840 con 0x7fd52815c7b0 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52f8a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280b4610 con 0x7fd5280d0fb0 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33434/0 (socket says 172.21.15.99:33434) 2024-09-10T11:32:09.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 -- 172.21.15.99:0/2314644766 learned_addr learned my addr 172.21.15.99:0/2314644766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 -- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5280d0fb0 msgr2=0x7fd5281e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5280d0fb0 0x7fd5281e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 -- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52815c7b0 msgr2=0x7fd52814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52815c7b0 0x7fd52814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.720+0000 7fd52d641640 1 -- 172.21.15.99:0/2314644766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5281e69f0 con 0x7fd52806f000 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd52d641640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd5240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524037040 con 0x7fd52806f000 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd524004250 con 0x7fd52806f000 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd524032e20 con 0x7fd52806f000 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5281e6cd0 con 0x7fd52806f000 2024-09-10T11:32:09.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.721+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5281e7290 con 0x7fd52806f000 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.722+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5280b50b0 con 0x7fd52806f000 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.722+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd524030770 con 0x7fd52806f000 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.723+0000 7fd51dffb640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 0x7fd50c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.723+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7fd5240ac060 con 0x7fd52806f000 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.723+0000 7fd52ce40640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 0x7fd50c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:09.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.723+0000 7fd52ce40640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 0x7fd50c06e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd52814d9d0 tx=0x7fd5180079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:09.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.727+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd5280b50b0 con 0x7fd52806f000 2024-09-10T11:32:09.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.929+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd52815ccf0 con 0x7fd52806f000 2024-09-10T11:32:09.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.929+0000 7fd51dffb640 1 -- 172.21.15.99:0/2314644766 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7fd52407abc0 con 0x7fd52806f000 2024-09-10T11:32:09.932 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:09.932 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:09.932 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 msgr2=0x7fd50c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 0x7fd50c06e320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd52814d9d0 tx=0x7fd5180079b0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 msgr2=0x7fd5281e7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd5240097a0 tx=0x7fd5240046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 shutdown_connections 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd50c06be60 0x7fd50c06e320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd52815c7b0 0x7fd52814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd5280d0fb0 0x7fd5281e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 --2- 172.21.15.99:0/2314644766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52806f000 0x7fd5281e7a90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:09.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 >> 172.21.15.99:0/2314644766 conn(0x7fd52806bc10 msgr2=0x7fd528157a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:09.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 shutdown_connections 2024-09-10T11:32:09.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:09.931+0000 7fd52f8a5640 1 -- 172.21.15.99:0/2314644766 wait complete. 2024-09-10T11:32:09.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:10.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.041+0000 7f8155f9d640 1 Processor -- start 2024-09-10T11:32:10.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.041+0000 7f8155f9d640 1 -- start start 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f8155f9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 0x7f81500d2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f8155f9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f81500b3020 0x7f81500cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f8155f9d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81500d0ba0 con 0x7f81500d1570 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f8155f9d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f81500d0e80 con 0x7f81500b3020 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f8155f9d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f81500d1160 con 0x7f815015eee0 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f814ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 0x7f81500d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.042+0000 7f815579c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f81500b3020 0x7f81500cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f8154f9b640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f81500d1570 0x7f81500d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:47448/0 (socket says 172.21.15.99:47448) 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f8154f9b640 1 -- 172.21.15.99:0/2154017894 learned_addr learned my addr 172.21.15.99:0/2154017894 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:10.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f814f7fe640 1 -- 172.21.15.99:0/2154017894 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87289743 0 0) 0x7f81500d0ba0 con 0x7f81500d1570 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f814f7fe640 1 -- 172.21.15.99:0/2154017894 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8138003520 con 0x7f81500d1570 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f815579c640 1 -- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 msgr2=0x7f81500d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f815579c640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 0x7f81500d2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f815579c640 1 -- 172.21.15.99:0/2154017894 >> v1:172.21.15.196:6789/0 conn(0x7f81500d1570 legacy=0x7f81500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f815579c640 1 -- 172.21.15.99:0/2154017894 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81500b1810 con 0x7f81500b3020 2024-09-10T11:32:10.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.043+0000 7f814ffff640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 0x7f81500d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f815579c640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81500b3020 0x7f81500cf590 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fed5d84a17827c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f814f7fe640 1 -- 172.21.15.99:0/2154017894 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148007740 con 0x7f81500b3020 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f814f7fe640 1 -- 172.21.15.99:0/2154017894 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81480078e0 con 0x7f81500b3020 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f814f7fe640 1 -- 172.21.15.99:0/2154017894 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148007cb0 con 0x7f81500b3020 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 -- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81500b3020 msgr2=0x7f81500cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81500b3020 0x7f81500cf590 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a660 comp rx=0 tx=0).stop 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 -- 172.21.15.99:0/2154017894 shutdown_connections 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81500b3020 0x7f81500cf590 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 --2- 172.21.15.99:0/2154017894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f815015eee0 0x7f81500d2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 -- 172.21.15.99:0/2154017894 >> 172.21.15.99:0/2154017894 conn(0x7f815009de40 msgr2=0x7f81500d4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 -- 172.21.15.99:0/2154017894 shutdown_connections 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.044+0000 7f8155f9d640 1 -- 172.21.15.99:0/2154017894 wait complete. 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 Processor -- start 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 -- start start 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81500d1570 0x7f81500cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f815015eee0 0x7f81500cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81500cfad0 con 0x7f81500b3020 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81500b3e80 con 0x7f815015eee0 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.046+0000 7f8155f9d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f81500cfc50 con 0x7f81500d1570 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33440/0 (socket says 172.21.15.99:33440) 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 -- 172.21.15.99:0/3313397551 learned_addr learned my addr 172.21.15.99:0/3313397551 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f814ffff640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81500d1570 0x7f81500cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f815579c640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f815015eee0 0x7f81500cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 -- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81500d1570 msgr2=0x7f81500cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81500d1570 0x7f81500cbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 -- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f815015eee0 msgr2=0x7f81500cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f815015eee0 0x7f81500cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 -- 172.21.15.99:0/3313397551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81500cc460 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8154f9b640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f814000c430 tx=0x7f814000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140017070 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8140005ce0 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81500cc740 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400072f0 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.047+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81500ccd00 con 0x7f81500b3020 2024-09-10T11:32:10.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.048+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81500cf590 con 0x7f81500b3020 2024-09-10T11:32:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.049+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f814001a070 con 0x7f81500b3020 2024-09-10T11:32:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.049+0000 7f814d7fa640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 0x7f813c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.049+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f814008a220 con 0x7f81500b3020 2024-09-10T11:32:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.049+0000 7f814ffff640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 0x7f813c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.050+0000 7f814ffff640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 0x7f813c06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8144009560 tx=0x7f8144009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.053+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8140058610 con 0x7f81500b3020 2024-09-10T11:32:10.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.251+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f815006cd00 con 0x7f81500b3020 2024-09-10T11:32:10.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.251+0000 7f814d7fa640 1 -- 172.21.15.99:0/3313397551 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7f814005c910 con 0x7f81500b3020 2024-09-10T11:32:10.253 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:10.253 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:10.254 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:10.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 msgr2=0x7f813c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 0x7f813c06e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8144009560 tx=0x7f8144009190 comp rx=0 tx=0).stop 2024-09-10T11:32:10.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 msgr2=0x7f8150159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f814000c430 tx=0x7f814000c8f0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 shutdown_connections 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f813c06be30 0x7f813c06e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f815015eee0 0x7f81500cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81500d1570 0x7f81500cbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 --2- 172.21.15.99:0/3313397551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81500b3020 0x7f8150159140 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 >> 172.21.15.99:0/3313397551 conn(0x7f815009de40 msgr2=0x7f81500d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 shutdown_connections 2024-09-10T11:32:10.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.253+0000 7f8155f9d640 1 -- 172.21.15.99:0/3313397551 wait complete. 2024-09-10T11:32:10.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:10.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.363+0000 7fb62d6c3640 1 Processor -- start 2024-09-10T11:32:10.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.363+0000 7fb62d6c3640 1 -- start start 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb62d6c3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6280b1430 0x7fb6280b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb62d6c3640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb628070c50 con 0x7fb62815cae0 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb62d6c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb628070ef0 con 0x7fb6280b6920 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb62d6c3640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb6280b6d90 con 0x7fb6280b1430 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb61e7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb6280b1430 0x7fb6280b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb626ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb62815cae0 0x7fb628070260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51950/0 (socket says 172.21.15.99:51950) 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb626ffd640 1 -- 172.21.15.99:0/3766581093 learned_addr learned my addr 172.21.15.99:0/3766581093 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:10.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb61e7fc640 1 -- 172.21.15.99:0/3766581093 >> v1:172.21.15.196:6789/0 conn(0x7fb6280b6920 legacy=0x7fb6281665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb61e7fc640 1 -- 172.21.15.99:0/3766581093 >> v1:172.21.15.99:6789/0 conn(0x7fb62815cae0 legacy=0x7fb628070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb61e7fc640 1 -- 172.21.15.99:0/3766581093 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628071190 con 0x7fb6280b1430 2024-09-10T11:32:10.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.364+0000 7fb6267fc640 1 -- 172.21.15.99:0/3766581093 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970795725 0 0) 0x7fb628070c50 con 0x7fb62815cae0 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.365+0000 7fb61e7fc640 1 --2- 172.21.15.99:0/3766581093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6280b1430 0x7fb6280b63e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb6100097a0 tx=0x7fb61002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baf47a23254411e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.365+0000 7fb6267fc640 1 -- 172.21.15.99:0/3766581093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610037070 con 0x7fb6280b1430 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.365+0000 7fb6267fc640 1 -- 172.21.15.99:0/3766581093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb610032440 con 0x7fb6280b1430 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.365+0000 7fb6267fc640 1 -- 172.21.15.99:0/3766581093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610032810 con 0x7fb6280b1430 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 -- 172.21.15.99:0/3766581093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6280b1430 msgr2=0x7fb6280b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/3766581093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6280b1430 0x7fb6280b63e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb6100097a0 tx=0x7fb61002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 -- 172.21.15.99:0/3766581093 shutdown_connections 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/3766581093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb6280b1430 0x7fb6280b63e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 -- 172.21.15.99:0/3766581093 >> 172.21.15.99:0/3766581093 conn(0x7fb62809ba30 msgr2=0x7fb62809be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 -- 172.21.15.99:0/3766581093 shutdown_connections 2024-09-10T11:32:10.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.366+0000 7fb62d6c3640 1 -- 172.21.15.99:0/3766581093 wait complete. 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.367+0000 7fb62d6c3640 1 Processor -- start 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.367+0000 7fb62d6c3640 1 -- start start 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6280b1430 0x7fb6280a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6280b6920 0x7fb628152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb628070970 con 0x7fb62815cae0 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6280b5b20 con 0x7fb6280b1430 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb62d6c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6280b5ca0 con 0x7fb6280b6920 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33456/0 (socket says 172.21.15.99:33456) 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 -- 172.21.15.99:0/34156980 learned_addr learned my addr 172.21.15.99:0/34156980 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:10.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb626ffd640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6280b1430 0x7fb6280a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb61e7fc640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6280b6920 0x7fb628152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 -- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6280b6920 msgr2=0x7fb628152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6280b6920 0x7fb628152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 -- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6280b1430 msgr2=0x7fb6280a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6280b1430 0x7fb6280a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 -- 172.21.15.99:0/34156980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6281537b0 con 0x7fb62815cae0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb6277fe640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c780 tx=0x7fb62000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620004b60 con 0x7fb62815cae0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.368+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb620004d00 con 0x7fb62815cae0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.369+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000aa90 con 0x7fb62815cae0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.369+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb628153a90 con 0x7fb62815cae0 2024-09-10T11:32:10.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.369+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb628154050 con 0x7fb62815cae0 2024-09-10T11:32:10.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.370+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6280b63e0 con 0x7fb62815cae0 2024-09-10T11:32:10.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.370+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb62000ac30 con 0x7fb62815cae0 2024-09-10T11:32:10.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.370+0000 7fb61ffff640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 0x7fb60806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.371+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7fb62008b000 con 0x7fb62815cae0 2024-09-10T11:32:10.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.371+0000 7fb626ffd640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 0x7fb60806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.371+0000 7fb626ffd640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 0x7fb60806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb618000f60 tx=0x7fb618009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.375+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb620020070 con 0x7fb62815cae0 2024-09-10T11:32:10.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.539+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6280b2890 con 0x7fb62815cae0 2024-09-10T11:32:10.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.541+0000 7fb61ffff640 1 -- 172.21.15.99:0/34156980 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16669 (secure 0 0 0) 0x7fb620059360 con 0x7fb62815cae0 2024-09-10T11:32:10.543 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:10.543 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":54,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:06.798356+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:37.584024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:37.949922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.542+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 msgr2=0x7fb60806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.542+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 0x7fb60806e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb618000f60 tx=0x7fb618009040 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.542+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 msgr2=0x7fb6280b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.542+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb62000c780 tx=0x7fb62000cc40 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 shutdown_connections 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb60806be30 0x7fb60806e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62815cae0 0x7fb6280b32a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6280b6920 0x7fb628152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 --2- 172.21.15.99:0/34156980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6280b1430 0x7fb6280a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 >> 172.21.15.99:0/34156980 conn(0x7fb62809ba30 msgr2=0x7fb6280d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 shutdown_connections 2024-09-10T11:32:10.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.543+0000 7fb62d6c3640 1 -- 172.21.15.99:0/34156980 wait complete. 2024-09-10T11:32:10.556 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:32:10.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:10.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 Processor -- start 2024-09-10T11:32:10.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 -- start start 2024-09-10T11:32:10.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec0d1200 0x7fc9ec073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9ec15c770 0x7fc9ec06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ec152e60 con 0x7fc9ec06e5d0 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec06ef50 con 0x7fc9ec15c770 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.653+0000 7fc9f1053640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec0d15e0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9ec15c770 0x7fc9ec06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9ec15c770 0x7fc9ec06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33458/0 (socket says 172.21.15.99:33458) 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 -- 172.21.15.99:0/4147507484 learned_addr learned my addr 172.21.15.99:0/4147507484 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9ead9d640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec0d1200 0x7fc9ec073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 -- 172.21.15.99:0/4147507484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec0d1200 msgr2=0x7fc9ec073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec0d1200 0x7fc9ec073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 -- 172.21.15.99:0/4147507484 >> v1:172.21.15.196:6789/0 conn(0x7fc9ec06e5d0 legacy=0x7fc9ec15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.654+0000 7fc9eb59e640 1 -- 172.21.15.99:0/4147507484 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec072680 con 0x7fc9ec15c770 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9eb59e640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec15c770 0x7fc9ec06ea10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009e50 tx=0x7fc9e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c583daf39488a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9e9d9b640 1 -- 172.21.15.99:0/4147507484 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4036070 con 0x7fc9ec15c770 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9e9d9b640 1 -- 172.21.15.99:0/4147507484 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e4031600 con 0x7fc9ec15c770 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9e9d9b640 1 -- 172.21.15.99:0/4147507484 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e40319d0 con 0x7fc9ec15c770 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 -- 172.21.15.99:0/4147507484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec15c770 msgr2=0x7fc9ec06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec15c770 0x7fc9ec06ea10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009e50 tx=0x7fc9e402a680 comp rx=0 tx=0).stop 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 -- 172.21.15.99:0/4147507484 shutdown_connections 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec15c770 0x7fc9ec06ea10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 --2- 172.21.15.99:0/4147507484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec0d1200 0x7fc9ec073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 -- 172.21.15.99:0/4147507484 >> 172.21.15.99:0/4147507484 conn(0x7fc9ec06bc10 msgr2=0x7fc9ec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 -- 172.21.15.99:0/4147507484 shutdown_connections 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.655+0000 7fc9f1053640 1 -- 172.21.15.99:0/4147507484 wait complete. 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.657+0000 7fc9f1053640 1 Processor -- start 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.657+0000 7fc9f1053640 1 -- start start 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.657+0000 7fc9f1053640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec06e5d0 0x7fc9ec14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ec15c770 0x7fc9ec15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec072f20 con 0x7fc9ec0d1200 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec072c40 con 0x7fc9ec15c770 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ec15cd90 con 0x7fc9ec06e5d0 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33460/0 (socket says 172.21.15.99:33460) 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 -- 172.21.15.99:0/1523186617 learned_addr learned my addr 172.21.15.99:0/1523186617 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ead9d640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec06e5d0 0x7fc9ec14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9eb59e640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ec15c770 0x7fc9ec15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 -- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec06e5d0 msgr2=0x7fc9ec14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec06e5d0 0x7fc9ec14c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 -- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ec15c770 msgr2=0x7fc9ec15b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ec15c770 0x7fc9ec15b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 -- 172.21.15.99:0/1523186617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec1558c0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9ea59c640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c440 tx=0x7fc9e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00052d0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e0005470 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec155b60 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0007260 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.658+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec156120 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.659+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec073440 con 0x7fc9ec0d1200 2024-09-10T11:32:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.660+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9e00074c0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.660+0000 7fc9d37fe640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.660+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7fc9e008a1b0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.660+0000 7fc9ead9d640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.661+0000 7fc9ead9d640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006fd0 tx=0x7fc9dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.665+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc9e00585a0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.866+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9ec0b42d0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.866+0000 7fc9d37fe640 1 -- 172.21.15.99:0/1523186617 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7fc9e005c8a0 con 0x7fc9ec0d1200 2024-09-10T11:32:10.869 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:10.869 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:10.869 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 msgr2=0x7fc9d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006fd0 tx=0x7fc9dc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 msgr2=0x7fc9ec1f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000c440 tx=0x7fc9e000c900 comp rx=0 tx=0).stop 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 shutdown_connections 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9ec15c770 0x7fc9ec15b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9ec0d1200 0x7fc9ec1f25f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 --2- 172.21.15.99:0/1523186617 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9ec06e5d0 0x7fc9ec14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.868+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 >> 172.21.15.99:0/1523186617 conn(0x7fc9ec06bc10 msgr2=0x7fc9ec0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.869+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 shutdown_connections 2024-09-10T11:32:10.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.869+0000 7fc9f1053640 1 -- 172.21.15.99:0/1523186617 wait complete. 2024-09-10T11:32:10.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:10.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.979+0000 7fb0e1e74640 1 Processor -- start 2024-09-10T11:32:10.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.979+0000 7fb0e1e74640 1 -- start start 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0e1e74640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb0dc15b2d0 0x7fb0dc0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0e1e74640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc0b0020 con 0x7fb0dc15aef0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0e1e74640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0dc06ee10 con 0x7fb0dc0afbb0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0e1e74640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb0dc0cd9a0 con 0x7fb0dc15b2d0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0db7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb0dc15aef0 0x7fb0dc0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51956/0 (socket says 172.21.15.99:51956) 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0db7fe640 1 -- 172.21.15.99:0/1603028551 learned_addr learned my addr 172.21.15.99:0/1603028551 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0daffd640 1 --2- 172.21.15.99:0/1603028551 >> v2:172.21.15.196:3300/0 conn(0x7fb0dc15b2d0 0x7fb0dc0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585619547 0 0) 0x7fb0dc0b0020 con 0x7fb0dc15aef0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0c4005730 con 0x7fb0dc15aef0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 302557023 0 0) 0x7fb0dc06ee10 con 0x7fb0dc0afbb0 2024-09-10T11:32:10.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0dc0b0020 con 0x7fb0dc0afbb0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3693982617 0 0) 0x7fb0c4005730 con 0x7fb0dc15aef0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 >> v2:172.21.15.196:3300/0 conn(0x7fb0dc15b2d0 msgr2=0x7fb0dc0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 --2- 172.21.15.99:0/1603028551 >> v2:172.21.15.196:3300/0 conn(0x7fb0dc15b2d0 0x7fb0dc0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 >> v1:172.21.15.196:6789/0 conn(0x7fb0dc0afbb0 legacy=0x7fb0dc0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc0b0290 con 0x7fb0dc15aef0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.980+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fb0cc001670 con 0x7fb0dc15aef0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 >> v1:172.21.15.99:6789/0 conn(0x7fb0dc15aef0 legacy=0x7fb0dc0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 0x7fb0c40061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0c4006a10 0x7fb0c4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c40073d0 0x7fb0c400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc0cd9a0 con 0x7fb0c4005da0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c400e2a0 con 0x7fb0c4006a10 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c400e4a0 con 0x7fb0c40073d0 2024-09-10T11:32:10.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 0x7fb0c40061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0daffd640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0c4006a10 0x7fb0c4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0dbfff640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c40073d0 0x7fb0c400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 -- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c40073d0 msgr2=0x7fb0c400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c40073d0 0x7fb0c400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 -- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0c4006a10 msgr2=0x7fb0c4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0c4006a10 0x7fb0c4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0db7fe640 1 -- 172.21.15.99:0/1603028551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c4005730 con 0x7fb0c4005da0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0dbfff640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c40073d0 0x7fb0c400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.981+0000 7fb0dbfff640 1 -- 172.21.15.99:0/1603028551 reap_dead start 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 0x7fb0c40061a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c0e0 tx=0x7fb0c4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb568240ebabb724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc007c40 con 0x7fb0c4005da0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0cc005ad0 con 0x7fb0c4005da0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0da7fc640 1 -- 172.21.15.99:0/1603028551 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc007070 con 0x7fb0c4005da0 2024-09-10T11:32:10.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 -- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 msgr2=0x7fb0c40061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 0x7fb0c40061a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00c0e0 tx=0x7fb0c4007a40 comp rx=0 tx=0).stop 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 -- 172.21.15.99:0/1603028551 shutdown_connections 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/1603028551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0c4005da0 0x7fb0c40061a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 -- 172.21.15.99:0/1603028551 >> 172.21.15.99:0/1603028551 conn(0x7fb0dc06bc10 msgr2=0x7fb0dc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 -- 172.21.15.99:0/1603028551 shutdown_connections 2024-09-10T11:32:10.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.982+0000 7fb0e1e74640 1 -- 172.21.15.99:0/1603028551 wait complete. 2024-09-10T11:32:10.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.984+0000 7fb0e1e74640 1 Processor -- start 2024-09-10T11:32:10.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.984+0000 7fb0e1e74640 1 -- start start 2024-09-10T11:32:10.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c4005da0 0x7fb0dc0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0dc15d430 0x7fb0dc1f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc06eb30 con 0x7fb0dc1567d0 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc06e850 con 0x7fb0dc15d430 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0e1e74640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0dc0b4510 con 0x7fb0c4005da0 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33478/0 (socket says 172.21.15.99:33478) 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 -- 172.21.15.99:0/2598753797 learned_addr learned my addr 172.21.15.99:0/2598753797 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0dbfff640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0dc15d430 0x7fb0dc1f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c4005da0 0x7fb0dc0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 -- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c4005da0 msgr2=0x7fb0dc0aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c4005da0 0x7fb0dc0aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 -- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0dc15d430 msgr2=0x7fb0dc1f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0dc15d430 0x7fb0dc1f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 -- 172.21.15.99:0/2598753797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc1f4b50 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.985+0000 7fb0daffd640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0011a70 tx=0x7fb0d0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.986+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000fb50 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.986+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0d000fcf0 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.986+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc1f8f10 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.986+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000fea0 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.986+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1f50f0 con 0x7fb0dc1567d0 2024-09-10T11:32:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.987+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc0b48a0 con 0x7fb0dc1567d0 2024-09-10T11:32:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.987+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0d0012040 con 0x7fb0dc1567d0 2024-09-10T11:32:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.988+0000 7fb0c3fff640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 0x7fb0c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.988+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 0x7fb0c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.988+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7fb0d008b500 con 0x7fb0dc1567d0 2024-09-10T11:32:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.988+0000 7fb0db7fe640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 0x7fb0c806e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc032000 tx=0x7fb0cc02d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:10.992+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0d0019030 con 0x7fb0dc1567d0 2024-09-10T11:32:11.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.193+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0dc15a710 con 0x7fb0dc1567d0 2024-09-10T11:32:11.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.193+0000 7fb0c3fff640 1 -- 172.21.15.99:0/2598753797 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+5229 (secure 0 0 0) 0x7fb0d0059860 con 0x7fb0dc1567d0 2024-09-10T11:32:11.195 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:11.195 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":21,"btime":"2024-09-10T11:31:41:658183+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:31:38.982189+0000","modified":"2024-09-10T11:31:40.985291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5211":{"gid":5211,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":2,"addr":"172.21.15.99:6836/3824950902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3824950902},{"type":"v1","addr":"172.21.15.99:6836","nonce":3824950902}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5211,"qdb_cluster":[5211]},"id":3}]} 2024-09-10T11:32:11.196 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 21 2024-09-10T11:32:11.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.195+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 msgr2=0x7fb0c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.195+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 0x7fb0c806e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc032000 tx=0x7fb0cc02d000 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.195+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 msgr2=0x7fb0dc1f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.195+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0011a70 tx=0x7fb0d0011f30 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 shutdown_connections 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb0c806be30 0x7fb0c806e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0dc15d430 0x7fb0dc1f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0dc1567d0 0x7fb0dc1f1bc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 --2- 172.21.15.99:0/2598753797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0c4005da0 0x7fb0dc0aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 >> 172.21.15.99:0/2598753797 conn(0x7fb0dc06bc10 msgr2=0x7fb0dc157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 shutdown_connections 2024-09-10T11:32:11.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.196+0000 7fb0e1e74640 1 -- 172.21.15.99:0/2598753797 wait complete. 2024-09-10T11:32:11.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:11.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.305+0000 7f7ee0781640 1 Processor -- start 2024-09-10T11:32:11.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.305+0000 7f7ee0781640 1 -- start start 2024-09-10T11:32:11.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ee0781640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed81609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ee0781640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ed815f210 0x7f7ed8160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ee0781640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed80d1bd0 con 0x7f7ed80ae020 2024-09-10T11:32:11.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ee0781640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed8075620 con 0x7f7ed815f210 2024-09-10T11:32:11.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ee0781640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed80752b0 con 0x7f7ed81605c0 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7ede51d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed81609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eded1e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ed815f210 0x7f7ed8160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eddd1c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7ed80ae020 0x7f7ed815cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:47484/0 (socket says 172.21.15.99:47484) 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eddd1c640 1 -- 172.21.15.99:0/3132017892 learned_addr learned my addr 172.21.15.99:0/3132017892 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eded1e640 1 -- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 msgr2=0x7f7ed81609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eded1e640 1 --2- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed81609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eded1e640 1 -- 172.21.15.99:0/3132017892 >> v1:172.21.15.196:6789/0 conn(0x7f7ed80ae020 legacy=0x7f7ed815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7eded1e640 1 -- 172.21.15.99:0/3132017892 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80d1eb0 con 0x7f7ed815f210 2024-09-10T11:32:11.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.306+0000 7f7edd51b640 1 -- 172.21.15.99:0/3132017892 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410329327 0 0) 0x7f7ed80d1bd0 con 0x7f7ed80ae020 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.307+0000 7f7eded1e640 1 --2- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed815f210 0x7f7ed8160ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009d90 tx=0x7f7ed002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48b71cad389c2231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.307+0000 7f7edd51b640 1 -- 172.21.15.99:0/3132017892 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007740 con 0x7f7ed815f210 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.307+0000 7f7edd51b640 1 -- 172.21.15.99:0/3132017892 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed00078e0 con 0x7f7ed815f210 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.307+0000 7f7edd51b640 1 -- 172.21.15.99:0/3132017892 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007cb0 con 0x7f7ed815f210 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 -- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed815f210 msgr2=0x7f7ed8160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 --2- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed815f210 0x7f7ed8160ee0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009d90 tx=0x7f7ed002a660 comp rx=0 tx=0).stop 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 -- 172.21.15.99:0/3132017892 shutdown_connections 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 --2- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed815f210 0x7f7ed8160ee0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 --2- 172.21.15.99:0/3132017892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed81609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 -- 172.21.15.99:0/3132017892 >> 172.21.15.99:0/3132017892 conn(0x7f7ed80730c0 msgr2=0x7f7ed80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 -- 172.21.15.99:0/3132017892 shutdown_connections 2024-09-10T11:32:11.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.308+0000 7f7ee0781640 1 -- 172.21.15.99:0/3132017892 wait complete. 2024-09-10T11:32:11.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.309+0000 7f7ee0781640 1 Processor -- start 2024-09-10T11:32:11.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.309+0000 7f7ee0781640 1 -- start start 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed80ae020 0x7f7ed81f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8073d00 con 0x7f7ed815f210 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed815a9e0 con 0x7f7ed80ae020 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ee0781640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed8073e80 con 0x7f7ed81605c0 2024-09-10T11:32:11.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33482/0 (socket says 172.21.15.99:33482) 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 -- 172.21.15.99:0/2444808817 learned_addr learned my addr 172.21.15.99:0/2444808817 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7ede51d640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed80ae020 0x7f7ed81f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eded1e640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 -- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 msgr2=0x7f7ed814b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed814b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 -- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed80ae020 msgr2=0x7f7ed81f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed80ae020 0x7f7ed81f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 -- 172.21.15.99:0/2444808817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8158410 con 0x7f7ed815f210 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eddd1c640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c410 tx=0x7f7ec800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80074e0 con 0x7f7ed815f210 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec80047e0 con 0x7f7ed815f210 2024-09-10T11:32:11.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.310+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8004bd0 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.311+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed80b4cf0 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.311+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed80b52b0 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.312+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed80b54b0 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.312+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ec8011070 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.312+0000 7f7eceffd640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 0x7f7ebc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.313+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec808a8f0 con 0x7f7ed815f210 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.313+0000 7f7ede51d640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 0x7f7ebc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.313+0000 7f7ede51d640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 0x7f7ebc06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009aa0 tx=0x7f7ed4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.317+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ed80b54b0 con 0x7f7ed815f210 2024-09-10T11:32:11.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.481+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ed81609a0 con 0x7f7ed815f210 2024-09-10T11:32:11.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.482+0000 7f7eceffd640 1 -- 172.21.15.99:0/2444808817 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16669 (secure 0 0 0) 0x7f7ec8058ce0 con 0x7f7ed815f210 2024-09-10T11:32:11.484 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:11.484 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":54,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:06.798356+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:31:37.584024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T11:31:37.949922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 msgr2=0x7f7ebc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 0x7f7ebc06e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009aa0 tx=0x7f7ed4009190 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 msgr2=0x7f7ed80b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800c410 tx=0x7f7ec800c8d0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 shutdown_connections 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7ebc06be30 0x7f7ebc06e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed81605c0 0x7f7ed814b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7ed815f210 0x7f7ed80b57d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 --2- 172.21.15.99:0/2444808817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed80ae020 0x7f7ed81f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 >> 172.21.15.99:0/2444808817 conn(0x7f7ed80730c0 msgr2=0x7f7ed815d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 shutdown_connections 2024-09-10T11:32:11.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.484+0000 7f7ee0781640 1 -- 172.21.15.99:0/2444808817 wait complete. 2024-09-10T11:32:11.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:32:11.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.595+0000 7f94c4eba640 1 Processor -- start 2024-09-10T11:32:11.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.595+0000 7f94c4eba640 1 -- start start 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.595+0000 7f94c4eba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94c015ee00 0x7f94c00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94c4eba640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c00b40e0 con 0x7f94c015f1e0 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94c4eba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c00b5ab0 con 0x7f94c00b0090 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94c4eba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f94c0073c50 con 0x7f94c015ee00 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bdd9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f94c015f1e0 0x7f94c00b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51966/0 (socket says 172.21.15.99:51966) 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bdd9b640 1 -- 172.21.15.99:0/591277046 learned_addr learned my addr 172.21.15.99:0/591277046 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:11.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94be59c640 1 --2- 172.21.15.99:0/591277046 >> v2:172.21.15.196:3300/0 conn(0x7f94c015ee00 0x7f94c00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280315750 0 0) 0x7f94c00b40e0 con 0x7f94c015f1e0 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94a8003520 con 0x7f94c015f1e0 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462789625 0 0) 0x7f94c00b5ab0 con 0x7f94c00b0090 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94c00b40e0 con 0x7f94c00b0090 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94be59c640 1 -- 172.21.15.99:0/591277046 >> v1:172.21.15.196:6789/0 conn(0x7f94c00b0090 legacy=0x7f94c00b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94be59c640 1 -- 172.21.15.99:0/591277046 >> v1:172.21.15.99:6789/0 conn(0x7f94c015f1e0 legacy=0x7f94c00b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.596+0000 7f94be59c640 1 -- 172.21.15.99:0/591277046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c00b4380 con 0x7f94c015ee00 2024-09-10T11:32:11.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.597+0000 7f94be59c640 1 --2- 172.21.15.99:0/591277046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c015ee00 0x7f94c00b5f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009d90 tx=0x7f94b002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=382cc897279c180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.597+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0007740 con 0x7f94c015ee00 2024-09-10T11:32:11.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.597+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b00078e0 con 0x7f94c015ee00 2024-09-10T11:32:11.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.597+0000 7f94bd59a640 1 -- 172.21.15.99:0/591277046 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0007c90 con 0x7f94c015ee00 2024-09-10T11:32:11.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 -- 172.21.15.99:0/591277046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c015ee00 msgr2=0x7f94c00b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 --2- 172.21.15.99:0/591277046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c015ee00 0x7f94c00b5f90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009d90 tx=0x7f94b002a6f0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 -- 172.21.15.99:0/591277046 shutdown_connections 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 --2- 172.21.15.99:0/591277046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c015ee00 0x7f94c00b5f90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 -- 172.21.15.99:0/591277046 >> 172.21.15.99:0/591277046 conn(0x7f94c00730c0 msgr2=0x7f94c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 -- 172.21.15.99:0/591277046 shutdown_connections 2024-09-10T11:32:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.598+0000 7f94c4eba640 1 -- 172.21.15.99:0/591277046 wait complete. 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.599+0000 7f94c4eba640 1 Processor -- start 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.599+0000 7f94c4eba640 1 -- start start 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c00b0090 0x7f94c0159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 0x7f94c0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00b0600 con 0x7f94c015ee00 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00afd60 con 0x7f94c00b0090 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94c4eba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f94c00b0780 con 0x7f94c015f1e0 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33488/0 (socket says 172.21.15.99:33488) 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 -- 172.21.15.99:0/2825499103 learned_addr learned my addr 172.21.15.99:0/2825499103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94be59c640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c00b0090 0x7f94c0159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bed9d640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 0x7f94c0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 -- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 msgr2=0x7f94c0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 0x7f94c0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 -- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c00b0090 msgr2=0x7f94c0159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c00b0090 0x7f94c0159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 -- 172.21.15.99:0/2825499103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c00cb8a0 con 0x7f94c015ee00 2024-09-10T11:32:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bed9d640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 0x7f94c0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94bdd9b640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c410 tx=0x7f94b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4017070 con 0x7f94c015ee00 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.600+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b4005ce0 con 0x7f94c015ee00 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.601+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c00cbb80 con 0x7f94c015ee00 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.601+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b40072d0 con 0x7f94c015ee00 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.601+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c00cc140 con 0x7f94c015ee00 2024-09-10T11:32:11.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.602+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c00b5f90 con 0x7f94c015ee00 2024-09-10T11:32:11.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.602+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f94b401a070 con 0x7f94c015ee00 2024-09-10T11:32:11.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.603+0000 7f94aeffd640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 0x7f94a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.603+0000 7f94be59c640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 0x7f94a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.603+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5709+0+0 (secure 0 0 0) 0x7f94b408a1c0 con 0x7f94c015ee00 2024-09-10T11:32:11.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.603+0000 7f94be59c640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 0x7f94a006e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94b0034000 tx=0x7f94b002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.607+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94b40584a0 con 0x7f94c015ee00 2024-09-10T11:32:11.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.812+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f94c00b1a20 con 0x7f94c015ee00 2024-09-10T11:32:11.814 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:32:11.812+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:32:11.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.814+0000 7f94aeffd640 1 -- 172.21.15.99:0/2825499103 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v22) ==== 161+0+0 (secure 0 0 0) 0x7f94b405c7a0 con 0x7f94c015ee00 2024-09-10T11:32:11.815 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 msgr2=0x7f94a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 0x7f94a006e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f94b0034000 tx=0x7f94b002f000 comp rx=0 tx=0).stop 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 msgr2=0x7f94c00cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c410 tx=0x7f94b400c8d0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 shutdown_connections 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f94a006be30 0x7f94a006e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94c015f1e0 0x7f94c0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94c015ee00 0x7f94c00cc940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 --2- 172.21.15.99:0/2825499103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94c00b0090 0x7f94c0159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 >> 172.21.15.99:0/2825499103 conn(0x7f94c00730c0 msgr2=0x7f94c00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 shutdown_connections 2024-09-10T11:32:11.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.818+0000 7f94c4eba640 1 -- 172.21.15.99:0/2825499103 wait complete. 2024-09-10T11:32:11.821 INFO:tasks.ceph.mds.a.smithi099.stderr: -9473> 2024-09-10T11:31:41.619+0000 7fee99eab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:32:11.833 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:32:11.855 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:32:11.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.933+0000 7f7ddc39f640 1 Processor -- start 2024-09-10T11:32:11.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.933+0000 7f7ddc39f640 1 -- start start 2024-09-10T11:32:11.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7ddc39f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415ee00 0x7f7dd40b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7ddc39f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7dd415f1e0 0x7f7dd40b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7ddc39f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dd40b40e0 con 0x7f7dd40b0090 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7ddc39f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd40b5ab0 con 0x7f7dd415f1e0 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7ddc39f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd4073c50 con 0x7f7dd415ee00 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7dd415f1e0 0x7f7dd40b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dda93c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7dd40b0090 0x7f7dd40b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51974/0 (socket says 172.21.15.99:51974) 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7dd415f1e0 0x7f7dd40b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33518/0 (socket says 172.21.15.99:33518) 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dda93c640 1 -- 172.21.15.99:0/870766642 learned_addr learned my addr 172.21.15.99:0/870766642 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dda13b640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415ee00 0x7f7dd40b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 -- 172.21.15.99:0/870766642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415ee00 msgr2=0x7f7dd40b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415ee00 0x7f7dd40b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 -- 172.21.15.99:0/870766642 >> v1:172.21.15.99:6789/0 conn(0x7f7dd40b0090 legacy=0x7f7dd40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd993a640 1 -- 172.21.15.99:0/870766642 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd40b4380 con 0x7f7dd415f1e0 2024-09-10T11:32:11.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.934+0000 7f7dd9139640 1 -- 172.21.15.99:0/870766642 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558117827 0 0) 0x7f7dd40b40e0 con 0x7f7dd40b0090 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.935+0000 7f7dd993a640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd415f1e0 0x7f7dd40b1560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097a0 tx=0x7f7dc402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d12f4765abb4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.935+0000 7f7dd9139640 1 -- 172.21.15.99:0/870766642 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4037070 con 0x7f7dd415f1e0 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.935+0000 7f7dd9139640 1 -- 172.21.15.99:0/870766642 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc4032440 con 0x7f7dd415f1e0 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.935+0000 7f7dd9139640 1 -- 172.21.15.99:0/870766642 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4032810 con 0x7f7dd415f1e0 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 -- 172.21.15.99:0/870766642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd415f1e0 msgr2=0x7f7dd40b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd415f1e0 0x7f7dd40b1560 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40097a0 tx=0x7f7dc402a180 comp rx=0 tx=0).stop 2024-09-10T11:32:11.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 -- 172.21.15.99:0/870766642 shutdown_connections 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd415f1e0 0x7f7dd40b1560 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/870766642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415ee00 0x7f7dd40b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 -- 172.21.15.99:0/870766642 >> 172.21.15.99:0/870766642 conn(0x7f7dd40730c0 msgr2=0x7f7dd40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 -- 172.21.15.99:0/870766642 shutdown_connections 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.936+0000 7f7ddc39f640 1 -- 172.21.15.99:0/870766642 wait complete. 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.937+0000 7f7ddc39f640 1 Processor -- start 2024-09-10T11:32:11.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.937+0000 7f7ddc39f640 1 -- start start 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dd415ee00 0x7f7dd409ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 0x7f7dd4161b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40b21b0 con 0x7f7dd40b0090 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40afd60 con 0x7f7dd415ee00 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7ddc39f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd40b0600 con 0x7f7dd415f1e0 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33522/0 (socket says 172.21.15.99:33522) 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 -- 172.21.15.99:0/1232411897 learned_addr learned my addr 172.21.15.99:0/1232411897 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dd993a640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dd415ee00 0x7f7dd409ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda93c640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 0x7f7dd4161b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 -- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 msgr2=0x7f7dd4161b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 0x7f7dd4161b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 -- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dd415ee00 msgr2=0x7f7dd409ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dd415ee00 0x7f7dd409ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 -- 172.21.15.99:0/1232411897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd409bbb0 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda93c640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 0x7f7dd4161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dda13b640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d800 tx=0x7f7dd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000b6c0 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.938+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd0019070 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.939+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000ba80 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.939+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd409be90 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.939+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd415f780 con 0x7f7dd40b0090 2024-09-10T11:32:11.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.939+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd409c3b0 con 0x7f7dd40b0090 2024-09-10T11:32:11.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.940+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7dd000bc20 con 0x7f7dd40b0090 2024-09-10T11:32:11.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.940+0000 7f7dcaffd640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 0x7f7db806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:11.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.940+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5795+0+0 (secure 0 0 0) 0x7f7dd008b270 con 0x7f7dd40b0090 2024-09-10T11:32:11.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.940+0000 7f7dd993a640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 0x7f7db806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:11.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.941+0000 7f7dd993a640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 0x7f7db806e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4034000 tx=0x7f7dc402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:11.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:11.945+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7dd0059c50 con 0x7f7dd40b0090 2024-09-10T11:32:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.141+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7dd40b5f90 con 0x7f7dd40b0090 2024-09-10T11:32:12.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.836+0000 7f7dcaffd640 1 -- 172.21.15.99:0/1232411897 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f7dd0059550 con 0x7f7dd40b0090 2024-09-10T11:32:12.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 msgr2=0x7f7db806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:12.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 0x7f7db806e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4034000 tx=0x7f7dc402e000 comp rx=0 tx=0).stop 2024-09-10T11:32:12.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 msgr2=0x7f7dd41516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:12.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000d800 tx=0x7f7dd000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 shutdown_connections 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7db806be30 0x7f7db806e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7dd415f1e0 0x7f7dd4161b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7dd415ee00 0x7f7dd409ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 --2- 172.21.15.99:0/1232411897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7dd40b0090 0x7f7dd41516c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 >> 172.21.15.99:0/1232411897 conn(0x7f7dd40730c0 msgr2=0x7f7dd40b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 shutdown_connections 2024-09-10T11:32:12.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:12.839+0000 7f7ddc39f640 1 -- 172.21.15.99:0/1232411897 wait complete. 2024-09-10T11:32:12.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:32:13.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.003+0000 7f5dd39b7640 1 Processor -- start 2024-09-10T11:32:13.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.003+0000 7f5dd39b7640 1 -- start start 2024-09-10T11:32:13.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd39b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5dcc15f1e0 0x7f5dcc0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd39b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dcc0b40e0 con 0x7f5dcc0b0090 2024-09-10T11:32:13.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd39b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dcc0b5ab0 con 0x7f5dcc15ee00 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd39b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5dcc073c50 con 0x7f5dcc15f1e0 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd1f54640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5dcc0b0090 0x7f5dcc0b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44016/0 (socket says 172.21.15.99:44016) 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd1f54640 1 -- 172.21.15.99:0/1796763187 learned_addr learned my addr 172.21.15.99:0/1796763187 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322749689 0 0) 0x7f5dcc0b40e0 con 0x7f5dcc0b0090 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5db8003520 con 0x7f5dcc0b0090 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/1796763187 >> v2:172.21.15.196:3300/0 conn(0x7f5dcc15f1e0 0x7f5dcc0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3370950485 0 0) 0x7f5db8003520 con 0x7f5dcc0b0090 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 >> v2:172.21.15.196:3300/0 conn(0x7f5dcc15f1e0 msgr2=0x7f5dcc0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 --2- 172.21.15.99:0/1796763187 >> v2:172.21.15.196:3300/0 conn(0x7f5dcc15f1e0 0x7f5dcc0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 >> v1:172.21.15.196:6789/0 conn(0x7f5dcc15ee00 legacy=0x7f5dcc0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc0b4380 con 0x7f5dcc0b0090 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5dc40019f0 con 0x7f5dcc0b0090 2024-09-10T11:32:13.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 >> v1:172.21.15.99:6789/0 conn(0x7f5dcc0b0090 legacy=0x7f5dcc0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5db80039b0 0x7f5db8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5db8004800 0x7f5db8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5db800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.004+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc073c50 con 0x7f5db80051c0 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5db800c060 con 0x7f5db8004800 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5db800c260 con 0x7f5db80039b0 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5db800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5db8004800 0x7f5db8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1753640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5db80039b0 0x7f5db8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 -- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5db80039b0 msgr2=0x7f5db8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5db80039b0 0x7f5db8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 -- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5db8004800 msgr2=0x7f5db8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5db8004800 0x7f5db8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 -- 172.21.15.99:0/1796763187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db8003520 con 0x7f5db80051c0 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.005+0000 7f5dd1f54640 1 -- 172.21.15.99:0/1796763187 reap_dead start 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5db800b9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400a850 tx=0x7f5db8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c45f5beb8fa8756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400a4a0 con 0x7f5db80051c0 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc402f070 con 0x7f5db80051c0 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dc3fff640 1 -- 172.21.15.99:0/1796763187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc402bce0 con 0x7f5db80051c0 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 -- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 msgr2=0x7f5db800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5db800b9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400a850 tx=0x7f5db8005830 comp rx=0 tx=0).stop 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 -- 172.21.15.99:0/1796763187 shutdown_connections 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/1796763187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5db800b9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 -- 172.21.15.99:0/1796763187 >> 172.21.15.99:0/1796763187 conn(0x7f5dcc0730c0 msgr2=0x7f5dcc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:13.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 -- 172.21.15.99:0/1796763187 shutdown_connections 2024-09-10T11:32:13.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.006+0000 7f5dd39b7640 1 -- 172.21.15.99:0/1796763187 wait complete. 2024-09-10T11:32:13.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.007+0000 7f5dd39b7640 1 Processor -- start 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 -- start start 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 0x7f5dcc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 0x7f5dcc0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc0b0600 con 0x7f5db80051c0 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dcc0afd60 con 0x7f5dcc156e00 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd39b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5dcc0b0780 con 0x7f5dcc0ca900 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd1753640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd1753640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57856/0 (socket says 172.21.15.99:57856) 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd1753640 1 -- 172.21.15.99:0/3193801904 learned_addr learned my addr 172.21.15.99:0/3193801904 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 0x7f5dcc0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.008+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 0x7f5dcc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 -- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 msgr2=0x7f5dcc0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 0x7f5dcc0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 -- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 msgr2=0x7f5dcc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 0x7f5dcc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 -- 172.21.15.99:0/3193801904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dcc157ce0 con 0x7f5db80051c0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1f54640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 0x7f5dcc0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 0x7f5dcc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd1753640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800f620 tx=0x7f5dc800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80052e0 con 0x7f5db80051c0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc8018070 con 0x7f5db80051c0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dcc157f70 con 0x7f5db80051c0 2024-09-10T11:32:13.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8003810 con 0x7f5db80051c0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.009+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dcc158530 con 0x7f5db80051c0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.010+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dcc0b5f90 con 0x7f5db80051c0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.010+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5dc80039b0 con 0x7f5db80051c0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.011+0000 7f5dc1ffb640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 0x7f5db006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.011+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5795+0+0 (secure 0 0 0) 0x7f5dc808aba0 con 0x7f5db80051c0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.011+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 0x7f5db006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.011+0000 7f5dd0f52640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 0x7f5db006e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0d1c70 tx=0x7f5dbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.015+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5dc80594f0 con 0x7f5db80051c0 2024-09-10T11:32:13.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.178+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5dcc15dac0 con 0x7f5db80051c0 2024-09-10T11:32:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.845+0000 7f5dc1ffb640 1 -- 172.21.15.99:0/3193801904 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) ==== 181+0+0 (secure 0 0 0) 0x7f5dc8058df0 con 0x7f5db80051c0 2024-09-10T11:32:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:32:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.847+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 msgr2=0x7f5db006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.847+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 0x7f5db006e320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0d1c70 tx=0x7f5dbc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.847+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 msgr2=0x7f5dcc147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.847+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800f620 tx=0x7f5dc800fae0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 shutdown_connections 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5db006be60 0x7f5db006e320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5dcc0ca900 0x7f5dcc0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dcc156e00 0x7f5dcc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 --2- 172.21.15.99:0/3193801904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5db80051c0 0x7f5dcc147500 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 >> 172.21.15.99:0/3193801904 conn(0x7f5dcc0730c0 msgr2=0x7f5dcc0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 shutdown_connections 2024-09-10T11:32:13.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.848+0000 7f5dd39b7640 1 -- 172.21.15.99:0/3193801904 wait complete. 2024-09-10T11:32:13.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:32:13.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.966+0000 7f4f78331640 1 Processor -- start 2024-09-10T11:32:13.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.966+0000 7f4f78331640 1 -- start start 2024-09-10T11:32:13.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f78331640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f7015cb70 0x7f4f70072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f78331640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f700b4150 0x7f4f700b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f78331640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f70072650 con 0x7f4f700b0df0 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f78331640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4f700b1240 con 0x7f4f7015cb70 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f78331640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4f7015db70 con 0x7f4f700b4150 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f7015cb70 0x7f4f70072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4f7015cb70 0x7f4f70072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57862/0 (socket says 172.21.15.99:57862) 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 -- 172.21.15.99:0/2504888116 learned_addr learned my addr 172.21.15.99:0/2504888116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f758cc640 1 --2- 172.21.15.99:0/2504888116 >> v2:172.21.15.196:3300/0 conn(0x7f4f700b4150 0x7f4f700b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 -- 172.21.15.99:0/2504888116 >> v2:172.21.15.196:3300/0 conn(0x7f4f700b4150 msgr2=0x7f4f700b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 --2- 172.21.15.99:0/2504888116 >> v2:172.21.15.196:3300/0 conn(0x7f4f700b4150 0x7f4f700b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 -- 172.21.15.99:0/2504888116 >> v1:172.21.15.196:6789/0 conn(0x7f4f700b0df0 legacy=0x7f4f7015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.967+0000 7f4f760cd640 1 -- 172.21.15.99:0/2504888116 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7015de50 con 0x7f4f7015cb70 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.968+0000 7f4f760cd640 1 --2- 172.21.15.99:0/2504888116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70072de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f7006e080 tx=0x7f4f6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5305d827fd4e4f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.968+0000 7f4f750cb640 1 -- 172.21.15.99:0/2504888116 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c037070 con 0x7f4f7015cb70 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.968+0000 7f4f750cb640 1 -- 172.21.15.99:0/2504888116 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c032440 con 0x7f4f7015cb70 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.968+0000 7f4f750cb640 1 -- 172.21.15.99:0/2504888116 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c032810 con 0x7f4f7015cb70 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 -- 172.21.15.99:0/2504888116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 msgr2=0x7f4f70072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 --2- 172.21.15.99:0/2504888116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70072de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f7006e080 tx=0x7f4f6c02a230 comp rx=0 tx=0).stop 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 -- 172.21.15.99:0/2504888116 shutdown_connections 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 --2- 172.21.15.99:0/2504888116 >> v2:172.21.15.196:3300/0 conn(0x7f4f700b4150 0x7f4f700b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 --2- 172.21.15.99:0/2504888116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70072de0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 -- 172.21.15.99:0/2504888116 >> 172.21.15.99:0/2504888116 conn(0x7f4f700b3630 msgr2=0x7f4f700b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 -- 172.21.15.99:0/2504888116 shutdown_connections 2024-09-10T11:32:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.969+0000 7f4f78331640 1 -- 172.21.15.99:0/2504888116 wait complete. 2024-09-10T11:32:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.970+0000 7f4f78331640 1 Processor -- start 2024-09-10T11:32:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.970+0000 7f4f78331640 1 -- start start 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f700b0df0 0x7f4f701605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 0x7f4f70145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f70073320 con 0x7f4f7015cb70 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015e130 con 0x7f4f700b4150 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f78331640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700734a0 con 0x7f4f700b0df0 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57872/0 (socket says 172.21.15.99:57872) 2024-09-10T11:32:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 -- 172.21.15.99:0/3475938366 learned_addr learned my addr 172.21.15.99:0/3475938366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 -- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f700b0df0 msgr2=0x7f4f701605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f700b0df0 0x7f4f701605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 -- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 msgr2=0x7f4f70145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f758cc640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 0x7f4f70145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 0x7f4f70145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f768ce640 1 -- 172.21.15.99:0/3475938366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700afec0 con 0x7f4f7015cb70 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.971+0000 7f4f758cc640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 0x7f4f70145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f768ce640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c780 tx=0x7f4f6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68004b60 con 0x7f4f7015cb70 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f68004d00 con 0x7f4f7015cb70 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800aa90 con 0x7f4f7015cb70 2024-09-10T11:32:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f700b4ad0 con 0x7f4f7015cb70 2024-09-10T11:32:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.972+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f700b5370 con 0x7f4f7015cb70 2024-09-10T11:32:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.973+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f700b5570 con 0x7f4f7015cb70 2024-09-10T11:32:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.973+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f6800ac30 con 0x7f4f7015cb70 2024-09-10T11:32:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.974+0000 7f4f66ffd640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 0x7f4f5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.974+0000 7f4f760cd640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 0x7f4f5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:13.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.974+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5357+0+0 (secure 0 0 0) 0x7f4f6808afa0 con 0x7f4f7015cb70 2024-09-10T11:32:13.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.974+0000 7f4f760cd640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 0x7f4f5406e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:13.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:13.978+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f700b5570 con 0x7f4f7015cb70 2024-09-10T11:32:14.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.144+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4f700b4ca0 con 0x7f4f7015cb70 2024-09-10T11:32:14.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.853+0000 7f4f66ffd640 1 -- 172.21.15.99:0/3475938366 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) ==== 169+0+0 (secure 0 0 0) 0x7f4f68020070 con 0x7f4f7015cb70 2024-09-10T11:32:14.854 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 msgr2=0x7f4f5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 0x7f4f5406e2f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c02c000 comp rx=0 tx=0).stop 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 msgr2=0x7f4f70146360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c780 tx=0x7f4f6800cc40 comp rx=0 tx=0).stop 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 shutdown_connections 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f5406be30 0x7f4f5406e2f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f7015cb70 0x7f4f70146360 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f700b4150 0x7f4f70145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 --2- 172.21.15.99:0/3475938366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f700b0df0 0x7f4f701605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 >> 172.21.15.99:0/3475938366 conn(0x7f4f700b3630 msgr2=0x7f4f7006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:14.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 shutdown_connections 2024-09-10T11:32:14.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.856+0000 7f4f78331640 1 -- 172.21.15.99:0/3475938366 wait complete. 2024-09-10T11:32:14.871 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:32:14.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.982+0000 7f7a9d6ba640 1 Processor -- start 2024-09-10T11:32:14.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 -- start start 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7a981626c0 0x7f7a980d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a980d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a9815ff70 con 0x7f7a9815eb80 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7a980d30c0 con 0x7f7a981626c0 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a9d6ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7a980d52d0 con 0x7f7a980d4e40 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a96ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7a9815eb80 0x7f7a980d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44018/0 (socket says 172.21.15.99:44018) 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.983+0000 7f7a96ffd640 1 -- 172.21.15.99:0/4110978234 learned_addr learned my addr 172.21.15.99:0/4110978234 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/4110978234 >> v2:172.21.15.196:3300/0 conn(0x7f7a981626c0 0x7f7a980d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a977fe640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a980d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286507061 0 0) 0x7f7a9815ff70 con 0x7f7a9815eb80 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a84003520 con 0x7f7a9815eb80 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2213234470 0 0) 0x7f7a84003520 con 0x7f7a9815eb80 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a8ffff640 1 -- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 msgr2=0x7f7a980d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a980d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a8ffff640 1 -- 172.21.15.99:0/4110978234 >> v1:172.21.15.99:6789/0 conn(0x7f7a9815eb80 legacy=0x7f7a980d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a8ffff640 1 -- 172.21.15.99:0/4110978234 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a980d1620 con 0x7f7a981626c0 2024-09-10T11:32:14.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.984+0000 7f7a977fe640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a980d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.985+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980d4900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a8002a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a741d3aa885a278a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.985+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037070 con 0x7f7a981626c0 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.985+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8002ab40 con 0x7f7a981626c0 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.985+0000 7f7a967fc640 1 -- 172.21.15.99:0/4110978234 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88001670 con 0x7f7a981626c0 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 msgr2=0x7f7a980d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980d4900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097d0 tx=0x7f7a8002a0b0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/4110978234 shutdown_connections 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a980d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/4110978234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980d4900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/4110978234 >> 172.21.15.99:0/4110978234 conn(0x7f7a9808e680 msgr2=0x7f7a9809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/4110978234 shutdown_connections 2024-09-10T11:32:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.986+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/4110978234 wait complete. 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.987+0000 7f7a9d6ba640 1 Processor -- start 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.987+0000 7f7a9d6ba640 1 -- start start 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a9815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98163180 con 0x7f7a9815eb80 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98162ea0 con 0x7f7a981626c0 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a9d6ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7a98163460 con 0x7f7a980d4e40 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57888/0 (socket says 172.21.15.99:57888) 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 -- 172.21.15.99:0/1656832467 learned_addr learned my addr 172.21.15.99:0/1656832467 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a977fe640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a96ffd640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a9815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 -- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 msgr2=0x7f7a9815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a9815a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 -- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 msgr2=0x7f7a980b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 -- 172.21.15.99:0/1656832467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a980ba6a0 con 0x7f7a9815eb80 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a96ffd640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a9815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.988+0000 7f7a8ffff640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a80035040 tx=0x7f7a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.989+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037050 con 0x7f7a9815eb80 2024-09-10T11:32:14.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.989+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a80043070 con 0x7f7a9815eb80 2024-09-10T11:32:14.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.989+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9815a740 con 0x7f7a9815eb80 2024-09-10T11:32:14.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.989+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80006e10 con 0x7f7a9815eb80 2024-09-10T11:32:14.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.989+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a981f7f30 con 0x7f7a9815eb80 2024-09-10T11:32:14.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.990+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a980d4900 con 0x7f7a9815eb80 2024-09-10T11:32:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.990+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a8003ca50 con 0x7f7a9815eb80 2024-09-10T11:32:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.991+0000 7f7a8f7fe640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 0x7f7a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.991+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7f7a800ab680 con 0x7f7a9815eb80 2024-09-10T11:32:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.991+0000 7f7a96ffd640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 0x7f7a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:14.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.991+0000 7f7a96ffd640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 0x7f7a7806e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a88005880 tx=0x7f7a88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:14.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:14.995+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a980ba6a0 con 0x7f7a9815eb80 2024-09-10T11:32:15.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.158+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7a980d0980 con 0x7f7a9815eb80 2024-09-10T11:32:15.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.161+0000 7f7a8f7fe640 1 -- 172.21.15.99:0/1656832467 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) ==== 104+0+0 (secure 0 0 0) 0x7f7a8007a4e0 con 0x7f7a9815eb80 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 msgr2=0x7f7a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 0x7f7a7806e2f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7a88005880 tx=0x7f7a88009040 comp rx=0 tx=0).stop 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 msgr2=0x7f7a980b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a80035040 tx=0x7f7a800046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 shutdown_connections 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7a7806be30 0x7f7a7806e2f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981626c0 0x7f7a980b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a9815eb80 0x7f7a980b6270 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 --2- 172.21.15.99:0/1656832467 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a980d4e40 0x7f7a9815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 >> 172.21.15.99:0/1656832467 conn(0x7f7a9808e680 msgr2=0x7f7a9815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:15.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 shutdown_connections 2024-09-10T11:32:15.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.164+0000 7f7a9d6ba640 1 -- 172.21.15.99:0/1656832467 wait complete. 2024-09-10T11:32:15.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-09-10T11:32:15.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.273+0000 7f0fd7f42640 1 Processor -- start 2024-09-10T11:32:15.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.273+0000 7f0fd7f42640 1 -- start start 2024-09-10T11:32:15.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd7f42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 0x7f0fd00d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd7f42640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0fd006f000 0x7f0fd006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd7f42640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fd00ceb10 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd7f42640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd00b3840 con 0x7f0fd006f000 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd7f42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd00d13f0 con 0x7f0fd015c7b0 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0fd006f000 0x7f0fd006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd54dd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0fd00d0fb0 0x7f0fd006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44034/0 (socket says 172.21.15.99:44034) 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0fd006f000 0x7f0fd006d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57898/0 (socket says 172.21.15.99:57898) 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd54dd640 1 -- 172.21.15.99:0/3435235693 learned_addr learned my addr 172.21.15.99:0/3435235693 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 0x7f0fd00d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 -- 172.21.15.99:0/3435235693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 msgr2=0x7f0fd00d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 0x7f0fd00d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 -- 172.21.15.99:0/3435235693 >> v1:172.21.15.99:6789/0 conn(0x7f0fd00d0fb0 legacy=0x7f0fd006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd4cdc640 1 -- 172.21.15.99:0/3435235693 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840585517 0 0) 0x7f0fc0001110 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd64df640 1 -- 172.21.15.99:0/3435235693 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd00b3a90 con 0x7f0fd006f000 2024-09-10T11:32:15.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.274+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 0x7f0fd00d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.275+0000 7f0fd64df640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd006f000 0x7f0fd006d4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009d90 tx=0x7f0fc802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e781b4644ad56c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.275+0000 7f0fd4cdc640 1 -- 172.21.15.99:0/3435235693 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8007740 con 0x7f0fd006f000 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.275+0000 7f0fd4cdc640 1 -- 172.21.15.99:0/3435235693 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc80078e0 con 0x7f0fd006f000 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.275+0000 7f0fd4cdc640 1 -- 172.21.15.99:0/3435235693 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8007cb0 con 0x7f0fd006f000 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3435235693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd006f000 msgr2=0x7f0fd006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd006f000 0x7f0fd006d4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009d90 tx=0x7f0fc802a660 comp rx=0 tx=0).stop 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3435235693 shutdown_connections 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd006f000 0x7f0fd006d4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3435235693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd015c7b0 0x7f0fd00d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3435235693 >> 172.21.15.99:0/3435235693 conn(0x7f0fd006bc10 msgr2=0x7f0fd006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:15.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3435235693 shutdown_connections 2024-09-10T11:32:15.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.276+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3435235693 wait complete. 2024-09-10T11:32:15.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.277+0000 7f0fd7f42640 1 Processor -- start 2024-09-10T11:32:15.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.277+0000 7f0fd7f42640 1 -- start start 2024-09-10T11:32:15.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 0x7f0fd00b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fd015c7b0 0x7f0fd00b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00b4330 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00b4050 con 0x7f0fd015c7b0 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd00b4610 con 0x7f0fd006f000 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57902/0 (socket says 172.21.15.99:57902) 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 -- 172.21.15.99:0/3393669861 learned_addr learned my addr 172.21.15.99:0/3393669861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 0x7f0fd00b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd64df640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fd015c7b0 0x7f0fd00b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 -- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 msgr2=0x7f0fd00b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 0x7f0fd00b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 -- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fd015c7b0 msgr2=0x7f0fd00b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fd015c7b0 0x7f0fd00b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 -- 172.21.15.99:0/3393669861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd0071cf0 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 0x7f0fd00b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd54dd640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c7b0 tx=0x7f0fc000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0018070 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc00072d0 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.278+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd0071fd0 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.279+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00047e0 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.279+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd0072590 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.280+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd015cc20 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.280+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fc001b070 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.280+0000 7f0fc67fc640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 0x7f0fb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.281+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 0x7f0fb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.281+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7f0fc0089ec0 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.281+0000 7f0fd5cde640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 0x7f0fb406e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.285+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0fd00ceb10 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.513+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7f0fd014b910 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.857+0000 7f0fc67fc640 1 -- 172.21.15.99:0/3393669861 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v115) ==== 142+0+0 (secure 0 0 0) 0x7f0fc0058b70 con 0x7f0fd00d0fb0 2024-09-10T11:32:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 msgr2=0x7f0fb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 0x7f0fb406e320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc006fd0 tx=0x7f0fcc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 msgr2=0x7f0fd00714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c7b0 tx=0x7f0fc000cc70 comp rx=0 tx=0).stop 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 shutdown_connections 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0fb406be60 0x7f0fb406e320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fd015c7b0 0x7f0fd00b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0fd00d0fb0 0x7f0fd00714a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 --2- 172.21.15.99:0/3393669861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fd006f000 0x7f0fd00b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 >> 172.21.15.99:0/3393669861 conn(0x7f0fd006bc10 msgr2=0x7f0fd0158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 shutdown_connections 2024-09-10T11:32:15.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.859+0000 7f0fd7f42640 1 -- 172.21.15.99:0/3393669861 wait complete. 2024-09-10T11:32:15.871 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-09-10T11:32:15.872 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-09-10T11:32:15.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:15.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.970+0000 7f8664467640 1 Processor -- start 2024-09-10T11:32:15.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.970+0000 7f8664467640 1 -- start start 2024-09-10T11:32:15.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8664467640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8664467640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f865c15c770 0x7f865c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8664467640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865c152e60 con 0x7f865c06e5d0 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8664467640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f865c06ef50 con 0x7f865c15c770 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8664467640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d15e0 con 0x7f865c0d1200 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8662a04640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f865c15c770 0x7f865c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8662a04640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f865c15c770 0x7f865c06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:33828/0 (socket says 172.21.15.99:33828) 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8662a04640 1 -- 172.21.15.99:0/692831210 learned_addr learned my addr 172.21.15.99:0/692831210 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8661a02640 1 --1- 172.21.15.99:0/692831210 >> v1:172.21.15.99:6789/0 conn(0x7f865c06e5d0 0x7f865c15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44038/0 (socket says 172.21.15.99:44038) 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8662203640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8662a04640 1 -- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 msgr2=0x7f865c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.972+0000 7f8662a04640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.972+0000 7f8662a04640 1 -- 172.21.15.99:0/692831210 >> v1:172.21.15.99:6789/0 conn(0x7f865c06e5d0 legacy=0x7f865c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.972+0000 7f8662a04640 1 -- 172.21.15.99:0/692831210 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c072680 con 0x7f865c15c770 2024-09-10T11:32:15.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.971+0000 7f8661201640 1 -- 172.21.15.99:0/692831210 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1171422665 0 0) 0x7f865c152e60 con 0x7f865c06e5d0 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.972+0000 7f8662a04640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c06ea10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8654009e50 tx=0x7f865402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6780e1dedf0db27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.972+0000 7f8661201640 1 -- 172.21.15.99:0/692831210 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654036070 con 0x7f865c15c770 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8661201640 1 -- 172.21.15.99:0/692831210 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8654031600 con 0x7f865c15c770 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8661201640 1 -- 172.21.15.99:0/692831210 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540319d0 con 0x7f865c15c770 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 -- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 msgr2=0x7f865c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c06ea10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8654009e50 tx=0x7f865402a680 comp rx=0 tx=0).stop 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 -- 172.21.15.99:0/692831210 shutdown_connections 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c06ea10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 --2- 172.21.15.99:0/692831210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 -- 172.21.15.99:0/692831210 >> 172.21.15.99:0/692831210 conn(0x7f865c06bc10 msgr2=0x7f865c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:15.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 -- 172.21.15.99:0/692831210 shutdown_connections 2024-09-10T11:32:15.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.973+0000 7f8664467640 1 -- 172.21.15.99:0/692831210 wait complete. 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 Processor -- start 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 -- start start 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f865c072f20 con 0x7f865c06e5d0 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f865c072c40 con 0x7f865c15c770 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8664467640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f865c15cd90 con 0x7f865c0d1200 2024-09-10T11:32:15.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8662203640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8662203640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57912/0 (socket says 172.21.15.99:57912) 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.975+0000 7f8662203640 1 -- 172.21.15.99:0/2893017358 learned_addr learned my addr 172.21.15.99:0/2893017358 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8661a02640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662a04640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 -- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 msgr2=0x7f865c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 -- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 msgr2=0x7f865c152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 -- 172.21.15.99:0/2893017358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c153d20 con 0x7f865c06e5d0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8662203640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f865800e8a0 tx=0x7f865800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658017070 con 0x7f865c06e5d0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865800c940 con 0x7f865c06e5d0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c153fc0 con 0x7f865c06e5d0 2024-09-10T11:32:15.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865800cd30 con 0x7f865c06e5d0 2024-09-10T11:32:15.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.976+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c154580 con 0x7f865c06e5d0 2024-09-10T11:32:15.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.977+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c073440 con 0x7f865c06e5d0 2024-09-10T11:32:15.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.978+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f865801a070 con 0x7f865c06e5d0 2024-09-10T11:32:15.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.978+0000 7f8652ffd640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 0x7f864006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.978+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7f865808a970 con 0x7f865c06e5d0 2024-09-10T11:32:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.978+0000 7f8661a02640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 0x7f864006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:15.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.978+0000 7f8661a02640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 0x7f864006e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cd7c0 tx=0x7f864c007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:15.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:15.982+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8658058fd0 con 0x7f865c06e5d0 2024-09-10T11:32:16.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.148+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f865c1548c0 con 0x7f865c06e5d0 2024-09-10T11:32:16.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.149+0000 7f8652ffd640 1 -- 172.21.15.99:0/2893017358 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13032 (secure 0 0 0) 0x7f865805d2d0 con 0x7f865c06e5d0 2024-09-10T11:32:16.150 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:16.150 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":57,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:14.847185+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6836/3824950902":"2024-09-11T11:32:11.813804+0000","172.21.15.99:6834/3824950902":"2024-09-11T11:32:11.813804+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 msgr2=0x7f864006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 0x7f864006e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cd7c0 tx=0x7f864c007de0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 msgr2=0x7f865c15e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f865800e8a0 tx=0x7f865800ed60 comp rx=0 tx=0).stop 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 shutdown_connections 2024-09-10T11:32:16.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f864006be30 0x7f864006e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f865c15c770 0x7f865c152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f865c0d1200 0x7f865c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 --2- 172.21.15.99:0/2893017358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f865c06e5d0 0x7f865c15e410 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 >> 172.21.15.99:0/2893017358 conn(0x7f865c06bc10 msgr2=0x7f865c1e7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 shutdown_connections 2024-09-10T11:32:16.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.151+0000 7f8664467640 1 -- 172.21.15.99:0/2893017358 wait complete. 2024-09-10T11:32:16.163 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:32:16.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.264+0000 7f0df6b31640 1 Processor -- start 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 -- start start 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0df00d1760 0x7f0df00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0df00d0130 0x7f0df00cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df00ce430 con 0x7f0df0160030 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0df00ce710 con 0x7f0df00d0130 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df6b31640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0df00d1240 con 0x7f0df00d1760 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.265+0000 7f0df50ce640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0df00d0130 0x7f0df00cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df48cd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0df0160030 0x7f0df00b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44054/0 (socket says 172.21.15.99:44054) 2024-09-10T11:32:16.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df50ce640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0df00d0130 0x7f0df00cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57924/0 (socket says 172.21.15.99:57924) 2024-09-10T11:32:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df48cd640 1 -- 172.21.15.99:0/1455304797 learned_addr learned my addr 172.21.15.99:0/1455304797 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0deffff640 1 --2- 172.21.15.99:0/1455304797 >> v2:172.21.15.196:3300/0 conn(0x7f0df00d1760 0x7f0df00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df50ce640 1 -- 172.21.15.99:0/1455304797 >> v2:172.21.15.196:3300/0 conn(0x7f0df00d1760 msgr2=0x7f0df00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df50ce640 1 --2- 172.21.15.99:0/1455304797 >> v2:172.21.15.196:3300/0 conn(0x7f0df00d1760 0x7f0df00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df50ce640 1 -- 172.21.15.99:0/1455304797 >> v1:172.21.15.99:6789/0 conn(0x7f0df0160030 legacy=0x7f0df00b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0df50ce640 1 -- 172.21.15.99:0/1455304797 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df00ce030 con 0x7f0df00d0130 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.266+0000 7f0def7fe640 1 -- 172.21.15.99:0/1455304797 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544229181 0 0) 0x7f0df00ce430 con 0x7f0df0160030 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df50ce640 1 --2- 172.21.15.99:0/1455304797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df00cdaf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cee70 tx=0x7f0de802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd36f0ff4f383e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0def7fe640 1 -- 172.21.15.99:0/1455304797 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8037070 con 0x7f0df00d0130 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0def7fe640 1 -- 172.21.15.99:0/1455304797 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de8032440 con 0x7f0df00d0130 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0def7fe640 1 -- 172.21.15.99:0/1455304797 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de8032810 con 0x7f0df00d0130 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 -- 172.21.15.99:0/1455304797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 msgr2=0x7f0df00cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 --2- 172.21.15.99:0/1455304797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df00cdaf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0df00cee70 tx=0x7f0de802a1a0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 -- 172.21.15.99:0/1455304797 shutdown_connections 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 --2- 172.21.15.99:0/1455304797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df00cdaf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 --2- 172.21.15.99:0/1455304797 >> v2:172.21.15.196:3300/0 conn(0x7f0df00d1760 0x7f0df00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 -- 172.21.15.99:0/1455304797 >> 172.21.15.99:0/1455304797 conn(0x7f0df00b4650 msgr2=0x7f0df00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:16.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 -- 172.21.15.99:0/1455304797 shutdown_connections 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.267+0000 7f0df6b31640 1 -- 172.21.15.99:0/1455304797 wait complete. 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.268+0000 7f0df6b31640 1 Processor -- start 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.268+0000 7f0df6b31640 1 -- start start 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0df00d1760 0x7f0df00af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0df0160030 0x7f0df0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0df01606a0 con 0x7f0df00d0130 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0df00b5950 con 0x7f0df00d1760 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df6b31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0df00b11c0 con 0x7f0df0160030 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df48cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df48cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57936/0 (socket says 172.21.15.99:57936) 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.269+0000 7f0df48cd640 1 -- 172.21.15.99:0/967844933 learned_addr learned my addr 172.21.15.99:0/967844933 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:16.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0deffff640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0df00d1760 0x7f0df00af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df50ce640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0df0160030 0x7f0df0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 -- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0df0160030 msgr2=0x7f0df0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0df0160030 0x7f0df0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 -- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0df00d1760 msgr2=0x7f0df00af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0df00d1760 0x7f0df00af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 -- 172.21.15.99:0/967844933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df00afe70 con 0x7f0df00d0130 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df48cd640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c430 tx=0x7f0de000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de00074e0 con 0x7f0df00d0130 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0de00047e0 con 0x7f0df00d0130 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df00b0110 con 0x7f0df00d0130 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de0004bd0 con 0x7f0df00d0130 2024-09-10T11:32:16.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.270+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df00b06d0 con 0x7f0df00d0130 2024-09-10T11:32:16.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.271+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df00d38e0 con 0x7f0df00d0130 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.272+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0de0011070 con 0x7f0df00d0130 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.272+0000 7f0ded7fa640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 0x7f0ddc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.272+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7f0de008a770 con 0x7f0df00d0130 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.272+0000 7f0deffff640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 0x7f0ddc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.272+0000 7f0deffff640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 0x7f0ddc06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0df00b2ef0 tx=0x7f0de40039b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.276+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0df00d38e0 con 0x7f0df00d0130 2024-09-10T11:32:16.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.442+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0df00b85b0 con 0x7f0df00d0130 2024-09-10T11:32:16.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.442+0000 7f0ded7fa640 1 -- 172.21.15.99:0/967844933 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7f0df00b85b0 con 0x7f0df00d0130 2024-09-10T11:32:16.444 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:32:16.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 msgr2=0x7f0ddc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 0x7f0ddc06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0df00b2ef0 tx=0x7f0de40039b0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 msgr2=0x7f0df015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0de000c430 tx=0x7f0de000c8f0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 shutdown_connections 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0ddc06be30 0x7f0ddc06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0df0160030 0x7f0df0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0df00d1760 0x7f0df00af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 --2- 172.21.15.99:0/967844933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0df00d0130 0x7f0df015d740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 >> 172.21.15.99:0/967844933 conn(0x7f0df00b4650 msgr2=0x7f0df00b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:16.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 shutdown_connections 2024-09-10T11:32:16.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.445+0000 7f0df6b31640 1 -- 172.21.15.99:0/967844933 wait complete. 2024-09-10T11:32:16.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-09-10T11:32:16.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.554+0000 7fb79f1f1640 1 Processor -- start 2024-09-10T11:32:16.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 -- start start 2024-09-10T11:32:16.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb79815eb80 0x7fb7980d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb7980d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 0x7fb7980d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb7980d52f0 con 0x7fb79815eb80 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb7980d1620 con 0x7fb7980d4e40 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79f1f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb798162bc0 con 0x7fb7981626c0 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79cf8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb79815eb80 0x7fb7980d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79cf8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb79815eb80 0x7fb7980d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57940/0 (socket says 172.21.15.99:57940) 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.555+0000 7fb79cf8d640 1 -- 172.21.15.99:0/1057961062 learned_addr learned my addr 172.21.15.99:0/1057961062 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79d78e640 1 --2- 172.21.15.99:0/1057961062 >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 0x7fb7980d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb797fff640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb7980d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 -- 172.21.15.99:0/1057961062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 msgr2=0x7fb7980d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb7980d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 -- 172.21.15.99:0/1057961062 >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 msgr2=0x7fb7980d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1057961062 >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 0x7fb7980d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 -- 172.21.15.99:0/1057961062 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb798162ea0 con 0x7fb79815eb80 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79d78e640 1 --2- 172.21.15.99:0/1057961062 >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 0x7fb7980d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb797fff640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb7980d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.556+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb7980d29b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb7980d5da0 tx=0x7fb78802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e257f9547207c896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb7977fe640 1 -- 172.21.15.99:0/1057961062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788037070 con 0x7fb79815eb80 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb7977fe640 1 -- 172.21.15.99:0/1057961062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb788032440 con 0x7fb79815eb80 2024-09-10T11:32:16.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb7977fe640 1 -- 172.21.15.99:0/1057961062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788032810 con 0x7fb79815eb80 2024-09-10T11:32:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1057961062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 msgr2=0x7fb7980d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb7980d29b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb7980d5da0 tx=0x7fb78802a140 comp rx=0 tx=0).stop 2024-09-10T11:32:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1057961062 shutdown_connections 2024-09-10T11:32:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1057961062 >> v2:172.21.15.196:3300/0 conn(0x7fb7980d4e40 0x7fb7980d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb7980d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1057961062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb7980d29b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1057961062 >> 172.21.15.99:0/1057961062 conn(0x7fb79808e680 msgr2=0x7fb79809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1057961062 shutdown_connections 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.557+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1057961062 wait complete. 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.558+0000 7fb79f1f1640 1 Processor -- start 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.558+0000 7fb79f1f1640 1 -- start start 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7980d4e40 0x7fb798152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb79815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7980d2ef0 con 0x7fb79815eb80 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb798163460 con 0x7fb7980d4e40 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb79f1f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7980d3070 con 0x7fb7981626c0 2024-09-10T11:32:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb797fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb797fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57944/0 (socket says 172.21.15.99:57944) 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.559+0000 7fb797fff640 1 -- 172.21.15.99:0/1318918718 learned_addr learned my addr 172.21.15.99:0/1318918718 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7980d4e40 0x7fb798152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb79d78e640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb79815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 -- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 msgr2=0x7fb79815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb79815bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 -- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7980d4e40 msgr2=0x7fb798152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7980d4e40 0x7fb798152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 -- 172.21.15.99:0/1318918718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7981f5870 con 0x7fb79815eb80 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb797fff640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00ce10 tx=0x7fb78c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00f890 con 0x7fb79815eb80 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c00fa30 con 0x7fb79815eb80 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb798153350 con 0x7fb79815eb80 2024-09-10T11:32:16.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c00fe20 con 0x7fb79815eb80 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.560+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981f5e10 con 0x7fb79815eb80 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.561+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7980d29b0 con 0x7fb79815eb80 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.562+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb78c018400 con 0x7fb79815eb80 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.562+0000 7fb7957fa640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 0x7fb78406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.562+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7fb78c08b6d0 con 0x7fb79815eb80 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.562+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 0x7fb78406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:16.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.563+0000 7fb79cf8d640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 0x7fb78406e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb788034000 tx=0x7fb78802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:16.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.566+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb78c014030 con 0x7fb79815eb80 2024-09-10T11:32:16.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.793+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7fb79809e180 con 0x7fb79815eb80 2024-09-10T11:32:16.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.858+0000 7fb7957fa640 1 -- 172.21.15.99:0/1318918718 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v116) ==== 137+0+0 (secure 0 0 0) 0x7fb78c00ae20 con 0x7fb79815eb80 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 msgr2=0x7fb78406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 0x7fb78406e320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb788034000 tx=0x7fb78802e000 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 msgr2=0x7fb798159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb78c00ce10 tx=0x7fb78c00a650 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 shutdown_connections 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb78406be60 0x7fb78406e320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7981626c0 0x7fb79815bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb79815eb80 0x7fb798159f20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 --2- 172.21.15.99:0/1318918718 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7980d4e40 0x7fb798152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:16.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 >> 172.21.15.99:0/1318918718 conn(0x7fb79808e680 msgr2=0x7fb7980b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:16.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 shutdown_connections 2024-09-10T11:32:16.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:16.861+0000 7fb79f1f1640 1 -- 172.21.15.99:0/1318918718 wait complete. 2024-09-10T11:32:16.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-09-10T11:32:16.983 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2024-09-10T11:32:16.983 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-09-10T11:32:16.983 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:32:17.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.084+0000 7f483e8dd640 1 Processor -- start 2024-09-10T11:32:17.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.084+0000 7f483e8dd640 1 -- start start 2024-09-10T11:32:17.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483e8dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f4838072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483e8dd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48380b4150 0x7f48380b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483e8dd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4838072650 con 0x7f48380b0df0 2024-09-10T11:32:17.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483e8dd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f48380b1260 con 0x7f48380b4150 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483e8dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f483815db70 con 0x7f483815cb70 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f48377fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f48380b4150 0x7f48380b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483ce7a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48380b0df0 0x7f483815cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44058/0 (socket says 172.21.15.99:44058) 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f483ce7a640 1 -- 172.21.15.99:0/2746308766 learned_addr learned my addr 172.21.15.99:0/2746308766 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f48377fe640 1 -- 172.21.15.99:0/2746308766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 msgr2=0x7f4838072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f48377fe640 1 --2- 172.21.15.99:0/2746308766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f4838072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f48377fe640 1 -- 172.21.15.99:0/2746308766 >> v1:172.21.15.99:6789/0 conn(0x7f48380b0df0 legacy=0x7f483815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f4836ffd640 1 -- 172.21.15.99:0/2746308766 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999305146 0 0) 0x7f4838072650 con 0x7f48380b0df0 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.085+0000 7f48377fe640 1 -- 172.21.15.99:0/2746308766 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483815de50 con 0x7f48380b4150 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f48377fe640 1 --2- 172.21.15.99:0/2746308766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380b08b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f482c0097a0 tx=0x7f482c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6e58ef8569fa2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f4836ffd640 1 -- 172.21.15.99:0/2746308766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c037070 con 0x7f48380b4150 2024-09-10T11:32:17.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f4836ffd640 1 -- 172.21.15.99:0/2746308766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c032440 con 0x7f48380b4150 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f4836ffd640 1 -- 172.21.15.99:0/2746308766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c032810 con 0x7f48380b4150 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 -- 172.21.15.99:0/2746308766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 msgr2=0x7f48380b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 --2- 172.21.15.99:0/2746308766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380b08b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f482c0097a0 tx=0x7f482c02a180 comp rx=0 tx=0).stop 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 -- 172.21.15.99:0/2746308766 shutdown_connections 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 --2- 172.21.15.99:0/2746308766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380b08b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 --2- 172.21.15.99:0/2746308766 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f4838072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.086+0000 7f483e8dd640 1 -- 172.21.15.99:0/2746308766 >> 172.21.15.99:0/2746308766 conn(0x7f48380b3630 msgr2=0x7f48380b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.087+0000 7f483e8dd640 1 -- 172.21.15.99:0/2746308766 shutdown_connections 2024-09-10T11:32:17.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.087+0000 7f483e8dd640 1 -- 172.21.15.99:0/2746308766 wait complete. 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.088+0000 7f483e8dd640 1 Processor -- start 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.088+0000 7f483e8dd640 1 -- start start 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48380b0df0 0x7f483814a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f483815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4838073320 con 0x7f48380b4150 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f483815e130 con 0x7f48380b0df0 2024-09-10T11:32:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f48380734a0 con 0x7f483815cb70 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57966/0 (socket says 172.21.15.99:57966) 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 -- 172.21.15.99:0/3464477018 learned_addr learned my addr 172.21.15.99:0/3464477018 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483ce7a640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f483815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f4837fff640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48380b0df0 0x7f483814a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 -- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 msgr2=0x7f483815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f483815a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 -- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48380b0df0 msgr2=0x7f483814a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48380b0df0 0x7f483814a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 -- 172.21.15.99:0/3464477018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838158eb0 con 0x7f48380b4150 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f48377fe640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f482c0098c0 tx=0x7f482c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c041070 con 0x7f48380b4150 2024-09-10T11:32:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f482c03c870 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c037040 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4838159190 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.089+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4838159750 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.090+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483815d6b0 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.091+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f482c044070 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.091+0000 7f4834ff9640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 0x7f482406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.091+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7f482c0ab6a0 con 0x7f48380b4150 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.091+0000 7f4837fff640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 0x7f482406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.092+0000 7f4837fff640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 0x7f482406e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4828009890 tx=0x7f4828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.095+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f482c07a470 con 0x7f48380b4150 2024-09-10T11:32:17.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.260+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f483806fcc0 con 0x7f48380b4150 2024-09-10T11:32:17.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.263+0000 7f4834ff9640 1 -- 172.21.15.99:0/3464477018 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f482c079d70 con 0x7f48380b4150 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 msgr2=0x7f482406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 0x7f482406e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4828009890 tx=0x7f4828008040 comp rx=0 tx=0).stop 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 msgr2=0x7f48380ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f482c0098c0 tx=0x7f482c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 shutdown_connections 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f482406be30 0x7f482406e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f483815cb70 0x7f483815a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48380b4150 0x7f48380ab1d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 --2- 172.21.15.99:0/3464477018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48380b0df0 0x7f483814a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 >> 172.21.15.99:0/3464477018 conn(0x7f48380b3630 msgr2=0x7f48381e66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 shutdown_connections 2024-09-10T11:32:17.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.265+0000 7f483e8dd640 1 -- 172.21.15.99:0/3464477018 wait complete. 2024-09-10T11:32:17.277 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:32:17.277 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:32:17.320 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:32:17.369 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:32:17.370 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:17.370 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:17.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:32:17.386 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:32:17.386 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:17.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:32:17.404 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:32:17.404 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:17.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:32:17.443 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:32:17.443 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:32:17.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:32:17.462 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:32:17.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:17.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.579+0000 7fdc8d0d7640 1 Processor -- start 2024-09-10T11:32:17.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.579+0000 7fdc8d0d7640 1 -- start start 2024-09-10T11:32:17.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc8d0d7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc880d1200 0x7fdc88073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc8d0d7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 0x7fdc8806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc8d0d7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc88152e60 con 0x7fdc8815c060 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc8d0d7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdc8806ef80 con 0x7fdc880d1200 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc8d0d7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdc880d15e0 con 0x7fdc8806e5d0 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc880d1200 0x7fdc88073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc880d1200 0x7fdc88073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57974/0 (socket says 172.21.15.99:57974) 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 -- 172.21.15.99:0/2257837505 learned_addr learned my addr 172.21.15.99:0/2257837505 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 -- 172.21.15.99:0/2257837505 >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 msgr2=0x7fdc8806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/2257837505 >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 0x7fdc8806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 --2- 172.21.15.99:0/2257837505 >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 0x7fdc8806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 -- 172.21.15.99:0/2257837505 >> v1:172.21.15.196:6789/0 conn(0x7fdc8815c060 legacy=0x7fdc8815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc86d9d640 1 -- 172.21.15.99:0/2257837505 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc88072680 con 0x7fdc880d1200 2024-09-10T11:32:17.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.580+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/2257837505 >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 0x7fdc8806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.581+0000 7fdc86d9d640 1 --2- 172.21.15.99:0/2257837505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc880d1200 0x7fdc88073440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdc88071bb0 tx=0x7fdc7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa21beee92790f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.581+0000 7fdc8659c640 1 -- 172.21.15.99:0/2257837505 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78037070 con 0x7fdc880d1200 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.581+0000 7fdc8659c640 1 -- 172.21.15.99:0/2257837505 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78032440 con 0x7fdc880d1200 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.581+0000 7fdc8659c640 1 -- 172.21.15.99:0/2257837505 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78032810 con 0x7fdc880d1200 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/2257837505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc880d1200 msgr2=0x7fdc88073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/2257837505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc880d1200 0x7fdc88073440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdc88071bb0 tx=0x7fdc7802a1a0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/2257837505 shutdown_connections 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/2257837505 >> v2:172.21.15.196:3300/0 conn(0x7fdc8806e5d0 0x7fdc8806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/2257837505 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc880d1200 0x7fdc88073440 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/2257837505 >> 172.21.15.99:0/2257837505 conn(0x7fdc8806bc10 msgr2=0x7fdc8806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/2257837505 shutdown_connections 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.582+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/2257837505 wait complete. 2024-09-10T11:32:17.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.583+0000 7fdc8d0d7640 1 Processor -- start 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.583+0000 7fdc8d0d7640 1 -- start start 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 0x7fdc881565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc8815c060 0x7fdc88156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88072f20 con 0x7fdc8806e5d0 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88072c40 con 0x7fdc880d1200 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8d0d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8815cd90 con 0x7fdc8815c060 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57982/0 (socket says 172.21.15.99:57982) 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 -- 172.21.15.99:0/1624064684 learned_addr learned my addr 172.21.15.99:0/1624064684 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc8759e640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc8815c060 0x7fdc88156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 0x7fdc881565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 -- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc8815c060 msgr2=0x7fdc88156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc8815c060 0x7fdc88156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 -- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 msgr2=0x7fdc881565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 0x7fdc881565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc86d9d640 1 -- 172.21.15.99:0/1624064684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc88157700 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.584+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 0x7fdc881565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc86d9d640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78041020 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc78030970 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc88157140 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78037040 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.585+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8815a280 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.586+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8815a6c0 con 0x7fdc8806e5d0 2024-09-10T11:32:17.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.586+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc78044020 con 0x7fdc8806e5d0 2024-09-10T11:32:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.587+0000 7fdc7ffff640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 0x7fdc6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.587+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 0x7fdc6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.587+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 4943+0+0 (secure 0 0 0) 0x7fdc780ab5a0 con 0x7fdc8806e5d0 2024-09-10T11:32:17.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.587+0000 7fdc7e59c640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 0x7fdc6806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc88157b90 tx=0x7fdc70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:17.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.591+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc7804b030 con 0x7fdc8806e5d0 2024-09-10T11:32:17.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.788+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc8806ea40 con 0x7fdc8806e5d0 2024-09-10T11:32:17.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.788+0000 7fdc7ffff640 1 -- 172.21.15.99:0/1624064684 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+3914 (secure 0 0 0) 0x7fdc7807a250 con 0x7fdc8806e5d0 2024-09-10T11:32:17.789 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:17.789 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":23,"btime":"2024-09-10T11:32:12:829676+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5080,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3285466089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3285466089},{"type":"v1","addr":"172.21.15.196:6834","nonce":3285466089}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5083,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2917206529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2917206529},{"type":"v1","addr":"172.21.15.196:6835","nonce":2917206529}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5208,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1780282853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1780282853},{"type":"v1","addr":"172.21.15.99:6837","nonce":1780282853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5496,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/932029629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":932029629},{"type":"v1","addr":"172.21.15.99:6836","nonce":932029629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[]} 2024-09-10T11:32:17.790 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 23 2024-09-10T11:32:17.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 msgr2=0x7fdc6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 0x7fdc6806e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc88157b90 tx=0x7fdc70008040 comp rx=0 tx=0).stop 2024-09-10T11:32:17.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 msgr2=0x7fdc880b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:17.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000c00 tx=0x7fdc780046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 shutdown_connections 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc6806be30 0x7fdc6806e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc8815c060 0x7fdc88156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc880d1200 0x7fdc881565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 --2- 172.21.15.99:0/1624064684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc8806e5d0 0x7fdc880b1a40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 >> 172.21.15.99:0/1624064684 conn(0x7fdc8806bc10 msgr2=0x7fdc8815fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 shutdown_connections 2024-09-10T11:32:17.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:17.790+0000 7fdc8d0d7640 1 -- 172.21.15.99:0/1624064684 wait complete. 2024-09-10T11:32:17.802 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:32:17.803 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:32:17.803 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:32:17.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:32:17.803 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:32:17.803 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:32:17.803 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:32:17.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:32:17.804 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:32:17.804 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:32:17.804 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:32:17.804 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:32:17.804 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:32:17.804 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:32:17.804 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:32:17.805 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:32:17.813 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:32:17.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:32:17.819 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:32:17.819 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:32:17.820 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:32:17.820 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:32:17.856 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:32:17.856 DEBUG:teuthology.parallel:result is None 2024-09-10T11:32:17.862 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:32:17.862 DEBUG:teuthology.parallel:result is None 2024-09-10T11:32:17.863 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:32:17.863 DEBUG:teuthology.parallel:result is None 2024-09-10T11:32:17.930 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:32:17.944 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:32:17.945 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:32:23.813 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:32:23.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:32:23.816 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:32:23.816 DEBUG:teuthology.parallel:result is None 2024-09-10T11:32:23.816 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:32:23.901 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:32:23.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.927+0000 7f4acc15b640 1 Processor -- start 2024-09-10T11:32:23.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.928+0000 7f4acc15b640 1 -- start start 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4acc15b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ac40c83a0 0x7f4ac406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4acc15b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac406d8a0 con 0x7f4ac4070f60 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4acc15b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac406db80 con 0x7f4ac40cfa90 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4acc15b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac406de60 con 0x7f4ac40c83a0 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4ac96f6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ac40c83a0 0x7f4ac406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4aca6f8640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4ac4070f60 0x7f4ac40d0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58570/0 (socket says 172.21.15.99:58570) 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4aca6f8640 1 -- 172.21.15.99:0/634441071 learned_addr learned my addr 172.21.15.99:0/634441071 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4ac96f6640 1 -- 172.21.15.99:0/634441071 >> v1:172.21.15.196:6789/0 conn(0x7f4ac40cfa90 legacy=0x7f4ac40c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4ac96f6640 1 -- 172.21.15.99:0/634441071 >> v1:172.21.15.99:6789/0 conn(0x7f4ac4070f60 legacy=0x7f4ac40d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4ac8ef5640 1 -- 172.21.15.99:0/634441071 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070272779 0 0) 0x7f4ac406d8a0 con 0x7f4ac4070f60 2024-09-10T11:32:23.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.929+0000 7f4ac96f6640 1 -- 172.21.15.99:0/634441071 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac406e140 con 0x7f4ac40c83a0 2024-09-10T11:32:23.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.930+0000 7f4ac96f6640 1 --2- 172.21.15.99:0/634441071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40c83a0 0x7f4ac406e940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd80eca94062fbaa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.930+0000 7f4ac8ef5640 1 -- 172.21.15.99:0/634441071 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4037070 con 0x7f4ac40c83a0 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.930+0000 7f4ac8ef5640 1 -- 172.21.15.99:0/634441071 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab4032440 con 0x7f4ac40c83a0 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.930+0000 7f4ac8ef5640 1 -- 172.21.15.99:0/634441071 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4032810 con 0x7f4ac40c83a0 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 -- 172.21.15.99:0/634441071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40c83a0 msgr2=0x7f4ac406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 --2- 172.21.15.99:0/634441071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40c83a0 0x7f4ac406e940 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40097a0 tx=0x7f4ab402a1a0 comp rx=0 tx=0).stop 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 -- 172.21.15.99:0/634441071 shutdown_connections 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 --2- 172.21.15.99:0/634441071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40c83a0 0x7f4ac406e940 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 -- 172.21.15.99:0/634441071 >> 172.21.15.99:0/634441071 conn(0x7f4ac40b4660 msgr2=0x7f4ac40b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 -- 172.21.15.99:0/634441071 shutdown_connections 2024-09-10T11:32:23.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.931+0000 7f4acc15b640 1 -- 172.21.15.99:0/634441071 wait complete. 2024-09-10T11:32:23.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.932+0000 7f4acc15b640 1 Processor -- start 2024-09-10T11:32:23.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.932+0000 7f4acc15b640 1 -- start start 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4070f60 0x7f4ac41f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac40c83a0 0x7f4ac41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac40c89a0 con 0x7f4ac40cfa90 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac4071370 con 0x7f4ac4070f60 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac40b7710 con 0x7f4ac40c83a0 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45384/0 (socket says 172.21.15.99:45384) 2024-09-10T11:32:23.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 -- 172.21.15.99:0/2814628933 learned_addr learned my addr 172.21.15.99:0/2814628933 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4ac9ef7640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4070f60 0x7f4ac41f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4ac96f6640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac40c83a0 0x7f4ac41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 -- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac40c83a0 msgr2=0x7f4ac41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac40c83a0 0x7f4ac41584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 -- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4070f60 msgr2=0x7f4ac41f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4070f60 0x7f4ac41f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 -- 172.21.15.99:0/2814628933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac41f8b30 con 0x7f4ac40cfa90 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aca6f8640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c7a0 tx=0x7f4abc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc018070 con 0x7f4ac40cfa90 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4abc00a640 con 0x7f4ac40cfa90 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.933+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac41f8e10 con 0x7f4ac40cfa90 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.934+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00aab0 con 0x7f4ac40cfa90 2024-09-10T11:32:23.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.934+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac41f90f0 con 0x7f4ac40cfa90 2024-09-10T11:32:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.934+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac4071700 con 0x7f4ac40cfa90 2024-09-10T11:32:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.935+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4abc01b070 con 0x7f4ac40cfa90 2024-09-10T11:32:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.935+0000 7f4aba7fc640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 0x7f4aa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:23.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.935+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f4abc08a190 con 0x7f4ac40cfa90 2024-09-10T11:32:23.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.936+0000 7f4ac9ef7640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 0x7f4aa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:23.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.936+0000 7f4ac9ef7640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 0x7f4aa806e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000f60 tx=0x7f4ac0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:23.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:23.940+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4abc058750 con 0x7f4ac40cfa90 2024-09-10T11:32:24.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:24.115+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4ac40b3740 con 0x7f4ac40cfa90 2024-09-10T11:32:24.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-10T11:32:25.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.883+0000 7f4aba7fc640 1 -- 172.21.15.99:0/2814628933 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) ==== 97+0+0 (secure 0 0 0) 0x7f4abc05ca50 con 0x7f4ac40cfa90 2024-09-10T11:32:25.884 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:32:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 msgr2=0x7f4aa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 0x7f4aa806e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0000f60 tx=0x7f4ac0009040 comp rx=0 tx=0).stop 2024-09-10T11:32:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 msgr2=0x7f4ac40b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:25.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00c7a0 tx=0x7f4abc00cc60 comp rx=0 tx=0).stop 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 shutdown_connections 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4aa806be30 0x7f4aa806e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ac40cfa90 0x7f4ac40b5960 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ac40c83a0 0x7f4ac41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 --2- 172.21.15.99:0/2814628933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ac4070f60 0x7f4ac41f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 >> 172.21.15.99:0/2814628933 conn(0x7f4ac40b4660 msgr2=0x7f4ac40b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 shutdown_connections 2024-09-10T11:32:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:25.886+0000 7f4acc15b640 1 -- 172.21.15.99:0/2814628933 wait complete. 2024-09-10T11:32:25.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:32:26.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.004+0000 7f3e7b986640 1 Processor -- start 2024-09-10T11:32:26.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.005+0000 7f3e7b986640 1 -- start start 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.005+0000 7f3e7b986640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e740d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.005+0000 7f3e7b986640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e740d52f0 con 0x7f3e740d4e40 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.005+0000 7f3e7b986640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7415ff70 con 0x7f3e7415eb80 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.005+0000 7f3e7b986640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e740d30c0 con 0x7f3e741626c0 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79f23640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3e740d4e40 0x7f3e740d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58578/0 (socket says 172.21.15.99:58578) 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79f23640 1 -- 172.21.15.99:0/730572555 learned_addr learned my addr 172.21.15.99:0/730572555 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073761953 0 0) 0x7f3e6c001490 con 0x7f3e740d4e40 2024-09-10T11:32:26.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e78f21640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e740d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e60003520 con 0x7f3e740d4e40 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 702767474 0 0) 0x7f3e60003520 con 0x7f3e740d4e40 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 msgr2=0x7f3e740d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e740d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 >> v1:172.21.15.196:6789/0 conn(0x7f3e7415eb80 legacy=0x7f3e740d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740d1620 con 0x7f3e740d4e40 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f3e6c001bc0 con 0x7f3e740d4e40 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e78f21640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e740d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 >> v1:172.21.15.99:6789/0 conn(0x7f3e740d4e40 legacy=0x7f3e740d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e60004800 0x7f3e60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e740d30c0 con 0x7f3e600039e0 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e60005510 con 0x7f3e60004800 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6000bf00 con 0x7f3e741626c0 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79722640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e78f21640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e60004800 0x7f3e60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79f23640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79722640 1 -- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 msgr2=0x7f3e6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79722640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.006+0000 7f3e79722640 1 -- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e60004800 msgr2=0x7f3e60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e79722640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e60004800 0x7f3e60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e79722640 1 -- 172.21.15.99:0/730572555 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e60003520 con 0x7f3e600039e0 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e79f23640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e741626c0 0x7f3e6000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e79722640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e60003e00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001750 tx=0x7f3e60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b6908288b9362e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70001370 con 0x7f3e600039e0 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e700052d0 con 0x7f3e600039e0 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.007+0000 7f3e6bfff640 1 -- 172.21.15.99:0/730572555 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7002abe0 con 0x7f3e600039e0 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 -- 172.21.15.99:0/730572555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 msgr2=0x7f3e60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 --2- 172.21.15.99:0/730572555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e60003e00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001750 tx=0x7f3e60005830 comp rx=0 tx=0).stop 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e79f23640 1 -- 172.21.15.99:0/730572555 reap_dead start 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 -- 172.21.15.99:0/730572555 shutdown_connections 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 -- 172.21.15.99:0/730572555 >> 172.21.15.99:0/730572555 conn(0x7f3e7408e680 msgr2=0x7f3e7409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 -- 172.21.15.99:0/730572555 shutdown_connections 2024-09-10T11:32:26.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.008+0000 7f3e7b986640 1 -- 172.21.15.99:0/730572555 wait complete. 2024-09-10T11:32:26.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.009+0000 7f3e7b986640 1 Processor -- start 2024-09-10T11:32:26.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.009+0000 7f3e7b986640 1 -- start start 2024-09-10T11:32:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 0x7f3e740b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e740b28d0 0x7f3e740b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74163180 con 0x7f3e600039e0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e74162ea0 con 0x7f3e741583b0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e7b986640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e74163460 con 0x7f3e740b28d0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45400/0 (socket says 172.21.15.99:45400) 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 -- 172.21.15.99:0/111708485 learned_addr learned my addr 172.21.15.99:0/111708485 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e78f21640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 0x7f3e740b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79f23640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e740b28d0 0x7f3e740b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 -- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e740b28d0 msgr2=0x7f3e740b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e740b28d0 0x7f3e740b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 -- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 msgr2=0x7f3e740b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 0x7f3e740b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 -- 172.21.15.99:0/111708485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e740b1b10 con 0x7f3e600039e0 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e78f21640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 0x7f3e740b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.010+0000 7f3e79722640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001870 tx=0x7f3e70001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.011+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70042070 con 0x7f3e600039e0 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.011+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e7002be50 con 0x7f3e600039e0 2024-09-10T11:32:26.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.011+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e740b1df0 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.011+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037040 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.011+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e740b23b0 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.012+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e740d4900 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.012+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e70045070 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.013+0000 7f3e69ffb640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 0x7f3e5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.013+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f3e700aa310 con 0x7f3e600039e0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.013+0000 7f3e78f21640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 0x7f3e5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.013+0000 7f3e78f21640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 0x7f3e5806e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e7406d760 tx=0x7f3e64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.017+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e70079010 con 0x7f3e600039e0 2024-09-10T11:32:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.244+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3e74158890 con 0x7f3e600039e0 2024-09-10T11:32:26.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.245+0000 7f3e69ffb640 1 -- 172.21.15.99:0/111708485 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f3e70078910 con 0x7f3e600039e0 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:32:26.246 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:32:26.248 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:32:26.249 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.251 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:32:26.252 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:32:26.254 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.255 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:32:26.256 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.257 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:32:26.258 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:32:26.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.259 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:32:26.259 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 msgr2=0x7f3e5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 0x7f3e5806e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3e7406d760 tx=0x7f3e64008040 comp rx=0 tx=0).stop 2024-09-10T11:32:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 msgr2=0x7f3e740ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3e70001870 tx=0x7f3e70001120 comp rx=0 tx=0).stop 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 shutdown_connections 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3e5806be30 0x7f3e5806e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e740b28d0 0x7f3e740b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e741583b0 0x7f3e740b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 --2- 172.21.15.99:0/111708485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e600039e0 0x7f3e740ce540 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 >> 172.21.15.99:0/111708485 conn(0x7f3e7408e680 msgr2=0x7f3e740cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 shutdown_connections 2024-09-10T11:32:26.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.247+0000 7f3e7b986640 1 -- 172.21.15.99:0/111708485 wait complete. 2024-09-10T11:32:26.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:32:26.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.359+0000 7fc2ce598640 1 Processor -- start 2024-09-10T11:32:26.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.359+0000 7fc2ce598640 1 -- start start 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ce598640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ce598640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2c80b0090 0x7fc2c80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ce598640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2c80b40e0 con 0x7fc2c815ee00 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ce598640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c80b5ab0 con 0x7fc2c80b0090 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ce598640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c8073c50 con 0x7fc2c815f1e0 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2c80b0090 0x7fc2c80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc2c80b0090 0x7fc2c80b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:59712/0 (socket says 172.21.15.99:59712) 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 -- 172.21.15.99:0/2791692439 learned_addr learned my addr 172.21.15.99:0/2791692439 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2c77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2c7fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc2c815ee00 0x7fc2c80b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:35140/0 (socket says 172.21.15.99:35140) 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 -- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 msgr2=0x7fc2c80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 -- 172.21.15.99:0/2791692439 >> v1:172.21.15.196:6789/0 conn(0x7fc2c815ee00 legacy=0x7fc2c80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2c6ffd640 1 -- 172.21.15.99:0/2791692439 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074735817 0 0) 0x7fc2c80b40e0 con 0x7fc2c815ee00 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2ccb35640 1 -- 172.21.15.99:0/2791692439 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c80b4380 con 0x7fc2c80b0090 2024-09-10T11:32:26.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.360+0000 7fc2c77fe640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.361+0000 7fc2ccb35640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c80b0090 0x7fc2c80b1c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14cb1a4c712b1698 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.361+0000 7fc2c6ffd640 1 -- 172.21.15.99:0/2791692439 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0036070 con 0x7fc2c80b0090 2024-09-10T11:32:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.361+0000 7fc2c6ffd640 1 -- 172.21.15.99:0/2791692439 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c0031600 con 0x7fc2c80b0090 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.361+0000 7fc2c6ffd640 1 -- 172.21.15.99:0/2791692439 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00319d0 con 0x7fc2c80b0090 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 -- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c80b0090 msgr2=0x7fc2c80b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c80b0090 0x7fc2c80b1c70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c002a710 comp rx=0 tx=0).stop 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 -- 172.21.15.99:0/2791692439 shutdown_connections 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c80b0090 0x7fc2c80b1c70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 --2- 172.21.15.99:0/2791692439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 -- 172.21.15.99:0/2791692439 >> 172.21.15.99:0/2791692439 conn(0x7fc2c80730c0 msgr2=0x7fc2c80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 -- 172.21.15.99:0/2791692439 shutdown_connections 2024-09-10T11:32:26.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.362+0000 7fc2ce598640 1 -- 172.21.15.99:0/2791692439 wait complete. 2024-09-10T11:32:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.363+0000 7fc2ce598640 1 Processor -- start 2024-09-10T11:32:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.363+0000 7fc2ce598640 1 -- start start 2024-09-10T11:32:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 0x7fc2c814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c81f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c80b21b0 con 0x7fc2c80b0090 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c80afd60 con 0x7fc2c815ee00 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ce598640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c80b0600 con 0x7fc2c815f1e0 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45404/0 (socket says 172.21.15.99:45404) 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 -- 172.21.15.99:0/561692890 learned_addr learned my addr 172.21.15.99:0/561692890 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ccb35640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c81f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 -- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 msgr2=0x7fc2c81f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c77fe640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 0x7fc2c814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c81f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 -- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 msgr2=0x7fc2c814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 0x7fc2c814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c7fff640 1 -- 172.21.15.99:0/561692890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c8156000 con 0x7fc2c80b0090 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2ccb35640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c81f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.364+0000 7fc2c77fe640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 0x7fc2c814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2c7fff640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800c4d0 tx=0x7fc2b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8018070 con 0x7fc2c80b0090 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b8007500 con 0x7fc2c80b0090 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004a20 con 0x7fc2c80b0090 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c81562e0 con 0x7fc2c80b0090 2024-09-10T11:32:26.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.365+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c81f4670 con 0x7fc2c80b0090 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.366+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c80b5f90 con 0x7fc2c80b0090 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.366+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc2b801b070 con 0x7fc2c80b0090 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.367+0000 7fc2c4ff9640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 0x7fc2b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.367+0000 7fc2c77fe640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 0x7fc2b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.367+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7fc2b8089de0 con 0x7fc2c80b0090 2024-09-10T11:32:26.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.367+0000 7fc2c77fe640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 0x7fc2b406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8145eb0 tx=0x7fc2bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.371+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc2b8058b70 con 0x7fc2c80b0090 2024-09-10T11:32:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.538+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc2c80ae3a0 con 0x7fc2c80b0090 2024-09-10T11:32:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.538+0000 7fc2c4ff9640 1 -- 172.21.15.99:0/561692890 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) ==== 84+0+65 (secure 0 0 0) 0x7fc2b8058470 con 0x7fc2c80b0090 2024-09-10T11:32:26.539 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:26.539 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:32:26.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:32:26.540 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:32:26.540 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:32:26.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:26.540 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:32:26.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 msgr2=0x7fc2b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 0x7fc2b406e2f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8145eb0 tx=0x7fc2bc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:26.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 msgr2=0x7fc2c8148a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800c4d0 tx=0x7fc2b800c990 comp rx=0 tx=0).stop 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 shutdown_connections 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc2b406be30 0x7fc2b406e2f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc2c815f1e0 0x7fc2c81f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2c815ee00 0x7fc2c814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 --2- 172.21.15.99:0/561692890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2c80b0090 0x7fc2c8148a20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 >> 172.21.15.99:0/561692890 conn(0x7fc2c80730c0 msgr2=0x7fc2c80af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 shutdown_connections 2024-09-10T11:32:26.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.540+0000 7fc2ce598640 1 -- 172.21.15.99:0/561692890 wait complete. 2024-09-10T11:32:26.552 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:32:26.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:32:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.654+0000 7f6e9f6eb640 1 Processor -- start 2024-09-10T11:32:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.654+0000 7f6e9f6eb640 1 -- start start 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9f6eb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e98073440 0x7f6e9815c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9f6eb640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e98152e60 con 0x7f6e9806ece0 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9f6eb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e980d15e0 con 0x7f6e980d1200 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9f6eb640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6e98072680 con 0x7f6e98073440 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e98073440 0x7f6e9815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e98073440 0x7f6e9815c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45406/0 (socket says 172.21.15.99:45406) 2024-09-10T11:32:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 -- 172.21.15.99:0/220060176 learned_addr learned my addr 172.21.15.99:0/220060176 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9cc86640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6e9806ece0 0x7f6e9815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58580/0 (socket says 172.21.15.99:58580) 2024-09-10T11:32:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 -- 172.21.15.99:0/220060176 >> v1:172.21.15.196:6789/0 conn(0x7f6e980d1200 legacy=0x7f6e9806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 -- 172.21.15.99:0/220060176 >> v1:172.21.15.99:6789/0 conn(0x7f6e9806ece0 legacy=0x7f6e9815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e8ffff640 1 -- 172.21.15.99:0/220060176 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1403751835 0 0) 0x7f6e98152e60 con 0x7f6e9806ece0 2024-09-10T11:32:26.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.655+0000 7f6e9dc88640 1 -- 172.21.15.99:0/220060176 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e98072960 con 0x7f6e98073440 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.656+0000 7f6e9dc88640 1 --2- 172.21.15.99:0/220060176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e98073440 0x7f6e9815c770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009e50 tx=0x7f6e9002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fed85ed06495a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.656+0000 7f6e8ffff640 1 -- 172.21.15.99:0/220060176 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90036070 con 0x7f6e98073440 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.656+0000 7f6e8ffff640 1 -- 172.21.15.99:0/220060176 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90031600 con 0x7f6e98073440 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.656+0000 7f6e8ffff640 1 -- 172.21.15.99:0/220060176 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e900319d0 con 0x7f6e98073440 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/220060176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e98073440 msgr2=0x7f6e9815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/220060176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e98073440 0x7f6e9815c770 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009e50 tx=0x7f6e9002a6d0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/220060176 shutdown_connections 2024-09-10T11:32:26.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/220060176 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e98073440 0x7f6e9815c770 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/220060176 >> 172.21.15.99:0/220060176 conn(0x7f6e9806bc10 msgr2=0x7f6e9806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/220060176 shutdown_connections 2024-09-10T11:32:26.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.657+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/220060176 wait complete. 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.658+0000 7f6e9f6eb640 1 Processor -- start 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.658+0000 7f6e9f6eb640 1 -- start start 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9806ece0 0x7f6e98154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e98073440 0x7f6e98147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9815cd90 con 0x7f6e980d1200 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e98072f20 con 0x7f6e98073440 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9f6eb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e98159b50 con 0x7f6e9806ece0 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45414/0 (socket says 172.21.15.99:45414) 2024-09-10T11:32:26.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 -- 172.21.15.99:0/785778298 learned_addr learned my addr 172.21.15.99:0/785778298 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9cc86640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e98073440 0x7f6e98147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9d487640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9806ece0 0x7f6e98154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 -- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9806ece0 msgr2=0x7f6e98154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9806ece0 0x7f6e98154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 -- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e98073440 msgr2=0x7f6e98147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e98073440 0x7f6e98147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 -- 172.21.15.99:0/785778298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e981f2b60 con 0x7f6e980d1200 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.659+0000 7f6e9dc88640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6e90034040 tx=0x7f6e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.660+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90031e60 con 0x7f6e980d1200 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.660+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90038600 con 0x7f6e980d1200 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.660+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e981f2e40 con 0x7f6e980d1200 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.660+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e900389d0 con 0x7f6e980d1200 2024-09-10T11:32:26.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.660+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e981f3400 con 0x7f6e980d1200 2024-09-10T11:32:26.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.661+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e9806c190 con 0x7f6e980d1200 2024-09-10T11:32:26.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.661+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e9004a060 con 0x7f6e980d1200 2024-09-10T11:32:26.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.662+0000 7f6e8dffb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 0x7f6e7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:26.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.662+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4857+0+0 (secure 0 0 0) 0x7f6e900ace00 con 0x7f6e980d1200 2024-09-10T11:32:26.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.662+0000 7f6e9d487640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 0x7f6e7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:26.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.662+0000 7f6e9d487640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 0x7f6e7c06e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006fd0 tx=0x7f6e94009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:26.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.666+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e9003e070 con 0x7f6e980d1200 2024-09-10T11:32:26.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.832+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6e980761b0 con 0x7f6e980d1200 2024-09-10T11:32:26.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.896+0000 7f6e8dffb640 1 -- 172.21.15.99:0/785778298 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) ==== 138+0+0 (secure 0 0 0) 0x7f6e980d15e0 con 0x7f6e980d1200 2024-09-10T11:32:26.897 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:32:26.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 msgr2=0x7f6e7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 0x7f6e7c06e2f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6e94006fd0 tx=0x7f6e94009060 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 msgr2=0x7f6e981482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6e90034040 tx=0x7f6e90004a40 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 shutdown_connections 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e7c06be30 0x7f6e7c06e2f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e980d1200 0x7f6e981482f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.898+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e98073440 0x7f6e98147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.899+0000 7f6e9f6eb640 1 --2- 172.21.15.99:0/785778298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9806ece0 0x7f6e98154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:26.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.899+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 >> 172.21.15.99:0/785778298 conn(0x7f6e9806bc10 msgr2=0x7f6e9815d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:26.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.900+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 shutdown_connections 2024-09-10T11:32:26.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:26.900+0000 7f6e9f6eb640 1 -- 172.21.15.99:0/785778298 wait complete. 2024-09-10T11:32:26.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:32:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.018+0000 7fc54db9f640 1 Processor -- start 2024-09-10T11:32:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.018+0000 7fc54db9f640 1 -- start start 2024-09-10T11:32:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54db9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc54815ed20 0x7fc5480d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54db9f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 0x7fc5480cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54db9f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc548163660 con 0x7fc5480d2f30 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54db9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc5480d1c30 con 0x7fc54815ed20 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54db9f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc5480d1f10 con 0x7fc54806d430 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc54815ed20 0x7fc5480d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc547fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc5480d2f30 0x7fc54806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58590/0 (socket says 172.21.15.99:58590) 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc547fff640 1 -- 172.21.15.99:0/2908678181 learned_addr learned my addr 172.21.15.99:0/2908678181 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc54815ed20 0x7fc5480d29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45424/0 (socket says 172.21.15.99:45424) 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54d39e640 1 --2- 172.21.15.99:0/2908678181 >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 0x7fc5480cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 -- 172.21.15.99:0/2908678181 >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 msgr2=0x7fc5480cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 --2- 172.21.15.99:0/2908678181 >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 0x7fc5480cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 -- 172.21.15.99:0/2908678181 >> v1:172.21.15.99:6789/0 conn(0x7fc5480d2f30 legacy=0x7fc54806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc5477fe640 1 -- 172.21.15.99:0/2908678181 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629239995 0 0) 0x7fc548163660 con 0x7fc5480d2f30 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54cb9d640 1 -- 172.21.15.99:0/2908678181 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5480d21f0 con 0x7fc54815ed20 2024-09-10T11:32:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.019+0000 7fc54d39e640 1 --2- 172.21.15.99:0/2908678181 >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 0x7fc5480cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.020+0000 7fc54cb9d640 1 --2- 172.21.15.99:0/2908678181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815ed20 0x7fc5480d29f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc54806eeb0 tx=0x7fc53c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf25b3702bbb9358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.020+0000 7fc5477fe640 1 -- 172.21.15.99:0/2908678181 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c037070 con 0x7fc54815ed20 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.020+0000 7fc5477fe640 1 -- 172.21.15.99:0/2908678181 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc53c032440 con 0x7fc54815ed20 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.020+0000 7fc5477fe640 1 -- 172.21.15.99:0/2908678181 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c032810 con 0x7fc54815ed20 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 -- 172.21.15.99:0/2908678181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815ed20 msgr2=0x7fc5480d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 --2- 172.21.15.99:0/2908678181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815ed20 0x7fc5480d29f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc54806eeb0 tx=0x7fc53c02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 -- 172.21.15.99:0/2908678181 shutdown_connections 2024-09-10T11:32:27.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 --2- 172.21.15.99:0/2908678181 >> v2:172.21.15.196:3300/0 conn(0x7fc54806d430 0x7fc5480cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 --2- 172.21.15.99:0/2908678181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815ed20 0x7fc5480d29f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 -- 172.21.15.99:0/2908678181 >> 172.21.15.99:0/2908678181 conn(0x7fc54809ba30 msgr2=0x7fc54809be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 -- 172.21.15.99:0/2908678181 shutdown_connections 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.021+0000 7fc54db9f640 1 -- 172.21.15.99:0/2908678181 wait complete. 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.022+0000 7fc54db9f640 1 Processor -- start 2024-09-10T11:32:27.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.022+0000 7fc54db9f640 1 -- start start 2024-09-10T11:32:27.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d2f30 0x7fc548153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 0x7fc5481570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480ccb70 con 0x7fc54815b3e0 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480cc9f0 con 0x7fc54815ed20 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54db9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc548162910 con 0x7fc5480d2f30 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45432/0 (socket says 172.21.15.99:45432) 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 -- 172.21.15.99:0/3923304276 learned_addr learned my addr 172.21.15.99:0/3923304276 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54cb9d640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d2f30 0x7fc548153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc547fff640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 0x7fc5481570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 -- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d2f30 msgr2=0x7fc548153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d2f30 0x7fc548153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 -- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 msgr2=0x7fc5481570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 0x7fc5481570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 -- 172.21.15.99:0/3923304276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54815bdc0 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc547fff640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 0x7fc5481570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc54d39e640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc54000ce80 tx=0x7fc54000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.023+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54001a070 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.024+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54000d880 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.024+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5480afda0 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.024+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000dc70 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.024+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5480b0360 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.025+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5480cc2a0 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.025+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc54001d070 con 0x7fc54815b3e0 2024-09-10T11:32:27.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.025+0000 7fc5457fa640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 0x7fc52c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.026+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5227+0+0 (secure 0 0 0) 0x7fc54008b3c0 con 0x7fc54815b3e0 2024-09-10T11:32:27.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.026+0000 7fc54cb9d640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 0x7fc52c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.026+0000 7fc54cb9d640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 0x7fc52c06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc53c034000 tx=0x7fc53c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:27.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.030+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc540059900 con 0x7fc54815b3e0 2024-09-10T11:32:27.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.195+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc5480d36b0 con 0x7fc54815b3e0 2024-09-10T11:32:27.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.903+0000 7fc5457fa640 1 -- 172.21.15.99:0/3923304276 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) ==== 170+0+0 (secure 0 0 0) 0x7fc540060120 con 0x7fc54815b3e0 2024-09-10T11:32:27.905 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 msgr2=0x7fc52c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 0x7fc52c06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc53c034000 tx=0x7fc53c02e000 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 msgr2=0x7fc5480b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc54000ce80 tx=0x7fc54000a710 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 shutdown_connections 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc52c06be30 0x7fc52c06e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc54815b3e0 0x7fc5480b0b60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815ed20 0x7fc5481570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 --2- 172.21.15.99:0/3923304276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d2f30 0x7fc548153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 >> 172.21.15.99:0/3923304276 conn(0x7fc54809ba30 msgr2=0x7fc5480ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:27.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 shutdown_connections 2024-09-10T11:32:27.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:27.913+0000 7fc54db9f640 1 -- 172.21.15.99:0/3923304276 wait complete. 2024-09-10T11:32:27.933 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:32:28.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.036+0000 7f92bc952640 1 Processor -- start 2024-09-10T11:32:28.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.036+0000 7f92bc952640 1 -- start start 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92bc952640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92b80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92bc952640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92b81626c0 0x7f92b80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92bc952640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b80d52d0 con 0x7f92b80d4e40 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92bc952640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f92b815ff70 con 0x7f92b81626c0 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92bc952640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80d1620 con 0x7f92b815eb80 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b6d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f92b80d4e40 0x7f92b80d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58606/0 (socket says 172.21.15.99:58606) 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b6d9d640 1 -- 172.21.15.99:0/4255947331 learned_addr learned my addr 172.21.15.99:0/4255947331 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b659c640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92b80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/4255947331 >> v2:172.21.15.196:3300/0 conn(0x7f92b81626c0 0x7f92b80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578463394 0 0) 0x7f92b80d52d0 con 0x7f92b80d4e40 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92a0003520 con 0x7f92b80d4e40 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2499640176 0 0) 0x7f92a0003520 con 0x7f92b80d4e40 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 msgr2=0x7f92b80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92b80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 >> v2:172.21.15.196:3300/0 conn(0x7f92b81626c0 msgr2=0x7f92b80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 --2- 172.21.15.99:0/4255947331 >> v2:172.21.15.196:3300/0 conn(0x7f92b81626c0 0x7f92b80d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b8162bc0 con 0x7f92b80d4e40 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f92b00019f0 con 0x7f92b80d4e40 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 >> v1:172.21.15.99:6789/0 conn(0x7f92b80d4e40 legacy=0x7f92b80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92a00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0003f30 0x7f92a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92b815ff70 con 0x7f92a0003f30 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92b80d1620 con 0x7f92a0004d40 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.037+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92a000ba10 con 0x7f92b815eb80 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0003f30 0x7f92a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b659c640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92a00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b6d9d640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 -- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 msgr2=0x7f92a00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b815eb80 0x7f92a00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 -- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0004d40 msgr2=0x7f92a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:28.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0004d40 0x7f92a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.038+0000 7f92b5d9b640 1 -- 172.21.15.99:0/4255947331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0003520 con 0x7f92a0003f30 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0003f30 0x7f92a0004800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009af0 tx=0x7f92a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9434a0fbc599e069 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac036070 con 0x7f92a0003f30 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac031400 con 0x7f92a0003f30 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92b559a640 1 -- 172.21.15.99:0/4255947331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac009690 con 0x7f92a0003f30 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 -- 172.21.15.99:0/4255947331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0003f30 msgr2=0x7f92a0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 --2- 172.21.15.99:0/4255947331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92a0003f30 0x7f92a0004800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f92ac009af0 tx=0x7f92a0005830 comp rx=0 tx=0).stop 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92b6d9d640 1 -- 172.21.15.99:0/4255947331 reap_dead start 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 -- 172.21.15.99:0/4255947331 shutdown_connections 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 -- 172.21.15.99:0/4255947331 >> 172.21.15.99:0/4255947331 conn(0x7f92b808e680 msgr2=0x7f92b809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 -- 172.21.15.99:0/4255947331 shutdown_connections 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.039+0000 7f92bc952640 1 -- 172.21.15.99:0/4255947331 wait complete. 2024-09-10T11:32:28.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 Processor -- start 2024-09-10T11:32:28.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 -- start start 2024-09-10T11:32:28.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 0x7f92b80ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b806d5e0 0x7f92b806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8163460 con 0x7f92b80b6cc0 2024-09-10T11:32:28.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8163180 con 0x7f92a0003f30 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.041+0000 7f92bc952640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80d2230 con 0x7f92b806d5e0 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45440/0 (socket says 172.21.15.99:45440) 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 -- 172.21.15.99:0/468329385 learned_addr learned my addr 172.21.15.99:0/468329385 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b6d9d640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b806d5e0 0x7f92b806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b659c640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 0x7f92b80ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 -- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b806d5e0 msgr2=0x7f92b806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b806d5e0 0x7f92b806da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 -- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 msgr2=0x7f92b80ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 0x7f92b80ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 -- 172.21.15.99:0/468329385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80b7140 con 0x7f92b80b6cc0 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b659c640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 0x7f92b80ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:28.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92b5d9b640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f92ac034040 tx=0x7f92ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac040020 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92ac003810 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac036040 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b81f7f10 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.042+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81f84d0 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.043+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b80d4900 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.044+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92ac043020 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.044+0000 7f92a6ffd640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 0x7f929806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.044+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5609+0+0 (secure 0 0 0) 0x7f92ac0aac30 con 0x7f92b80b6cc0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.044+0000 7f92b659c640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 0x7f929806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:28.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.045+0000 7f92b659c640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 0x7f929806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92b8161760 tx=0x7f92a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:28.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.049+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92ac04a070 con 0x7f92b80b6cc0 2024-09-10T11:32:28.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:32:28.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:32:28.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.252+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f92b81613d0 con 0x7f92b80b6cc0 2024-09-10T11:32:28.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.905+0000 7f92a6ffd640 1 -- 172.21.15.99:0/468329385 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) ==== 172+0+0 (secure 0 0 0) 0x7f92ac0796d0 con 0x7f92b80b6cc0 2024-09-10T11:32:28.906 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 msgr2=0x7f929806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 0x7f929806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f92b8161760 tx=0x7f92a8008040 comp rx=0 tx=0).stop 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 msgr2=0x7f92b81f51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f92ac034040 tx=0x7f92ac004730 comp rx=0 tx=0).stop 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 shutdown_connections 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f929806be30 0x7f929806e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b806d5e0 0x7f92b806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80b6cc0 0x7f92b81f51d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 --2- 172.21.15.99:0/468329385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92a0003f30 0x7f92b80ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 >> 172.21.15.99:0/468329385 conn(0x7f92b808e680 msgr2=0x7f92b80d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:28.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 shutdown_connections 2024-09-10T11:32:28.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:28.908+0000 7f92bc952640 1 -- 172.21.15.99:0/468329385 wait complete. 2024-09-10T11:32:28.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.036+0000 7fe5c9aae640 1 Processor -- start 2024-09-10T11:32:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.036+0000 7fe5c9aae640 1 -- start start 2024-09-10T11:32:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c9aae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c9aae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c9aae640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5c40b3620 con 0x7fe5c40b3960 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c9aae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe5c406d960 con 0x7fe5c40b3160 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c9aae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c406dc40 con 0x7fe5c4163580 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c3fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c37fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:33160/0 (socket says 172.21.15.99:33160) 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c3fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:59774/0 (socket says 172.21.15.99:59774) 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c2ffd640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe5c40b3960 0x7fe5c40d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:35156/0 (socket says 172.21.15.99:35156) 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.037+0000 7fe5c37fe640 1 -- 172.21.15.99:0/245860235 learned_addr learned my addr 172.21.15.99:0/245860235 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c37fe640 1 -- 172.21.15.99:0/245860235 >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 msgr2=0x7fe5c406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c37fe640 1 --2- 172.21.15.99:0/245860235 >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c37fe640 1 -- 172.21.15.99:0/245860235 >> v1:172.21.15.196:6789/0 conn(0x7fe5c40b3960 legacy=0x7fe5c40d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c27fc640 1 -- 172.21.15.99:0/245860235 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1109714235 0 0) 0x7fe5c40b3620 con 0x7fe5c40b3960 2024-09-10T11:32:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c37fe640 1 -- 172.21.15.99:0/245860235 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c406df20 con 0x7fe5c4163580 2024-09-10T11:32:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c3fff640 1 --2- 172.21.15.99:0/245860235 >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c37fe640 1 --2- 172.21.15.99:0/245860235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009e50 tx=0x7fe5b402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef87ba5bb1a21d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c27fc640 1 -- 172.21.15.99:0/245860235 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b4036070 con 0x7fe5c4163580 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c27fc640 1 -- 172.21.15.99:0/245860235 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5b4031600 con 0x7fe5c4163580 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.038+0000 7fe5c27fc640 1 -- 172.21.15.99:0/245860235 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b40319d0 con 0x7fe5c4163580 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 -- 172.21.15.99:0/245860235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 msgr2=0x7fe5c40b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/245860235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009e50 tx=0x7fe5b402a5f0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 -- 172.21.15.99:0/245860235 shutdown_connections 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/245860235 >> v2:172.21.15.196:3300/0 conn(0x7fe5c40b3160 0x7fe5c406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/245860235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c4163580 0x7fe5c40b5090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 -- 172.21.15.99:0/245860235 >> 172.21.15.99:0/245860235 conn(0x7fe5c40b4570 msgr2=0x7fe5c40b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 -- 172.21.15.99:0/245860235 shutdown_connections 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.039+0000 7fe5c9aae640 1 -- 172.21.15.99:0/245860235 wait complete. 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.040+0000 7fe5c9aae640 1 Processor -- start 2024-09-10T11:32:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.040+0000 7fe5c9aae640 1 -- start start 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c40b3160 0x7fe5c40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c40b3960 0x7fe5c4154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c406d3e0 con 0x7fe5c4163580 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c406d260 con 0x7fe5c40b3960 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c9aae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c406d560 con 0x7fe5c40b3160 2024-09-10T11:32:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45446/0 (socket says 172.21.15.99:45446) 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c3fff640 1 -- 172.21.15.99:0/2704223941 learned_addr learned my addr 172.21.15.99:0/2704223941 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c2ffd640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c40b3960 0x7fe5c4154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.041+0000 7fe5c37fe640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c40b3160 0x7fe5c40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 -- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c40b3160 msgr2=0x7fe5c40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c40b3160 0x7fe5c40cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 -- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c40b3960 msgr2=0x7fe5c4154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c40b3960 0x7fe5c4154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 -- 172.21.15.99:0/2704223941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c40c9c90 con 0x7fe5c4163580 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c3fff640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e880 tx=0x7fe5bc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc017070 con 0x7fe5c4163580 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00c940 con 0x7fe5c4163580 2024-09-10T11:32:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc00cd30 con 0x7fe5c4163580 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c40c9f70 con 0x7fe5c4163580 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.042+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c40ca530 con 0x7fe5c4163580 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.043+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c406cd20 con 0x7fe5c4163580 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.043+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe5bc01a070 con 0x7fe5c4163580 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.044+0000 7fe5abfff640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 0x7fe5b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.044+0000 7fe5c37fe640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 0x7fe5b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.044+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7fe5bc08af90 con 0x7fe5c4163580 2024-09-10T11:32:29.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.044+0000 7fe5c37fe640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 0x7fe5b006e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4033000 tx=0x7fe5b4009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.048+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe5bc0592f0 con 0x7fe5c4163580 2024-09-10T11:32:29.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.215+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5c40b2d40 con 0x7fe5c4163580 2024-09-10T11:32:29.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.216+0000 7fe5abfff640 1 -- 172.21.15.99:0/2704223941 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16738 (secure 0 0 0) 0x7fe5bc082e20 con 0x7fe5c4163580 2024-09-10T11:32:29.217 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:29.217 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":65,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:28.253414+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:32:26.850541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T11:32:27.213215+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:29.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 msgr2=0x7fe5b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 0x7fe5b006e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4033000 tx=0x7fe5b4009000 comp rx=0 tx=0).stop 2024-09-10T11:32:29.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 msgr2=0x7fe5c40b7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00e880 tx=0x7fe5bc00ed40 comp rx=0 tx=0).stop 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 shutdown_connections 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe5b006be30 0x7fe5b006e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5c4163580 0x7fe5c40b7660 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5c40b3960 0x7fe5c4154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 --2- 172.21.15.99:0/2704223941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5c40b3160 0x7fe5c40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 >> 172.21.15.99:0/2704223941 conn(0x7fe5c40b4570 msgr2=0x7fe5c40b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 shutdown_connections 2024-09-10T11:32:29.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.218+0000 7fe5c9aae640 1 -- 172.21.15.99:0/2704223941 wait complete. 2024-09-10T11:32:29.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:29.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.327+0000 7f45cf8c9640 1 Processor -- start 2024-09-10T11:32:29.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.327+0000 7f45cf8c9640 1 -- start start 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c815c7b0 0x7f45c80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 0x7f45c806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45c806ee30 0x7f45c81604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f45c806f280 con 0x7f45c806ee30 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f45c80b3840 con 0x7f45c80d0fb0 2024-09-10T11:32:29.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cf8c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80d13f0 con 0x7f45c815c7b0 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cde66640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45c806ee30 0x7f45c81604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cde66640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f45c806ee30 0x7f45c81604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45460/0 (socket says 172.21.15.99:45460) 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.328+0000 7f45cde66640 1 -- 172.21.15.99:0/2572785610 learned_addr learned my addr 172.21.15.99:0/2572785610 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 -- 172.21.15.99:0/2572785610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c815c7b0 msgr2=0x7f45c80d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cce64640 1 --2- 172.21.15.99:0/2572785610 >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 0x7f45c806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cd665640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c815c7b0 0x7f45c80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c815c7b0 0x7f45c80d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 -- 172.21.15.99:0/2572785610 >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 msgr2=0x7f45c806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 --2- 172.21.15.99:0/2572785610 >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 0x7f45c806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 -- 172.21.15.99:0/2572785610 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c80ceb10 con 0x7f45c806ee30 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cce64640 1 --2- 172.21.15.99:0/2572785610 >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 0x7f45c806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45cde66640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c806ee30 0x7f45c81604b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f45c806f720 tx=0x7f45c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c1941ac2bf0a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45bffff640 1 -- 172.21.15.99:0/2572785610 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0036070 con 0x7f45c806ee30 2024-09-10T11:32:29.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45bffff640 1 -- 172.21.15.99:0/2572785610 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c0031600 con 0x7f45c806ee30 2024-09-10T11:32:29.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.329+0000 7f45bffff640 1 -- 172.21.15.99:0/2572785610 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c00319d0 con 0x7f45c806ee30 2024-09-10T11:32:29.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 -- 172.21.15.99:0/2572785610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c806ee30 msgr2=0x7f45c81604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c806ee30 0x7f45c81604b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f45c806f720 tx=0x7f45c002a6a0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 -- 172.21.15.99:0/2572785610 shutdown_connections 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c806ee30 0x7f45c81604b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/2572785610 >> v2:172.21.15.196:3300/0 conn(0x7f45c80d0fb0 0x7f45c806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/2572785610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c815c7b0 0x7f45c80d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 -- 172.21.15.99:0/2572785610 >> 172.21.15.99:0/2572785610 conn(0x7f45c806bc10 msgr2=0x7f45c806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 -- 172.21.15.99:0/2572785610 shutdown_connections 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.330+0000 7f45cf8c9640 1 -- 172.21.15.99:0/2572785610 wait complete. 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.331+0000 7f45cf8c9640 1 Processor -- start 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.331+0000 7f45cf8c9640 1 -- start start 2024-09-10T11:32:29.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c806ee30 0x7f45c81463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45c815c7b0 0x7f45c8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b4050 con 0x7f45c80d0fb0 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f45c80b3d70 con 0x7f45c815c7b0 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cf8c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f45c80b4330 con 0x7f45c806ee30 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45476/0 (socket says 172.21.15.99:45476) 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 -- 172.21.15.99:0/963608712 learned_addr learned my addr 172.21.15.99:0/963608712 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cd665640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c806ee30 0x7f45c81463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cde66640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45c815c7b0 0x7f45c8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 -- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c806ee30 msgr2=0x7f45c81463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c806ee30 0x7f45c81463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 -- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45c815c7b0 msgr2=0x7f45c8159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45c815c7b0 0x7f45c8159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.332+0000 7f45cce64640 1 -- 172.21.15.99:0/963608712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c814eff0 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45cce64640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c820 tx=0x7f45b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8015070 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b8007a30 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c814f2d0 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8007e00 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.333+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c814f890 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.334+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8069640 con 0x7f45c80d0fb0 2024-09-10T11:32:29.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.334+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45c80d0fb0 2024-09-10T11:32:29.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.334+0000 7f45bdffb640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.335+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7f45b808ad10 con 0x7f45c80d0fb0 2024-09-10T11:32:29.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.335+0000 7f45cd665640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.335+0000 7f45cd665640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45c400a9a0 tx=0x7f45c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.339+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45c8069640 con 0x7f45c80d0fb0 2024-09-10T11:32:29.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.505+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45c8073410 con 0x7f45c80d0fb0 2024-09-10T11:32:29.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.506+0000 7f45bdffb640 1 -- 172.21.15.99:0/963608712 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16738 (secure 0 0 0) 0x7f45b8059100 con 0x7f45c80d0fb0 2024-09-10T11:32:29.507 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:29.507 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":65,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:28.253414+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:32:26.850541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T11:32:27.213215+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 msgr2=0x7f45ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 0x7f45ac06e2f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45c400a9a0 tx=0x7f45c4008040 comp rx=0 tx=0).stop 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 msgr2=0x7f45c814e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45b800c820 tx=0x7f45b800cc20 comp rx=0 tx=0).stop 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 shutdown_connections 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f45ac06be30 0x7f45ac06e2f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45c815c7b0 0x7f45c8159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45c80d0fb0 0x7f45c814e610 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 --2- 172.21.15.99:0/963608712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45c806ee30 0x7f45c81463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 >> 172.21.15.99:0/963608712 conn(0x7f45c806bc10 msgr2=0x7f45c80b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 shutdown_connections 2024-09-10T11:32:29.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.508+0000 7f45cf8c9640 1 -- 172.21.15.99:0/963608712 wait complete. 2024-09-10T11:32:29.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:32:29.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.620+0000 7f86ae65e640 1 Processor -- start 2024-09-10T11:32:29.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.620+0000 7f86ae65e640 1 -- start start 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86ae65e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86a8160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86ae65e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86a80b4930 0x7f86a806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86ae65e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a80b5140 con 0x7f86a806bc10 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86ae65e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86a80b5420 con 0x7f86a80b4930 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86ae65e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86a8162b40 con 0x7f86a8163620 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86a7fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f86a806bc10 0x7f86a806bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58620/0 (socket says 172.21.15.99:58620) 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86a7fff640 1 -- 172.21.15.99:0/1355025950 learned_addr learned my addr 172.21.15.99:0/1355025950 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210677739 0 0) 0x7f86a80b5140 con 0x7f86a806bc10 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8690003520 con 0x7f86a806bc10 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86a77fe640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86a8160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.621+0000 7f86acbfb640 1 --2- 172.21.15.99:0/1355025950 >> v2:172.21.15.196:3300/0 conn(0x7f86a80b4930 0x7f86a806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3890014091 0 0) 0x7f8690003520 con 0x7f86a806bc10 2024-09-10T11:32:29.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 msgr2=0x7f86a8160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86a8160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 >> v2:172.21.15.196:3300/0 conn(0x7f86a80b4930 msgr2=0x7f86a806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 --2- 172.21.15.99:0/1355025950 >> v2:172.21.15.196:3300/0 conn(0x7f86a80b4930 0x7f86a806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a8162e20 con 0x7f86a806bc10 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8698001670 con 0x7f86a806bc10 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 >> v1:172.21.15.99:6789/0 conn(0x7f86a806bc10 legacy=0x7f86a806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8690003f30 0x7f8690004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8690004d40 0x7f86900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80b5420 con 0x7f8690004d40 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8162b40 con 0x7f8690003f30 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f869000ba10 con 0x7f86a8163620 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8690004d40 0x7f86900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a7fff640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a77fe640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8690003f30 0x7f8690004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 -- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 msgr2=0x7f86900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 -- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8690003f30 msgr2=0x7f8690004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8690003f30 0x7f8690004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86acbfb640 1 -- 172.21.15.99:0/1355025950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8690003520 con 0x7f8690004d40 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.622+0000 7f86a7fff640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a8163620 0x7f86900039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.623+0000 7f86acbfb640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8690004d40 0x7f86900051c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e170 tx=0x7f8690005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ce3233094cec6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.623+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0034020 con 0x7f8690004d40 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.623+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a0030430 con 0x7f8690004d40 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.623+0000 7f86a6ffd640 1 -- 172.21.15.99:0/1355025950 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0009c10 con 0x7f8690004d40 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 -- 172.21.15.99:0/1355025950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8690004d40 msgr2=0x7f86900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 --2- 172.21.15.99:0/1355025950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8690004d40 0x7f86900051c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e170 tx=0x7f8690005830 comp rx=0 tx=0).stop 2024-09-10T11:32:29.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86acbfb640 1 -- 172.21.15.99:0/1355025950 reap_dead start 2024-09-10T11:32:29.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 -- 172.21.15.99:0/1355025950 shutdown_connections 2024-09-10T11:32:29.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 -- 172.21.15.99:0/1355025950 >> 172.21.15.99:0/1355025950 conn(0x7f86a80b3630 msgr2=0x7f86a80b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 -- 172.21.15.99:0/1355025950 shutdown_connections 2024-09-10T11:32:29.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.624+0000 7f86ae65e640 1 -- 172.21.15.99:0/1355025950 wait complete. 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.625+0000 7f86ae65e640 1 Processor -- start 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.625+0000 7f86ae65e640 1 -- start start 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a815dff0 0x7f86a81e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a815e3d0 0x7f86a81f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80b4d90 con 0x7f86a81f8b10 2024-09-10T11:32:29.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86a80d1560 con 0x7f86a815e3d0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86a813fe80 con 0x7f86a815dff0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45494/0 (socket says 172.21.15.99:45494) 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 -- 172.21.15.99:0/45462964 learned_addr learned my addr 172.21.15.99:0/45462964 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86a77fe640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a815e3d0 0x7f86a81f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86a7fff640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a815dff0 0x7f86a81e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 -- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a815dff0 msgr2=0x7f86a81e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a815dff0 0x7f86a81e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 -- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a815e3d0 msgr2=0x7f86a81f85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a815e3d0 0x7f86a81f85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 -- 172.21.15.99:0/45462964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a813fb50 con 0x7f86a81f8b10 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86acbfb640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e290 tx=0x7f86a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0034070 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86a0003b90 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0009c10 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.626+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a81e96e0 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.627+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81e9ca0 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.628+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a80b5140 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.628+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86a0037020 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.628+0000 7f86a4ff9640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.629+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5709+0+0 (secure 0 0 0) 0x7f86a00ab2a0 con 0x7f86a81f8b10 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.629+0000 7f86a7fff640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.629+0000 7f86a7fff640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 0x7f869406e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f86980017e0 tx=0x7f8698007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:29.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.633+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86a0048030 con 0x7f86a81f8b10 2024-09-10T11:32:29.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.831+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f86a81e9ea0 con 0x7f86a81f8b10 2024-09-10T11:32:29.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.928+0000 7f86a4ff9640 1 -- 172.21.15.99:0/45462964 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f86a003f070 con 0x7f86a81f8b10 2024-09-10T11:32:29.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 msgr2=0x7f869406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 0x7f869406e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f86980017e0 tx=0x7f8698007040 comp rx=0 tx=0).stop 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 msgr2=0x7f86a81f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e290 tx=0x7f86a0009750 comp rx=0 tx=0).stop 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 shutdown_connections 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f869406be30 0x7f869406e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86a81f8b10 0x7f86a81f8fa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86a815e3d0 0x7f86a81f85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 --2- 172.21.15.99:0/45462964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86a815dff0 0x7f86a81e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:29.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 >> 172.21.15.99:0/45462964 conn(0x7f86a80b3630 msgr2=0x7f86a8147be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:29.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 shutdown_connections 2024-09-10T11:32:29.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:29.930+0000 7f86ae65e640 1 -- 172.21.15.99:0/45462964 wait complete. 2024-09-10T11:32:29.944 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:32:29.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:32:30.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.064+0000 7f0025fe8640 1 Processor -- start 2024-09-10T11:32:30.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.064+0000 7f0025fe8640 1 -- start start 2024-09-10T11:32:30.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f0025fe8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f002006e4d0 0x7f0020167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f0025fe8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00200ceb10 con 0x7f00200d1180 2024-09-10T11:32:30.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f0025fe8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00200b3a90 con 0x7f002015c7b0 2024-09-10T11:32:30.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f0025fe8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f00200b3840 con 0x7f002006e4d0 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001effd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f00200d1180 0x7f002006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58634/0 (socket says 172.21.15.99:58634) 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001effd640 1 -- 172.21.15.99:0/120259750 learned_addr learned my addr 172.21.15.99:0/120259750 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001ffff640 1 --2- 172.21.15.99:0/120259750 >> v2:172.21.15.196:3300/0 conn(0x7f002006e4d0 0x7f0020167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 128445706 0 0) 0x7f0008001110 con 0x7f00200d1180 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f000c003520 con 0x7f00200d1180 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2533669088 0 0) 0x7f000c003520 con 0x7f00200d1180 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 >> v2:172.21.15.196:3300/0 conn(0x7f002006e4d0 msgr2=0x7f0020167610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 --2- 172.21.15.99:0/120259750 >> v2:172.21.15.196:3300/0 conn(0x7f002006e4d0 0x7f0020167610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 >> v1:172.21.15.196:6789/0 conn(0x7f002015c7b0 legacy=0x7f00200d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00200b3d70 con 0x7f00200d1180 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f0008001840 con 0x7f00200d1180 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 >> v1:172.21.15.99:6789/0 conn(0x7f00200d1180 legacy=0x7f002006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00200b3840 con 0x7f000c0051c0 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f000c00c060 con 0x7f000c0039b0 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f000c00c260 con 0x7f000c004800 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.065+0000 7f001ffff640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001f7fe640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001effd640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 -- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c004800 msgr2=0x7f000c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c004800 0x7f000c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 -- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c0039b0 msgr2=0x7f000c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 -- 172.21.15.99:0/120259750 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c003520 con 0x7f000c0051c0 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001f7fe640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c0039b0 0x7f000c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 -- 172.21.15.99:0/120259750 reap_dead start 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001ffff640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 0x7f000c00b9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0018009fc0 tx=0x7f000c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd223c96b0e997d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.066+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018035020 con 0x7f000c0051c0 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0018003bd0 con 0x7f000c0051c0 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f001e7fc640 1 -- 172.21.15.99:0/120259750 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0018009c10 con 0x7f000c0051c0 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 -- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 msgr2=0x7f000c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 0x7f000c00b9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0018009fc0 tx=0x7f000c005830 comp rx=0 tx=0).stop 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 -- 172.21.15.99:0/120259750 shutdown_connections 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 --2- 172.21.15.99:0/120259750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f000c0051c0 0x7f000c00b9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 -- 172.21.15.99:0/120259750 >> 172.21.15.99:0/120259750 conn(0x7f002006bc10 msgr2=0x7f002006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:30.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 -- 172.21.15.99:0/120259750 shutdown_connections 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.067+0000 7f0025fe8640 1 -- 172.21.15.99:0/120259750 wait complete. 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.068+0000 7f0025fe8640 1 Processor -- start 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.068+0000 7f0025fe8640 1 -- start start 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 0x7f00200aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 0x7f002015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f00200b4610 con 0x7f00200aaea0 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f00200b4330 con 0x7f00201f25c0 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f0025fe8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f002015cb90 con 0x7f000c0051c0 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f001effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f001effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45520/0 (socket says 172.21.15.99:45520) 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f001effd640 1 -- 172.21.15.99:0/1251499133 learned_addr learned my addr 172.21.15.99:0/1251499133 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f001f7fe640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 0x7f00200aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.069+0000 7f001ffff640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 0x7f002015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 -- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 msgr2=0x7f00200aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 0x7f00200aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 -- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 msgr2=0x7f002015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 0x7f002015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 -- 172.21.15.99:0/1251499133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020158740 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001f7fe640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 0x7f00200aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001ffff640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 0x7f002015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f001effd640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f000800e730 tx=0x7f000800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000800c6c0 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0008018070 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00200abb80 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000800caa0 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.070+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00200ac140 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.071+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00200d0a70 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.071+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f000800cc40 con 0x7f00200aaea0 2024-09-10T11:32:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.072+0000 7f0017fff640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 0x7f000006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.072+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f000808ac50 con 0x7f00200aaea0 2024-09-10T11:32:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.072+0000 7f001f7fe640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 0x7f000006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.072+0000 7f001f7fe640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 0x7f000006e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0010007fd0 tx=0x7f0010009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.076+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00200ceb10 con 0x7f00200aaea0 2024-09-10T11:32:30.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.275+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0020159260 con 0x7f00200aaea0 2024-09-10T11:32:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.929+0000 7f0017fff640 1 -- 172.21.15.99:0/1251499133 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7f0008059600 con 0x7f00200aaea0 2024-09-10T11:32:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 msgr2=0x7f000006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 0x7f000006e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0010007fd0 tx=0x7f0010009040 comp rx=0 tx=0).stop 2024-09-10T11:32:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 msgr2=0x7f0020158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f000800e730 tx=0x7f000800ebf0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 shutdown_connections 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f000006be30 0x7f000006e2f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00201f25c0 0x7f002015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00200aaea0 0x7f0020158200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 --2- 172.21.15.99:0/1251499133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f000c0051c0 0x7f00200aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 >> 172.21.15.99:0/1251499133 conn(0x7f002006bc10 msgr2=0x7f00200b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 shutdown_connections 2024-09-10T11:32:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:30.932+0000 7f0025fe8640 1 -- 172.21.15.99:0/1251499133 wait complete. 2024-09-10T11:32:30.947 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:32:30.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:30.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-10T11:32:31.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.050+0000 7f96bf3c4640 1 Processor -- start 2024-09-10T11:32:31.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.051+0000 7f96bf3c4640 1 -- start start 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.051+0000 7f96bf3c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96b815eb80 0x7f96b80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.051+0000 7f96bf3c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 0x7f96b80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.051+0000 7f96bf3c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b815ff70 con 0x7f96b81626c0 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bf3c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f96b80d1620 con 0x7f96b815eb80 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bf3c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f96b8162bc0 con 0x7f96b80d5010 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96b815eb80 0x7f96b80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96b815eb80 0x7f96b80d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45530/0 (socket says 172.21.15.99:45530) 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 -- 172.21.15.99:0/3607935700 learned_addr learned my addr 172.21.15.99:0/3607935700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd961640 1 --2- 172.21.15.99:0/3607935700 >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 0x7f96b80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 -- 172.21.15.99:0/3607935700 >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 msgr2=0x7f96b80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 --2- 172.21.15.99:0/3607935700 >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 0x7f96b80d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 -- 172.21.15.99:0/3607935700 >> v1:172.21.15.196:6789/0 conn(0x7f96b81626c0 legacy=0x7f96b80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd160640 1 -- 172.21.15.99:0/3607935700 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8162ea0 con 0x7f96b815eb80 2024-09-10T11:32:31.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.052+0000 7f96bd961640 1 --2- 172.21.15.99:0/3607935700 >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 0x7f96b80d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:31.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bd160640 1 --2- 172.21.15.99:0/3607935700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b80d29b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009870 tx=0x7f96b402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c323d32e3d6eedf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96abfff640 1 -- 172.21.15.99:0/3607935700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4037070 con 0x7f96b815eb80 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96abfff640 1 -- 172.21.15.99:0/3607935700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4032440 con 0x7f96b815eb80 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96abfff640 1 -- 172.21.15.99:0/3607935700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4032810 con 0x7f96b815eb80 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 -- 172.21.15.99:0/3607935700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 msgr2=0x7f96b80d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/3607935700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b80d29b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009870 tx=0x7f96b402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 -- 172.21.15.99:0/3607935700 shutdown_connections 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/3607935700 >> v2:172.21.15.196:3300/0 conn(0x7f96b80d5010 0x7f96b80d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/3607935700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b80d29b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.053+0000 7f96bf3c4640 1 -- 172.21.15.99:0/3607935700 >> 172.21.15.99:0/3607935700 conn(0x7f96b808e680 msgr2=0x7f96b809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.054+0000 7f96bf3c4640 1 -- 172.21.15.99:0/3607935700 shutdown_connections 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.054+0000 7f96bf3c4640 1 -- 172.21.15.99:0/3607935700 wait complete. 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 Processor -- start 2024-09-10T11:32:31.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 -- start start 2024-09-10T11:32:31.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b80d5010 0x7f96b80b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b81626c0 0x7f96b81e9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f96b80d2ef0 con 0x7f96b815eb80 2024-09-10T11:32:31.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f96b8163460 con 0x7f96b81626c0 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.055+0000 7f96bf3c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f96b80d3070 con 0x7f96b80d5010 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45544/0 (socket says 172.21.15.99:45544) 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 -- 172.21.15.99:0/4273919550 learned_addr learned my addr 172.21.15.99:0/4273919550 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bd961640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b81626c0 0x7f96b81e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bd160640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b80d5010 0x7f96b80b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 -- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b80d5010 msgr2=0x7f96b80b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b80d5010 0x7f96b80b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 -- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b81626c0 msgr2=0x7f96b81e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b81626c0 0x7f96b81e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 -- 172.21.15.99:0/4273919550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8159ee0 con 0x7f96b815eb80 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bc95f640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b3c0 tx=0x7f96ac00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac011830 con 0x7f96b815eb80 2024-09-10T11:32:31.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac016070 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b815cc70 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac011c50 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.056+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b8160640 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.057+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b80d29b0 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.058+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96ac01b460 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.058+0000 7f96a9ffb640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 0x7f969c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.058+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f96ac08a7e0 con 0x7f96b815eb80 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.058+0000 7f96bd160640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 0x7f969c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.059+0000 7f96bd160640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 0x7f969c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96b4034000 tx=0x7f96b402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.063+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f96ac020020 con 0x7f96b815eb80 2024-09-10T11:32:31.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.261+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b80b5a20 con 0x7f96b815eb80 2024-09-10T11:32:31.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.262+0000 7f96a9ffb640 1 -- 172.21.15.99:0/4273919550 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f96ac059220 con 0x7f96b815eb80 2024-09-10T11:32:31.263 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:31.263 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:32:31.263 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 msgr2=0x7f969c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 0x7f969c06e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f96b4034000 tx=0x7f96b402e000 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 msgr2=0x7f96b81599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00b3c0 tx=0x7f96ac00b880 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 shutdown_connections 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f969c06be30 0x7f969c06e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b81626c0 0x7f96b81e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96b815eb80 0x7f96b81599a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 --2- 172.21.15.99:0/4273919550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b80d5010 0x7f96b80b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 >> 172.21.15.99:0/4273919550 conn(0x7f96b808e680 msgr2=0x7f96b815d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 shutdown_connections 2024-09-10T11:32:31.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.264+0000 7f96bf3c4640 1 -- 172.21.15.99:0/4273919550 wait complete. 2024-09-10T11:32:31.276 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:32:31.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.374+0000 7fbc36d3b640 1 Processor -- start 2024-09-10T11:32:31.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.374+0000 7fbc36d3b640 1 -- start start 2024-09-10T11:32:31.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc36d3b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbc300d1200 0x7fbc30073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc36d3b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc30152e60 con 0x7fbc3015c770 2024-09-10T11:32:31.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc36d3b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc300d15e0 con 0x7fbc3006e5d0 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc36d3b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbc30072680 con 0x7fbc300d1200 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc34ad7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbc300d1200 0x7fbc30073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc352d8640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbc3015c770 0x7fbc30159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:58648/0 (socket says 172.21.15.99:58648) 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc352d8640 1 -- 172.21.15.99:0/1863503031 learned_addr learned my addr 172.21.15.99:0/1863503031 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.375+0000 7fbc34ad7640 1 -- 172.21.15.99:0/1863503031 >> v1:172.21.15.196:6789/0 conn(0x7fbc3006e5d0 legacy=0x7fbc3015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc2f7fe640 1 -- 172.21.15.99:0/1863503031 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1650875770 0 0) 0x7fbc30152e60 con 0x7fbc3015c770 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc34ad7640 1 -- 172.21.15.99:0/1863503031 >> v1:172.21.15.99:6789/0 conn(0x7fbc3015c770 legacy=0x7fbc30159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc34ad7640 1 -- 172.21.15.99:0/1863503031 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc30072960 con 0x7fbc300d1200 2024-09-10T11:32:31.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc34ad7640 1 --2- 172.21.15.99:0/1863503031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc300d1200 0x7fbc30073440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097a0 tx=0x7fbc2402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f789bee08a81de71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc2f7fe640 1 -- 172.21.15.99:0/1863503031 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24037070 con 0x7fbc300d1200 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc2f7fe640 1 -- 172.21.15.99:0/1863503031 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2402ae50 con 0x7fbc300d1200 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.376+0000 7fbc2f7fe640 1 -- 172.21.15.99:0/1863503031 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24032660 con 0x7fbc300d1200 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 -- 172.21.15.99:0/1863503031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc300d1200 msgr2=0x7fbc30073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/1863503031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc300d1200 0x7fbc30073440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097a0 tx=0x7fbc2402a140 comp rx=0 tx=0).stop 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 -- 172.21.15.99:0/1863503031 shutdown_connections 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/1863503031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc300d1200 0x7fbc30073440 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 -- 172.21.15.99:0/1863503031 >> 172.21.15.99:0/1863503031 conn(0x7fbc3006bc10 msgr2=0x7fbc3006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 -- 172.21.15.99:0/1863503031 shutdown_connections 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.377+0000 7fbc36d3b640 1 -- 172.21.15.99:0/1863503031 wait complete. 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.378+0000 7fbc36d3b640 1 Processor -- start 2024-09-10T11:32:31.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.378+0000 7fbc36d3b640 1 -- start start 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 0x7fbc30143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 0x7fbc301e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3015cd90 con 0x7fbc3006e5d0 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30072f20 con 0x7fbc3015c770 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc36d3b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3006ea40 con 0x7fbc300d1200 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45564/0 (socket says 172.21.15.99:45564) 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 -- 172.21.15.99:0/3375203231 learned_addr learned my addr 172.21.15.99:0/3375203231 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc352d8640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 0x7fbc301e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc2ffff640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 0x7fbc30143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 -- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 msgr2=0x7fbc30143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 0x7fbc30143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 -- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 msgr2=0x7fbc301e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 0x7fbc301e5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 -- 172.21.15.99:0/3375203231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc30143740 con 0x7fbc3006e5d0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc352d8640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 0x7fbc301e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc2ffff640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 0x7fbc30143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.379+0000 7fbc34ad7640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097a0 tx=0x7fbc240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.380+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24037040 con 0x7fbc3006e5d0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.380+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc24004030 con 0x7fbc3006e5d0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.380+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc301e53c0 con 0x7fbc3006e5d0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.380+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24006e10 con 0x7fbc3006e5d0 2024-09-10T11:32:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.380+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc301e5660 con 0x7fbc3006e5d0 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.381+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3006eda0 con 0x7fbc3006e5d0 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.381+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc2403d650 con 0x7fbc3006e5d0 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.381+0000 7fbc2d7fa640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 0x7fbc1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.382+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fbc240ab650 con 0x7fbc3006e5d0 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.382+0000 7fbc2ffff640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 0x7fbc1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.382+0000 7fbc2ffff640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 0x7fbc1406e320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc3015d560 tx=0x7fbc1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.386+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc30143740 con 0x7fbc3006e5d0 2024-09-10T11:32:31.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.552+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc300b4510 con 0x7fbc3006e5d0 2024-09-10T11:32:31.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.553+0000 7fbc2d7fa640 1 -- 172.21.15.99:0/3375203231 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16725 (secure 0 0 0) 0x7fbc24042020 con 0x7fbc3006e5d0 2024-09-10T11:32:31.555 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:31.555 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":66,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:32:29.905800+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:32:26.850541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T11:32:27.213215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:32:31.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 msgr2=0x7fbc1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 0x7fbc1406e320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc3015d560 tx=0x7fbc1c009040 comp rx=0 tx=0).stop 2024-09-10T11:32:31.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 msgr2=0x7fbc301e77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbc240097a0 tx=0x7fbc240046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 shutdown_connections 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbc1406be60 0x7fbc1406e320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc3015c770 0x7fbc301e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc300d1200 0x7fbc30143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 --2- 172.21.15.99:0/3375203231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc3006e5d0 0x7fbc301e77e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 >> 172.21.15.99:0/3375203231 conn(0x7fbc3006bc10 msgr2=0x7fbc3015d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 shutdown_connections 2024-09-10T11:32:31.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.555+0000 7fbc36d3b640 1 -- 172.21.15.99:0/3375203231 wait complete. 2024-09-10T11:32:31.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:32:31.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:32:31.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.666+0000 7ff8580f8640 1 Processor -- start 2024-09-10T11:32:31.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.666+0000 7ff8580f8640 1 -- start start 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff8580f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff85015c7b0 0x7ff8500d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff8580f8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8500d0fb0 0x7ff85006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff8580f8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8500ceb10 con 0x7ff85006ee30 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff8580f8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff8500d1420 con 0x7ff8500d0fb0 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff8580f8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff8500b3840 con 0x7ff85015c7b0 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8500d0fb0 0x7ff85006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff8500d0fb0 0x7ff85006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45576/0 (socket says 172.21.15.99:45576) 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 -- 172.21.15.99:0/1635477563 learned_addr learned my addr 172.21.15.99:0/1635477563 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855e94640 1 --2- 172.21.15.99:0/1635477563 >> v2:172.21.15.196:3300/0 conn(0x7ff85015c7b0 0x7ff8500d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 -- 172.21.15.99:0/1635477563 >> v2:172.21.15.196:3300/0 conn(0x7ff85015c7b0 msgr2=0x7ff8500d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 --2- 172.21.15.99:0/1635477563 >> v2:172.21.15.196:3300/0 conn(0x7ff85015c7b0 0x7ff8500d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 -- 172.21.15.99:0/1635477563 >> v1:172.21.15.196:6789/0 conn(0x7ff85006ee30 legacy=0x7ff8501604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.667+0000 7ff855693640 1 -- 172.21.15.99:0/1635477563 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500b3a90 con 0x7ff8500d0fb0 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.668+0000 7ff855693640 1 --2- 172.21.15.99:0/1635477563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8500d0fb0 0x7ff85006e8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff8500cd7c0 tx=0x7ff84002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe3f7693f83a515 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.668+0000 7ff854e92640 1 -- 172.21.15.99:0/1635477563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840037070 con 0x7ff8500d0fb0 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.668+0000 7ff854e92640 1 -- 172.21.15.99:0/1635477563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff840032440 con 0x7ff8500d0fb0 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.668+0000 7ff854e92640 1 -- 172.21.15.99:0/1635477563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840032810 con 0x7ff8500d0fb0 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 -- 172.21.15.99:0/1635477563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8500d0fb0 msgr2=0x7ff85006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 --2- 172.21.15.99:0/1635477563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8500d0fb0 0x7ff85006e8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff8500cd7c0 tx=0x7ff84002a270 comp rx=0 tx=0).stop 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 -- 172.21.15.99:0/1635477563 shutdown_connections 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 --2- 172.21.15.99:0/1635477563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff8500d0fb0 0x7ff85006e8f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 --2- 172.21.15.99:0/1635477563 >> v2:172.21.15.196:3300/0 conn(0x7ff85015c7b0 0x7ff8500d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 -- 172.21.15.99:0/1635477563 >> 172.21.15.99:0/1635477563 conn(0x7ff85006bc10 msgr2=0x7ff85006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 -- 172.21.15.99:0/1635477563 shutdown_connections 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.669+0000 7ff8580f8640 1 -- 172.21.15.99:0/1635477563 wait complete. 2024-09-10T11:32:31.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.670+0000 7ff8580f8640 1 Processor -- start 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.670+0000 7ff8580f8640 1 -- start start 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8500d0fb0 0x7ff850152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff85015c7b0 0x7ff8500b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500b4050 con 0x7ff85006ee30 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff85006f260 con 0x7ff85015c7b0 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500b4330 con 0x7ff8500d0fb0 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45586/0 (socket says 172.21.15.99:45586) 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 -- 172.21.15.99:0/3530732631 learned_addr learned my addr 172.21.15.99:0/3530732631 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855693640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8500d0fb0 0x7ff850152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff856695640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff85015c7b0 0x7ff8500b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 -- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8500d0fb0 msgr2=0x7ff850152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8500d0fb0 0x7ff850152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 -- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff85015c7b0 msgr2=0x7ff8500b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff85015c7b0 0x7ff8500b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 -- 172.21.15.99:0/3530732631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8501f2a00 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff855e94640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e870 tx=0x7ff84c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c017070 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84c00c940 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00cd30 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.671+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff850152f40 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.672+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501f2f90 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.672+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85015cb90 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.673+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff84c01a070 con 0x7ff85006ee30 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.674+0000 7ff8467fc640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 0x7ff83406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.674+0000 7ff855693640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 0x7ff83406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.674+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7ff84c08abd0 con 0x7ff85006ee30 2024-09-10T11:32:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.674+0000 7ff855693640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 0x7ff83406e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff840034000 tx=0x7ff84002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.678+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff84c058fc0 con 0x7ff85006ee30 2024-09-10T11:32:31.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.838+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff85014e4b0 con 0x7ff83406be60 2024-09-10T11:32:31.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.843+0000 7ff8467fc640 1 -- 172.21.15.99:0/3530732631 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237410 (secure 0 0 0) 0x7ff85014e4b0 con 0x7ff83406be60 2024-09-10T11:32:31.846 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:32:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 msgr2=0x7ff83406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 0x7ff83406e320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff840034000 tx=0x7ff84002e000 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 msgr2=0x7ff8501f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e870 tx=0x7ff84c00ed30 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 shutdown_connections 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff83406be60 0x7ff83406e320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff85015c7b0 0x7ff8500b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8500d0fb0 0x7ff850152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 --2- 172.21.15.99:0/3530732631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff85006ee30 0x7ff8501f3790 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 >> 172.21.15.99:0/3530732631 conn(0x7ff85006bc10 msgr2=0x7ff8501593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 shutdown_connections 2024-09-10T11:32:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.846+0000 7ff8580f8640 1 -- 172.21.15.99:0/3530732631 wait complete. 2024-09-10T11:32:31.858 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T11:32:30.933394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3590,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558909,"statfs":{"total":1600000000,"available":1596323182,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035880"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899242+0000","last_change":"2024-09-10T11:32:26.899242+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899242+0000","last_undegraded":"2024-09-10T11:32:26.899242+0000","last_fullsized":"2024-09-10T11:32:26.899242+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900897+0000","last_change":"2024-09-10T11:32:26.900897+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900897+0000","last_undegraded":"2024-09-10T11:32:26.900897+0000","last_fullsized":"2024-09-10T11:32:26.900897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901152+0000","last_change":"2024-09-10T11:32:26.901152+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901152+0000","last_undegraded":"2024-09-10T11:32:26.901152+0000","last_fullsized":"2024-09-10T11:32:26.901152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899560+0000","last_change":"2024-09-10T11:32:26.899560+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899560+0000","last_undegraded":"2024-09-10T11:32:26.899560+0000","last_fullsized":"2024-09-10T11:32:26.899560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898894+0000","last_change":"2024-09-10T11:32:26.898894+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898894+0000","last_undegraded":"2024-09-10T11:32:26.898894+0000","last_fullsized":"2024-09-10T11:32:26.898894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900904+0000","last_change":"2024-09-10T11:32:26.900904+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900904+0000","last_undegraded":"2024-09-10T11:32:26.900904+0000","last_fullsized":"2024-09-10T11:32:26.900904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900524+0000","last_change":"2024-09-10T11:32:26.900524+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900524+0000","last_undegraded":"2024-09-10T11:32:26.900524+0000","last_fullsized":"2024-09-10T11:32:26.900524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900363+0000","last_change":"2024-09-10T11:32:26.900363+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900363+0000","last_undegraded":"2024-09-10T11:32:26.900363+0000","last_fullsized":"2024-09-10T11:32:26.900363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900797+0000","last_change":"2024-09-10T11:32:26.900797+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900797+0000","last_undegraded":"2024-09-10T11:32:26.900797+0000","last_fullsized":"2024-09-10T11:32:26.900797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902120+0000","last_change":"2024-09-10T11:32:26.902120+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902120+0000","last_undegraded":"2024-09-10T11:32:26.902120+0000","last_fullsized":"2024-09-10T11:32:26.902120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900084+0000","last_change":"2024-09-10T11:32:26.900084+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900084+0000","last_undegraded":"2024-09-10T11:32:26.900084+0000","last_fullsized":"2024-09-10T11:32:26.900084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899972+0000","last_change":"2024-09-10T11:32:26.899972+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899972+0000","last_undegraded":"2024-09-10T11:32:26.899972+0000","last_fullsized":"2024-09-10T11:32:26.899972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902158+0000","last_change":"2024-09-10T11:32:26.902158+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902158+0000","last_undegraded":"2024-09-10T11:32:26.902158+0000","last_fullsized":"2024-09-10T11:32:26.902158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899246+0000","last_change":"2024-09-10T11:32:26.899246+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899246+0000","last_undegraded":"2024-09-10T11:32:26.899246+0000","last_fullsized":"2024-09-10T11:32:26.899246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899236+0000","last_change":"2024-09-10T11:32:26.899236+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899236+0000","last_undegraded":"2024-09-10T11:32:26.899236+0000","last_fullsized":"2024-09-10T11:32:26.899236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899253+0000","last_change":"2024-09-10T11:32:26.899253+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899253+0000","last_undegraded":"2024-09-10T11:32:26.899253+0000","last_fullsized":"2024-09-10T11:32:26.899253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900572+0000","last_change":"2024-09-10T11:32:26.900572+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900572+0000","last_undegraded":"2024-09-10T11:32:26.900572+0000","last_fullsized":"2024-09-10T11:32:26.900572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901741+0000","last_change":"2024-09-10T11:32:26.901741+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901741+0000","last_undegraded":"2024-09-10T11:32:26.901741+0000","last_fullsized":"2024-09-10T11:32:26.901741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898535+0000","last_change":"2024-09-10T11:32:26.898535+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898535+0000","last_undegraded":"2024-09-10T11:32:26.898535+0000","last_fullsized":"2024-09-10T11:32:26.898535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901742+0000","last_change":"2024-09-10T11:32:26.901742+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901742+0000","last_undegraded":"2024-09-10T11:32:26.901742+0000","last_fullsized":"2024-09-10T11:32:26.901742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898838+0000","last_change":"2024-09-10T11:32:26.898838+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898838+0000","last_undegraded":"2024-09-10T11:32:26.898838+0000","last_fullsized":"2024-09-10T11:32:26.898838+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902491+0000","last_change":"2024-09-10T11:32:26.902491+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902491+0000","last_undegraded":"2024-09-10T11:32:26.902491+0000","last_fullsized":"2024-09-10T11:32:26.902491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900352+0000","last_change":"2024-09-10T11:32:26.900352+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900352+0000","last_undegraded":"2024-09-10T11:32:26.900352+0000","last_fullsized":"2024-09-10T11:32:26.900352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900182+0000","last_change":"2024-09-10T11:32:26.900182+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900182+0000","last_undegraded":"2024-09-10T11:32:26.900182+0000","last_fullsized":"2024-09-10T11:32:26.900182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900591+0000","last_change":"2024-09-10T11:32:26.900591+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900591+0000","last_undegraded":"2024-09-10T11:32:26.900591+0000","last_fullsized":"2024-09-10T11:32:26.900591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900164+0000","last_change":"2024-09-10T11:32:26.900164+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900164+0000","last_undegraded":"2024-09-10T11:32:26.900164+0000","last_fullsized":"2024-09-10T11:32:26.900164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898512+0000","last_change":"2024-09-10T11:32:26.898512+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898512+0000","last_undegraded":"2024-09-10T11:32:26.898512+0000","last_fullsized":"2024-09-10T11:32:26.898512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900150+0000","last_change":"2024-09-10T11:32:26.900150+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900150+0000","last_undegraded":"2024-09-10T11:32:26.900150+0000","last_fullsized":"2024-09-10T11:32:26.900150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898555+0000","last_change":"2024-09-10T11:32:26.898555+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898555+0000","last_undegraded":"2024-09-10T11:32:26.898555+0000","last_fullsized":"2024-09-10T11:32:26.898555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900181+0000","last_change":"2024-09-10T11:32:26.900181+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900181+0000","last_undegraded":"2024-09-10T11:32:26.900181+0000","last_fullsized":"2024-09-10T11:32:26.900181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901768+0000","last_change":"2024-09-10T11:32:26.901768+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901768+0000","last_undegraded":"2024-09-10T11:32:26.901768+0000","last_fullsized":"2024-09-10T11:32:26.901768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900018+0000","last_change":"2024-09-10T11:32:26.900018+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900018+0000","last_undegraded":"2024-09-10T11:32:26.900018+0000","last_fullsized":"2024-09-10T11:32:26.900018+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":255,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T11:32:26.901380+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:32:26.901380+0000","last_peered":"2024-09-10T11:32:26.901380+0000","last_clean":"2024-09-10T11:32:26.901380+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:32:26.901380+0000","last_undegraded":"2024-09-10T11:32:26.901380+0000","last_fullsized":"2024-09-10T11:32:26.901380+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:31.673361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:32:31.862 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:32:31.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.995+0000 7f2ce3931640 1 Processor -- start 2024-09-10T11:32:31.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 -- start start 2024-09-10T11:32:31.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 0x7f2cdc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2cdc0d52d0 con 0x7f2cdc0d4e40 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2cdc0d1620 con 0x7f2cdc1626c0 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.996+0000 7f2ce3931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2cdc162bc0 con 0x7f2cdc15eb80 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45592/0 (socket says 172.21.15.99:45592) 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 -- 172.21.15.99:0/2758560858 learned_addr learned my addr 172.21.15.99:0/2758560858 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce16cd640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce0ecc640 1 --2- 172.21.15.99:0/2758560858 >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 0x7f2cdc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 -- 172.21.15.99:0/2758560858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 msgr2=0x7f2cdc0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 -- 172.21.15.99:0/2758560858 >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 msgr2=0x7f2cdc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:31.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 --2- 172.21.15.99:0/2758560858 >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 0x7f2cdc0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 -- 172.21.15.99:0/2758560858 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc162ea0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce16cd640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce0ecc640 1 --2- 172.21.15.99:0/2758560858 >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 0x7f2cdc0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.997+0000 7f2ce1ece640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc08db70 tx=0x7f2cd402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8828dd14c4f3bca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2cd3fff640 1 -- 172.21.15.99:0/2758560858 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4036070 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2cd3fff640 1 -- 172.21.15.99:0/2758560858 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd4031600 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2cd3fff640 1 -- 172.21.15.99:0/2758560858 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40319d0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 -- 172.21.15.99:0/2758560858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 msgr2=0x7f2cdc0d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc08db70 tx=0x7f2cd402a610 comp rx=0 tx=0).stop 2024-09-10T11:32:32.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 -- 172.21.15.99:0/2758560858 shutdown_connections 2024-09-10T11:32:32.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0d1b20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 --2- 172.21.15.99:0/2758560858 >> v2:172.21.15.196:3300/0 conn(0x7f2cdc1626c0 0x7f2cdc0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 --2- 172.21.15.99:0/2758560858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 -- 172.21.15.99:0/2758560858 >> 172.21.15.99:0/2758560858 conn(0x7f2cdc08e680 msgr2=0x7f2cdc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 -- 172.21.15.99:0/2758560858 shutdown_connections 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.998+0000 7f2ce3931640 1 -- 172.21.15.99:0/2758560858 wait complete. 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.999+0000 7f2ce3931640 1 Processor -- start 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:31.999+0000 7f2ce3931640 1 -- start start 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cdc1626c0 0x7f2cdc0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc0d2ef0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc163460 con 0x7f2cdc1626c0 2024-09-10T11:32:32.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce3931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2cdc0d3070 con 0x7f2cdc15eb80 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce16cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce16cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45602/0 (socket says 172.21.15.99:45602) 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce16cd640 1 -- 172.21.15.99:0/1199358168 learned_addr learned my addr 172.21.15.99:0/1199358168 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce0ecc640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.000+0000 7f2ce1ece640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cdc1626c0 0x7f2cdc0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 -- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 msgr2=0x7f2cdc0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 -- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cdc1626c0 msgr2=0x7f2cdc0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cdc1626c0 0x7f2cdc0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 -- 172.21.15.99:0/1199358168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc0b7f10 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce16cd640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c7f0 tx=0x7f2cd800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8015070 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8007820 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc0b81f0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8007bd0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.001+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc0b87b0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.002+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc0d29b0 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.002+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2cd8018070 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.003+0000 7f2cd1ffb640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.003+0000 7f2ce0ecc640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.003+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f2cd808af20 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.004+0000 7f2ce0ecc640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 0x7f2cc006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00aa40 tx=0x7f2ccc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.007+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cd8059280 con 0x7f2cdc0d4e40 2024-09-10T11:32:32.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.168+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cdc09e040 con 0x7f2cc006be30 2024-09-10T11:32:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.173+0000 7f2cd1ffb640 1 -- 172.21.15.99:0/1199358168 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237410 (secure 0 0 0) 0x7f2cdc09e040 con 0x7f2cc006be30 2024-09-10T11:32:32.176 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 msgr2=0x7f2cc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 0x7f2cc006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00aa40 tx=0x7f2ccc008040 comp rx=0 tx=0).stop 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 msgr2=0x7f2cdc0b8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2cd800c7f0 tx=0x7f2cd800ccb0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 shutdown_connections 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2cc006be30 0x7f2cc006e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cdc1626c0 0x7f2cdc0b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cdc15eb80 0x7f2cdc0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 --2- 172.21.15.99:0/1199358168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cdc0d4e40 0x7f2cdc0b8fb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 >> 172.21.15.99:0/1199358168 conn(0x7f2cdc08e680 msgr2=0x7f2cdc0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 shutdown_connections 2024-09-10T11:32:32.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.176+0000 7f2ce3931640 1 -- 172.21.15.99:0/1199358168 wait complete. 2024-09-10T11:32:32.188 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T11:32:30.933394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3590,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558909,"statfs":{"total":1600000000,"available":1596323182,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035880"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899242+0000","last_change":"2024-09-10T11:32:26.899242+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899242+0000","last_undegraded":"2024-09-10T11:32:26.899242+0000","last_fullsized":"2024-09-10T11:32:26.899242+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900897+0000","last_change":"2024-09-10T11:32:26.900897+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900897+0000","last_undegraded":"2024-09-10T11:32:26.900897+0000","last_fullsized":"2024-09-10T11:32:26.900897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901152+0000","last_change":"2024-09-10T11:32:26.901152+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901152+0000","last_undegraded":"2024-09-10T11:32:26.901152+0000","last_fullsized":"2024-09-10T11:32:26.901152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899560+0000","last_change":"2024-09-10T11:32:26.899560+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899560+0000","last_undegraded":"2024-09-10T11:32:26.899560+0000","last_fullsized":"2024-09-10T11:32:26.899560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898894+0000","last_change":"2024-09-10T11:32:26.898894+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898894+0000","last_undegraded":"2024-09-10T11:32:26.898894+0000","last_fullsized":"2024-09-10T11:32:26.898894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900904+0000","last_change":"2024-09-10T11:32:26.900904+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900904+0000","last_undegraded":"2024-09-10T11:32:26.900904+0000","last_fullsized":"2024-09-10T11:32:26.900904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900524+0000","last_change":"2024-09-10T11:32:26.900524+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900524+0000","last_undegraded":"2024-09-10T11:32:26.900524+0000","last_fullsized":"2024-09-10T11:32:26.900524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900363+0000","last_change":"2024-09-10T11:32:26.900363+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900363+0000","last_undegraded":"2024-09-10T11:32:26.900363+0000","last_fullsized":"2024-09-10T11:32:26.900363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900797+0000","last_change":"2024-09-10T11:32:26.900797+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900797+0000","last_undegraded":"2024-09-10T11:32:26.900797+0000","last_fullsized":"2024-09-10T11:32:26.900797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902120+0000","last_change":"2024-09-10T11:32:26.902120+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902120+0000","last_undegraded":"2024-09-10T11:32:26.902120+0000","last_fullsized":"2024-09-10T11:32:26.902120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900084+0000","last_change":"2024-09-10T11:32:26.900084+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900084+0000","last_undegraded":"2024-09-10T11:32:26.900084+0000","last_fullsized":"2024-09-10T11:32:26.900084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899972+0000","last_change":"2024-09-10T11:32:26.899972+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899972+0000","last_undegraded":"2024-09-10T11:32:26.899972+0000","last_fullsized":"2024-09-10T11:32:26.899972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902158+0000","last_change":"2024-09-10T11:32:26.902158+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902158+0000","last_undegraded":"2024-09-10T11:32:26.902158+0000","last_fullsized":"2024-09-10T11:32:26.902158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899246+0000","last_change":"2024-09-10T11:32:26.899246+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899246+0000","last_undegraded":"2024-09-10T11:32:26.899246+0000","last_fullsized":"2024-09-10T11:32:26.899246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899236+0000","last_change":"2024-09-10T11:32:26.899236+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899236+0000","last_undegraded":"2024-09-10T11:32:26.899236+0000","last_fullsized":"2024-09-10T11:32:26.899236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899253+0000","last_change":"2024-09-10T11:32:26.899253+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899253+0000","last_undegraded":"2024-09-10T11:32:26.899253+0000","last_fullsized":"2024-09-10T11:32:26.899253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900572+0000","last_change":"2024-09-10T11:32:26.900572+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900572+0000","last_undegraded":"2024-09-10T11:32:26.900572+0000","last_fullsized":"2024-09-10T11:32:26.900572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901741+0000","last_change":"2024-09-10T11:32:26.901741+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901741+0000","last_undegraded":"2024-09-10T11:32:26.901741+0000","last_fullsized":"2024-09-10T11:32:26.901741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898535+0000","last_change":"2024-09-10T11:32:26.898535+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898535+0000","last_undegraded":"2024-09-10T11:32:26.898535+0000","last_fullsized":"2024-09-10T11:32:26.898535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901742+0000","last_change":"2024-09-10T11:32:26.901742+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901742+0000","last_undegraded":"2024-09-10T11:32:26.901742+0000","last_fullsized":"2024-09-10T11:32:26.901742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898838+0000","last_change":"2024-09-10T11:32:26.898838+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898838+0000","last_undegraded":"2024-09-10T11:32:26.898838+0000","last_fullsized":"2024-09-10T11:32:26.898838+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902491+0000","last_change":"2024-09-10T11:32:26.902491+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902491+0000","last_undegraded":"2024-09-10T11:32:26.902491+0000","last_fullsized":"2024-09-10T11:32:26.902491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900352+0000","last_change":"2024-09-10T11:32:26.900352+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900352+0000","last_undegraded":"2024-09-10T11:32:26.900352+0000","last_fullsized":"2024-09-10T11:32:26.900352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900182+0000","last_change":"2024-09-10T11:32:26.900182+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900182+0000","last_undegraded":"2024-09-10T11:32:26.900182+0000","last_fullsized":"2024-09-10T11:32:26.900182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900591+0000","last_change":"2024-09-10T11:32:26.900591+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900591+0000","last_undegraded":"2024-09-10T11:32:26.900591+0000","last_fullsized":"2024-09-10T11:32:26.900591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900164+0000","last_change":"2024-09-10T11:32:26.900164+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900164+0000","last_undegraded":"2024-09-10T11:32:26.900164+0000","last_fullsized":"2024-09-10T11:32:26.900164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898512+0000","last_change":"2024-09-10T11:32:26.898512+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898512+0000","last_undegraded":"2024-09-10T11:32:26.898512+0000","last_fullsized":"2024-09-10T11:32:26.898512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900150+0000","last_change":"2024-09-10T11:32:26.900150+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900150+0000","last_undegraded":"2024-09-10T11:32:26.900150+0000","last_fullsized":"2024-09-10T11:32:26.900150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898555+0000","last_change":"2024-09-10T11:32:26.898555+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898555+0000","last_undegraded":"2024-09-10T11:32:26.898555+0000","last_fullsized":"2024-09-10T11:32:26.898555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900181+0000","last_change":"2024-09-10T11:32:26.900181+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900181+0000","last_undegraded":"2024-09-10T11:32:26.900181+0000","last_fullsized":"2024-09-10T11:32:26.900181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901768+0000","last_change":"2024-09-10T11:32:26.901768+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901768+0000","last_undegraded":"2024-09-10T11:32:26.901768+0000","last_fullsized":"2024-09-10T11:32:26.901768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900018+0000","last_change":"2024-09-10T11:32:26.900018+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900018+0000","last_undegraded":"2024-09-10T11:32:26.900018+0000","last_fullsized":"2024-09-10T11:32:26.900018+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":255,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T11:32:26.901380+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:32:26.901380+0000","last_peered":"2024-09-10T11:32:26.901380+0000","last_clean":"2024-09-10T11:32:26.901380+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:32:26.901380+0000","last_undegraded":"2024-09-10T11:32:26.901380+0000","last_fullsized":"2024-09-10T11:32:26.901380+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:31.673361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:32:32.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:32:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.326+0000 7f751fad9640 1 Processor -- start 2024-09-10T11:32:32.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.326+0000 7f751fad9640 1 -- start start 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.327+0000 7f751fad9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7518160030 0x7f75180b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.327+0000 7f751fad9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75180d0130 0x7f75180cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.327+0000 7f751fad9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75180ce430 con 0x7f75180d1760 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.327+0000 7f751fad9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f75180ce200 con 0x7f75180d0130 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.327+0000 7f751fad9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f75180ce710 con 0x7f7518160030 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75180d0130 0x7f75180cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751d875640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7518160030 0x7f75180b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75180d0130 0x7f75180cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:59918/0 (socket says 172.21.15.99:59918) 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751d074640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f75180d1760 0x7f75180cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:35200/0 (socket says 172.21.15.99:35200) 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 -- 172.21.15.99:0/2693348163 learned_addr learned my addr 172.21.15.99:0/2693348163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:32.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 -- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7518160030 msgr2=0x7f75180b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 --2- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7518160030 0x7f75180b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 -- 172.21.15.99:0/2693348163 >> v1:172.21.15.196:6789/0 conn(0x7f75180d1760 legacy=0x7f75180cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.328+0000 7f751e076640 1 -- 172.21.15.99:0/2693348163 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75180d1240 con 0x7f75180d0130 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.329+0000 7f751e076640 1 --2- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75180d0130 0x7f75180cdcc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7510009870 tx=0x7f751002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b290f7b9ccae8e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.329+0000 7f751c873640 1 -- 172.21.15.99:0/2693348163 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510037070 con 0x7f75180d0130 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.329+0000 7f751c873640 1 -- 172.21.15.99:0/2693348163 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7510032440 con 0x7f75180d0130 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.329+0000 7f751c873640 1 -- 172.21.15.99:0/2693348163 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510032810 con 0x7f75180d0130 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 -- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75180d0130 msgr2=0x7f75180cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 --2- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75180d0130 0x7f75180cdcc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7510009870 tx=0x7f751002a270 comp rx=0 tx=0).stop 2024-09-10T11:32:32.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 -- 172.21.15.99:0/2693348163 shutdown_connections 2024-09-10T11:32:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 --2- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75180d0130 0x7f75180cdcc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 --2- 172.21.15.99:0/2693348163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7518160030 0x7f75180b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 -- 172.21.15.99:0/2693348163 >> 172.21.15.99:0/2693348163 conn(0x7f75180b4650 msgr2=0x7f75180b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 -- 172.21.15.99:0/2693348163 shutdown_connections 2024-09-10T11:32:32.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.330+0000 7f751fad9640 1 -- 172.21.15.99:0/2693348163 wait complete. 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.331+0000 7f751fad9640 1 Processor -- start 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.331+0000 7f751fad9640 1 -- start start 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75180d1760 0x7f75180c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7518160030 0x7f75180cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7518160590 con 0x7f75180d0130 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7518160410 con 0x7f7518160030 2024-09-10T11:32:32.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7518160710 con 0x7f75180d1760 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45614/0 (socket says 172.21.15.99:45614) 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 -- 172.21.15.99:0/901410574 learned_addr learned my addr 172.21.15.99:0/901410574 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d074640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75180d1760 0x7f75180c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751e076640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7518160030 0x7f75180cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 -- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75180d1760 msgr2=0x7f75180c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75180d1760 0x7f75180c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 -- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7518160030 msgr2=0x7f75180cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7518160030 0x7f75180cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 -- 172.21.15.99:0/901410574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75180c6ae0 con 0x7f75180d0130 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751d875640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f751400efc0 tx=0x7f751400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514018020 con 0x7f75180d0130 2024-09-10T11:32:32.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7514014480 con 0x7f75180d0130 2024-09-10T11:32:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75180c6dc0 con 0x7f75180d0130 2024-09-10T11:32:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514014850 con 0x7f75180d0130 2024-09-10T11:32:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.332+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75180c7380 con 0x7f75180d0130 2024-09-10T11:32:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.333+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75180b38f0 con 0x7f75180d0130 2024-09-10T11:32:32.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.334+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f751401b020 con 0x7f75180d0130 2024-09-10T11:32:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.334+0000 7f750e7fc640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 0x7f74fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.334+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f751408bd10 con 0x7f75180d0130 2024-09-10T11:32:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.334+0000 7f751d074640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 0x7f74fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.335+0000 7f751d074640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 0x7f74fc06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75180b74f0 tx=0x7f7508007c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.338+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7514022020 con 0x7f75180d0130 2024-09-10T11:32:32.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.499+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f751815f040 con 0x7f74fc06be30 2024-09-10T11:32:32.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.505+0000 7f750e7fc640 1 -- 172.21.15.99:0/901410574 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237410 (secure 0 0 0) 0x7f751815f040 con 0x7f74fc06be30 2024-09-10T11:32:32.508 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:32.509 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:32:32.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 msgr2=0x7f74fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 0x7f74fc06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75180b74f0 tx=0x7f7508007c80 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 msgr2=0x7f751815d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f751400efc0 tx=0x7f751400c490 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 shutdown_connections 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f74fc06be30 0x7f74fc06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7518160030 0x7f75180cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75180d1760 0x7f75180c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 --2- 172.21.15.99:0/901410574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75180d0130 0x7f751815d420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 >> 172.21.15.99:0/901410574 conn(0x7f75180b4650 msgr2=0x7f75180b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.507+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 shutdown_connections 2024-09-10T11:32:32.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.508+0000 7f751fad9640 1 -- 172.21.15.99:0/901410574 wait complete. 2024-09-10T11:32:32.519 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-10T11:32:30.933394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3590,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558909,"statfs":{"total":1600000000,"available":1596323182,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.035880"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899242+0000","last_change":"2024-09-10T11:32:26.899242+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899242+0000","last_undegraded":"2024-09-10T11:32:26.899242+0000","last_fullsized":"2024-09-10T11:32:26.899242+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900897+0000","last_change":"2024-09-10T11:32:26.900897+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900897+0000","last_undegraded":"2024-09-10T11:32:26.900897+0000","last_fullsized":"2024-09-10T11:32:26.900897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901152+0000","last_change":"2024-09-10T11:32:26.901152+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901152+0000","last_undegraded":"2024-09-10T11:32:26.901152+0000","last_fullsized":"2024-09-10T11:32:26.901152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899560+0000","last_change":"2024-09-10T11:32:26.899560+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899560+0000","last_undegraded":"2024-09-10T11:32:26.899560+0000","last_fullsized":"2024-09-10T11:32:26.899560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898894+0000","last_change":"2024-09-10T11:32:26.898894+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898894+0000","last_undegraded":"2024-09-10T11:32:26.898894+0000","last_fullsized":"2024-09-10T11:32:26.898894+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900904+0000","last_change":"2024-09-10T11:32:26.900904+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900904+0000","last_undegraded":"2024-09-10T11:32:26.900904+0000","last_fullsized":"2024-09-10T11:32:26.900904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900524+0000","last_change":"2024-09-10T11:32:26.900524+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900524+0000","last_undegraded":"2024-09-10T11:32:26.900524+0000","last_fullsized":"2024-09-10T11:32:26.900524+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900363+0000","last_change":"2024-09-10T11:32:26.900363+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900363+0000","last_undegraded":"2024-09-10T11:32:26.900363+0000","last_fullsized":"2024-09-10T11:32:26.900363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900797+0000","last_change":"2024-09-10T11:32:26.900797+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900797+0000","last_undegraded":"2024-09-10T11:32:26.900797+0000","last_fullsized":"2024-09-10T11:32:26.900797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902120+0000","last_change":"2024-09-10T11:32:26.902120+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902120+0000","last_undegraded":"2024-09-10T11:32:26.902120+0000","last_fullsized":"2024-09-10T11:32:26.902120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900084+0000","last_change":"2024-09-10T11:32:26.900084+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900084+0000","last_undegraded":"2024-09-10T11:32:26.900084+0000","last_fullsized":"2024-09-10T11:32:26.900084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899972+0000","last_change":"2024-09-10T11:32:26.899972+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899972+0000","last_undegraded":"2024-09-10T11:32:26.899972+0000","last_fullsized":"2024-09-10T11:32:26.899972+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902158+0000","last_change":"2024-09-10T11:32:26.902158+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902158+0000","last_undegraded":"2024-09-10T11:32:26.902158+0000","last_fullsized":"2024-09-10T11:32:26.902158+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899246+0000","last_change":"2024-09-10T11:32:26.899246+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899246+0000","last_undegraded":"2024-09-10T11:32:26.899246+0000","last_fullsized":"2024-09-10T11:32:26.899246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899236+0000","last_change":"2024-09-10T11:32:26.899236+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899236+0000","last_undegraded":"2024-09-10T11:32:26.899236+0000","last_fullsized":"2024-09-10T11:32:26.899236+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.899253+0000","last_change":"2024-09-10T11:32:26.899253+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.899253+0000","last_undegraded":"2024-09-10T11:32:26.899253+0000","last_fullsized":"2024-09-10T11:32:26.899253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900572+0000","last_change":"2024-09-10T11:32:26.900572+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900572+0000","last_undegraded":"2024-09-10T11:32:26.900572+0000","last_fullsized":"2024-09-10T11:32:26.900572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901741+0000","last_change":"2024-09-10T11:32:26.901741+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901741+0000","last_undegraded":"2024-09-10T11:32:26.901741+0000","last_fullsized":"2024-09-10T11:32:26.901741+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898535+0000","last_change":"2024-09-10T11:32:26.898535+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898535+0000","last_undegraded":"2024-09-10T11:32:26.898535+0000","last_fullsized":"2024-09-10T11:32:26.898535+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901742+0000","last_change":"2024-09-10T11:32:26.901742+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901742+0000","last_undegraded":"2024-09-10T11:32:26.901742+0000","last_fullsized":"2024-09-10T11:32:26.901742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898838+0000","last_change":"2024-09-10T11:32:26.898838+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898838+0000","last_undegraded":"2024-09-10T11:32:26.898838+0000","last_fullsized":"2024-09-10T11:32:26.898838+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.902491+0000","last_change":"2024-09-10T11:32:26.902491+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.902491+0000","last_undegraded":"2024-09-10T11:32:26.902491+0000","last_fullsized":"2024-09-10T11:32:26.902491+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900352+0000","last_change":"2024-09-10T11:32:26.900352+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900352+0000","last_undegraded":"2024-09-10T11:32:26.900352+0000","last_fullsized":"2024-09-10T11:32:26.900352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900182+0000","last_change":"2024-09-10T11:32:26.900182+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900182+0000","last_undegraded":"2024-09-10T11:32:26.900182+0000","last_fullsized":"2024-09-10T11:32:26.900182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900591+0000","last_change":"2024-09-10T11:32:26.900591+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900591+0000","last_undegraded":"2024-09-10T11:32:26.900591+0000","last_fullsized":"2024-09-10T11:32:26.900591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900164+0000","last_change":"2024-09-10T11:32:26.900164+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900164+0000","last_undegraded":"2024-09-10T11:32:26.900164+0000","last_fullsized":"2024-09-10T11:32:26.900164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898512+0000","last_change":"2024-09-10T11:32:26.898512+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898512+0000","last_undegraded":"2024-09-10T11:32:26.898512+0000","last_fullsized":"2024-09-10T11:32:26.898512+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900150+0000","last_change":"2024-09-10T11:32:26.900150+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900150+0000","last_undegraded":"2024-09-10T11:32:26.900150+0000","last_fullsized":"2024-09-10T11:32:26.900150+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.898555+0000","last_change":"2024-09-10T11:32:26.898555+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.898555+0000","last_undegraded":"2024-09-10T11:32:26.898555+0000","last_fullsized":"2024-09-10T11:32:26.898555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900181+0000","last_change":"2024-09-10T11:32:26.900181+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900181+0000","last_undegraded":"2024-09-10T11:32:26.900181+0000","last_fullsized":"2024-09-10T11:32:26.900181+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.901768+0000","last_change":"2024-09-10T11:32:26.901768+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.901768+0000","last_undegraded":"2024-09-10T11:32:26.901768+0000","last_fullsized":"2024-09-10T11:32:26.901768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:32:27.897264+0000","last_change":"2024-09-10T11:32:27.897264+0000","last_active":"2024-09-10T11:32:27.897264+0000","last_peered":"2024-09-10T11:32:27.897264+0000","last_clean":"2024-09-10T11:32:27.897264+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:27.897264+0000","last_undegraded":"2024-09-10T11:32:27.897264+0000","last_fullsized":"2024-09-10T11:32:27.897264+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-10T11:32:26.900018+0000","last_change":"2024-09-10T11:32:26.900018+0000","last_active":"2024-09-10T11:32:26.891726+0000","last_peered":"2024-09-10T11:32:26.891726+0000","last_clean":"2024-09-10T11:32:26.891726+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:32:26.900018+0000","last_undegraded":"2024-09-10T11:32:26.900018+0000","last_fullsized":"2024-09-10T11:32:26.900018+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":255,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-10T11:32:26.901380+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:32:26.901380+0000","last_peered":"2024-09-10T11:32:26.901380+0000","last_clean":"2024-09-10T11:32:26.901380+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:32:26.901380+0000","last_undegraded":"2024-09-10T11:32:26.901380+0000","last_fullsized":"2024-09-10T11:32:26.901380+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:31.673361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":300,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195011,"statfs":{"total":200000000,"available":199691835,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":882,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194430,"statfs":{"total":200000000,"available":199096340,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":305,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195006,"statfs":{"total":200000000,"available":199686708,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3f is not active+clean 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3d is not active+clean 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3c is not active+clean 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.523 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3b is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3a is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:PG 9.39 is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:PG 9.38 is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:PG 9.37 is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:PG 9.36 is not active+clean 2024-09-10T11:32:32.524 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.35 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.34 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.32 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.31 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:PG 9.30 is not active+clean 2024-09-10T11:32:32.525 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2f is not active+clean 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2e is not active+clean 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2c is not active+clean 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2b is not active+clean 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.526 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2a is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.29 is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.26 is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-10T11:32:32.527 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.24 is not active+clean 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.23 is not active+clean 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.22 is not active+clean 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:PG 9.20 is not active+clean 2024-09-10T11:32:32.528 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1e is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1f is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899242+0000', 'last_change': '2024-09-10T11:32:26.899242+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899242+0000', 'last_undegraded': '2024-09-10T11:32:26.899242+0000', 'last_fullsized': '2024-09-10T11:32:26.899242+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1f is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900897+0000', 'last_change': '2024-09-10T11:32:26.900897+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900897+0000', 'last_undegraded': '2024-09-10T11:32:26.900897+0000', 'last_fullsized': '2024-09-10T11:32:26.900897+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-09-10T11:32:32.529 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.901152+0000', 'last_change': '2024-09-10T11:32:26.901152+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.901152+0000', 'last_undegraded': '2024-09-10T11:32:26.901152+0000', 'last_fullsized': '2024-09-10T11:32:26.901152+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1d is not active+clean 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1c is not active+clean 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899560+0000', 'last_change': '2024-09-10T11:32:26.899560+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899560+0000', 'last_undegraded': '2024-09-10T11:32:26.899560+0000', 'last_fullsized': '2024-09-10T11:32:26.899560+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.898894+0000', 'last_change': '2024-09-10T11:32:26.898894+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.898894+0000', 'last_undegraded': '2024-09-10T11:32:26.898894+0000', 'last_fullsized': '2024-09-10T11:32:26.898894+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1b is not active+clean 2024-09-10T11:32:32.530 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900904+0000', 'last_change': '2024-09-10T11:32:26.900904+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900904+0000', 'last_undegraded': '2024-09-10T11:32:26.900904+0000', 'last_fullsized': '2024-09-10T11:32:26.900904+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 8.19 is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900524+0000', 'last_change': '2024-09-10T11:32:26.900524+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900524+0000', 'last_undegraded': '2024-09-10T11:32:26.900524+0000', 'last_fullsized': '2024-09-10T11:32:26.900524+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.19 is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900363+0000', 'last_change': '2024-09-10T11:32:26.900363+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900363+0000', 'last_undegraded': '2024-09-10T11:32:26.900363+0000', 'last_fullsized': '2024-09-10T11:32:26.900363+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:PG 9.16 is not active+clean 2024-09-10T11:32:32.531 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:PG 8.17 is not active+clean 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900797+0000', 'last_change': '2024-09-10T11:32:26.900797+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900797+0000', 'last_undegraded': '2024-09-10T11:32:26.900797+0000', 'last_fullsized': '2024-09-10T11:32:26.900797+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.17 is not active+clean 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:PG 8.16 is not active+clean 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.902120+0000', 'last_change': '2024-09-10T11:32:26.902120+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.902120+0000', 'last_undegraded': '2024-09-10T11:32:26.902120+0000', 'last_fullsized': '2024-09-10T11:32:26.902120+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:PG 8.15 is not active+clean 2024-09-10T11:32:32.532 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900084+0000', 'last_change': '2024-09-10T11:32:26.900084+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900084+0000', 'last_undegraded': '2024-09-10T11:32:26.900084+0000', 'last_fullsized': '2024-09-10T11:32:26.900084+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.15 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899972+0000', 'last_change': '2024-09-10T11:32:26.899972+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899972+0000', 'last_undegraded': '2024-09-10T11:32:26.899972+0000', 'last_fullsized': '2024-09-10T11:32:26.899972+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.12 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 8.13 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.902158+0000', 'last_change': '2024-09-10T11:32:26.902158+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.902158+0000', 'last_undegraded': '2024-09-10T11:32:26.902158+0000', 'last_fullsized': '2024-09-10T11:32:26.902158+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 9.13 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-09-10T11:32:32.533 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899246+0000', 'last_change': '2024-09-10T11:32:26.899246+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899246+0000', 'last_undegraded': '2024-09-10T11:32:26.899246+0000', 'last_fullsized': '2024-09-10T11:32:26.899246+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.10 is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 8.11 is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899236+0000', 'last_change': '2024-09-10T11:32:26.899236+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899236+0000', 'last_undegraded': '2024-09-10T11:32:26.899236+0000', 'last_fullsized': '2024-09-10T11:32:26.899236+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.11 is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 8.10 is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.899253+0000', 'last_change': '2024-09-10T11:32:26.899253+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.899253+0000', 'last_undegraded': '2024-09-10T11:32:26.899253+0000', 'last_fullsized': '2024-09-10T11:32:26.899253+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 9.e is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-09-10T11:32:32.534 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900572+0000', 'last_change': '2024-09-10T11:32:26.900572+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900572+0000', 'last_undegraded': '2024-09-10T11:32:26.900572+0000', 'last_fullsized': '2024-09-10T11:32:26.900572+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.f is not active+clean 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.901741+0000', 'last_change': '2024-09-10T11:32:26.901741+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.901741+0000', 'last_undegraded': '2024-09-10T11:32:26.901741+0000', 'last_fullsized': '2024-09-10T11:32:26.901741+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 8.d is not active+clean 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.898535+0000', 'last_change': '2024-09-10T11:32:26.898535+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.898535+0000', 'last_undegraded': '2024-09-10T11:32:26.898535+0000', 'last_fullsized': '2024-09-10T11:32:26.898535+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 9.d is not active+clean 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.535 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.901742+0000', 'last_change': '2024-09-10T11:32:26.901742+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.901742+0000', 'last_undegraded': '2024-09-10T11:32:26.901742+0000', 'last_fullsized': '2024-09-10T11:32:26.901742+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.a is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:PG 8.b is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.898838+0000', 'last_change': '2024-09-10T11:32:26.898838+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.898838+0000', 'last_undegraded': '2024-09-10T11:32:26.898838+0000', 'last_fullsized': '2024-09-10T11:32:26.898838+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:PG 8.a is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.902491+0000', 'last_change': '2024-09-10T11:32:26.902491+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.902491+0000', 'last_undegraded': '2024-09-10T11:32:26.902491+0000', 'last_fullsized': '2024-09-10T11:32:26.902491+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-10T11:32:32.536 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:PG 8.8 is not active+clean 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900352+0000', 'last_change': '2024-09-10T11:32:26.900352+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900352+0000', 'last_undegraded': '2024-09-10T11:32:26.900352+0000', 'last_fullsized': '2024-09-10T11:32:26.900352+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.6 is not active+clean 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900182+0000', 'last_change': '2024-09-10T11:32:26.900182+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900182+0000', 'last_undegraded': '2024-09-10T11:32:26.900182+0000', 'last_fullsized': '2024-09-10T11:32:26.900182+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:PG 9.7 is not active+clean 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-09-10T11:32:32.537 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900591+0000', 'last_change': '2024-09-10T11:32:26.900591+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900591+0000', 'last_undegraded': '2024-09-10T11:32:26.900591+0000', 'last_fullsized': '2024-09-10T11:32:26.900591+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 8.5 is not active+clean 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900164+0000', 'last_change': '2024-09-10T11:32:26.900164+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900164+0000', 'last_undegraded': '2024-09-10T11:32:26.900164+0000', 'last_fullsized': '2024-09-10T11:32:26.900164+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 8.4 is not active+clean 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.898512+0000', 'last_change': '2024-09-10T11:32:26.898512+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.898512+0000', 'last_undegraded': '2024-09-10T11:32:26.898512+0000', 'last_fullsized': '2024-09-10T11:32:26.898512+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2 is not active+clean 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.538 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900150+0000', 'last_change': '2024-09-10T11:32:26.900150+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900150+0000', 'last_undegraded': '2024-09-10T11:32:26.900150+0000', 'last_fullsized': '2024-09-10T11:32:26.900150+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.898555+0000', 'last_change': '2024-09-10T11:32:26.898555+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.898555+0000', 'last_undegraded': '2024-09-10T11:32:26.898555+0000', 'last_fullsized': '2024-09-10T11:32:26.898555+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900181+0000', 'last_change': '2024-09-10T11:32:26.900181+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900181+0000', 'last_undegraded': '2024-09-10T11:32:26.900181+0000', 'last_fullsized': '2024-09-10T11:32:26.900181+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-10T11:32:32.539 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:PG 8.0 is not active+clean 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.901768+0000', 'last_change': '2024-09-10T11:32:26.901768+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.901768+0000', 'last_undegraded': '2024-09-10T11:32:26.901768+0000', 'last_fullsized': '2024-09-10T11:32:26.901768+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T11:32:27.897264+0000', 'last_change': '2024-09-10T11:32:27.897264+0000', 'last_active': '2024-09-10T11:32:27.897264+0000', 'last_peered': '2024-09-10T11:32:27.897264+0000', 'last_clean': '2024-09-10T11:32:27.897264+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:27.897264+0000', 'last_undegraded': '2024-09-10T11:32:27.897264+0000', 'last_fullsized': '2024-09-10T11:32:27.897264+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:27.897264+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-09-10T11:32:32.540 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:32:26.900018+0000', 'last_change': '2024-09-10T11:32:26.900018+0000', 'last_active': '2024-09-10T11:32:26.891726+0000', 'last_peered': '2024-09-10T11:32:26.891726+0000', 'last_clean': '2024-09-10T11:32:26.891726+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:32:26.900018+0000', 'last_undegraded': '2024-09-10T11:32:26.900018+0000', 'last_fullsized': '2024-09-10T11:32:26.900018+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'last_clean_scrub_stamp': '2024-09-10T11:32:26.891726+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:32:32.541 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T11:32:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.655+0000 7fc9fc253640 1 Processor -- start 2024-09-10T11:32:32.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.655+0000 7fc9fc253640 1 -- start start 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fc253640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9f406ece0 0x7fc9f4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fc253640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 0x7fc9f415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fc253640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f4152e60 con 0x7fc9f40d1200 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fc253640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f40d15e0 con 0x7fc9f406ece0 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fc253640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f4072680 con 0x7fc9f415c060 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9f406ece0 0x7fc9f4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc9f406ece0 0x7fc9f4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42896/0 (socket says 172.21.15.99:42896) 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 -- 172.21.15.99:0/3676027021 learned_addr learned my addr 172.21.15.99:0/3676027021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fa7f0640 1 --2- 172.21.15.99:0/3676027021 >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 0x7fc9f415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 -- 172.21.15.99:0/3676027021 >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 msgr2=0x7fc9f415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 --2- 172.21.15.99:0/3676027021 >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 0x7fc9f415c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 -- 172.21.15.99:0/3676027021 >> v1:172.21.15.196:6789/0 conn(0x7fc9f40d1200 legacy=0x7fc9f406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:32.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9f97ee640 1 -- 172.21.15.99:0/3676027021 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f4072960 con 0x7fc9f406ece0 2024-09-10T11:32:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.656+0000 7fc9fa7f0640 1 --2- 172.21.15.99:0/3676027021 >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 0x7fc9f415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.657+0000 7fc9f97ee640 1 --2- 172.21.15.99:0/3676027021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f4073440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40cd570 tx=0x7fc9e402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39cc23c02b894f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.657+0000 7fc9f8fed640 1 -- 172.21.15.99:0/3676027021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4032710 con 0x7fc9f406ece0 2024-09-10T11:32:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.657+0000 7fc9f8fed640 1 -- 172.21.15.99:0/3676027021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e40328b0 con 0x7fc9f406ece0 2024-09-10T11:32:32.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.657+0000 7fc9f8fed640 1 -- 172.21.15.99:0/3676027021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4032ca0 con 0x7fc9f406ece0 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 -- 172.21.15.99:0/3676027021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 msgr2=0x7fc9f4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3676027021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f4073440 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40cd570 tx=0x7fc9e402c850 comp rx=0 tx=0).stop 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 -- 172.21.15.99:0/3676027021 shutdown_connections 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3676027021 >> v2:172.21.15.196:3300/0 conn(0x7fc9f415c060 0x7fc9f415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3676027021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f4073440 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 -- 172.21.15.99:0/3676027021 >> 172.21.15.99:0/3676027021 conn(0x7fc9f406bc10 msgr2=0x7fc9f406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 -- 172.21.15.99:0/3676027021 shutdown_connections 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.658+0000 7fc9fc253640 1 -- 172.21.15.99:0/3676027021 wait complete. 2024-09-10T11:32:32.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.659+0000 7fc9fc253640 1 Processor -- start 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.659+0000 7fc9fc253640 1 -- start start 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f40d1200 0x7fc9f413f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 0x7fc9f4142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f415cd90 con 0x7fc9f406ece0 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4072f20 con 0x7fc9f40d1200 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f415ca30 con 0x7fc9f415c060 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42910/0 (socket says 172.21.15.99:42910) 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 -- 172.21.15.99:0/3292284223 learned_addr learned my addr 172.21.15.99:0/3292284223 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f97ee640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f40d1200 0x7fc9f413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fa7f0640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 0x7fc9f4142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 -- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 msgr2=0x7fc9f4142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 0x7fc9f4142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 -- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f40d1200 msgr2=0x7fc9f413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f40d1200 0x7fc9f413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 -- 172.21.15.99:0/3292284223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f4140320 con 0x7fc9f406ece0 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fa7f0640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 0x7fc9f4142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9f9fef640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b3f0 tx=0x7fc9f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0015020 con 0x7fc9f406ece0 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f0011830 con 0x7fc9f406ece0 2024-09-10T11:32:32.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f40c5e00 con 0x7fc9f406ece0 2024-09-10T11:32:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011c20 con 0x7fc9f406ece0 2024-09-10T11:32:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.660+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f40c5b20 con 0x7fc9f406ece0 2024-09-10T11:32:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.661+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f4073440 con 0x7fc9f406ece0 2024-09-10T11:32:32.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.662+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc9f0018020 con 0x7fc9f406ece0 2024-09-10T11:32:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.662+0000 7fc9ea7fc640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.662+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fc9f008a9c0 con 0x7fc9f406ece0 2024-09-10T11:32:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.662+0000 7fc9f97ee640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.663+0000 7fc9f97ee640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4037000 tx=0x7fc9e4036040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:32.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.667+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc9f008ce20 con 0x7fc9f406ece0 2024-09-10T11:32:32.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.894+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc9f406de10 con 0x7fc9f406ece0 2024-09-10T11:32:32.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.895+0000 7fc9ea7fc640 1 -- 172.21.15.99:0/3292284223 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1693 (secure 0 0 0) 0x7fc9f005f070 con 0x7fc9f406ece0 2024-09-10T11:32:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:32.896 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1725967815,"num_in_osds":8,"osd_in_since":1725967812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":3676818,"bytes_avail":1596323182,"bytes_total":1600000000,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5592}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-10T11:31:16.920785+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 msgr2=0x7fc9d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4037000 tx=0x7fc9e4036040 comp rx=0 tx=0).stop 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 msgr2=0x7fc9f415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000b3f0 tx=0x7fc9f000b8b0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 shutdown_connections 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc9d806be30 0x7fc9d806e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9f415c060 0x7fc9f4142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc9f40d1200 0x7fc9f413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 --2- 172.21.15.99:0/3292284223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc9f406ece0 0x7fc9f415b1c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:32.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 >> 172.21.15.99:0/3292284223 conn(0x7fc9f406bc10 msgr2=0x7fc9f4152200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:32.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 shutdown_connections 2024-09-10T11:32:32.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:32.897+0000 7fc9fc253640 1 -- 172.21.15.99:0/3292284223 wait complete. 2024-09-10T11:32:32.909 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T11:32:32.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:32:33.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.007+0000 7f52395a8640 1 Processor -- start 2024-09-10T11:32:33.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.007+0000 7f52395a8640 1 -- start start 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52395a8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 0x7f5234163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52395a8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52340b4930 0x7f523406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52395a8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52340b5140 con 0x7f5234160030 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52395a8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f52340b5420 con 0x7f52340b4930 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52395a8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5234162b40 con 0x7f523406bc10 2024-09-10T11:32:33.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52337fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52340b4930 0x7f523406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52337fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52340b4930 0x7f523406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42926/0 (socket says 172.21.15.99:42926) 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52337fe640 1 -- 172.21.15.99:0/129274346 learned_addr learned my addr 172.21.15.99:0/129274346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f5232ffd640 1 --2- 172.21.15.99:0/129274346 >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 0x7f5234163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.008+0000 7f52337fe640 1 -- 172.21.15.99:0/129274346 >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 msgr2=0x7f5234163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f52337fe640 1 --2- 172.21.15.99:0/129274346 >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 0x7f5234163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f52337fe640 1 -- 172.21.15.99:0/129274346 >> v1:172.21.15.196:6789/0 conn(0x7f5234160030 legacy=0x7f523406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f52337fe640 1 -- 172.21.15.99:0/129274346 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234162e20 con 0x7f52340b4930 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f5232ffd640 1 --2- 172.21.15.99:0/129274346 >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 0x7f5234163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f52337fe640 1 --2- 172.21.15.99:0/129274346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52340b4930 0x7f523406e5d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5234073870 tx=0x7f522c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170f15f16297459d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f5231ffb640 1 -- 172.21.15.99:0/129274346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c036070 con 0x7f52340b4930 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f5231ffb640 1 -- 172.21.15.99:0/129274346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c031600 con 0x7f52340b4930 2024-09-10T11:32:33.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.009+0000 7f5231ffb640 1 -- 172.21.15.99:0/129274346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c0319d0 con 0x7f52340b4930 2024-09-10T11:32:33.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 -- 172.21.15.99:0/129274346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52340b4930 msgr2=0x7f523406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 --2- 172.21.15.99:0/129274346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52340b4930 0x7f523406e5d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5234073870 tx=0x7f522c02a6a0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 -- 172.21.15.99:0/129274346 shutdown_connections 2024-09-10T11:32:33.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 --2- 172.21.15.99:0/129274346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52340b4930 0x7f523406e5d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 --2- 172.21.15.99:0/129274346 >> v2:172.21.15.196:3300/0 conn(0x7f523406bc10 0x7f5234163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 -- 172.21.15.99:0/129274346 >> 172.21.15.99:0/129274346 conn(0x7f52340b3630 msgr2=0x7f52340b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 -- 172.21.15.99:0/129274346 shutdown_connections 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.010+0000 7f52395a8640 1 -- 172.21.15.99:0/129274346 wait complete. 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.011+0000 7f52395a8640 1 Processor -- start 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.011+0000 7f52395a8640 1 -- start start 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f523406bc10 0x7f5234140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 0x7f5234142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52340d1560 con 0x7f5234160030 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f52340b4dc0 con 0x7f52340b4930 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52395a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f52340d16e0 con 0x7f523406bc10 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42942/0 (socket says 172.21.15.99:42942) 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 -- 172.21.15.99:0/750913972 learned_addr learned my addr 172.21.15.99:0/750913972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52327fc640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 0x7f5234142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 -- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f523406bc10 msgr2=0x7f5234140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f523406bc10 0x7f5234140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 -- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 msgr2=0x7f5234142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 0x7f5234142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52337fe640 1 -- 172.21.15.99:0/750913972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5234140550 con 0x7f5234160030 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.012+0000 7f52327fc640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 0x7f5234142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f52337fe640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f522c034040 tx=0x7f522c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c036050 con 0x7f5234160030 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f522c042070 con 0x7f5234160030 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52340ae940 con 0x7f5234160030 2024-09-10T11:32:33.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c009510 con 0x7f5234160030 2024-09-10T11:32:33.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.013+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5234143a30 con 0x7f5234160030 2024-09-10T11:32:33.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.014+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5234163620 con 0x7f5234160030 2024-09-10T11:32:33.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.014+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f522c007bd0 con 0x7f5234160030 2024-09-10T11:32:33.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.015+0000 7f521b7fe640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 0x7f522006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.015+0000 7f5232ffd640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 0x7f522006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.015+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f522c0abc60 con 0x7f5234160030 2024-09-10T11:32:33.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.015+0000 7f5232ffd640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 0x7f522006e320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5224006e80 tx=0x7f5224008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:33.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.019+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f522c045360 con 0x7f5234160030 2024-09-10T11:32:33.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.179+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5234073d40 con 0x7f522006be60 2024-09-10T11:32:33.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.185+0000 7f521b7fe640 1 -- 172.21.15.99:0/750913972 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247492 (secure 0 0 0) 0x7f5234073d40 con 0x7f522006be60 2024-09-10T11:32:33.188 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:33.190 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 msgr2=0x7f522006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 0x7f522006e320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5224006e80 tx=0x7f5224008040 comp rx=0 tx=0).stop 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 msgr2=0x7f52340725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f522c034040 tx=0x7f522c004a40 comp rx=0 tx=0).stop 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 shutdown_connections 2024-09-10T11:32:33.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f522006be60 0x7f522006e320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5234160030 0x7f52340725b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52340b4930 0x7f5234142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 --2- 172.21.15.99:0/750913972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f523406bc10 0x7f5234140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 >> 172.21.15.99:0/750913972 conn(0x7f52340b3630 msgr2=0x7f523406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 shutdown_connections 2024-09-10T11:32:33.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:33.188+0000 7f52395a8640 1 -- 172.21.15.99:0/750913972 wait complete. 2024-09-10T11:32:33.200 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-10T11:32:32.933734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558710,"statfs":{"total":1600000000,"available":1596119372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000962"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908695+0000","last_change":"2024-09-10T11:32:28.919685+0000","last_active":"2024-09-10T11:32:29.908695+0000","last_peered":"2024-09-10T11:32:29.908695+0000","last_clean":"2024-09-10T11:32:29.908695+0000","last_became_active":"2024-09-10T11:32:28.919057+0000","last_became_peered":"2024-09-10T11:32:28.919057+0000","last_unstale":"2024-09-10T11:32:29.908695+0000","last_undegraded":"2024-09-10T11:32:29.908695+0000","last_fullsized":"2024-09-10T11:32:29.908695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:09.895137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177116+0000","last_change":"2024-09-10T11:32:28.918781+0000","last_active":"2024-09-10T11:32:31.177116+0000","last_peered":"2024-09-10T11:32:31.177116+0000","last_clean":"2024-09-10T11:32:31.177116+0000","last_became_active":"2024-09-10T11:32:28.917900+0000","last_became_peered":"2024-09-10T11:32:28.917900+0000","last_unstale":"2024-09-10T11:32:31.177116+0000","last_undegraded":"2024-09-10T11:32:31.177116+0000","last_fullsized":"2024-09-10T11:32:31.177116+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:19.258335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910776+0000","last_change":"2024-09-10T11:32:28.924199+0000","last_active":"2024-09-10T11:32:30.910776+0000","last_peered":"2024-09-10T11:32:30.910776+0000","last_clean":"2024-09-10T11:32:30.910776+0000","last_became_active":"2024-09-10T11:32:28.923967+0000","last_became_peered":"2024-09-10T11:32:28.923967+0000","last_unstale":"2024-09-10T11:32:30.910776+0000","last_undegraded":"2024-09-10T11:32:30.910776+0000","last_fullsized":"2024-09-10T11:32:30.910776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:01.566893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387659+0000","last_change":"2024-09-10T11:32:28.925901+0000","last_active":"2024-09-10T11:32:31.387659+0000","last_peered":"2024-09-10T11:32:31.387659+0000","last_clean":"2024-09-10T11:32:31.387659+0000","last_became_active":"2024-09-10T11:32:28.925638+0000","last_became_peered":"2024-09-10T11:32:28.925638+0000","last_unstale":"2024-09-10T11:32:31.387659+0000","last_undegraded":"2024-09-10T11:32:31.387659+0000","last_fullsized":"2024-09-10T11:32:31.387659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:42.089791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908831+0000","last_change":"2024-09-10T11:32:28.926133+0000","last_active":"2024-09-10T11:32:29.908831+0000","last_peered":"2024-09-10T11:32:29.908831+0000","last_clean":"2024-09-10T11:32:29.908831+0000","last_became_active":"2024-09-10T11:32:28.925909+0000","last_became_peered":"2024-09-10T11:32:28.925909+0000","last_unstale":"2024-09-10T11:32:29.908831+0000","last_undegraded":"2024-09-10T11:32:29.908831+0000","last_fullsized":"2024-09-10T11:32:29.908831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:39.267880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908869+0000","last_change":"2024-09-10T11:32:28.919299+0000","last_active":"2024-09-10T11:32:29.908869+0000","last_peered":"2024-09-10T11:32:29.908869+0000","last_clean":"2024-09-10T11:32:29.908869+0000","last_became_active":"2024-09-10T11:32:28.917949+0000","last_became_peered":"2024-09-10T11:32:28.917949+0000","last_unstale":"2024-09-10T11:32:29.908869+0000","last_undegraded":"2024-09-10T11:32:29.908869+0000","last_fullsized":"2024-09-10T11:32:29.908869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:27.823063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387610+0000","last_change":"2024-09-10T11:32:28.928710+0000","last_active":"2024-09-10T11:32:31.387610+0000","last_peered":"2024-09-10T11:32:31.387610+0000","last_clean":"2024-09-10T11:32:31.387610+0000","last_became_active":"2024-09-10T11:32:28.928352+0000","last_became_peered":"2024-09-10T11:32:28.928352+0000","last_unstale":"2024-09-10T11:32:31.387610+0000","last_undegraded":"2024-09-10T11:32:31.387610+0000","last_fullsized":"2024-09-10T11:32:31.387610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:24.008047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387845+0000","last_change":"2024-09-10T11:32:28.922119+0000","last_active":"2024-09-10T11:32:31.387845+0000","last_peered":"2024-09-10T11:32:31.387845+0000","last_clean":"2024-09-10T11:32:31.387845+0000","last_became_active":"2024-09-10T11:32:28.921820+0000","last_became_peered":"2024-09-10T11:32:28.921820+0000","last_unstale":"2024-09-10T11:32:31.387845+0000","last_undegraded":"2024-09-10T11:32:31.387845+0000","last_fullsized":"2024-09-10T11:32:31.387845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:00.104608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910926+0000","last_change":"2024-09-10T11:32:28.919247+0000","last_active":"2024-09-10T11:32:29.910926+0000","last_peered":"2024-09-10T11:32:29.910926+0000","last_clean":"2024-09-10T11:32:29.910926+0000","last_became_active":"2024-09-10T11:32:28.918268+0000","last_became_peered":"2024-09-10T11:32:28.918268+0000","last_unstale":"2024-09-10T11:32:29.910926+0000","last_undegraded":"2024-09-10T11:32:29.910926+0000","last_fullsized":"2024-09-10T11:32:29.910926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:49.907599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908922+0000","last_change":"2024-09-10T11:32:28.915631+0000","last_active":"2024-09-10T11:32:29.908922+0000","last_peered":"2024-09-10T11:32:29.908922+0000","last_clean":"2024-09-10T11:32:29.908922+0000","last_became_active":"2024-09-10T11:32:28.915111+0000","last_became_peered":"2024-09-10T11:32:28.915111+0000","last_unstale":"2024-09-10T11:32:29.908922+0000","last_undegraded":"2024-09-10T11:32:29.908922+0000","last_fullsized":"2024-09-10T11:32:29.908922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:02.159286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911181+0000","last_change":"2024-09-10T11:32:28.921597+0000","last_active":"2024-09-10T11:32:29.911181+0000","last_peered":"2024-09-10T11:32:29.911181+0000","last_clean":"2024-09-10T11:32:29.911181+0000","last_became_active":"2024-09-10T11:32:28.921258+0000","last_became_peered":"2024-09-10T11:32:28.921258+0000","last_unstale":"2024-09-10T11:32:29.911181+0000","last_undegraded":"2024-09-10T11:32:29.911181+0000","last_fullsized":"2024-09-10T11:32:29.911181+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:06.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910880+0000","last_change":"2024-09-10T11:32:28.915312+0000","last_active":"2024-09-10T11:32:29.910880+0000","last_peered":"2024-09-10T11:32:29.910880+0000","last_clean":"2024-09-10T11:32:29.910880+0000","last_became_active":"2024-09-10T11:32:28.914948+0000","last_became_peered":"2024-09-10T11:32:28.914948+0000","last_unstale":"2024-09-10T11:32:29.910880+0000","last_undegraded":"2024-09-10T11:32:29.910880+0000","last_fullsized":"2024-09-10T11:32:29.910880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:28.310413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157334+0000","last_change":"2024-09-10T11:32:28.916893+0000","last_active":"2024-09-10T11:32:30.157334+0000","last_peered":"2024-09-10T11:32:30.157334+0000","last_clean":"2024-09-10T11:32:30.157334+0000","last_became_active":"2024-09-10T11:32:28.916353+0000","last_became_peered":"2024-09-10T11:32:28.916353+0000","last_unstale":"2024-09-10T11:32:30.157334+0000","last_undegraded":"2024-09-10T11:32:30.157334+0000","last_fullsized":"2024-09-10T11:32:30.157334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:19.519642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910969+0000","last_change":"2024-09-10T11:32:28.927901+0000","last_active":"2024-09-10T11:32:30.910969+0000","last_peered":"2024-09-10T11:32:30.910969+0000","last_clean":"2024-09-10T11:32:30.910969+0000","last_became_active":"2024-09-10T11:32:28.927536+0000","last_became_peered":"2024-09-10T11:32:28.927536+0000","last_unstale":"2024-09-10T11:32:30.910969+0000","last_undegraded":"2024-09-10T11:32:30.910969+0000","last_fullsized":"2024-09-10T11:32:30.910969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:52.040179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157516+0000","last_change":"2024-09-10T11:32:28.916345+0000","last_active":"2024-09-10T11:32:30.157516+0000","last_peered":"2024-09-10T11:32:30.157516+0000","last_clean":"2024-09-10T11:32:30.157516+0000","last_became_active":"2024-09-10T11:32:28.916084+0000","last_became_peered":"2024-09-10T11:32:28.916084+0000","last_unstale":"2024-09-10T11:32:30.157516+0000","last_undegraded":"2024-09-10T11:32:30.157516+0000","last_fullsized":"2024-09-10T11:32:30.157516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:14.467235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.909016+0000","last_change":"2024-09-10T11:32:28.925069+0000","last_active":"2024-09-10T11:32:29.909016+0000","last_peered":"2024-09-10T11:32:29.909016+0000","last_clean":"2024-09-10T11:32:29.909016+0000","last_became_active":"2024-09-10T11:32:28.924846+0000","last_became_peered":"2024-09-10T11:32:28.924846+0000","last_unstale":"2024-09-10T11:32:29.909016+0000","last_undegraded":"2024-09-10T11:32:29.909016+0000","last_fullsized":"2024-09-10T11:32:29.909016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:41.844007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908434+0000","last_change":"2024-09-10T11:32:28.921320+0000","last_active":"2024-09-10T11:32:29.908434+0000","last_peered":"2024-09-10T11:32:29.908434+0000","last_clean":"2024-09-10T11:32:29.908434+0000","last_became_active":"2024-09-10T11:32:28.921100+0000","last_became_peered":"2024-09-10T11:32:28.921100+0000","last_unstale":"2024-09-10T11:32:29.908434+0000","last_undegraded":"2024-09-10T11:32:29.908434+0000","last_fullsized":"2024-09-10T11:32:29.908434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:50.171686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910955+0000","last_change":"2024-09-10T11:32:28.927274+0000","last_active":"2024-09-10T11:32:30.910955+0000","last_peered":"2024-09-10T11:32:30.910955+0000","last_clean":"2024-09-10T11:32:30.910955+0000","last_became_active":"2024-09-10T11:32:28.927049+0000","last_became_peered":"2024-09-10T11:32:28.927049+0000","last_unstale":"2024-09-10T11:32:30.910955+0000","last_undegraded":"2024-09-10T11:32:30.910955+0000","last_fullsized":"2024-09-10T11:32:30.910955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:17.168093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.176898+0000","last_change":"2024-09-10T11:32:28.925682+0000","last_active":"2024-09-10T11:32:31.176898+0000","last_peered":"2024-09-10T11:32:31.176898+0000","last_clean":"2024-09-10T11:32:31.176898+0000","last_became_active":"2024-09-10T11:32:28.925144+0000","last_became_peered":"2024-09-10T11:32:28.925144+0000","last_unstale":"2024-09-10T11:32:31.176898+0000","last_undegraded":"2024-09-10T11:32:31.176898+0000","last_fullsized":"2024-09-10T11:32:31.176898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:48.901968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911454+0000","last_change":"2024-09-10T11:32:28.916505+0000","last_active":"2024-09-10T11:32:29.911454+0000","last_peered":"2024-09-10T11:32:29.911454+0000","last_clean":"2024-09-10T11:32:29.911454+0000","last_became_active":"2024-09-10T11:32:28.916271+0000","last_became_peered":"2024-09-10T11:32:28.916271+0000","last_unstale":"2024-09-10T11:32:29.911454+0000","last_undegraded":"2024-09-10T11:32:29.911454+0000","last_fullsized":"2024-09-10T11:32:29.911454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:25.154266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157337+0000","last_change":"2024-09-10T11:32:28.923121+0000","last_active":"2024-09-10T11:32:30.157337+0000","last_peered":"2024-09-10T11:32:30.157337+0000","last_clean":"2024-09-10T11:32:30.157337+0000","last_became_active":"2024-09-10T11:32:28.922878+0000","last_became_peered":"2024-09-10T11:32:28.922878+0000","last_unstale":"2024-09-10T11:32:30.157337+0000","last_undegraded":"2024-09-10T11:32:30.157337+0000","last_fullsized":"2024-09-10T11:32:30.157337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:29.689180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911314+0000","last_change":"2024-09-10T11:32:28.919842+0000","last_active":"2024-09-10T11:32:29.911314+0000","last_peered":"2024-09-10T11:32:29.911314+0000","last_clean":"2024-09-10T11:32:29.911314+0000","last_became_active":"2024-09-10T11:32:28.918964+0000","last_became_peered":"2024-09-10T11:32:28.918964+0000","last_unstale":"2024-09-10T11:32:29.911314+0000","last_undegraded":"2024-09-10T11:32:29.911314+0000","last_fullsized":"2024-09-10T11:32:29.911314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:55.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387509+0000","last_change":"2024-09-10T11:32:28.928259+0000","last_active":"2024-09-10T11:32:31.387509+0000","last_peered":"2024-09-10T11:32:31.387509+0000","last_clean":"2024-09-10T11:32:31.387509+0000","last_became_active":"2024-09-10T11:32:28.922888+0000","last_became_peered":"2024-09-10T11:32:28.922888+0000","last_unstale":"2024-09-10T11:32:31.387509+0000","last_undegraded":"2024-09-10T11:32:31.387509+0000","last_fullsized":"2024-09-10T11:32:31.387509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:30.764399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911050+0000","last_change":"2024-09-10T11:32:28.919002+0000","last_active":"2024-09-10T11:32:29.911050+0000","last_peered":"2024-09-10T11:32:29.911050+0000","last_clean":"2024-09-10T11:32:29.911050+0000","last_became_active":"2024-09-10T11:32:28.917834+0000","last_became_peered":"2024-09-10T11:32:28.917834+0000","last_unstale":"2024-09-10T11:32:29.911050+0000","last_undegraded":"2024-09-10T11:32:29.911050+0000","last_fullsized":"2024-09-10T11:32:29.911050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:07.627027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177180+0000","last_change":"2024-09-10T11:32:28.921152+0000","last_active":"2024-09-10T11:32:31.177180+0000","last_peered":"2024-09-10T11:32:31.177180+0000","last_clean":"2024-09-10T11:32:31.177180+0000","last_became_active":"2024-09-10T11:32:28.920515+0000","last_became_peered":"2024-09-10T11:32:28.920515+0000","last_unstale":"2024-09-10T11:32:31.177180+0000","last_undegraded":"2024-09-10T11:32:31.177180+0000","last_fullsized":"2024-09-10T11:32:31.177180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:03.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157505+0000","last_change":"2024-09-10T11:32:28.925784+0000","last_active":"2024-09-10T11:32:30.157505+0000","last_peered":"2024-09-10T11:32:30.157505+0000","last_clean":"2024-09-10T11:32:30.157505+0000","last_became_active":"2024-09-10T11:32:28.925543+0000","last_became_peered":"2024-09-10T11:32:28.925543+0000","last_unstale":"2024-09-10T11:32:30.157505+0000","last_undegraded":"2024-09-10T11:32:30.157505+0000","last_fullsized":"2024-09-10T11:32:30.157505+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:15.277875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177314+0000","last_change":"2024-09-10T11:32:28.921681+0000","last_active":"2024-09-10T11:32:31.177314+0000","last_peered":"2024-09-10T11:32:31.177314+0000","last_clean":"2024-09-10T11:32:31.177314+0000","last_became_active":"2024-09-10T11:32:28.921400+0000","last_became_peered":"2024-09-10T11:32:28.921400+0000","last_unstale":"2024-09-10T11:32:31.177314+0000","last_undegraded":"2024-09-10T11:32:31.177314+0000","last_fullsized":"2024-09-10T11:32:31.177314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:11.028184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388056+0000","last_change":"2024-09-10T11:32:28.924430+0000","last_active":"2024-09-10T11:32:31.388056+0000","last_peered":"2024-09-10T11:32:31.388056+0000","last_clean":"2024-09-10T11:32:31.388056+0000","last_became_active":"2024-09-10T11:32:28.924202+0000","last_became_peered":"2024-09-10T11:32:28.924202+0000","last_unstale":"2024-09-10T11:32:31.388056+0000","last_undegraded":"2024-09-10T11:32:31.388056+0000","last_fullsized":"2024-09-10T11:32:31.388056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:38.282568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387995+0000","last_change":"2024-09-10T11:32:28.926316+0000","last_active":"2024-09-10T11:32:31.387995+0000","last_peered":"2024-09-10T11:32:31.387995+0000","last_clean":"2024-09-10T11:32:31.387995+0000","last_became_active":"2024-09-10T11:32:28.925468+0000","last_became_peered":"2024-09-10T11:32:28.925468+0000","last_unstale":"2024-09-10T11:32:31.387995+0000","last_undegraded":"2024-09-10T11:32:31.387995+0000","last_fullsized":"2024-09-10T11:32:31.387995+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:11.330797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387877+0000","last_change":"2024-09-10T11:32:28.925325+0000","last_active":"2024-09-10T11:32:31.387877+0000","last_peered":"2024-09-10T11:32:31.387877+0000","last_clean":"2024-09-10T11:32:31.387877+0000","last_became_active":"2024-09-10T11:32:28.924919+0000","last_became_peered":"2024-09-10T11:32:28.924919+0000","last_unstale":"2024-09-10T11:32:31.387877+0000","last_undegraded":"2024-09-10T11:32:31.387877+0000","last_fullsized":"2024-09-10T11:32:31.387877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:24.176717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177215+0000","last_change":"2024-09-10T11:32:28.918165+0000","last_active":"2024-09-10T11:32:31.177215+0000","last_peered":"2024-09-10T11:32:31.177215+0000","last_clean":"2024-09-10T11:32:31.177215+0000","last_became_active":"2024-09-10T11:32:28.917204+0000","last_became_peered":"2024-09-10T11:32:28.917204+0000","last_unstale":"2024-09-10T11:32:31.177215+0000","last_undegraded":"2024-09-10T11:32:31.177215+0000","last_fullsized":"2024-09-10T11:32:31.177215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:49.144106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911458+0000","last_change":"2024-09-10T11:32:28.920093+0000","last_active":"2024-09-10T11:32:29.911458+0000","last_peered":"2024-09-10T11:32:29.911458+0000","last_clean":"2024-09-10T11:32:29.911458+0000","last_became_active":"2024-09-10T11:32:28.919277+0000","last_became_peered":"2024-09-10T11:32:28.919277+0000","last_unstale":"2024-09-10T11:32:29.911458+0000","last_undegraded":"2024-09-10T11:32:29.911458+0000","last_fullsized":"2024-09-10T11:32:29.911458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:22.583567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908455+0000","last_change":"2024-09-10T11:32:28.916763+0000","last_active":"2024-09-10T11:32:29.908455+0000","last_peered":"2024-09-10T11:32:29.908455+0000","last_clean":"2024-09-10T11:32:29.908455+0000","last_became_active":"2024-09-10T11:32:28.916486+0000","last_became_peered":"2024-09-10T11:32:28.916486+0000","last_unstale":"2024-09-10T11:32:29.908455+0000","last_undegraded":"2024-09-10T11:32:29.908455+0000","last_fullsized":"2024-09-10T11:32:29.908455+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:27.817260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911241+0000","last_change":"2024-09-10T11:32:27.912191+0000","last_active":"2024-09-10T11:32:29.911241+0000","last_peered":"2024-09-10T11:32:29.911241+0000","last_clean":"2024-09-10T11:32:29.911241+0000","last_became_active":"2024-09-10T11:32:27.911967+0000","last_became_peered":"2024-09-10T11:32:27.911967+0000","last_unstale":"2024-09-10T11:32:29.911241+0000","last_undegraded":"2024-09-10T11:32:29.911241+0000","last_fullsized":"2024-09-10T11:32:29.911241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:20.619592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908567+0000","last_change":"2024-09-10T11:32:28.914101+0000","last_active":"2024-09-10T11:32:29.908567+0000","last_peered":"2024-09-10T11:32:29.908567+0000","last_clean":"2024-09-10T11:32:29.908567+0000","last_became_active":"2024-09-10T11:32:28.913622+0000","last_became_peered":"2024-09-10T11:32:28.913622+0000","last_unstale":"2024-09-10T11:32:29.908567+0000","last_undegraded":"2024-09-10T11:32:29.908567+0000","last_fullsized":"2024-09-10T11:32:29.908567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:07.073954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.176974+0000","last_change":"2024-09-10T11:32:27.913855+0000","last_active":"2024-09-10T11:32:31.176974+0000","last_peered":"2024-09-10T11:32:31.176974+0000","last_clean":"2024-09-10T11:32:31.176974+0000","last_became_active":"2024-09-10T11:32:27.913606+0000","last_became_peered":"2024-09-10T11:32:27.913606+0000","last_unstale":"2024-09-10T11:32:31.176974+0000","last_undegraded":"2024-09-10T11:32:31.176974+0000","last_fullsized":"2024-09-10T11:32:31.176974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:54.825875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388075+0000","last_change":"2024-09-10T11:32:28.923480+0000","last_active":"2024-09-10T11:32:31.388075+0000","last_peered":"2024-09-10T11:32:31.388075+0000","last_clean":"2024-09-10T11:32:31.388075+0000","last_became_active":"2024-09-10T11:32:28.923013+0000","last_became_peered":"2024-09-10T11:32:28.923013+0000","last_unstale":"2024-09-10T11:32:31.388075+0000","last_undegraded":"2024-09-10T11:32:31.388075+0000","last_fullsized":"2024-09-10T11:32:31.388075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:38.753869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908955+0000","last_change":"2024-09-10T11:32:27.910254+0000","last_active":"2024-09-10T11:32:29.908955+0000","last_peered":"2024-09-10T11:32:29.908955+0000","last_clean":"2024-09-10T11:32:29.908955+0000","last_became_active":"2024-09-10T11:32:27.909522+0000","last_became_peered":"2024-09-10T11:32:27.909522+0000","last_unstale":"2024-09-10T11:32:29.908955+0000","last_undegraded":"2024-09-10T11:32:29.908955+0000","last_fullsized":"2024-09-10T11:32:29.908955+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:39.946197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908628+0000","last_change":"2024-09-10T11:32:28.915365+0000","last_active":"2024-09-10T11:32:29.908628+0000","last_peered":"2024-09-10T11:32:29.908628+0000","last_clean":"2024-09-10T11:32:29.908628+0000","last_became_active":"2024-09-10T11:32:28.914284+0000","last_became_peered":"2024-09-10T11:32:28.914284+0000","last_unstale":"2024-09-10T11:32:29.908628+0000","last_undegraded":"2024-09-10T11:32:29.908628+0000","last_fullsized":"2024-09-10T11:32:29.908628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:07.819464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387720+0000","last_change":"2024-09-10T11:32:27.912688+0000","last_active":"2024-09-10T11:32:31.387720+0000","last_peered":"2024-09-10T11:32:31.387720+0000","last_clean":"2024-09-10T11:32:31.387720+0000","last_became_active":"2024-09-10T11:32:27.912475+0000","last_became_peered":"2024-09-10T11:32:27.912475+0000","last_unstale":"2024-09-10T11:32:31.387720+0000","last_undegraded":"2024-09-10T11:32:31.387720+0000","last_fullsized":"2024-09-10T11:32:31.387720+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:25.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388230+0000","last_change":"2024-09-10T11:32:28.923672+0000","last_active":"2024-09-10T11:32:31.388230+0000","last_peered":"2024-09-10T11:32:31.388230+0000","last_clean":"2024-09-10T11:32:31.388230+0000","last_became_active":"2024-09-10T11:32:28.923432+0000","last_became_peered":"2024-09-10T11:32:28.923432+0000","last_unstale":"2024-09-10T11:32:31.388230+0000","last_undegraded":"2024-09-10T11:32:31.388230+0000","last_fullsized":"2024-09-10T11:32:31.388230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:24.067073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911133+0000","last_change":"2024-09-10T11:32:27.919406+0000","last_active":"2024-09-10T11:32:30.911133+0000","last_peered":"2024-09-10T11:32:30.911133+0000","last_clean":"2024-09-10T11:32:30.911133+0000","last_became_active":"2024-09-10T11:32:27.919144+0000","last_became_peered":"2024-09-10T11:32:27.919144+0000","last_unstale":"2024-09-10T11:32:30.911133+0000","last_undegraded":"2024-09-10T11:32:30.911133+0000","last_fullsized":"2024-09-10T11:32:30.911133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:07.561720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911599+0000","last_change":"2024-09-10T11:32:28.920340+0000","last_active":"2024-09-10T11:32:29.911599+0000","last_peered":"2024-09-10T11:32:29.911599+0000","last_clean":"2024-09-10T11:32:29.911599+0000","last_became_active":"2024-09-10T11:32:28.919600+0000","last_became_peered":"2024-09-10T11:32:28.919600+0000","last_unstale":"2024-09-10T11:32:29.911599+0000","last_undegraded":"2024-09-10T11:32:29.911599+0000","last_fullsized":"2024-09-10T11:32:29.911599+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:56.702146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908189+0000","last_change":"2024-09-10T11:32:27.917365+0000","last_active":"2024-09-10T11:32:29.908189+0000","last_peered":"2024-09-10T11:32:29.908189+0000","last_clean":"2024-09-10T11:32:29.908189+0000","last_became_active":"2024-09-10T11:32:27.916293+0000","last_became_peered":"2024-09-10T11:32:27.916293+0000","last_unstale":"2024-09-10T11:32:29.908189+0000","last_undegraded":"2024-09-10T11:32:29.908189+0000","last_fullsized":"2024-09-10T11:32:29.908189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:25.818896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910968+0000","last_change":"2024-09-10T11:32:28.926149+0000","last_active":"2024-09-10T11:32:30.910968+0000","last_peered":"2024-09-10T11:32:30.910968+0000","last_clean":"2024-09-10T11:32:30.910968+0000","last_became_active":"2024-09-10T11:32:28.925920+0000","last_became_peered":"2024-09-10T11:32:28.925920+0000","last_unstale":"2024-09-10T11:32:30.910968+0000","last_undegraded":"2024-09-10T11:32:30.910968+0000","last_fullsized":"2024-09-10T11:32:30.910968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:55.406309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911233+0000","last_change":"2024-09-10T11:32:27.916209+0000","last_active":"2024-09-10T11:32:29.911233+0000","last_peered":"2024-09-10T11:32:29.911233+0000","last_clean":"2024-09-10T11:32:29.911233+0000","last_became_active":"2024-09-10T11:32:27.915221+0000","last_became_peered":"2024-09-10T11:32:27.915221+0000","last_unstale":"2024-09-10T11:32:29.911233+0000","last_undegraded":"2024-09-10T11:32:29.911233+0000","last_fullsized":"2024-09-10T11:32:29.911233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:50.911341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908781+0000","last_change":"2024-09-10T11:32:28.928160+0000","last_active":"2024-09-10T11:32:29.908781+0000","last_peered":"2024-09-10T11:32:29.908781+0000","last_clean":"2024-09-10T11:32:29.908781+0000","last_became_active":"2024-09-10T11:32:28.927877+0000","last_became_peered":"2024-09-10T11:32:28.927877+0000","last_unstale":"2024-09-10T11:32:29.908781+0000","last_undegraded":"2024-09-10T11:32:29.908781+0000","last_fullsized":"2024-09-10T11:32:29.908781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.324873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157542+0000","last_change":"2024-09-10T11:32:27.911435+0000","last_active":"2024-09-10T11:32:30.157542+0000","last_peered":"2024-09-10T11:32:30.157542+0000","last_clean":"2024-09-10T11:32:30.157542+0000","last_became_active":"2024-09-10T11:32:27.910879+0000","last_became_peered":"2024-09-10T11:32:27.910879+0000","last_unstale":"2024-09-10T11:32:30.157542+0000","last_undegraded":"2024-09-10T11:32:30.157542+0000","last_fullsized":"2024-09-10T11:32:30.157542+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:09.144058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157401+0000","last_change":"2024-09-10T11:32:28.916127+0000","last_active":"2024-09-10T11:32:30.157401+0000","last_peered":"2024-09-10T11:32:30.157401+0000","last_clean":"2024-09-10T11:32:30.157401+0000","last_became_active":"2024-09-10T11:32:28.915852+0000","last_became_peered":"2024-09-10T11:32:28.915852+0000","last_unstale":"2024-09-10T11:32:30.157401+0000","last_undegraded":"2024-09-10T11:32:30.157401+0000","last_fullsized":"2024-09-10T11:32:30.157401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:54.431522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911024+0000","last_change":"2024-09-10T11:32:27.915364+0000","last_active":"2024-09-10T11:32:29.911024+0000","last_peered":"2024-09-10T11:32:29.911024+0000","last_clean":"2024-09-10T11:32:29.911024+0000","last_became_active":"2024-09-10T11:32:27.914935+0000","last_became_peered":"2024-09-10T11:32:27.914935+0000","last_unstale":"2024-09-10T11:32:29.911024+0000","last_undegraded":"2024-09-10T11:32:29.911024+0000","last_fullsized":"2024-09-10T11:32:29.911024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:51.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910926+0000","last_change":"2024-09-10T11:32:28.919547+0000","last_active":"2024-09-10T11:32:30.910926+0000","last_peered":"2024-09-10T11:32:30.910926+0000","last_clean":"2024-09-10T11:32:30.910926+0000","last_became_active":"2024-09-10T11:32:28.919304+0000","last_became_peered":"2024-09-10T11:32:28.919304+0000","last_unstale":"2024-09-10T11:32:30.910926+0000","last_undegraded":"2024-09-10T11:32:30.910926+0000","last_fullsized":"2024-09-10T11:32:30.910926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:19.649704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387820+0000","last_change":"2024-09-10T11:32:27.916688+0000","last_active":"2024-09-10T11:32:31.387820+0000","last_peered":"2024-09-10T11:32:31.387820+0000","last_clean":"2024-09-10T11:32:31.387820+0000","last_became_active":"2024-09-10T11:32:27.915700+0000","last_became_peered":"2024-09-10T11:32:27.915700+0000","last_unstale":"2024-09-10T11:32:31.387820+0000","last_undegraded":"2024-09-10T11:32:31.387820+0000","last_fullsized":"2024-09-10T11:32:31.387820+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:26.238719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388158+0000","last_change":"2024-09-10T11:32:28.926115+0000","last_active":"2024-09-10T11:32:31.388158+0000","last_peered":"2024-09-10T11:32:31.388158+0000","last_clean":"2024-09-10T11:32:31.388158+0000","last_became_active":"2024-09-10T11:32:28.925145+0000","last_became_peered":"2024-09-10T11:32:28.925145+0000","last_unstale":"2024-09-10T11:32:31.388158+0000","last_undegraded":"2024-09-10T11:32:31.388158+0000","last_fullsized":"2024-09-10T11:32:31.388158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:19.497028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387687+0000","last_change":"2024-09-10T11:32:27.917336+0000","last_active":"2024-09-10T11:32:31.387687+0000","last_peered":"2024-09-10T11:32:31.387687+0000","last_clean":"2024-09-10T11:32:31.387687+0000","last_became_active":"2024-09-10T11:32:27.917107+0000","last_became_peered":"2024-09-10T11:32:27.917107+0000","last_unstale":"2024-09-10T11:32:31.387687+0000","last_undegraded":"2024-09-10T11:32:31.387687+0000","last_fullsized":"2024-09-10T11:32:31.387687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:38.891665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387652+0000","last_change":"2024-09-10T11:32:28.920991+0000","last_active":"2024-09-10T11:32:31.387652+0000","last_peered":"2024-09-10T11:32:31.387652+0000","last_clean":"2024-09-10T11:32:31.387652+0000","last_became_active":"2024-09-10T11:32:28.920729+0000","last_became_peered":"2024-09-10T11:32:28.920729+0000","last_unstale":"2024-09-10T11:32:31.387652+0000","last_undegraded":"2024-09-10T11:32:31.387652+0000","last_fullsized":"2024-09-10T11:32:31.387652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:36.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157296+0000","last_change":"2024-09-10T11:32:27.912712+0000","last_active":"2024-09-10T11:32:30.157296+0000","last_peered":"2024-09-10T11:32:30.157296+0000","last_clean":"2024-09-10T11:32:30.157296+0000","last_became_active":"2024-09-10T11:32:27.912422+0000","last_became_peered":"2024-09-10T11:32:27.912422+0000","last_unstale":"2024-09-10T11:32:30.157296+0000","last_undegraded":"2024-09-10T11:32:30.157296+0000","last_fullsized":"2024-09-10T11:32:30.157296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:16.893039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910980+0000","last_change":"2024-09-10T11:32:28.914223+0000","last_active":"2024-09-10T11:32:29.910980+0000","last_peered":"2024-09-10T11:32:29.910980+0000","last_clean":"2024-09-10T11:32:29.910980+0000","last_became_active":"2024-09-10T11:32:28.913991+0000","last_became_peered":"2024-09-10T11:32:28.913991+0000","last_unstale":"2024-09-10T11:32:29.910980+0000","last_undegraded":"2024-09-10T11:32:29.910980+0000","last_fullsized":"2024-09-10T11:32:29.910980+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:49.547445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388007+0000","last_change":"2024-09-10T11:32:27.917530+0000","last_active":"2024-09-10T11:32:31.388007+0000","last_peered":"2024-09-10T11:32:31.388007+0000","last_clean":"2024-09-10T11:32:31.388007+0000","last_became_active":"2024-09-10T11:32:27.917064+0000","last_became_peered":"2024-09-10T11:32:27.917064+0000","last_unstale":"2024-09-10T11:32:31.388007+0000","last_undegraded":"2024-09-10T11:32:31.388007+0000","last_fullsized":"2024-09-10T11:32:31.388007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:49.040919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911140+0000","last_change":"2024-09-10T11:32:28.920392+0000","last_active":"2024-09-10T11:32:30.911140+0000","last_peered":"2024-09-10T11:32:30.911140+0000","last_clean":"2024-09-10T11:32:30.911140+0000","last_became_active":"2024-09-10T11:32:28.920143+0000","last_became_peered":"2024-09-10T11:32:28.920143+0000","last_unstale":"2024-09-10T11:32:30.911140+0000","last_undegraded":"2024-09-10T11:32:30.911140+0000","last_fullsized":"2024-09-10T11:32:30.911140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:02.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387813+0000","last_change":"2024-09-10T11:32:27.909637+0000","last_active":"2024-09-10T11:32:31.387813+0000","last_peered":"2024-09-10T11:32:31.387813+0000","last_clean":"2024-09-10T11:32:31.387813+0000","last_became_active":"2024-09-10T11:32:27.909320+0000","last_became_peered":"2024-09-10T11:32:27.909320+0000","last_unstale":"2024-09-10T11:32:31.387813+0000","last_undegraded":"2024-09-10T11:32:31.387813+0000","last_fullsized":"2024-09-10T11:32:31.387813+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:14.679136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910944+0000","last_change":"2024-09-10T11:32:28.916372+0000","last_active":"2024-09-10T11:32:29.910944+0000","last_peered":"2024-09-10T11:32:29.910944+0000","last_clean":"2024-09-10T11:32:29.910944+0000","last_became_active":"2024-09-10T11:32:28.916155+0000","last_became_peered":"2024-09-10T11:32:28.916155+0000","last_unstale":"2024-09-10T11:32:29.910944+0000","last_undegraded":"2024-09-10T11:32:29.910944+0000","last_fullsized":"2024-09-10T11:32:29.910944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:30:48.396127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387797+0000","last_change":"2024-09-10T11:32:27.912984+0000","last_active":"2024-09-10T11:32:31.387797+0000","last_peered":"2024-09-10T11:32:31.387797+0000","last_clean":"2024-09-10T11:32:31.387797+0000","last_became_active":"2024-09-10T11:32:27.912763+0000","last_became_peered":"2024-09-10T11:32:27.912763+0000","last_unstale":"2024-09-10T11:32:31.387797+0000","last_undegraded":"2024-09-10T11:32:31.387797+0000","last_fullsized":"2024-09-10T11:32:31.387797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:26.180721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387753+0000","last_change":"2024-09-10T11:32:28.929562+0000","last_active":"2024-09-10T11:32:31.387753+0000","last_peered":"2024-09-10T11:32:31.387753+0000","last_clean":"2024-09-10T11:32:31.387753+0000","last_became_active":"2024-09-10T11:32:28.929284+0000","last_became_peered":"2024-09-10T11:32:28.929284+0000","last_unstale":"2024-09-10T11:32:31.387753+0000","last_undegraded":"2024-09-10T11:32:31.387753+0000","last_fullsized":"2024-09-10T11:32:31.387753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:52.197825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387745+0000","last_change":"2024-09-10T11:32:27.910409+0000","last_active":"2024-09-10T11:32:31.387745+0000","last_peered":"2024-09-10T11:32:31.387745+0000","last_clean":"2024-09-10T11:32:31.387745+0000","last_became_active":"2024-09-10T11:32:27.910164+0000","last_became_peered":"2024-09-10T11:32:27.910164+0000","last_unstale":"2024-09-10T11:32:31.387745+0000","last_undegraded":"2024-09-10T11:32:31.387745+0000","last_fullsized":"2024-09-10T11:32:31.387745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:19.665894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911255+0000","last_change":"2024-09-10T11:32:28.920876+0000","last_active":"2024-09-10T11:32:30.911255+0000","last_peered":"2024-09-10T11:32:30.911255+0000","last_clean":"2024-09-10T11:32:30.911255+0000","last_became_active":"2024-09-10T11:32:28.920652+0000","last_became_peered":"2024-09-10T11:32:28.920652+0000","last_unstale":"2024-09-10T11:32:30.911255+0000","last_undegraded":"2024-09-10T11:32:30.911255+0000","last_fullsized":"2024-09-10T11:32:30.911255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:21.524810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177097+0000","last_change":"2024-09-10T11:32:27.915031+0000","last_active":"2024-09-10T11:32:31.177097+0000","last_peered":"2024-09-10T11:32:31.177097+0000","last_clean":"2024-09-10T11:32:31.177097+0000","last_became_active":"2024-09-10T11:32:27.914759+0000","last_became_peered":"2024-09-10T11:32:27.914759+0000","last_unstale":"2024-09-10T11:32:31.177097+0000","last_undegraded":"2024-09-10T11:32:31.177097+0000","last_fullsized":"2024-09-10T11:32:31.177097+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:34.528205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911139+0000","last_change":"2024-09-10T11:32:28.927745+0000","last_active":"2024-09-10T11:32:30.911139+0000","last_peered":"2024-09-10T11:32:30.911139+0000","last_clean":"2024-09-10T11:32:30.911139+0000","last_became_active":"2024-09-10T11:32:28.927195+0000","last_became_peered":"2024-09-10T11:32:28.927195+0000","last_unstale":"2024-09-10T11:32:30.911139+0000","last_undegraded":"2024-09-10T11:32:30.911139+0000","last_fullsized":"2024-09-10T11:32:30.911139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:48.968800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388127+0000","last_change":"2024-09-10T11:32:27.917338+0000","last_active":"2024-09-10T11:32:31.388127+0000","last_peered":"2024-09-10T11:32:31.388127+0000","last_clean":"2024-09-10T11:32:31.388127+0000","last_became_active":"2024-09-10T11:32:27.916552+0000","last_became_peered":"2024-09-10T11:32:27.916552+0000","last_unstale":"2024-09-10T11:32:31.388127+0000","last_undegraded":"2024-09-10T11:32:31.388127+0000","last_fullsized":"2024-09-10T11:32:31.388127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:51.122272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177414+0000","last_change":"2024-09-10T11:32:28.918408+0000","last_active":"2024-09-10T11:32:31.177414+0000","last_peered":"2024-09-10T11:32:31.177414+0000","last_clean":"2024-09-10T11:32:31.177414+0000","last_became_active":"2024-09-10T11:32:28.917381+0000","last_became_peered":"2024-09-10T11:32:28.917381+0000","last_unstale":"2024-09-10T11:32:31.177414+0000","last_undegraded":"2024-09-10T11:32:31.177414+0000","last_fullsized":"2024-09-10T11:32:31.177414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:51.451751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911052+0000","last_change":"2024-09-10T11:32:27.911190+0000","last_active":"2024-09-10T11:32:30.911052+0000","last_peered":"2024-09-10T11:32:30.911052+0000","last_clean":"2024-09-10T11:32:30.911052+0000","last_became_active":"2024-09-10T11:32:27.910943+0000","last_became_peered":"2024-09-10T11:32:27.910943+0000","last_unstale":"2024-09-10T11:32:30.911052+0000","last_undegraded":"2024-09-10T11:32:30.911052+0000","last_fullsized":"2024-09-10T11:32:30.911052+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:44.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387674+0000","last_change":"2024-09-10T11:32:28.925502+0000","last_active":"2024-09-10T11:32:31.387674+0000","last_peered":"2024-09-10T11:32:31.387674+0000","last_clean":"2024-09-10T11:32:31.387674+0000","last_became_active":"2024-09-10T11:32:28.925008+0000","last_became_peered":"2024-09-10T11:32:28.925008+0000","last_unstale":"2024-09-10T11:32:31.387674+0000","last_undegraded":"2024-09-10T11:32:31.387674+0000","last_fullsized":"2024-09-10T11:32:31.387674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:36.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387916+0000","last_change":"2024-09-10T11:32:27.915945+0000","last_active":"2024-09-10T11:32:31.387916+0000","last_peered":"2024-09-10T11:32:31.387916+0000","last_clean":"2024-09-10T11:32:31.387916+0000","last_became_active":"2024-09-10T11:32:27.915250+0000","last_became_peered":"2024-09-10T11:32:27.915250+0000","last_unstale":"2024-09-10T11:32:31.387916+0000","last_undegraded":"2024-09-10T11:32:31.387916+0000","last_fullsized":"2024-09-10T11:32:31.387916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911208+0000","last_change":"2024-09-10T11:32:28.918778+0000","last_active":"2024-09-10T11:32:29.911208+0000","last_peered":"2024-09-10T11:32:29.911208+0000","last_clean":"2024-09-10T11:32:29.911208+0000","last_became_active":"2024-09-10T11:32:28.917535+0000","last_became_peered":"2024-09-10T11:32:28.917535+0000","last_unstale":"2024-09-10T11:32:29.911208+0000","last_undegraded":"2024-09-10T11:32:29.911208+0000","last_fullsized":"2024-09-10T11:32:29.911208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:04.331758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911364+0000","last_change":"2024-09-10T11:32:27.911324+0000","last_active":"2024-09-10T11:32:29.911364+0000","last_peered":"2024-09-10T11:32:29.911364+0000","last_clean":"2024-09-10T11:32:29.911364+0000","last_became_active":"2024-09-10T11:32:27.911108+0000","last_became_peered":"2024-09-10T11:32:27.911108+0000","last_unstale":"2024-09-10T11:32:29.911364+0000","last_undegraded":"2024-09-10T11:32:29.911364+0000","last_fullsized":"2024-09-10T11:32:29.911364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:56.827113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388458+0000","last_change":"2024-09-10T11:32:28.920979+0000","last_active":"2024-09-10T11:32:31.388458+0000","last_peered":"2024-09-10T11:32:31.388458+0000","last_clean":"2024-09-10T11:32:31.388458+0000","last_became_active":"2024-09-10T11:32:28.920739+0000","last_became_peered":"2024-09-10T11:32:28.920739+0000","last_unstale":"2024-09-10T11:32:31.388458+0000","last_undegraded":"2024-09-10T11:32:31.388458+0000","last_fullsized":"2024-09-10T11:32:31.388458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:30.702845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388383+0000","last_change":"2024-09-10T11:32:27.917853+0000","last_active":"2024-09-10T11:32:31.388383+0000","last_peered":"2024-09-10T11:32:31.388383+0000","last_clean":"2024-09-10T11:32:31.388383+0000","last_became_active":"2024-09-10T11:32:27.916037+0000","last_became_peered":"2024-09-10T11:32:27.916037+0000","last_unstale":"2024-09-10T11:32:31.388383+0000","last_undegraded":"2024-09-10T11:32:31.388383+0000","last_fullsized":"2024-09-10T11:32:31.388383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:13.213394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387934+0000","last_change":"2024-09-10T11:32:28.923043+0000","last_active":"2024-09-10T11:32:31.387934+0000","last_peered":"2024-09-10T11:32:31.387934+0000","last_clean":"2024-09-10T11:32:31.387934+0000","last_became_active":"2024-09-10T11:32:28.922785+0000","last_became_peered":"2024-09-10T11:32:28.922785+0000","last_unstale":"2024-09-10T11:32:31.387934+0000","last_undegraded":"2024-09-10T11:32:31.387934+0000","last_fullsized":"2024-09-10T11:32:31.387934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.630557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911105+0000","last_change":"2024-09-10T11:32:27.914687+0000","last_active":"2024-09-10T11:32:29.911105+0000","last_peered":"2024-09-10T11:32:29.911105+0000","last_clean":"2024-09-10T11:32:29.911105+0000","last_became_active":"2024-09-10T11:32:27.914415+0000","last_became_peered":"2024-09-10T11:32:27.914415+0000","last_unstale":"2024-09-10T11:32:29.911105+0000","last_undegraded":"2024-09-10T11:32:29.911105+0000","last_fullsized":"2024-09-10T11:32:29.911105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:49.868334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911135+0000","last_change":"2024-09-10T11:32:28.926691+0000","last_active":"2024-09-10T11:32:30.911135+0000","last_peered":"2024-09-10T11:32:30.911135+0000","last_clean":"2024-09-10T11:32:30.911135+0000","last_became_active":"2024-09-10T11:32:28.926424+0000","last_became_peered":"2024-09-10T11:32:28.926424+0000","last_unstale":"2024-09-10T11:32:30.911135+0000","last_undegraded":"2024-09-10T11:32:30.911135+0000","last_fullsized":"2024-09-10T11:32:30.911135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:53.220598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177503+0000","last_change":"2024-09-10T11:32:27.914458+0000","last_active":"2024-09-10T11:32:31.177503+0000","last_peered":"2024-09-10T11:32:31.177503+0000","last_clean":"2024-09-10T11:32:31.177503+0000","last_became_active":"2024-09-10T11:32:27.914189+0000","last_became_peered":"2024-09-10T11:32:27.914189+0000","last_unstale":"2024-09-10T11:32:31.177503+0000","last_undegraded":"2024-09-10T11:32:31.177503+0000","last_fullsized":"2024-09-10T11:32:31.177503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:42.807711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911075+0000","last_change":"2024-09-10T11:32:28.927573+0000","last_active":"2024-09-10T11:32:30.911075+0000","last_peered":"2024-09-10T11:32:30.911075+0000","last_clean":"2024-09-10T11:32:30.911075+0000","last_became_active":"2024-09-10T11:32:28.927293+0000","last_became_peered":"2024-09-10T11:32:28.927293+0000","last_unstale":"2024-09-10T11:32:30.911075+0000","last_undegraded":"2024-09-10T11:32:30.911075+0000","last_fullsized":"2024-09-10T11:32:30.911075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:35.905304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908381+0000","last_change":"2024-09-10T11:32:27.916531+0000","last_active":"2024-09-10T11:32:29.908381+0000","last_peered":"2024-09-10T11:32:29.908381+0000","last_clean":"2024-09-10T11:32:29.908381+0000","last_became_active":"2024-09-10T11:32:27.915984+0000","last_became_peered":"2024-09-10T11:32:27.915984+0000","last_unstale":"2024-09-10T11:32:29.908381+0000","last_undegraded":"2024-09-10T11:32:29.908381+0000","last_fullsized":"2024-09-10T11:32:29.908381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:12.939020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177366+0000","last_change":"2024-09-10T11:32:28.927379+0000","last_active":"2024-09-10T11:32:31.177366+0000","last_peered":"2024-09-10T11:32:31.177366+0000","last_clean":"2024-09-10T11:32:31.177366+0000","last_became_active":"2024-09-10T11:32:28.927017+0000","last_became_peered":"2024-09-10T11:32:28.927017+0000","last_unstale":"2024-09-10T11:32:31.177366+0000","last_undegraded":"2024-09-10T11:32:31.177366+0000","last_fullsized":"2024-09-10T11:32:31.177366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:30.905206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911279+0000","last_change":"2024-09-10T11:32:27.916453+0000","last_active":"2024-09-10T11:32:29.911279+0000","last_peered":"2024-09-10T11:32:29.911279+0000","last_clean":"2024-09-10T11:32:29.911279+0000","last_became_active":"2024-09-10T11:32:27.915841+0000","last_became_peered":"2024-09-10T11:32:27.915841+0000","last_unstale":"2024-09-10T11:32:29.911279+0000","last_undegraded":"2024-09-10T11:32:29.911279+0000","last_fullsized":"2024-09-10T11:32:29.911279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:53.154878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388271+0000","last_change":"2024-09-10T11:32:28.925887+0000","last_active":"2024-09-10T11:32:31.388271+0000","last_peered":"2024-09-10T11:32:31.388271+0000","last_clean":"2024-09-10T11:32:31.388271+0000","last_became_active":"2024-09-10T11:32:28.924747+0000","last_became_peered":"2024-09-10T11:32:28.924747+0000","last_unstale":"2024-09-10T11:32:31.388271+0000","last_undegraded":"2024-09-10T11:32:31.388271+0000","last_fullsized":"2024-09-10T11:32:31.388271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:00.050958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911289+0000","last_change":"2024-09-10T11:32:27.913504+0000","last_active":"2024-09-10T11:32:30.911289+0000","last_peered":"2024-09-10T11:32:30.911289+0000","last_clean":"2024-09-10T11:32:30.911289+0000","last_became_active":"2024-09-10T11:32:27.913056+0000","last_became_peered":"2024-09-10T11:32:27.913056+0000","last_unstale":"2024-09-10T11:32:30.911289+0000","last_undegraded":"2024-09-10T11:32:30.911289+0000","last_fullsized":"2024-09-10T11:32:30.911289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:30.598671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908622+0000","last_change":"2024-09-10T11:32:28.913342+0000","last_active":"2024-09-10T11:32:29.908622+0000","last_peered":"2024-09-10T11:32:29.908622+0000","last_clean":"2024-09-10T11:32:29.908622+0000","last_became_active":"2024-09-10T11:32:28.912948+0000","last_became_peered":"2024-09-10T11:32:28.912948+0000","last_unstale":"2024-09-10T11:32:29.908622+0000","last_undegraded":"2024-09-10T11:32:29.908622+0000","last_fullsized":"2024-09-10T11:32:29.908622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.513720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177433+0000","last_change":"2024-09-10T11:32:27.914126+0000","last_active":"2024-09-10T11:32:31.177433+0000","last_peered":"2024-09-10T11:32:31.177433+0000","last_clean":"2024-09-10T11:32:31.177433+0000","last_became_active":"2024-09-10T11:32:27.913799+0000","last_became_peered":"2024-09-10T11:32:27.913799+0000","last_unstale":"2024-09-10T11:32:31.177433+0000","last_undegraded":"2024-09-10T11:32:31.177433+0000","last_fullsized":"2024-09-10T11:32:31.177433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.654320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157647+0000","last_change":"2024-09-10T11:32:28.921803+0000","last_active":"2024-09-10T11:32:30.157647+0000","last_peered":"2024-09-10T11:32:30.157647+0000","last_clean":"2024-09-10T11:32:30.157647+0000","last_became_active":"2024-09-10T11:32:28.921585+0000","last_became_peered":"2024-09-10T11:32:28.921585+0000","last_unstale":"2024-09-10T11:32:30.157647+0000","last_undegraded":"2024-09-10T11:32:30.157647+0000","last_fullsized":"2024-09-10T11:32:30.157647+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:55.729598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911253+0000","last_change":"2024-09-10T11:32:27.920438+0000","last_active":"2024-09-10T11:32:30.911253+0000","last_peered":"2024-09-10T11:32:30.911253+0000","last_clean":"2024-09-10T11:32:30.911253+0000","last_became_active":"2024-09-10T11:32:27.920179+0000","last_became_peered":"2024-09-10T11:32:27.920179+0000","last_unstale":"2024-09-10T11:32:30.911253+0000","last_undegraded":"2024-09-10T11:32:30.911253+0000","last_fullsized":"2024-09-10T11:32:30.911253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:34.207814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388352+0000","last_change":"2024-09-10T11:32:28.925697+0000","last_active":"2024-09-10T11:32:31.388352+0000","last_peered":"2024-09-10T11:32:31.388352+0000","last_clean":"2024-09-10T11:32:31.388352+0000","last_became_active":"2024-09-10T11:32:28.924175+0000","last_became_peered":"2024-09-10T11:32:28.924175+0000","last_unstale":"2024-09-10T11:32:31.388352+0000","last_undegraded":"2024-09-10T11:32:31.388352+0000","last_fullsized":"2024-09-10T11:32:31.388352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:45.362113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908638+0000","last_change":"2024-09-10T11:32:27.909076+0000","last_active":"2024-09-10T11:32:29.908638+0000","last_peered":"2024-09-10T11:32:29.908638+0000","last_clean":"2024-09-10T11:32:29.908638+0000","last_became_active":"2024-09-10T11:32:27.908530+0000","last_became_peered":"2024-09-10T11:32:27.908530+0000","last_unstale":"2024-09-10T11:32:29.908638+0000","last_undegraded":"2024-09-10T11:32:29.908638+0000","last_fullsized":"2024-09-10T11:32:29.908638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:38.130838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177253+0000","last_change":"2024-09-10T11:32:28.913230+0000","last_active":"2024-09-10T11:32:31.177253+0000","last_peered":"2024-09-10T11:32:31.177253+0000","last_clean":"2024-09-10T11:32:31.177253+0000","last_became_active":"2024-09-10T11:32:28.912723+0000","last_became_peered":"2024-09-10T11:32:28.912723+0000","last_unstale":"2024-09-10T11:32:31.177253+0000","last_undegraded":"2024-09-10T11:32:31.177253+0000","last_fullsized":"2024-09-10T11:32:31.177253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:02.066151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388438+0000","last_change":"2024-09-10T11:32:27.917517+0000","last_active":"2024-09-10T11:32:31.388438+0000","last_peered":"2024-09-10T11:32:31.388438+0000","last_clean":"2024-09-10T11:32:31.388438+0000","last_became_active":"2024-09-10T11:32:27.915521+0000","last_became_peered":"2024-09-10T11:32:27.915521+0000","last_unstale":"2024-09-10T11:32:31.388438+0000","last_undegraded":"2024-09-10T11:32:31.388438+0000","last_fullsized":"2024-09-10T11:32:31.388438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:24.467428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388204+0000","last_change":"2024-09-10T11:32:28.920658+0000","last_active":"2024-09-10T11:32:31.388204+0000","last_peered":"2024-09-10T11:32:31.388204+0000","last_clean":"2024-09-10T11:32:31.388204+0000","last_became_active":"2024-09-10T11:32:28.920428+0000","last_became_peered":"2024-09-10T11:32:28.920428+0000","last_unstale":"2024-09-10T11:32:31.388204+0000","last_undegraded":"2024-09-10T11:32:31.388204+0000","last_fullsized":"2024-09-10T11:32:31.388204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:38.032533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157591+0000","last_change":"2024-09-10T11:32:27.911731+0000","last_active":"2024-09-10T11:32:30.157591+0000","last_peered":"2024-09-10T11:32:30.157591+0000","last_clean":"2024-09-10T11:32:30.157591+0000","last_became_active":"2024-09-10T11:32:27.911178+0000","last_became_peered":"2024-09-10T11:32:27.911178+0000","last_unstale":"2024-09-10T11:32:30.157591+0000","last_undegraded":"2024-09-10T11:32:30.157591+0000","last_fullsized":"2024-09-10T11:32:30.157591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:23.877257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":261,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911326+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:32:29.911326+0000","last_peered":"2024-09-10T11:32:29.911326+0000","last_clean":"2024-09-10T11:32:29.911326+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:32:29.911326+0000","last_undegraded":"2024-09-10T11:32:29.911326+0000","last_fullsized":"2024-09-10T11:32:29.911326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:31.673361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199661449,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199661519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":900,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194411,"statfs":{"total":200000000,"available":199077439,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199660925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199667751,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659769,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":907,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194404,"statfs":{"total":200000000,"available":199070591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:32:34.890 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:32:34.888+0000 7f88a58e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:32:36.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:32:36.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.314+0000 7f70f0447640 1 Processor -- start 2024-09-10T11:32:36.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.314+0000 7f70f0447640 1 -- start start 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70f0447640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f70e80d1200 0x7f70e8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70f0447640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e8152e60 con 0x7f70e815c770 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70f0447640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e80d15e0 con 0x7f70e806e5d0 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70f0447640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f70e8072680 con 0x7f70e80d1200 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee1e3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f70e80d1200 0x7f70e8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee9e4640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f70e815c770 0x7f70e8159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47048/0 (socket says 172.21.15.99:47048) 2024-09-10T11:32:36.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee9e4640 1 -- 172.21.15.99:0/2926284253 learned_addr learned my addr 172.21.15.99:0/2926284253 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:36.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee1e3640 1 -- 172.21.15.99:0/2926284253 >> v1:172.21.15.196:6789/0 conn(0x7f70e806e5d0 legacy=0x7f70e815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:36.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee1e3640 1 -- 172.21.15.99:0/2926284253 >> v1:172.21.15.99:6789/0 conn(0x7f70e815c770 legacy=0x7f70e8159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ed1e1640 1 -- 172.21.15.99:0/2926284253 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870731077 0 0) 0x7f70e8152e60 con 0x7f70e815c770 2024-09-10T11:32:36.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.315+0000 7f70ee1e3640 1 -- 172.21.15.99:0/2926284253 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e8072960 con 0x7f70e80d1200 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.316+0000 7f70ee1e3640 1 --2- 172.21.15.99:0/2926284253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e80d1200 0x7f70e8073440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f70e40097a0 tx=0x7f70e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c554bbd6e6a06a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.316+0000 7f70ed1e1640 1 -- 172.21.15.99:0/2926284253 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4037070 con 0x7f70e80d1200 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.316+0000 7f70ed1e1640 1 -- 172.21.15.99:0/2926284253 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e4032440 con 0x7f70e80d1200 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.316+0000 7f70ed1e1640 1 -- 172.21.15.99:0/2926284253 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4032810 con 0x7f70e80d1200 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 -- 172.21.15.99:0/2926284253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e80d1200 msgr2=0x7f70e8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 --2- 172.21.15.99:0/2926284253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e80d1200 0x7f70e8073440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f70e40097a0 tx=0x7f70e402a1a0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 -- 172.21.15.99:0/2926284253 shutdown_connections 2024-09-10T11:32:36.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 --2- 172.21.15.99:0/2926284253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e80d1200 0x7f70e8073440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 -- 172.21.15.99:0/2926284253 >> 172.21.15.99:0/2926284253 conn(0x7f70e806bc10 msgr2=0x7f70e806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:36.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 -- 172.21.15.99:0/2926284253 shutdown_connections 2024-09-10T11:32:36.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.317+0000 7f70f0447640 1 -- 172.21.15.99:0/2926284253 wait complete. 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.318+0000 7f70f0447640 1 Processor -- start 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.318+0000 7f70f0447640 1 -- start start 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 0x7f70e8146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 0x7f70e8149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70e815cd90 con 0x7f70e806e5d0 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8072f20 con 0x7f70e80d1200 2024-09-10T11:32:36.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70f0447640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70e806ea40 con 0x7f70e815c770 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42956/0 (socket says 172.21.15.99:42956) 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 -- 172.21.15.99:0/1048481608 learned_addr learned my addr 172.21.15.99:0/1048481608 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee9e4640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 0x7f70e8149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ed9e2640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 0x7f70e8146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 -- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 msgr2=0x7f70e8149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 0x7f70e8149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 -- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 msgr2=0x7f70e8146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 0x7f70e8146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 -- 172.21.15.99:0/1048481608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e8141680 con 0x7f70e806e5d0 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee9e4640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 0x7f70e8149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ed9e2640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 0x7f70e8146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:36.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.319+0000 7f70ee1e3640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f70e4035040 tx=0x7f70e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.320+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4037050 con 0x7f70e806e5d0 2024-09-10T11:32:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.320+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e81426a0 con 0x7f70e806e5d0 2024-09-10T11:32:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.320+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e4038040 con 0x7f70e806e5d0 2024-09-10T11:32:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.320+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004490 con 0x7f70e806e5d0 2024-09-10T11:32:36.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.320+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e814c270 con 0x7f70e806e5d0 2024-09-10T11:32:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.321+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e81424c0 con 0x7f70e806e5d0 2024-09-10T11:32:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.321+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70e4030900 con 0x7f70e806e5d0 2024-09-10T11:32:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.322+0000 7f70deffd640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 0x7f70cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.322+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f70e40aba90 con 0x7f70e806e5d0 2024-09-10T11:32:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.322+0000 7f70ed9e2640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 0x7f70cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.322+0000 7f70ed9e2640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 0x7f70cc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007fd0 tx=0x7f70d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.326+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70e81424c0 con 0x7f70e806e5d0 2024-09-10T11:32:36.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.488+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70e806eda0 con 0x7f70cc06be30 2024-09-10T11:32:36.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.494+0000 7f70deffd640 1 -- 172.21.15.99:0/1048481608 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247492 (secure 0 0 0) 0x7f70e806eda0 con 0x7f70cc06be30 2024-09-10T11:32:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:36.500 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:32:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 msgr2=0x7f70cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 0x7f70cc06e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007fd0 tx=0x7f70d8009040 comp rx=0 tx=0).stop 2024-09-10T11:32:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 msgr2=0x7f70e814bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f70e4035040 tx=0x7f70e40046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 shutdown_connections 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70cc06be30 0x7f70cc06e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70e815c770 0x7f70e8149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e80d1200 0x7f70e8146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 --2- 172.21.15.99:0/1048481608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e806e5d0 0x7f70e814bd30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 >> 172.21.15.99:0/1048481608 conn(0x7f70e806bc10 msgr2=0x7f70e80cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 shutdown_connections 2024-09-10T11:32:36.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.497+0000 7f70f0447640 1 -- 172.21.15.99:0/1048481608 wait complete. 2024-09-10T11:32:36.510 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-10T11:32:34.934156+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3789,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558710,"statfs":{"total":1600000000,"available":1596119372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.029327"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908695+0000","last_change":"2024-09-10T11:32:28.919685+0000","last_active":"2024-09-10T11:32:29.908695+0000","last_peered":"2024-09-10T11:32:29.908695+0000","last_clean":"2024-09-10T11:32:29.908695+0000","last_became_active":"2024-09-10T11:32:28.919057+0000","last_became_peered":"2024-09-10T11:32:28.919057+0000","last_unstale":"2024-09-10T11:32:29.908695+0000","last_undegraded":"2024-09-10T11:32:29.908695+0000","last_fullsized":"2024-09-10T11:32:29.908695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:09.895137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177116+0000","last_change":"2024-09-10T11:32:28.918781+0000","last_active":"2024-09-10T11:32:31.177116+0000","last_peered":"2024-09-10T11:32:31.177116+0000","last_clean":"2024-09-10T11:32:31.177116+0000","last_became_active":"2024-09-10T11:32:28.917900+0000","last_became_peered":"2024-09-10T11:32:28.917900+0000","last_unstale":"2024-09-10T11:32:31.177116+0000","last_undegraded":"2024-09-10T11:32:31.177116+0000","last_fullsized":"2024-09-10T11:32:31.177116+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:19.258335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910776+0000","last_change":"2024-09-10T11:32:28.924199+0000","last_active":"2024-09-10T11:32:30.910776+0000","last_peered":"2024-09-10T11:32:30.910776+0000","last_clean":"2024-09-10T11:32:30.910776+0000","last_became_active":"2024-09-10T11:32:28.923967+0000","last_became_peered":"2024-09-10T11:32:28.923967+0000","last_unstale":"2024-09-10T11:32:30.910776+0000","last_undegraded":"2024-09-10T11:32:30.910776+0000","last_fullsized":"2024-09-10T11:32:30.910776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:01.566893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387659+0000","last_change":"2024-09-10T11:32:28.925901+0000","last_active":"2024-09-10T11:32:31.387659+0000","last_peered":"2024-09-10T11:32:31.387659+0000","last_clean":"2024-09-10T11:32:31.387659+0000","last_became_active":"2024-09-10T11:32:28.925638+0000","last_became_peered":"2024-09-10T11:32:28.925638+0000","last_unstale":"2024-09-10T11:32:31.387659+0000","last_undegraded":"2024-09-10T11:32:31.387659+0000","last_fullsized":"2024-09-10T11:32:31.387659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:42.089791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908831+0000","last_change":"2024-09-10T11:32:28.926133+0000","last_active":"2024-09-10T11:32:29.908831+0000","last_peered":"2024-09-10T11:32:29.908831+0000","last_clean":"2024-09-10T11:32:29.908831+0000","last_became_active":"2024-09-10T11:32:28.925909+0000","last_became_peered":"2024-09-10T11:32:28.925909+0000","last_unstale":"2024-09-10T11:32:29.908831+0000","last_undegraded":"2024-09-10T11:32:29.908831+0000","last_fullsized":"2024-09-10T11:32:29.908831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:39.267880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908869+0000","last_change":"2024-09-10T11:32:28.919299+0000","last_active":"2024-09-10T11:32:29.908869+0000","last_peered":"2024-09-10T11:32:29.908869+0000","last_clean":"2024-09-10T11:32:29.908869+0000","last_became_active":"2024-09-10T11:32:28.917949+0000","last_became_peered":"2024-09-10T11:32:28.917949+0000","last_unstale":"2024-09-10T11:32:29.908869+0000","last_undegraded":"2024-09-10T11:32:29.908869+0000","last_fullsized":"2024-09-10T11:32:29.908869+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:27.823063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387610+0000","last_change":"2024-09-10T11:32:28.928710+0000","last_active":"2024-09-10T11:32:31.387610+0000","last_peered":"2024-09-10T11:32:31.387610+0000","last_clean":"2024-09-10T11:32:31.387610+0000","last_became_active":"2024-09-10T11:32:28.928352+0000","last_became_peered":"2024-09-10T11:32:28.928352+0000","last_unstale":"2024-09-10T11:32:31.387610+0000","last_undegraded":"2024-09-10T11:32:31.387610+0000","last_fullsized":"2024-09-10T11:32:31.387610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:24.008047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387845+0000","last_change":"2024-09-10T11:32:28.922119+0000","last_active":"2024-09-10T11:32:31.387845+0000","last_peered":"2024-09-10T11:32:31.387845+0000","last_clean":"2024-09-10T11:32:31.387845+0000","last_became_active":"2024-09-10T11:32:28.921820+0000","last_became_peered":"2024-09-10T11:32:28.921820+0000","last_unstale":"2024-09-10T11:32:31.387845+0000","last_undegraded":"2024-09-10T11:32:31.387845+0000","last_fullsized":"2024-09-10T11:32:31.387845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:00.104608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910926+0000","last_change":"2024-09-10T11:32:28.919247+0000","last_active":"2024-09-10T11:32:29.910926+0000","last_peered":"2024-09-10T11:32:29.910926+0000","last_clean":"2024-09-10T11:32:29.910926+0000","last_became_active":"2024-09-10T11:32:28.918268+0000","last_became_peered":"2024-09-10T11:32:28.918268+0000","last_unstale":"2024-09-10T11:32:29.910926+0000","last_undegraded":"2024-09-10T11:32:29.910926+0000","last_fullsized":"2024-09-10T11:32:29.910926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:49.907599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908922+0000","last_change":"2024-09-10T11:32:28.915631+0000","last_active":"2024-09-10T11:32:29.908922+0000","last_peered":"2024-09-10T11:32:29.908922+0000","last_clean":"2024-09-10T11:32:29.908922+0000","last_became_active":"2024-09-10T11:32:28.915111+0000","last_became_peered":"2024-09-10T11:32:28.915111+0000","last_unstale":"2024-09-10T11:32:29.908922+0000","last_undegraded":"2024-09-10T11:32:29.908922+0000","last_fullsized":"2024-09-10T11:32:29.908922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:02.159286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911181+0000","last_change":"2024-09-10T11:32:28.921597+0000","last_active":"2024-09-10T11:32:29.911181+0000","last_peered":"2024-09-10T11:32:29.911181+0000","last_clean":"2024-09-10T11:32:29.911181+0000","last_became_active":"2024-09-10T11:32:28.921258+0000","last_became_peered":"2024-09-10T11:32:28.921258+0000","last_unstale":"2024-09-10T11:32:29.911181+0000","last_undegraded":"2024-09-10T11:32:29.911181+0000","last_fullsized":"2024-09-10T11:32:29.911181+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:06.319647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910880+0000","last_change":"2024-09-10T11:32:28.915312+0000","last_active":"2024-09-10T11:32:29.910880+0000","last_peered":"2024-09-10T11:32:29.910880+0000","last_clean":"2024-09-10T11:32:29.910880+0000","last_became_active":"2024-09-10T11:32:28.914948+0000","last_became_peered":"2024-09-10T11:32:28.914948+0000","last_unstale":"2024-09-10T11:32:29.910880+0000","last_undegraded":"2024-09-10T11:32:29.910880+0000","last_fullsized":"2024-09-10T11:32:29.910880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:39:28.310413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157334+0000","last_change":"2024-09-10T11:32:28.916893+0000","last_active":"2024-09-10T11:32:30.157334+0000","last_peered":"2024-09-10T11:32:30.157334+0000","last_clean":"2024-09-10T11:32:30.157334+0000","last_became_active":"2024-09-10T11:32:28.916353+0000","last_became_peered":"2024-09-10T11:32:28.916353+0000","last_unstale":"2024-09-10T11:32:30.157334+0000","last_undegraded":"2024-09-10T11:32:30.157334+0000","last_fullsized":"2024-09-10T11:32:30.157334+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:19.519642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910969+0000","last_change":"2024-09-10T11:32:28.927901+0000","last_active":"2024-09-10T11:32:30.910969+0000","last_peered":"2024-09-10T11:32:30.910969+0000","last_clean":"2024-09-10T11:32:30.910969+0000","last_became_active":"2024-09-10T11:32:28.927536+0000","last_became_peered":"2024-09-10T11:32:28.927536+0000","last_unstale":"2024-09-10T11:32:30.910969+0000","last_undegraded":"2024-09-10T11:32:30.910969+0000","last_fullsized":"2024-09-10T11:32:30.910969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:52.040179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157516+0000","last_change":"2024-09-10T11:32:28.916345+0000","last_active":"2024-09-10T11:32:30.157516+0000","last_peered":"2024-09-10T11:32:30.157516+0000","last_clean":"2024-09-10T11:32:30.157516+0000","last_became_active":"2024-09-10T11:32:28.916084+0000","last_became_peered":"2024-09-10T11:32:28.916084+0000","last_unstale":"2024-09-10T11:32:30.157516+0000","last_undegraded":"2024-09-10T11:32:30.157516+0000","last_fullsized":"2024-09-10T11:32:30.157516+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:14.467235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.909016+0000","last_change":"2024-09-10T11:32:28.925069+0000","last_active":"2024-09-10T11:32:29.909016+0000","last_peered":"2024-09-10T11:32:29.909016+0000","last_clean":"2024-09-10T11:32:29.909016+0000","last_became_active":"2024-09-10T11:32:28.924846+0000","last_became_peered":"2024-09-10T11:32:28.924846+0000","last_unstale":"2024-09-10T11:32:29.909016+0000","last_undegraded":"2024-09-10T11:32:29.909016+0000","last_fullsized":"2024-09-10T11:32:29.909016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:41.844007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908434+0000","last_change":"2024-09-10T11:32:28.921320+0000","last_active":"2024-09-10T11:32:29.908434+0000","last_peered":"2024-09-10T11:32:29.908434+0000","last_clean":"2024-09-10T11:32:29.908434+0000","last_became_active":"2024-09-10T11:32:28.921100+0000","last_became_peered":"2024-09-10T11:32:28.921100+0000","last_unstale":"2024-09-10T11:32:29.908434+0000","last_undegraded":"2024-09-10T11:32:29.908434+0000","last_fullsized":"2024-09-10T11:32:29.908434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:50.171686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910955+0000","last_change":"2024-09-10T11:32:28.927274+0000","last_active":"2024-09-10T11:32:30.910955+0000","last_peered":"2024-09-10T11:32:30.910955+0000","last_clean":"2024-09-10T11:32:30.910955+0000","last_became_active":"2024-09-10T11:32:28.927049+0000","last_became_peered":"2024-09-10T11:32:28.927049+0000","last_unstale":"2024-09-10T11:32:30.910955+0000","last_undegraded":"2024-09-10T11:32:30.910955+0000","last_fullsized":"2024-09-10T11:32:30.910955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:17.168093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.176898+0000","last_change":"2024-09-10T11:32:28.925682+0000","last_active":"2024-09-10T11:32:31.176898+0000","last_peered":"2024-09-10T11:32:31.176898+0000","last_clean":"2024-09-10T11:32:31.176898+0000","last_became_active":"2024-09-10T11:32:28.925144+0000","last_became_peered":"2024-09-10T11:32:28.925144+0000","last_unstale":"2024-09-10T11:32:31.176898+0000","last_undegraded":"2024-09-10T11:32:31.176898+0000","last_fullsized":"2024-09-10T11:32:31.176898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:45:48.901968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911454+0000","last_change":"2024-09-10T11:32:28.916505+0000","last_active":"2024-09-10T11:32:29.911454+0000","last_peered":"2024-09-10T11:32:29.911454+0000","last_clean":"2024-09-10T11:32:29.911454+0000","last_became_active":"2024-09-10T11:32:28.916271+0000","last_became_peered":"2024-09-10T11:32:28.916271+0000","last_unstale":"2024-09-10T11:32:29.911454+0000","last_undegraded":"2024-09-10T11:32:29.911454+0000","last_fullsized":"2024-09-10T11:32:29.911454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:25.154266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157337+0000","last_change":"2024-09-10T11:32:28.923121+0000","last_active":"2024-09-10T11:32:30.157337+0000","last_peered":"2024-09-10T11:32:30.157337+0000","last_clean":"2024-09-10T11:32:30.157337+0000","last_became_active":"2024-09-10T11:32:28.922878+0000","last_became_peered":"2024-09-10T11:32:28.922878+0000","last_unstale":"2024-09-10T11:32:30.157337+0000","last_undegraded":"2024-09-10T11:32:30.157337+0000","last_fullsized":"2024-09-10T11:32:30.157337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:29.689180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911314+0000","last_change":"2024-09-10T11:32:28.919842+0000","last_active":"2024-09-10T11:32:29.911314+0000","last_peered":"2024-09-10T11:32:29.911314+0000","last_clean":"2024-09-10T11:32:29.911314+0000","last_became_active":"2024-09-10T11:32:28.918964+0000","last_became_peered":"2024-09-10T11:32:28.918964+0000","last_unstale":"2024-09-10T11:32:29.911314+0000","last_undegraded":"2024-09-10T11:32:29.911314+0000","last_fullsized":"2024-09-10T11:32:29.911314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:55.977984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387509+0000","last_change":"2024-09-10T11:32:28.928259+0000","last_active":"2024-09-10T11:32:31.387509+0000","last_peered":"2024-09-10T11:32:31.387509+0000","last_clean":"2024-09-10T11:32:31.387509+0000","last_became_active":"2024-09-10T11:32:28.922888+0000","last_became_peered":"2024-09-10T11:32:28.922888+0000","last_unstale":"2024-09-10T11:32:31.387509+0000","last_undegraded":"2024-09-10T11:32:31.387509+0000","last_fullsized":"2024-09-10T11:32:31.387509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:30.764399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911050+0000","last_change":"2024-09-10T11:32:28.919002+0000","last_active":"2024-09-10T11:32:29.911050+0000","last_peered":"2024-09-10T11:32:29.911050+0000","last_clean":"2024-09-10T11:32:29.911050+0000","last_became_active":"2024-09-10T11:32:28.917834+0000","last_became_peered":"2024-09-10T11:32:28.917834+0000","last_unstale":"2024-09-10T11:32:29.911050+0000","last_undegraded":"2024-09-10T11:32:29.911050+0000","last_fullsized":"2024-09-10T11:32:29.911050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:07.627027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177180+0000","last_change":"2024-09-10T11:32:28.921152+0000","last_active":"2024-09-10T11:32:31.177180+0000","last_peered":"2024-09-10T11:32:31.177180+0000","last_clean":"2024-09-10T11:32:31.177180+0000","last_became_active":"2024-09-10T11:32:28.920515+0000","last_became_peered":"2024-09-10T11:32:28.920515+0000","last_unstale":"2024-09-10T11:32:31.177180+0000","last_undegraded":"2024-09-10T11:32:31.177180+0000","last_fullsized":"2024-09-10T11:32:31.177180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:03.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157505+0000","last_change":"2024-09-10T11:32:28.925784+0000","last_active":"2024-09-10T11:32:30.157505+0000","last_peered":"2024-09-10T11:32:30.157505+0000","last_clean":"2024-09-10T11:32:30.157505+0000","last_became_active":"2024-09-10T11:32:28.925543+0000","last_became_peered":"2024-09-10T11:32:28.925543+0000","last_unstale":"2024-09-10T11:32:30.157505+0000","last_undegraded":"2024-09-10T11:32:30.157505+0000","last_fullsized":"2024-09-10T11:32:30.157505+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:15.277875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177314+0000","last_change":"2024-09-10T11:32:28.921681+0000","last_active":"2024-09-10T11:32:31.177314+0000","last_peered":"2024-09-10T11:32:31.177314+0000","last_clean":"2024-09-10T11:32:31.177314+0000","last_became_active":"2024-09-10T11:32:28.921400+0000","last_became_peered":"2024-09-10T11:32:28.921400+0000","last_unstale":"2024-09-10T11:32:31.177314+0000","last_undegraded":"2024-09-10T11:32:31.177314+0000","last_fullsized":"2024-09-10T11:32:31.177314+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:48:11.028184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388056+0000","last_change":"2024-09-10T11:32:28.924430+0000","last_active":"2024-09-10T11:32:31.388056+0000","last_peered":"2024-09-10T11:32:31.388056+0000","last_clean":"2024-09-10T11:32:31.388056+0000","last_became_active":"2024-09-10T11:32:28.924202+0000","last_became_peered":"2024-09-10T11:32:28.924202+0000","last_unstale":"2024-09-10T11:32:31.388056+0000","last_undegraded":"2024-09-10T11:32:31.388056+0000","last_fullsized":"2024-09-10T11:32:31.388056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:38.282568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387995+0000","last_change":"2024-09-10T11:32:28.926316+0000","last_active":"2024-09-10T11:32:31.387995+0000","last_peered":"2024-09-10T11:32:31.387995+0000","last_clean":"2024-09-10T11:32:31.387995+0000","last_became_active":"2024-09-10T11:32:28.925468+0000","last_became_peered":"2024-09-10T11:32:28.925468+0000","last_unstale":"2024-09-10T11:32:31.387995+0000","last_undegraded":"2024-09-10T11:32:31.387995+0000","last_fullsized":"2024-09-10T11:32:31.387995+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:11.330797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387877+0000","last_change":"2024-09-10T11:32:28.925325+0000","last_active":"2024-09-10T11:32:31.387877+0000","last_peered":"2024-09-10T11:32:31.387877+0000","last_clean":"2024-09-10T11:32:31.387877+0000","last_became_active":"2024-09-10T11:32:28.924919+0000","last_became_peered":"2024-09-10T11:32:28.924919+0000","last_unstale":"2024-09-10T11:32:31.387877+0000","last_undegraded":"2024-09-10T11:32:31.387877+0000","last_fullsized":"2024-09-10T11:32:31.387877+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:24.176717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177215+0000","last_change":"2024-09-10T11:32:28.918165+0000","last_active":"2024-09-10T11:32:31.177215+0000","last_peered":"2024-09-10T11:32:31.177215+0000","last_clean":"2024-09-10T11:32:31.177215+0000","last_became_active":"2024-09-10T11:32:28.917204+0000","last_became_peered":"2024-09-10T11:32:28.917204+0000","last_unstale":"2024-09-10T11:32:31.177215+0000","last_undegraded":"2024-09-10T11:32:31.177215+0000","last_fullsized":"2024-09-10T11:32:31.177215+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:49.144106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911458+0000","last_change":"2024-09-10T11:32:28.920093+0000","last_active":"2024-09-10T11:32:29.911458+0000","last_peered":"2024-09-10T11:32:29.911458+0000","last_clean":"2024-09-10T11:32:29.911458+0000","last_became_active":"2024-09-10T11:32:28.919277+0000","last_became_peered":"2024-09-10T11:32:28.919277+0000","last_unstale":"2024-09-10T11:32:29.911458+0000","last_undegraded":"2024-09-10T11:32:29.911458+0000","last_fullsized":"2024-09-10T11:32:29.911458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:22.583567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908455+0000","last_change":"2024-09-10T11:32:28.916763+0000","last_active":"2024-09-10T11:32:29.908455+0000","last_peered":"2024-09-10T11:32:29.908455+0000","last_clean":"2024-09-10T11:32:29.908455+0000","last_became_active":"2024-09-10T11:32:28.916486+0000","last_became_peered":"2024-09-10T11:32:28.916486+0000","last_unstale":"2024-09-10T11:32:29.908455+0000","last_undegraded":"2024-09-10T11:32:29.908455+0000","last_fullsized":"2024-09-10T11:32:29.908455+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:27.817260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"65'4","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911241+0000","last_change":"2024-09-10T11:32:27.912191+0000","last_active":"2024-09-10T11:32:29.911241+0000","last_peered":"2024-09-10T11:32:29.911241+0000","last_clean":"2024-09-10T11:32:29.911241+0000","last_became_active":"2024-09-10T11:32:27.911967+0000","last_became_peered":"2024-09-10T11:32:27.911967+0000","last_unstale":"2024-09-10T11:32:29.911241+0000","last_undegraded":"2024-09-10T11:32:29.911241+0000","last_fullsized":"2024-09-10T11:32:29.911241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:43:20.619592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908567+0000","last_change":"2024-09-10T11:32:28.914101+0000","last_active":"2024-09-10T11:32:29.908567+0000","last_peered":"2024-09-10T11:32:29.908567+0000","last_clean":"2024-09-10T11:32:29.908567+0000","last_became_active":"2024-09-10T11:32:28.913622+0000","last_became_peered":"2024-09-10T11:32:28.913622+0000","last_unstale":"2024-09-10T11:32:29.908567+0000","last_undegraded":"2024-09-10T11:32:29.908567+0000","last_fullsized":"2024-09-10T11:32:29.908567+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:07.073954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.176974+0000","last_change":"2024-09-10T11:32:27.913855+0000","last_active":"2024-09-10T11:32:31.176974+0000","last_peered":"2024-09-10T11:32:31.176974+0000","last_clean":"2024-09-10T11:32:31.176974+0000","last_became_active":"2024-09-10T11:32:27.913606+0000","last_became_peered":"2024-09-10T11:32:27.913606+0000","last_unstale":"2024-09-10T11:32:31.176974+0000","last_undegraded":"2024-09-10T11:32:31.176974+0000","last_fullsized":"2024-09-10T11:32:31.176974+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:54.825875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388075+0000","last_change":"2024-09-10T11:32:28.923480+0000","last_active":"2024-09-10T11:32:31.388075+0000","last_peered":"2024-09-10T11:32:31.388075+0000","last_clean":"2024-09-10T11:32:31.388075+0000","last_became_active":"2024-09-10T11:32:28.923013+0000","last_became_peered":"2024-09-10T11:32:28.923013+0000","last_unstale":"2024-09-10T11:32:31.388075+0000","last_undegraded":"2024-09-10T11:32:31.388075+0000","last_fullsized":"2024-09-10T11:32:31.388075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:38.753869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908955+0000","last_change":"2024-09-10T11:32:27.910254+0000","last_active":"2024-09-10T11:32:29.908955+0000","last_peered":"2024-09-10T11:32:29.908955+0000","last_clean":"2024-09-10T11:32:29.908955+0000","last_became_active":"2024-09-10T11:32:27.909522+0000","last_became_peered":"2024-09-10T11:32:27.909522+0000","last_unstale":"2024-09-10T11:32:29.908955+0000","last_undegraded":"2024-09-10T11:32:29.908955+0000","last_fullsized":"2024-09-10T11:32:29.908955+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:39.946197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908628+0000","last_change":"2024-09-10T11:32:28.915365+0000","last_active":"2024-09-10T11:32:29.908628+0000","last_peered":"2024-09-10T11:32:29.908628+0000","last_clean":"2024-09-10T11:32:29.908628+0000","last_became_active":"2024-09-10T11:32:28.914284+0000","last_became_peered":"2024-09-10T11:32:28.914284+0000","last_unstale":"2024-09-10T11:32:29.908628+0000","last_undegraded":"2024-09-10T11:32:29.908628+0000","last_fullsized":"2024-09-10T11:32:29.908628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:07.819464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387720+0000","last_change":"2024-09-10T11:32:27.912688+0000","last_active":"2024-09-10T11:32:31.387720+0000","last_peered":"2024-09-10T11:32:31.387720+0000","last_clean":"2024-09-10T11:32:31.387720+0000","last_became_active":"2024-09-10T11:32:27.912475+0000","last_became_peered":"2024-09-10T11:32:27.912475+0000","last_unstale":"2024-09-10T11:32:31.387720+0000","last_undegraded":"2024-09-10T11:32:31.387720+0000","last_fullsized":"2024-09-10T11:32:31.387720+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:25.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388230+0000","last_change":"2024-09-10T11:32:28.923672+0000","last_active":"2024-09-10T11:32:31.388230+0000","last_peered":"2024-09-10T11:32:31.388230+0000","last_clean":"2024-09-10T11:32:31.388230+0000","last_became_active":"2024-09-10T11:32:28.923432+0000","last_became_peered":"2024-09-10T11:32:28.923432+0000","last_unstale":"2024-09-10T11:32:31.388230+0000","last_undegraded":"2024-09-10T11:32:31.388230+0000","last_fullsized":"2024-09-10T11:32:31.388230+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:24.067073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911133+0000","last_change":"2024-09-10T11:32:27.919406+0000","last_active":"2024-09-10T11:32:30.911133+0000","last_peered":"2024-09-10T11:32:30.911133+0000","last_clean":"2024-09-10T11:32:30.911133+0000","last_became_active":"2024-09-10T11:32:27.919144+0000","last_became_peered":"2024-09-10T11:32:27.919144+0000","last_unstale":"2024-09-10T11:32:30.911133+0000","last_undegraded":"2024-09-10T11:32:30.911133+0000","last_fullsized":"2024-09-10T11:32:30.911133+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:07.561720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911599+0000","last_change":"2024-09-10T11:32:28.920340+0000","last_active":"2024-09-10T11:32:29.911599+0000","last_peered":"2024-09-10T11:32:29.911599+0000","last_clean":"2024-09-10T11:32:29.911599+0000","last_became_active":"2024-09-10T11:32:28.919600+0000","last_became_peered":"2024-09-10T11:32:28.919600+0000","last_unstale":"2024-09-10T11:32:29.911599+0000","last_undegraded":"2024-09-10T11:32:29.911599+0000","last_fullsized":"2024-09-10T11:32:29.911599+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:56.702146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908189+0000","last_change":"2024-09-10T11:32:27.917365+0000","last_active":"2024-09-10T11:32:29.908189+0000","last_peered":"2024-09-10T11:32:29.908189+0000","last_clean":"2024-09-10T11:32:29.908189+0000","last_became_active":"2024-09-10T11:32:27.916293+0000","last_became_peered":"2024-09-10T11:32:27.916293+0000","last_unstale":"2024-09-10T11:32:29.908189+0000","last_undegraded":"2024-09-10T11:32:29.908189+0000","last_fullsized":"2024-09-10T11:32:29.908189+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:25.818896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910968+0000","last_change":"2024-09-10T11:32:28.926149+0000","last_active":"2024-09-10T11:32:30.910968+0000","last_peered":"2024-09-10T11:32:30.910968+0000","last_clean":"2024-09-10T11:32:30.910968+0000","last_became_active":"2024-09-10T11:32:28.925920+0000","last_became_peered":"2024-09-10T11:32:28.925920+0000","last_unstale":"2024-09-10T11:32:30.910968+0000","last_undegraded":"2024-09-10T11:32:30.910968+0000","last_fullsized":"2024-09-10T11:32:30.910968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:55.406309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911233+0000","last_change":"2024-09-10T11:32:27.916209+0000","last_active":"2024-09-10T11:32:29.911233+0000","last_peered":"2024-09-10T11:32:29.911233+0000","last_clean":"2024-09-10T11:32:29.911233+0000","last_became_active":"2024-09-10T11:32:27.915221+0000","last_became_peered":"2024-09-10T11:32:27.915221+0000","last_unstale":"2024-09-10T11:32:29.911233+0000","last_undegraded":"2024-09-10T11:32:29.911233+0000","last_fullsized":"2024-09-10T11:32:29.911233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:50.911341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908781+0000","last_change":"2024-09-10T11:32:28.928160+0000","last_active":"2024-09-10T11:32:29.908781+0000","last_peered":"2024-09-10T11:32:29.908781+0000","last_clean":"2024-09-10T11:32:29.908781+0000","last_became_active":"2024-09-10T11:32:28.927877+0000","last_became_peered":"2024-09-10T11:32:28.927877+0000","last_unstale":"2024-09-10T11:32:29.908781+0000","last_undegraded":"2024-09-10T11:32:29.908781+0000","last_fullsized":"2024-09-10T11:32:29.908781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:38.324873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157542+0000","last_change":"2024-09-10T11:32:27.911435+0000","last_active":"2024-09-10T11:32:30.157542+0000","last_peered":"2024-09-10T11:32:30.157542+0000","last_clean":"2024-09-10T11:32:30.157542+0000","last_became_active":"2024-09-10T11:32:27.910879+0000","last_became_peered":"2024-09-10T11:32:27.910879+0000","last_unstale":"2024-09-10T11:32:30.157542+0000","last_undegraded":"2024-09-10T11:32:30.157542+0000","last_fullsized":"2024-09-10T11:32:30.157542+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:09.144058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157401+0000","last_change":"2024-09-10T11:32:28.916127+0000","last_active":"2024-09-10T11:32:30.157401+0000","last_peered":"2024-09-10T11:32:30.157401+0000","last_clean":"2024-09-10T11:32:30.157401+0000","last_became_active":"2024-09-10T11:32:28.915852+0000","last_became_peered":"2024-09-10T11:32:28.915852+0000","last_unstale":"2024-09-10T11:32:30.157401+0000","last_undegraded":"2024-09-10T11:32:30.157401+0000","last_fullsized":"2024-09-10T11:32:30.157401+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:54.431522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911024+0000","last_change":"2024-09-10T11:32:27.915364+0000","last_active":"2024-09-10T11:32:29.911024+0000","last_peered":"2024-09-10T11:32:29.911024+0000","last_clean":"2024-09-10T11:32:29.911024+0000","last_became_active":"2024-09-10T11:32:27.914935+0000","last_became_peered":"2024-09-10T11:32:27.914935+0000","last_unstale":"2024-09-10T11:32:29.911024+0000","last_undegraded":"2024-09-10T11:32:29.911024+0000","last_fullsized":"2024-09-10T11:32:29.911024+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:28:51.168685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.910926+0000","last_change":"2024-09-10T11:32:28.919547+0000","last_active":"2024-09-10T11:32:30.910926+0000","last_peered":"2024-09-10T11:32:30.910926+0000","last_clean":"2024-09-10T11:32:30.910926+0000","last_became_active":"2024-09-10T11:32:28.919304+0000","last_became_peered":"2024-09-10T11:32:28.919304+0000","last_unstale":"2024-09-10T11:32:30.910926+0000","last_undegraded":"2024-09-10T11:32:30.910926+0000","last_fullsized":"2024-09-10T11:32:30.910926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:19.649704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387820+0000","last_change":"2024-09-10T11:32:27.916688+0000","last_active":"2024-09-10T11:32:31.387820+0000","last_peered":"2024-09-10T11:32:31.387820+0000","last_clean":"2024-09-10T11:32:31.387820+0000","last_became_active":"2024-09-10T11:32:27.915700+0000","last_became_peered":"2024-09-10T11:32:27.915700+0000","last_unstale":"2024-09-10T11:32:31.387820+0000","last_undegraded":"2024-09-10T11:32:31.387820+0000","last_fullsized":"2024-09-10T11:32:31.387820+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:26.238719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388158+0000","last_change":"2024-09-10T11:32:28.926115+0000","last_active":"2024-09-10T11:32:31.388158+0000","last_peered":"2024-09-10T11:32:31.388158+0000","last_clean":"2024-09-10T11:32:31.388158+0000","last_became_active":"2024-09-10T11:32:28.925145+0000","last_became_peered":"2024-09-10T11:32:28.925145+0000","last_unstale":"2024-09-10T11:32:31.388158+0000","last_undegraded":"2024-09-10T11:32:31.388158+0000","last_fullsized":"2024-09-10T11:32:31.388158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:19.497028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"65'3","reported_seq":17,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387687+0000","last_change":"2024-09-10T11:32:27.917336+0000","last_active":"2024-09-10T11:32:31.387687+0000","last_peered":"2024-09-10T11:32:31.387687+0000","last_clean":"2024-09-10T11:32:31.387687+0000","last_became_active":"2024-09-10T11:32:27.917107+0000","last_became_peered":"2024-09-10T11:32:27.917107+0000","last_unstale":"2024-09-10T11:32:31.387687+0000","last_undegraded":"2024-09-10T11:32:31.387687+0000","last_fullsized":"2024-09-10T11:32:31.387687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:38.891665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387652+0000","last_change":"2024-09-10T11:32:28.920991+0000","last_active":"2024-09-10T11:32:31.387652+0000","last_peered":"2024-09-10T11:32:31.387652+0000","last_clean":"2024-09-10T11:32:31.387652+0000","last_became_active":"2024-09-10T11:32:28.920729+0000","last_became_peered":"2024-09-10T11:32:28.920729+0000","last_unstale":"2024-09-10T11:32:31.387652+0000","last_undegraded":"2024-09-10T11:32:31.387652+0000","last_fullsized":"2024-09-10T11:32:31.387652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:25:36.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"65'6","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157296+0000","last_change":"2024-09-10T11:32:27.912712+0000","last_active":"2024-09-10T11:32:30.157296+0000","last_peered":"2024-09-10T11:32:30.157296+0000","last_clean":"2024-09-10T11:32:30.157296+0000","last_became_active":"2024-09-10T11:32:27.912422+0000","last_became_peered":"2024-09-10T11:32:27.912422+0000","last_unstale":"2024-09-10T11:32:30.157296+0000","last_undegraded":"2024-09-10T11:32:30.157296+0000","last_fullsized":"2024-09-10T11:32:30.157296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:23:16.893039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910980+0000","last_change":"2024-09-10T11:32:28.914223+0000","last_active":"2024-09-10T11:32:29.910980+0000","last_peered":"2024-09-10T11:32:29.910980+0000","last_clean":"2024-09-10T11:32:29.910980+0000","last_became_active":"2024-09-10T11:32:28.913991+0000","last_became_peered":"2024-09-10T11:32:28.913991+0000","last_unstale":"2024-09-10T11:32:29.910980+0000","last_undegraded":"2024-09-10T11:32:29.910980+0000","last_fullsized":"2024-09-10T11:32:29.910980+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:49.547445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388007+0000","last_change":"2024-09-10T11:32:27.917530+0000","last_active":"2024-09-10T11:32:31.388007+0000","last_peered":"2024-09-10T11:32:31.388007+0000","last_clean":"2024-09-10T11:32:31.388007+0000","last_became_active":"2024-09-10T11:32:27.917064+0000","last_became_peered":"2024-09-10T11:32:27.917064+0000","last_unstale":"2024-09-10T11:32:31.388007+0000","last_undegraded":"2024-09-10T11:32:31.388007+0000","last_fullsized":"2024-09-10T11:32:31.388007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:44:49.040919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911140+0000","last_change":"2024-09-10T11:32:28.920392+0000","last_active":"2024-09-10T11:32:30.911140+0000","last_peered":"2024-09-10T11:32:30.911140+0000","last_clean":"2024-09-10T11:32:30.911140+0000","last_became_active":"2024-09-10T11:32:28.920143+0000","last_became_peered":"2024-09-10T11:32:28.920143+0000","last_unstale":"2024-09-10T11:32:30.911140+0000","last_undegraded":"2024-09-10T11:32:30.911140+0000","last_fullsized":"2024-09-10T11:32:30.911140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:02.766871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387813+0000","last_change":"2024-09-10T11:32:27.909637+0000","last_active":"2024-09-10T11:32:31.387813+0000","last_peered":"2024-09-10T11:32:31.387813+0000","last_clean":"2024-09-10T11:32:31.387813+0000","last_became_active":"2024-09-10T11:32:27.909320+0000","last_became_peered":"2024-09-10T11:32:27.909320+0000","last_unstale":"2024-09-10T11:32:31.387813+0000","last_undegraded":"2024-09-10T11:32:31.387813+0000","last_fullsized":"2024-09-10T11:32:31.387813+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:14.679136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.910944+0000","last_change":"2024-09-10T11:32:28.916372+0000","last_active":"2024-09-10T11:32:29.910944+0000","last_peered":"2024-09-10T11:32:29.910944+0000","last_clean":"2024-09-10T11:32:29.910944+0000","last_became_active":"2024-09-10T11:32:28.916155+0000","last_became_peered":"2024-09-10T11:32:28.916155+0000","last_unstale":"2024-09-10T11:32:29.910944+0000","last_undegraded":"2024-09-10T11:32:29.910944+0000","last_fullsized":"2024-09-10T11:32:29.910944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:30:48.396127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387797+0000","last_change":"2024-09-10T11:32:27.912984+0000","last_active":"2024-09-10T11:32:31.387797+0000","last_peered":"2024-09-10T11:32:31.387797+0000","last_clean":"2024-09-10T11:32:31.387797+0000","last_became_active":"2024-09-10T11:32:27.912763+0000","last_became_peered":"2024-09-10T11:32:27.912763+0000","last_unstale":"2024-09-10T11:32:31.387797+0000","last_undegraded":"2024-09-10T11:32:31.387797+0000","last_fullsized":"2024-09-10T11:32:31.387797+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:26.180721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387753+0000","last_change":"2024-09-10T11:32:28.929562+0000","last_active":"2024-09-10T11:32:31.387753+0000","last_peered":"2024-09-10T11:32:31.387753+0000","last_clean":"2024-09-10T11:32:31.387753+0000","last_became_active":"2024-09-10T11:32:28.929284+0000","last_became_peered":"2024-09-10T11:32:28.929284+0000","last_unstale":"2024-09-10T11:32:31.387753+0000","last_undegraded":"2024-09-10T11:32:31.387753+0000","last_fullsized":"2024-09-10T11:32:31.387753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:04:52.197825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"65'8","reported_seq":22,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387745+0000","last_change":"2024-09-10T11:32:27.910409+0000","last_active":"2024-09-10T11:32:31.387745+0000","last_peered":"2024-09-10T11:32:31.387745+0000","last_clean":"2024-09-10T11:32:31.387745+0000","last_became_active":"2024-09-10T11:32:27.910164+0000","last_became_peered":"2024-09-10T11:32:27.910164+0000","last_unstale":"2024-09-10T11:32:31.387745+0000","last_undegraded":"2024-09-10T11:32:31.387745+0000","last_fullsized":"2024-09-10T11:32:31.387745+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:19.665894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911255+0000","last_change":"2024-09-10T11:32:28.920876+0000","last_active":"2024-09-10T11:32:30.911255+0000","last_peered":"2024-09-10T11:32:30.911255+0000","last_clean":"2024-09-10T11:32:30.911255+0000","last_became_active":"2024-09-10T11:32:28.920652+0000","last_became_peered":"2024-09-10T11:32:28.920652+0000","last_unstale":"2024-09-10T11:32:30.911255+0000","last_undegraded":"2024-09-10T11:32:30.911255+0000","last_fullsized":"2024-09-10T11:32:30.911255+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:21.524810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177097+0000","last_change":"2024-09-10T11:32:27.915031+0000","last_active":"2024-09-10T11:32:31.177097+0000","last_peered":"2024-09-10T11:32:31.177097+0000","last_clean":"2024-09-10T11:32:31.177097+0000","last_became_active":"2024-09-10T11:32:27.914759+0000","last_became_peered":"2024-09-10T11:32:27.914759+0000","last_unstale":"2024-09-10T11:32:31.177097+0000","last_undegraded":"2024-09-10T11:32:31.177097+0000","last_fullsized":"2024-09-10T11:32:31.177097+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:34.528205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911139+0000","last_change":"2024-09-10T11:32:28.927745+0000","last_active":"2024-09-10T11:32:30.911139+0000","last_peered":"2024-09-10T11:32:30.911139+0000","last_clean":"2024-09-10T11:32:30.911139+0000","last_became_active":"2024-09-10T11:32:28.927195+0000","last_became_peered":"2024-09-10T11:32:28.927195+0000","last_unstale":"2024-09-10T11:32:30.911139+0000","last_undegraded":"2024-09-10T11:32:30.911139+0000","last_fullsized":"2024-09-10T11:32:30.911139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:48.968800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388127+0000","last_change":"2024-09-10T11:32:27.917338+0000","last_active":"2024-09-10T11:32:31.388127+0000","last_peered":"2024-09-10T11:32:31.388127+0000","last_clean":"2024-09-10T11:32:31.388127+0000","last_became_active":"2024-09-10T11:32:27.916552+0000","last_became_peered":"2024-09-10T11:32:27.916552+0000","last_unstale":"2024-09-10T11:32:31.388127+0000","last_undegraded":"2024-09-10T11:32:31.388127+0000","last_fullsized":"2024-09-10T11:32:31.388127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:51.122272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177414+0000","last_change":"2024-09-10T11:32:28.918408+0000","last_active":"2024-09-10T11:32:31.177414+0000","last_peered":"2024-09-10T11:32:31.177414+0000","last_clean":"2024-09-10T11:32:31.177414+0000","last_became_active":"2024-09-10T11:32:28.917381+0000","last_became_peered":"2024-09-10T11:32:28.917381+0000","last_unstale":"2024-09-10T11:32:31.177414+0000","last_undegraded":"2024-09-10T11:32:31.177414+0000","last_fullsized":"2024-09-10T11:32:31.177414+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:51.451751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"65'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911052+0000","last_change":"2024-09-10T11:32:27.911190+0000","last_active":"2024-09-10T11:32:30.911052+0000","last_peered":"2024-09-10T11:32:30.911052+0000","last_clean":"2024-09-10T11:32:30.911052+0000","last_became_active":"2024-09-10T11:32:27.910943+0000","last_became_peered":"2024-09-10T11:32:27.910943+0000","last_unstale":"2024-09-10T11:32:30.911052+0000","last_undegraded":"2024-09-10T11:32:30.911052+0000","last_fullsized":"2024-09-10T11:32:30.911052+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:44.463051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387674+0000","last_change":"2024-09-10T11:32:28.925502+0000","last_active":"2024-09-10T11:32:31.387674+0000","last_peered":"2024-09-10T11:32:31.387674+0000","last_clean":"2024-09-10T11:32:31.387674+0000","last_became_active":"2024-09-10T11:32:28.925008+0000","last_became_peered":"2024-09-10T11:32:28.925008+0000","last_unstale":"2024-09-10T11:32:31.387674+0000","last_undegraded":"2024-09-10T11:32:31.387674+0000","last_fullsized":"2024-09-10T11:32:31.387674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:36.190625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387916+0000","last_change":"2024-09-10T11:32:27.915945+0000","last_active":"2024-09-10T11:32:31.387916+0000","last_peered":"2024-09-10T11:32:31.387916+0000","last_clean":"2024-09-10T11:32:31.387916+0000","last_became_active":"2024-09-10T11:32:27.915250+0000","last_became_peered":"2024-09-10T11:32:27.915250+0000","last_unstale":"2024-09-10T11:32:31.387916+0000","last_undegraded":"2024-09-10T11:32:31.387916+0000","last_fullsized":"2024-09-10T11:32:31.387916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:14.229823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911208+0000","last_change":"2024-09-10T11:32:28.918778+0000","last_active":"2024-09-10T11:32:29.911208+0000","last_peered":"2024-09-10T11:32:29.911208+0000","last_clean":"2024-09-10T11:32:29.911208+0000","last_became_active":"2024-09-10T11:32:28.917535+0000","last_became_peered":"2024-09-10T11:32:28.917535+0000","last_unstale":"2024-09-10T11:32:29.911208+0000","last_undegraded":"2024-09-10T11:32:29.911208+0000","last_fullsized":"2024-09-10T11:32:29.911208+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:04.331758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911364+0000","last_change":"2024-09-10T11:32:27.911324+0000","last_active":"2024-09-10T11:32:29.911364+0000","last_peered":"2024-09-10T11:32:29.911364+0000","last_clean":"2024-09-10T11:32:29.911364+0000","last_became_active":"2024-09-10T11:32:27.911108+0000","last_became_peered":"2024-09-10T11:32:27.911108+0000","last_unstale":"2024-09-10T11:32:29.911364+0000","last_undegraded":"2024-09-10T11:32:29.911364+0000","last_fullsized":"2024-09-10T11:32:29.911364+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:56.827113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388458+0000","last_change":"2024-09-10T11:32:28.920979+0000","last_active":"2024-09-10T11:32:31.388458+0000","last_peered":"2024-09-10T11:32:31.388458+0000","last_clean":"2024-09-10T11:32:31.388458+0000","last_became_active":"2024-09-10T11:32:28.920739+0000","last_became_peered":"2024-09-10T11:32:28.920739+0000","last_unstale":"2024-09-10T11:32:31.388458+0000","last_undegraded":"2024-09-10T11:32:31.388458+0000","last_fullsized":"2024-09-10T11:32:31.388458+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:30.702845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388383+0000","last_change":"2024-09-10T11:32:27.917853+0000","last_active":"2024-09-10T11:32:31.388383+0000","last_peered":"2024-09-10T11:32:31.388383+0000","last_clean":"2024-09-10T11:32:31.388383+0000","last_became_active":"2024-09-10T11:32:27.916037+0000","last_became_peered":"2024-09-10T11:32:27.916037+0000","last_unstale":"2024-09-10T11:32:31.388383+0000","last_undegraded":"2024-09-10T11:32:31.388383+0000","last_fullsized":"2024-09-10T11:32:31.388383+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:13.213394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.387934+0000","last_change":"2024-09-10T11:32:28.923043+0000","last_active":"2024-09-10T11:32:31.387934+0000","last_peered":"2024-09-10T11:32:31.387934+0000","last_clean":"2024-09-10T11:32:31.387934+0000","last_became_active":"2024-09-10T11:32:28.922785+0000","last_became_peered":"2024-09-10T11:32:28.922785+0000","last_unstale":"2024-09-10T11:32:31.387934+0000","last_undegraded":"2024-09-10T11:32:31.387934+0000","last_fullsized":"2024-09-10T11:32:31.387934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.630557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911105+0000","last_change":"2024-09-10T11:32:27.914687+0000","last_active":"2024-09-10T11:32:29.911105+0000","last_peered":"2024-09-10T11:32:29.911105+0000","last_clean":"2024-09-10T11:32:29.911105+0000","last_became_active":"2024-09-10T11:32:27.914415+0000","last_became_peered":"2024-09-10T11:32:27.914415+0000","last_unstale":"2024-09-10T11:32:29.911105+0000","last_undegraded":"2024-09-10T11:32:29.911105+0000","last_fullsized":"2024-09-10T11:32:29.911105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:49.868334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911135+0000","last_change":"2024-09-10T11:32:28.926691+0000","last_active":"2024-09-10T11:32:30.911135+0000","last_peered":"2024-09-10T11:32:30.911135+0000","last_clean":"2024-09-10T11:32:30.911135+0000","last_became_active":"2024-09-10T11:32:28.926424+0000","last_became_peered":"2024-09-10T11:32:28.926424+0000","last_unstale":"2024-09-10T11:32:30.911135+0000","last_undegraded":"2024-09-10T11:32:30.911135+0000","last_fullsized":"2024-09-10T11:32:30.911135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:53.220598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"65'2","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177503+0000","last_change":"2024-09-10T11:32:27.914458+0000","last_active":"2024-09-10T11:32:31.177503+0000","last_peered":"2024-09-10T11:32:31.177503+0000","last_clean":"2024-09-10T11:32:31.177503+0000","last_became_active":"2024-09-10T11:32:27.914189+0000","last_became_peered":"2024-09-10T11:32:27.914189+0000","last_unstale":"2024-09-10T11:32:31.177503+0000","last_undegraded":"2024-09-10T11:32:31.177503+0000","last_fullsized":"2024-09-10T11:32:31.177503+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:42.807711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911075+0000","last_change":"2024-09-10T11:32:28.927573+0000","last_active":"2024-09-10T11:32:30.911075+0000","last_peered":"2024-09-10T11:32:30.911075+0000","last_clean":"2024-09-10T11:32:30.911075+0000","last_became_active":"2024-09-10T11:32:28.927293+0000","last_became_peered":"2024-09-10T11:32:28.927293+0000","last_unstale":"2024-09-10T11:32:30.911075+0000","last_undegraded":"2024-09-10T11:32:30.911075+0000","last_fullsized":"2024-09-10T11:32:30.911075+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:35.905304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"66'1","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908381+0000","last_change":"2024-09-10T11:32:27.916531+0000","last_active":"2024-09-10T11:32:29.908381+0000","last_peered":"2024-09-10T11:32:29.908381+0000","last_clean":"2024-09-10T11:32:29.908381+0000","last_became_active":"2024-09-10T11:32:27.915984+0000","last_became_peered":"2024-09-10T11:32:27.915984+0000","last_unstale":"2024-09-10T11:32:29.908381+0000","last_undegraded":"2024-09-10T11:32:29.908381+0000","last_fullsized":"2024-09-10T11:32:29.908381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:12.939020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177366+0000","last_change":"2024-09-10T11:32:28.927379+0000","last_active":"2024-09-10T11:32:31.177366+0000","last_peered":"2024-09-10T11:32:31.177366+0000","last_clean":"2024-09-10T11:32:31.177366+0000","last_became_active":"2024-09-10T11:32:28.927017+0000","last_became_peered":"2024-09-10T11:32:28.927017+0000","last_unstale":"2024-09-10T11:32:31.177366+0000","last_undegraded":"2024-09-10T11:32:31.177366+0000","last_fullsized":"2024-09-10T11:32:31.177366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:30.905206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911279+0000","last_change":"2024-09-10T11:32:27.916453+0000","last_active":"2024-09-10T11:32:29.911279+0000","last_peered":"2024-09-10T11:32:29.911279+0000","last_clean":"2024-09-10T11:32:29.911279+0000","last_became_active":"2024-09-10T11:32:27.915841+0000","last_became_peered":"2024-09-10T11:32:27.915841+0000","last_unstale":"2024-09-10T11:32:29.911279+0000","last_undegraded":"2024-09-10T11:32:29.911279+0000","last_fullsized":"2024-09-10T11:32:29.911279+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:53.154878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388271+0000","last_change":"2024-09-10T11:32:28.925887+0000","last_active":"2024-09-10T11:32:31.388271+0000","last_peered":"2024-09-10T11:32:31.388271+0000","last_clean":"2024-09-10T11:32:31.388271+0000","last_became_active":"2024-09-10T11:32:28.924747+0000","last_became_peered":"2024-09-10T11:32:28.924747+0000","last_unstale":"2024-09-10T11:32:31.388271+0000","last_undegraded":"2024-09-10T11:32:31.388271+0000","last_fullsized":"2024-09-10T11:32:31.388271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:00.050958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911289+0000","last_change":"2024-09-10T11:32:27.913504+0000","last_active":"2024-09-10T11:32:30.911289+0000","last_peered":"2024-09-10T11:32:30.911289+0000","last_clean":"2024-09-10T11:32:30.911289+0000","last_became_active":"2024-09-10T11:32:27.913056+0000","last_became_peered":"2024-09-10T11:32:27.913056+0000","last_unstale":"2024-09-10T11:32:30.911289+0000","last_undegraded":"2024-09-10T11:32:30.911289+0000","last_fullsized":"2024-09-10T11:32:30.911289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:30.598671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908622+0000","last_change":"2024-09-10T11:32:28.913342+0000","last_active":"2024-09-10T11:32:29.908622+0000","last_peered":"2024-09-10T11:32:29.908622+0000","last_clean":"2024-09-10T11:32:29.908622+0000","last_became_active":"2024-09-10T11:32:28.912948+0000","last_became_peered":"2024-09-10T11:32:28.912948+0000","last_unstale":"2024-09-10T11:32:29.908622+0000","last_undegraded":"2024-09-10T11:32:29.908622+0000","last_fullsized":"2024-09-10T11:32:29.908622+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:11.513720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177433+0000","last_change":"2024-09-10T11:32:27.914126+0000","last_active":"2024-09-10T11:32:31.177433+0000","last_peered":"2024-09-10T11:32:31.177433+0000","last_clean":"2024-09-10T11:32:31.177433+0000","last_became_active":"2024-09-10T11:32:27.913799+0000","last_became_peered":"2024-09-10T11:32:27.913799+0000","last_unstale":"2024-09-10T11:32:31.177433+0000","last_undegraded":"2024-09-10T11:32:31.177433+0000","last_fullsized":"2024-09-10T11:32:31.177433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:20.654320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157647+0000","last_change":"2024-09-10T11:32:28.921803+0000","last_active":"2024-09-10T11:32:30.157647+0000","last_peered":"2024-09-10T11:32:30.157647+0000","last_clean":"2024-09-10T11:32:30.157647+0000","last_became_active":"2024-09-10T11:32:28.921585+0000","last_became_peered":"2024-09-10T11:32:28.921585+0000","last_unstale":"2024-09-10T11:32:30.157647+0000","last_undegraded":"2024-09-10T11:32:30.157647+0000","last_fullsized":"2024-09-10T11:32:30.157647+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:55.729598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.911253+0000","last_change":"2024-09-10T11:32:27.920438+0000","last_active":"2024-09-10T11:32:30.911253+0000","last_peered":"2024-09-10T11:32:30.911253+0000","last_clean":"2024-09-10T11:32:30.911253+0000","last_became_active":"2024-09-10T11:32:27.920179+0000","last_became_peered":"2024-09-10T11:32:27.920179+0000","last_unstale":"2024-09-10T11:32:30.911253+0000","last_undegraded":"2024-09-10T11:32:30.911253+0000","last_fullsized":"2024-09-10T11:32:30.911253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:34.207814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388352+0000","last_change":"2024-09-10T11:32:28.925697+0000","last_active":"2024-09-10T11:32:31.388352+0000","last_peered":"2024-09-10T11:32:31.388352+0000","last_clean":"2024-09-10T11:32:31.388352+0000","last_became_active":"2024-09-10T11:32:28.924175+0000","last_became_peered":"2024-09-10T11:32:28.924175+0000","last_unstale":"2024-09-10T11:32:31.388352+0000","last_undegraded":"2024-09-10T11:32:31.388352+0000","last_fullsized":"2024-09-10T11:32:31.388352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:45.362113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.908638+0000","last_change":"2024-09-10T11:32:27.909076+0000","last_active":"2024-09-10T11:32:29.908638+0000","last_peered":"2024-09-10T11:32:29.908638+0000","last_clean":"2024-09-10T11:32:29.908638+0000","last_became_active":"2024-09-10T11:32:27.908530+0000","last_became_peered":"2024-09-10T11:32:27.908530+0000","last_unstale":"2024-09-10T11:32:29.908638+0000","last_undegraded":"2024-09-10T11:32:29.908638+0000","last_fullsized":"2024-09-10T11:32:29.908638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:38.130838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.177253+0000","last_change":"2024-09-10T11:32:28.913230+0000","last_active":"2024-09-10T11:32:31.177253+0000","last_peered":"2024-09-10T11:32:31.177253+0000","last_clean":"2024-09-10T11:32:31.177253+0000","last_became_active":"2024-09-10T11:32:28.912723+0000","last_became_peered":"2024-09-10T11:32:28.912723+0000","last_unstale":"2024-09-10T11:32:31.177253+0000","last_undegraded":"2024-09-10T11:32:31.177253+0000","last_fullsized":"2024-09-10T11:32:31.177253+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:02.066151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388438+0000","last_change":"2024-09-10T11:32:27.917517+0000","last_active":"2024-09-10T11:32:31.388438+0000","last_peered":"2024-09-10T11:32:31.388438+0000","last_clean":"2024-09-10T11:32:31.388438+0000","last_became_active":"2024-09-10T11:32:27.915521+0000","last_became_peered":"2024-09-10T11:32:27.915521+0000","last_unstale":"2024-09-10T11:32:31.388438+0000","last_undegraded":"2024-09-10T11:32:31.388438+0000","last_fullsized":"2024-09-10T11:32:31.388438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:24.467428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:31.388204+0000","last_change":"2024-09-10T11:32:28.920658+0000","last_active":"2024-09-10T11:32:31.388204+0000","last_peered":"2024-09-10T11:32:31.388204+0000","last_clean":"2024-09-10T11:32:31.388204+0000","last_became_active":"2024-09-10T11:32:28.920428+0000","last_became_peered":"2024-09-10T11:32:28.920428+0000","last_unstale":"2024-09-10T11:32:31.388204+0000","last_undegraded":"2024-09-10T11:32:31.388204+0000","last_fullsized":"2024-09-10T11:32:31.388204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:27.897264+0000","last_clean_scrub_stamp":"2024-09-10T11:32:27.897264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:38.032533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:30.157591+0000","last_change":"2024-09-10T11:32:27.911731+0000","last_active":"2024-09-10T11:32:30.157591+0000","last_peered":"2024-09-10T11:32:30.157591+0000","last_clean":"2024-09-10T11:32:30.157591+0000","last_became_active":"2024-09-10T11:32:27.911178+0000","last_became_peered":"2024-09-10T11:32:27.911178+0000","last_unstale":"2024-09-10T11:32:30.157591+0000","last_undegraded":"2024-09-10T11:32:30.157591+0000","last_fullsized":"2024-09-10T11:32:30.157591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:32:26.891726+0000","last_clean_scrub_stamp":"2024-09-10T11:32:26.891726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:23.877257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":261,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-10T11:32:29.911326+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:32:29.911326+0000","last_peered":"2024-09-10T11:32:29.911326+0000","last_clean":"2024-09-10T11:32:29.911326+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:32:29.911326+0000","last_undegraded":"2024-09-10T11:32:29.911326+0000","last_fullsized":"2024-09-10T11:32:29.911326+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:31.673361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199661449,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199661519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":900,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194411,"statfs":{"total":200000000,"available":199077439,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":331,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194981,"statfs":{"total":200000000,"available":199660925,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":324,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194988,"statfs":{"total":200000000,"available":199667751,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659769,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":907,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194404,"statfs":{"total":200000000,"available":199070591,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":332,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194980,"statfs":{"total":200000000,"available":199659929,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:32:36.514 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:32:36.514 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:32:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.653+0000 7f6d6be96640 1 Processor -- start 2024-09-10T11:32:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.653+0000 7f6d6be96640 1 -- start start 2024-09-10T11:32:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6be96640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d640d1200 0x7f6d64073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6be96640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d6406e5d0 0x7f6d6406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6be96640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d64152e60 con 0x7f6d6415c060 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6be96640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6d6406ef80 con 0x7f6d6406e5d0 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6be96640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6d640d15e0 con 0x7f6d640d1200 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d69431640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d6406e5d0 0x7f6d6406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6a433640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6d6415c060 0x7f6d6415c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47050/0 (socket says 172.21.15.99:47050) 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d6a433640 1 -- 172.21.15.99:0/3488747323 learned_addr learned my addr 172.21.15.99:0/3488747323 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.654+0000 7f6d69c32640 1 --2- 172.21.15.99:0/3488747323 >> v2:172.21.15.196:3300/0 conn(0x7f6d640d1200 0x7f6d64073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d69431640 1 -- 172.21.15.99:0/3488747323 >> v2:172.21.15.196:3300/0 conn(0x7f6d640d1200 msgr2=0x7f6d64073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:36.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d69431640 1 --2- 172.21.15.99:0/3488747323 >> v2:172.21.15.196:3300/0 conn(0x7f6d640d1200 0x7f6d64073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d69431640 1 -- 172.21.15.99:0/3488747323 >> v1:172.21.15.99:6789/0 conn(0x7f6d6415c060 legacy=0x7f6d6415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d69431640 1 -- 172.21.15.99:0/3488747323 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d64072680 con 0x7f6d6406e5d0 2024-09-10T11:32:36.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d68c30640 1 -- 172.21.15.99:0/3488747323 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870952675 0 0) 0x7f6d64152e60 con 0x7f6d6415c060 2024-09-10T11:32:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d69431640 1 --2- 172.21.15.99:0/3488747323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6406e5d0 0x7f6d6406ea40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d1850 tx=0x7f6d5402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a4bb30cd6bfd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d68c30640 1 -- 172.21.15.99:0/3488747323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54031710 con 0x7f6d6406e5d0 2024-09-10T11:32:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d68c30640 1 -- 172.21.15.99:0/3488747323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d540318b0 con 0x7f6d6406e5d0 2024-09-10T11:32:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.655+0000 7f6d68c30640 1 -- 172.21.15.99:0/3488747323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54031c80 con 0x7f6d6406e5d0 2024-09-10T11:32:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 -- 172.21.15.99:0/3488747323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6406e5d0 msgr2=0x7f6d6406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 --2- 172.21.15.99:0/3488747323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6406e5d0 0x7f6d6406ea40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d1850 tx=0x7f6d5402c7e0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 -- 172.21.15.99:0/3488747323 shutdown_connections 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 --2- 172.21.15.99:0/3488747323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6406e5d0 0x7f6d6406ea40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 --2- 172.21.15.99:0/3488747323 >> v2:172.21.15.196:3300/0 conn(0x7f6d640d1200 0x7f6d64073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 -- 172.21.15.99:0/3488747323 >> 172.21.15.99:0/3488747323 conn(0x7f6d6406bc10 msgr2=0x7f6d6406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 -- 172.21.15.99:0/3488747323 shutdown_connections 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.656+0000 7f6d6be96640 1 -- 172.21.15.99:0/3488747323 wait complete. 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.657+0000 7f6d6be96640 1 Processor -- start 2024-09-10T11:32:36.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.657+0000 7f6d6be96640 1 -- start start 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d6406e5d0 0x7f6d641430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d640d1200 0x7f6d64155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64072f20 con 0x7f6d6415c060 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64072c40 con 0x7f6d6406e5d0 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6be96640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6415cd90 con 0x7f6d640d1200 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42968/0 (socket says 172.21.15.99:42968) 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 -- 172.21.15.99:0/64539115 learned_addr learned my addr 172.21.15.99:0/64539115 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d69c32640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d6406e5d0 0x7f6d641430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d69431640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d640d1200 0x7f6d64155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 -- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d640d1200 msgr2=0x7f6d64155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d640d1200 0x7f6d64155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 -- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d6406e5d0 msgr2=0x7f6d641430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d6406e5d0 0x7f6d641430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.658+0000 7f6d6a433640 1 -- 172.21.15.99:0/64539115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d641565d0 con 0x7f6d6415c060 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d6a433640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c7a0 tx=0x7f6d5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c018070 con 0x7f6d6415c060 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00a640 con 0x7f6d6415c060 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d641568b0 con 0x7f6d6415c060 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00aa70 con 0x7f6d6415c060 2024-09-10T11:32:36.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.659+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d64156e70 con 0x7f6d6415c060 2024-09-10T11:32:36.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.660+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6406ea40 con 0x7f6d6415c060 2024-09-10T11:32:36.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.660+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d5c01b070 con 0x7f6d6415c060 2024-09-10T11:32:36.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.661+0000 7f6d5a7fc640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 0x7f6d4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:36.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.661+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f6d5c08a430 con 0x7f6d6415c060 2024-09-10T11:32:36.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.661+0000 7f6d69c32640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 0x7f6d4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:36.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.661+0000 7f6d69c32640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 0x7f6d4806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006fd0 tx=0x7f6d60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:36.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.665+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d5c058e70 con 0x7f6d6415c060 2024-09-10T11:32:36.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.892+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6d640d0420 con 0x7f6d6415c060 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.894+0000 7f6d5a7fc640 1 -- 172.21.15.99:0/64539115 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f6d5c058770 con 0x7f6d6415c060 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:32:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:32:36.897 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:32:36.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 msgr2=0x7f6d4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 0x7f6d4806e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006fd0 tx=0x7f6d60008040 comp rx=0 tx=0).stop 2024-09-10T11:32:36.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 msgr2=0x7f6d641559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:36.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c7a0 tx=0x7f6d5c00cc60 comp rx=0 tx=0).stop 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 shutdown_connections 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d4806be30 0x7f6d4806e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d6415c060 0x7f6d641559d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d640d1200 0x7f6d64155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 --2- 172.21.15.99:0/64539115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d6406e5d0 0x7f6d641430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 >> 172.21.15.99:0/64539115 conn(0x7f6d6406bc10 msgr2=0x7f6d640b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 shutdown_connections 2024-09-10T11:32:36.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:36.896+0000 7f6d6be96640 1 -- 172.21.15.99:0/64539115 wait complete. 2024-09-10T11:32:36.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:37.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.007+0000 7fb35b2e4640 1 Processor -- start 2024-09-10T11:32:37.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.007+0000 7fb35b2e4640 1 -- start start 2024-09-10T11:32:37.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb35b2e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb35415eb80 0x7fb3540d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb35b2e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 0x7fb3540d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb35b2e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35415ff70 con 0x7fb3540d4e40 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb35b2e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb3540d1620 con 0x7fb35415eb80 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb35b2e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb354162bc0 con 0x7fb3541626c0 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb359080640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb35415eb80 0x7fb3540d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb359881640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb3540d4e40 0x7fb3540d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47058/0 (socket says 172.21.15.99:47058) 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.008+0000 7fb359881640 1 -- 172.21.15.99:0/1802498652 learned_addr learned my addr 172.21.15.99:0/1802498652 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb35887f640 1 --2- 172.21.15.99:0/1802498652 >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 0x7fb3540d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb359080640 1 -- 172.21.15.99:0/1802498652 >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 msgr2=0x7fb3540d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb359080640 1 --2- 172.21.15.99:0/1802498652 >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 0x7fb3540d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb359080640 1 -- 172.21.15.99:0/1802498652 >> v1:172.21.15.99:6789/0 conn(0x7fb3540d4e40 legacy=0x7fb3540d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb359080640 1 -- 172.21.15.99:0/1802498652 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb354162ea0 con 0x7fb35415eb80 2024-09-10T11:32:37.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb353fff640 1 -- 172.21.15.99:0/1802498652 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629674423 0 0) 0x7fb35415ff70 con 0x7fb3540d4e40 2024-09-10T11:32:37.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb35887f640 1 --2- 172.21.15.99:0/1802498652 >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 0x7fb3540d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:37.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.009+0000 7fb359080640 1 --2- 172.21.15.99:0/1802498652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb35415eb80 0x7fb3540d29b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb344009d90 tx=0x7fb34402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e98f74d1edc1da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb353fff640 1 -- 172.21.15.99:0/1802498652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344031600 con 0x7fb35415eb80 2024-09-10T11:32:37.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb353fff640 1 -- 172.21.15.99:0/1802498652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3440317a0 con 0x7fb35415eb80 2024-09-10T11:32:37.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb353fff640 1 -- 172.21.15.99:0/1802498652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb344031b70 con 0x7fb35415eb80 2024-09-10T11:32:37.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 -- 172.21.15.99:0/1802498652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb35415eb80 msgr2=0x7fb3540d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/1802498652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb35415eb80 0x7fb3540d29b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb344009d90 tx=0x7fb34402a610 comp rx=0 tx=0).stop 2024-09-10T11:32:37.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 -- 172.21.15.99:0/1802498652 shutdown_connections 2024-09-10T11:32:37.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/1802498652 >> v2:172.21.15.196:3300/0 conn(0x7fb3541626c0 0x7fb3540d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/1802498652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb35415eb80 0x7fb3540d29b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 -- 172.21.15.99:0/1802498652 >> 172.21.15.99:0/1802498652 conn(0x7fb35408e680 msgr2=0x7fb35409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 -- 172.21.15.99:0/1802498652 shutdown_connections 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.010+0000 7fb35b2e4640 1 -- 172.21.15.99:0/1802498652 wait complete. 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.011+0000 7fb35b2e4640 1 Processor -- start 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.011+0000 7fb35b2e4640 1 -- start start 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 0x7fb3540cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35415eb80 0x7fb35415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb3540d5290 con 0x7fb3541626c0 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb354163460 con 0x7fb3540d4e40 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35b2e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb3540d2ef0 con 0x7fb35415eb80 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42988/0 (socket says 172.21.15.99:42988) 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 -- 172.21.15.99:0/2612154774 learned_addr learned my addr 172.21.15.99:0/2612154774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359080640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 0x7fb3540cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb35887f640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35415eb80 0x7fb35415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 -- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35415eb80 msgr2=0x7fb35415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35415eb80 0x7fb35415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 -- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 msgr2=0x7fb3540cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 0x7fb3540cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359881640 1 -- 172.21.15.99:0/2612154774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3540d0420 con 0x7fb3541626c0 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.012+0000 7fb359080640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 0x7fb3540cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb359881640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00c440 tx=0x7fb34c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0074e0 con 0x7fb3541626c0 2024-09-10T11:32:37.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c0047e0 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3540d06a0 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c004bd0 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.013+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3540b1fd0 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.014+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3540d29b0 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.014+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb34c011070 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.015+0000 7fb351ffb640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 0x7fb34006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.015+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fb34c08a930 con 0x7fb3541626c0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.015+0000 7fb359080640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 0x7fb34006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.015+0000 7fb359080640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 0x7fb34006e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb344034000 tx=0x7fb344033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.019+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3540d0420 con 0x7fb3541626c0 2024-09-10T11:32:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.219+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb35415dbd0 con 0x7fb3541626c0 2024-09-10T11:32:37.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.220+0000 7fb351ffb640 1 -- 172.21.15.99:0/2612154774 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7fb34c05f020 con 0x7fb3541626c0 2024-09-10T11:32:37.222 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:37.222 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:32:37.223 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:32:37.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 msgr2=0x7fb34006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 0x7fb34006e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb344034000 tx=0x7fb344033040 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 msgr2=0x7fb354159cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00c440 tx=0x7fb34c00c900 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 shutdown_connections 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb34006be30 0x7fb34006e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541626c0 0x7fb354159cd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35415eb80 0x7fb35415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 --2- 172.21.15.99:0/2612154774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3540d4e40 0x7fb3540cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 >> 172.21.15.99:0/2612154774 conn(0x7fb35408e680 msgr2=0x7fb3540d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 shutdown_connections 2024-09-10T11:32:37.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.222+0000 7fb35b2e4640 1 -- 172.21.15.99:0/2612154774 wait complete. 2024-09-10T11:32:37.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:32:28.253730+0000', 'modified': '2024-09-10T11:32:30.276228+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5592}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5592': {'gid': 5592, 'name': 'a', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6837/2576628819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 2576628819}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 2576628819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5592, 'qdb_cluster': [5592]} 2024-09-10T11:32:37.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:32:37.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:32:37.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.740+0000 7ff449e80640 1 Processor -- start 2024-09-10T11:32:37.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.740+0000 7ff449e80640 1 -- start start 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff449e80640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff4440b0090 0x7ff4440b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff449e80640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4440b40e0 con 0x7ff44415f1e0 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff449e80640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4440b5ab0 con 0x7ff44415ee00 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff449e80640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff444073c50 con 0x7ff4440b0090 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff442ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff44415f1e0 0x7ff4440b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47060/0 (socket says 172.21.15.99:47060) 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff442ffd640 1 -- 172.21.15.99:0/3920164225 learned_addr learned my addr 172.21.15.99:0/3920164225 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.741+0000 7ff443fff640 1 --2- 172.21.15.99:0/3920164225 >> v2:172.21.15.196:3300/0 conn(0x7ff4440b0090 0x7ff4440b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234012468 0 0) 0x7ff4440b40e0 con 0x7ff44415f1e0 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff42c003520 con 0x7ff44415f1e0 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740363079 0 0) 0x7ff4440b5ab0 con 0x7ff44415ee00 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4440b40e0 con 0x7ff44415ee00 2024-09-10T11:32:37.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1963076077 0 0) 0x7ff42c003520 con 0x7ff44415f1e0 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 >> v2:172.21.15.196:3300/0 conn(0x7ff4440b0090 msgr2=0x7ff4440b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 --2- 172.21.15.99:0/3920164225 >> v2:172.21.15.196:3300/0 conn(0x7ff4440b0090 0x7ff4440b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 >> v1:172.21.15.196:6789/0 conn(0x7ff44415ee00 legacy=0x7ff4440b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440b4380 con 0x7ff44415f1e0 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7ff438001670 con 0x7ff44415f1e0 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff443fff640 1 --2- 172.21.15.99:0/3920164225 >> v2:172.21.15.196:3300/0 conn(0x7ff4440b0090 0x7ff4440b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 >> v1:172.21.15.99:6789/0 conn(0x7ff44415f1e0 legacy=0x7ff4440b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff42c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42c004800 0x7ff42c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42c0051c0 0x7ff42c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff444073c50 con 0x7ff42c003b40 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c00c040 con 0x7ff42c0051c0 2024-09-10T11:32:37.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c00c240 con 0x7ff42c004800 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff42c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff443fff640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42c0051c0 0x7ff42c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42c004800 0x7ff42c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 -- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42c004800 msgr2=0x7ff42c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42c004800 0x7ff42c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 -- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42c0051c0 msgr2=0x7ff42c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42c0051c0 0x7ff42c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff4437fe640 1 -- 172.21.15.99:0/3920164225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c003520 con 0x7ff42c003b40 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff443fff640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42c0051c0 0x7ff42c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.742+0000 7ff443fff640 1 -- 172.21.15.99:0/3920164225 reap_dead start 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.743+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff42c003f40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4340018d0 tx=0x7ff42c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271dacacfdf11c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.743+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340014f0 con 0x7ff42c003b40 2024-09-10T11:32:37.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.743+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4340052d0 con 0x7ff42c003b40 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.743+0000 7ff4427fc640 1 -- 172.21.15.99:0/3920164225 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43402ac40 con 0x7ff42c003b40 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 -- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 msgr2=0x7ff42c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff42c003f40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff4340018d0 tx=0x7ff42c005830 comp rx=0 tx=0).stop 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 -- 172.21.15.99:0/3920164225 shutdown_connections 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 --2- 172.21.15.99:0/3920164225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff42c003f40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 -- 172.21.15.99:0/3920164225 >> 172.21.15.99:0/3920164225 conn(0x7ff4440730c0 msgr2=0x7ff4440734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 -- 172.21.15.99:0/3920164225 shutdown_connections 2024-09-10T11:32:37.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.744+0000 7ff449e80640 1 -- 172.21.15.99:0/3920164225 wait complete. 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.745+0000 7ff449e80640 1 Processor -- start 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.745+0000 7ff449e80640 1 -- start start 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4440cb410 0x7ff444158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff444155f20 0x7ff4441563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440b0600 con 0x7ff42c003b40 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4440afd60 con 0x7ff444155f20 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff449e80640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4440b0780 con 0x7ff4440cb410 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43000/0 (socket says 172.21.15.99:43000) 2024-09-10T11:32:37.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 -- 172.21.15.99:0/3932024444 learned_addr learned my addr 172.21.15.99:0/3932024444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4440cb410 0x7ff444158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff443fff640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff444155f20 0x7ff4441563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 -- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4440cb410 msgr2=0x7ff444158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4440cb410 0x7ff444158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 -- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff444155f20 msgr2=0x7ff4441563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff444155f20 0x7ff4441563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 -- 172.21.15.99:0/3932024444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4440cb910 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff434005040 tx=0x7ff4340012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.746+0000 7ff42bfff640 1 -- 172.21.15.99:0/3932024444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434040030 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.747+0000 7ff42bfff640 1 -- 172.21.15.99:0/3932024444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff434005480 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.747+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4440cbbf0 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.747+0000 7ff42bfff640 1 -- 172.21.15.99:0/3932024444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340014f0 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.747+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4440cc1b0 con 0x7ff42c003b40 2024-09-10T11:32:37.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.748+0000 7ff42bfff640 1 -- 172.21.15.99:0/3932024444 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff434043070 con 0x7ff42c003b40 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.749+0000 7ff42bfff640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.749+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.749+0000 7ff42bfff640 1 -- 172.21.15.99:0/3932024444 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7ff4340aad40 con 0x7ff42c003b40 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.749+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 0x7ff43006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff4440d1de0 tx=0x7ff438007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.750+0000 7ff44cc14740 1 Processor -- start 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.750+0000 7ff44cc14740 1 -- start start 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 0x555ad8c2f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ad8b91520 con 0x555ad8c2f2c0 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ad8b393e0 con 0x555ad8c2ca80 2024-09-10T11:32:37.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ad8b8d4e0 con 0x555ad8c30a50 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43012/0 (socket says 172.21.15.99:43012) 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 -- 172.21.15.99:0/2747913900 learned_addr learned my addr 172.21.15.99:0/2747913900 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff443fff640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff442ffd640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 0x555ad8c2f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 -- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 msgr2=0x555ad8c298b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c298b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 -- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 msgr2=0x555ad8c2f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 0x555ad8c2f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 -- 172.21.15.99:0/2747913900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad8b542f0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff442ffd640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 0x555ad8c2f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4437fe640 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff434000c00 tx=0x7ff434078c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7346451247d2b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4297fa640 1 -- 172.21.15.99:0/2747913900 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340a8070 con 0x555ad8c2f2c0 2024-09-10T11:32:37.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4297fa640 1 -- 172.21.15.99:0/2747913900 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff434091030 con 0x555ad8c2f2c0 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.751+0000 7ff4297fa640 1 -- 172.21.15.99:0/2747913900 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434081bd0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 -- 172.21.15.99:0/2747913900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 msgr2=0x555ad8c74a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff434000c00 tx=0x7ff434078c60 comp rx=0 tx=0).stop 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 -- 172.21.15.99:0/2747913900 shutdown_connections 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c298b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2ca80 0x555ad8c2f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 --2- 172.21.15.99:0/2747913900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c74a60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 -- 172.21.15.99:0/2747913900 >> 172.21.15.99:0/2747913900 conn(0x555ad8c2ba80 msgr2=0x555ad8c73f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 -- 172.21.15.99:0/2747913900 shutdown_connections 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.752+0000 7ff44cc14740 1 -- 172.21.15.99:0/2747913900 wait complete. 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 -- start start 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 Processor -- start 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c58c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ad8b8d4e0 con 0x555ad8c30a50 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ad8b1a450 con 0x555ad8c2f2c0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.753+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ad8b91520 con 0x555ad8c587e0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff44cc14740 10 client.0 _ll_register_callbacks cb 0x555ad88a3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43024/0 (socket says 172.21.15.99:43024) 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 -- 172.21.15.99:0/620752099 learned_addr learned my addr 172.21.15.99:0/620752099 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4437fe640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff443fff640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c58c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 -- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 msgr2=0x555ad8c58c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c58c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 -- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 msgr2=0x555ad8c4e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 -- 172.21.15.99:0/620752099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad88a42d0 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff442ffd640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff438004e70 tx=0x7ff4380019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affddea54853ce0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4227fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438029060 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7ff438024ab0 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ad8b9ec50 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff438024cb0 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff44cc14740 10 client.5736 fetch_fsmap learned FSMap version 29 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x555ad896e6b0 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.754+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x555ad8b9ec50 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 3502+0+0 (secure 0 0 0) 0x7ff438015740 con 0x555ad8c30a50 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 10 client.5736 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T11:32:37.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 10 client.5736 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 20 client.5736 populate_metadata read hostname 'smithi099' 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8c28850 0x555ad8c28c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 4 client.5736 mds_command: new command op to 5592 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 --> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ad8b98fa0 con 0x555ad8c28850 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff4437fe640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8c28850 0x555ad8c28c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff4437fe640 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8c28850 0x555ad8c28c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0ded834693c1a0 server_cookie=4525794da9e9066d in_seq=0 out_seq=0 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.755+0000 7ff4227fc640 10 client.5736 ms_handle_connect on v2:172.21.15.99:6836/2576628819 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff4227fc640 1 -- 172.21.15.99:0/620752099 <== mds.0 v2:172.21.15.99:6836/2576628819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ad8b98fa0 con 0x555ad8c28850 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff4227fc640 10 client.5736 handle_command_reply: tid=0 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 client.5736 shutdown 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 msgr2=0x555ad8c4da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff438004e70 tx=0x7ff4380019c0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 shutdown_connections 2024-09-10T11:32:37.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8c28850 0x555ad8c28c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c58c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff4227fc640 0 client.5736 ms_handle_reset on v2:172.21.15.99:6836/2576628819 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4da40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 --2- 172.21.15.99:0/620752099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 >> 172.21.15.99:0/620752099 conn(0x555ad8c2ca80 msgr2=0x555ad8c71fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 shutdown_connections 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 1 -- 172.21.15.99:0/620752099 wait complete. 2024-09-10T11:32:37.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.756+0000 7ff44cc14740 20 client.5736 trim_cache size 0 max 16384 2024-09-10T11:32:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.773+0000 7ff44cc14740 1 Processor -- start 2024-09-10T11:32:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 -- start start 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c72ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c587e0 0x555ad8c2d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4442009b0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff444148c10 con 0x555ad8c587e0 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4442003e0 con 0x555ad8c30a50 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.774+0000 7ff4437fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43026/0 (socket says 172.21.15.99:43026) 2024-09-10T11:32:37.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 -- 172.21.15.99:0/503598300 learned_addr learned my addr 172.21.15.99:0/503598300 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff443fff640 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c587e0 0x555ad8c2d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff442ffd640 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c72ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 -- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 msgr2=0x555ad8c72ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c72ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 -- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c587e0 msgr2=0x555ad8c2d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c587e0 0x555ad8c2d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 -- 172.21.15.99:0/503598300 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ad8c517b0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff4437fe640 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff434078cf0 tx=0x7ff434090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb920993f335d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff428ff9640 1 -- 172.21.15.99:0/503598300 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4340a8040 con 0x555ad8c2f2c0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff428ff9640 1 -- 172.21.15.99:0/503598300 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4340772f0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.775+0000 7ff428ff9640 1 -- 172.21.15.99:0/503598300 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434075290 con 0x555ad8c2f2c0 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- 172.21.15.99:0/503598300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 msgr2=0x555ad8c25310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff434078cf0 tx=0x7ff434090000 comp rx=0 tx=0).stop 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- 172.21.15.99:0/503598300 shutdown_connections 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c587e0 0x555ad8c2d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c30a50 0x555ad8c72ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 --2- 172.21.15.99:0/503598300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c2f2c0 0x555ad8c25310 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- 172.21.15.99:0/503598300 >> 172.21.15.99:0/503598300 conn(0x555ad8c2ca80 msgr2=0x555ad8c290f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- 172.21.15.99:0/503598300 shutdown_connections 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- 172.21.15.99:0/503598300 wait complete. 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 -- start start 2024-09-10T11:32:37.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.776+0000 7ff44cc14740 1 Processor -- start 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ad8c4e0c0 con 0x555ad8c30a50 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ad8b393e0 con 0x555ad8c2f2c0 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x555ad8c4e420 con 0x555ad8c587e0 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff44cc14740 10 client.0 _ll_register_callbacks cb 0x7ff438004a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43038/0 (socket says 172.21.15.99:43038) 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 -- 172.21.15.99:0/3994021945 learned_addr learned my addr 172.21.15.99:0/3994021945 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff443fff640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 -- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 msgr2=0x555ad8c977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 -- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 msgr2=0x555ad8c4d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff442ffd640 1 -- 172.21.15.99:0/3994021945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ad8c8dd50 con 0x555ad8c30a50 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff443fff640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.777+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff442ffd640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff4380018f0 tx=0x7ff438015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967440d27ac8da38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438025e60 con 0x555ad8c30a50 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7ff438010020 con 0x555ad8c30a50 2024-09-10T11:32:37.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4340772f0 con 0x555ad8c30a50 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff438042050 con 0x555ad8c30a50 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 10 client.5742 fetch_fsmap learned FSMap version 29 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x555ad8c724e0 con 0x555ad8c30a50 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4340772f0 con 0x555ad8c30a50 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 29) ==== 3502+0+0 (secure 0 0 0) 0x7ff438045020 con 0x555ad8c30a50 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 10 client.5742 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 10 client.5742 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 20 client.5742 populate_metadata read hostname 'smithi099' 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8cf1500 0x555ad8c85c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 4 client.5742 mds_command: new command op to 5592 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 --> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] -- command(tid 0: {"prefix": "status"}) -- 0x555ad8b15b80 con 0x555ad8cf1500 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.778+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8cf1500 0x555ad8c85c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff4227fc640 10 client.5742 ms_handle_connect on v2:172.21.15.99:6836/2576628819 2024-09-10T11:32:37.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff4437fe640 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8cf1500 0x555ad8c85c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=464c14cb40a33262 server_cookie=21cdb2ca87f3f402 in_seq=0 out_seq=0 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff4227fc640 1 -- 172.21.15.99:0/3994021945 <== mds.0 v2:172.21.15.99:6836/2576628819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x555ad8b15b80 con 0x555ad8cf1500 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff4227fc640 10 client.5742 handle_command_reply: tid=0 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 client.5742 shutdown 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 msgr2=0x555ad8c4dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff4380018f0 tx=0x7ff438015000 comp rx=0 tx=0).stop 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 shutdown_connections 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:6836/2576628819,v1:172.21.15.99:6837/2576628819] conn(0x555ad8cf1500 0x555ad8c85c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555ad8c587e0 0x555ad8c977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ad8c30a50 0x555ad8c4dab0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 --2- 172.21.15.99:0/3994021945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ad8c2f2c0 0x555ad8c4d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff4227fc640 0 client.5742 ms_handle_reset on v2:172.21.15.99:6836/2576628819 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 >> 172.21.15.99:0/3994021945 conn(0x555ad8c2ca80 msgr2=0x555ad8c71660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 shutdown_connections 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 1 -- 172.21.15.99:0/3994021945 wait complete. 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.779+0000 7ff44cc14740 20 client.5742 trim_cache size 0 max 16384 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:32:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5592, 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 8.8742435200000003, 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 29, 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 66, 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 66, 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 13.877632688 2024-09-10T11:32:37.789 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 msgr2=0x7ff43006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 0x7ff43006e2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff4440d1de0 tx=0x7ff438007040 comp rx=0 tx=0).stop 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 msgr2=0x7ff4440cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff434005040 tx=0x7ff4340012d0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 shutdown_connections 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff43006be30 0x7ff43006e2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff444155f20 0x7ff4441563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4440cb410 0x7ff444158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 --2- 172.21.15.99:0/3932024444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42c003b40 0x7ff4440cc9b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 >> 172.21.15.99:0/3932024444 conn(0x7ff4440730c0 msgr2=0x7ff4440aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 shutdown_connections 2024-09-10T11:32:37.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.780+0000 7ff449e80640 1 -- 172.21.15.99:0/3932024444 wait complete. 2024-09-10T11:32:37.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:32:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 Processor -- start 2024-09-10T11:32:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 -- start start 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d0415ee00 0x7f0d040b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0d0406c320 0x7f0d0406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d040b6bb0 con 0x7f0d04070dd0 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0d0415f1e0 con 0x7f0d0415ee00 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.889+0000 7f0d09ed0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0d04073530 con 0x7f0d0406c320 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d0415ee00 0x7f0d040b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d02ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0d04070dd0 0x7f0d0406bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47074/0 (socket says 172.21.15.99:47074) 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d0415ee00 0x7f0d040b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43048/0 (socket says 172.21.15.99:43048) 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d02ffd640 1 -- 172.21.15.99:0/2422277477 learned_addr learned my addr 172.21.15.99:0/2422277477 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d03fff640 1 --2- 172.21.15.99:0/2422277477 >> v2:172.21.15.196:3300/0 conn(0x7f0d0406c320 0x7f0d0406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 -- 172.21.15.99:0/2422277477 >> v2:172.21.15.196:3300/0 conn(0x7f0d0406c320 msgr2=0x7f0d0406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2422277477 >> v2:172.21.15.196:3300/0 conn(0x7f0d0406c320 0x7f0d0406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 -- 172.21.15.99:0/2422277477 >> v1:172.21.15.99:6789/0 conn(0x7f0d04070dd0 legacy=0x7f0d0406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d027fc640 1 -- 172.21.15.99:0/2422277477 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412928269 0 0) 0x7f0d040b6bb0 con 0x7f0d04070dd0 2024-09-10T11:32:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.890+0000 7f0d037fe640 1 -- 172.21.15.99:0/2422277477 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d04073810 con 0x7f0d0415ee00 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2422277477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d040b6670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0d040d14a0 tx=0x7f0cf402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5730926e0ef9fdd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d027fc640 1 -- 172.21.15.99:0/2422277477 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4037070 con 0x7f0d0415ee00 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d027fc640 1 -- 172.21.15.99:0/2422277477 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf4032440 con 0x7f0d0415ee00 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d027fc640 1 -- 172.21.15.99:0/2422277477 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf402ae60 con 0x7f0d0415ee00 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2422277477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 msgr2=0x7f0d040b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2422277477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d040b6670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0d040d14a0 tx=0x7f0cf402a110 comp rx=0 tx=0).stop 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2422277477 shutdown_connections 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2422277477 >> v2:172.21.15.196:3300/0 conn(0x7f0d0406c320 0x7f0d0406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2422277477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d040b6670 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2422277477 >> 172.21.15.99:0/2422277477 conn(0x7f0d04072680 msgr2=0x7f0d04072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2422277477 shutdown_connections 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.891+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2422277477 wait complete. 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.892+0000 7f0d09ed0640 1 Processor -- start 2024-09-10T11:32:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 -- start start 2024-09-10T11:32:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 0x7f0d040d0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04070dd0 0x7f0d0414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d040b6150 con 0x7f0d0415ee00 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d040b5e70 con 0x7f0d04070dd0 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d09ed0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d04073170 con 0x7f0d0406c320 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.893+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43058/0 (socket says 172.21.15.99:43058) 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 -- 172.21.15.99:0/2453926652 learned_addr learned my addr 172.21.15.99:0/2453926652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d02ffd640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04070dd0 0x7f0d0414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 0x7f0d040d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 -- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 msgr2=0x7f0d040d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 0x7f0d040d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 -- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04070dd0 msgr2=0x7f0d0414acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04070dd0 0x7f0d0414acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 -- 172.21.15.99:0/2453926652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d040d0c80 con 0x7f0d0415ee00 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 0x7f0d040d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:32:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d03fff640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc007e40 tx=0x7f0cfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc014620 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc019070 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0414b210 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc014a20 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.894+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d040cfc60 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.895+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d040706f0 con 0x7f0d0415ee00 2024-09-10T11:32:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.896+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cfc014bc0 con 0x7f0d0415ee00 2024-09-10T11:32:37.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.896+0000 7f0cebfff640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 0x7f0cf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:37.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.896+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f0cfc08b720 con 0x7f0d0415ee00 2024-09-10T11:32:37.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.896+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 0x7f0cf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:37.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.897+0000 7f0d037fe640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 0x7f0cf006e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4034000 tx=0x7f0cf402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:37.901+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d040d0c80 con 0x7f0d0415ee00 2024-09-10T11:32:38.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.101+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0d040b3320 con 0x7f0d0415ee00 2024-09-10T11:32:38.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.102+0000 7f0cebfff640 1 -- 172.21.15.99:0/2453926652 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7f0cfc05a0d0 con 0x7f0d0415ee00 2024-09-10T11:32:38.103 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.103+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 msgr2=0x7f0cf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.103+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 0x7f0cf006e320 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4034000 tx=0x7f0cf402e000 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.103+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 msgr2=0x7f0d041f8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.103+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc007e40 tx=0x7f0cfc00e4b0 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 shutdown_connections 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0cf006be60 0x7f0cf006e320 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d0415ee00 0x7f0d041f8b50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04070dd0 0x7f0d0414acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 --2- 172.21.15.99:0/2453926652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d0406c320 0x7f0d040d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 >> 172.21.15.99:0/2453926652 conn(0x7f0d04072680 msgr2=0x7f0d0406e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 shutdown_connections 2024-09-10T11:32:38.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38.104+0000 7f0d09ed0640 1 -- 172.21.15.99:0/2453926652 wait complete. 2024-09-10T11:32:38.116 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:32:38.117 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:32:38.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.118 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:32:38.133 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:32:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:32:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:32:38.133 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:32:38.134 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42691sec preferred_lft 42691sec 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:32:38.135 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:32:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:32:38.136 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:38.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.136 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:32:38.197 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:32:38.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.199 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:32:38.254 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:32:38.255 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:32:38.255 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:32:38.255 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:32:38.255 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:32:38.255 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:32:38.255 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:32:38.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:32:38.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:32:38.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:32:38.256 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:32:38.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:32:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:32:38.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.380 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:32:38.396 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:32:38.396 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:32:38.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:32:38.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.396 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:32:38.460 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:32:38.461 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:32:38.461 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:38.516 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:32:38.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:32:38.516 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:38.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:32:38.637 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:32:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:32:38.693 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:32:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:32:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:32:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:32:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:32:38.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:32:38.695 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:32:38.696 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:32:38.697 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:32:38.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.698 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:32:38.749 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:32:38.749 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:32:38.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:32:38.816 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:32:38.874 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:32:38.874 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:32:38.874 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:32:38.874 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:32:38.874 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:32:38.875 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:32:38.876 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:32:38.876 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:32:38.876 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:32:38.876 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:32:38.876 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:32:38.877 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:32:38.878 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:32:38.879 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:32:38.879 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:32:38.879 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:32:38.879 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:32:38.879 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:32:38.880 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:32:38.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:38.880 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:32:38.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:32:38.872+0000 7f5d8c749480 -1 init, newargv = 0x558a78ecff20 newargc=15 2024-09-10T11:32:38.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[44575]: starting ceph client 2024-09-10T11:32:38.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[44575]: starting fuse 2024-09-10T11:32:38.896 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:32:38.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:38.897 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:32:38.898 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:32:38.899 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:32:39.014 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.44575.asok 2024-09-10T11:32:39.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:32:39.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44575.asok 2024-09-10T11:32:39.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44575.asok status 2024-09-10T11:32:39.157 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:32:39.157 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "44575", 2024-09-10T11:32:39.158 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5757, 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5757 2024-09-10T11:32:39.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 868729781 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 868729781 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:32:39.160 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5757 192.168.0.1:0/868729781", 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/868729781", 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 29, 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 66, 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:32:39.161 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:32:39.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.166 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:32:39.166 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:39.182 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:32:39.182 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:39.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:32:39.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:32:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.382+0000 7fa0deaf1640 1 Processor -- start 2024-09-10T11:32:39.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.382+0000 7fa0deaf1640 1 -- start start 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0deaf1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0d815aef0 0x7fa0d80b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0deaf1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa0d815b2d0 0x7fa0d80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0deaf1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0d806ee10 con 0x7fa0d80cd290 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0deaf1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d80b0290 con 0x7fa0d815b2d0 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0deaf1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa0d806e570 con 0x7fa0d815aef0 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa0d815b2d0 0x7fa0d80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0dd08e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa0d80cd290 0x7fa0d80714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47076/0 (socket says 172.21.15.99:47076) 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0dd08e640 1 -- 172.21.15.99:0/2916420803 learned_addr learned my addr 172.21.15.99:0/2916420803 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:39.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0dc88d640 1 --2- 172.21.15.99:0/2916420803 >> v2:172.21.15.196:3300/0 conn(0x7fa0d815aef0 0x7fa0d80b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cf7fe640 1 -- 172.21.15.99:0/2916420803 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 824040272 0 0) 0x7fa0d806ee10 con 0x7fa0d80cd290 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cf7fe640 1 -- 172.21.15.99:0/2916420803 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c0003520 con 0x7fa0d80cd290 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cffff640 1 -- 172.21.15.99:0/2916420803 >> v2:172.21.15.196:3300/0 conn(0x7fa0d815aef0 msgr2=0x7fa0d80b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cffff640 1 --2- 172.21.15.99:0/2916420803 >> v2:172.21.15.196:3300/0 conn(0x7fa0d815aef0 0x7fa0d80b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cffff640 1 -- 172.21.15.99:0/2916420803 >> v1:172.21.15.99:6789/0 conn(0x7fa0d80cd290 legacy=0x7fa0d80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.383+0000 7fa0cffff640 1 -- 172.21.15.99:0/2916420803 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d806e850 con 0x7fa0d815b2d0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.384+0000 7fa0cffff640 1 --2- 172.21.15.99:0/2916420803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80afbb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80097a0 tx=0x7fa0c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acacaaa4453dfdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.384+0000 7fa0cf7fe640 1 -- 172.21.15.99:0/2916420803 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8037070 con 0x7fa0d815b2d0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.384+0000 7fa0cf7fe640 1 -- 172.21.15.99:0/2916420803 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c8032440 con 0x7fa0d815b2d0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.384+0000 7fa0cf7fe640 1 -- 172.21.15.99:0/2916420803 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8032810 con 0x7fa0d815b2d0 2024-09-10T11:32:39.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 -- 172.21.15.99:0/2916420803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815b2d0 msgr2=0x7fa0d80afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/2916420803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80afbb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fa0c80097a0 tx=0x7fa0c802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 -- 172.21.15.99:0/2916420803 shutdown_connections 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/2916420803 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80afbb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/2916420803 >> v2:172.21.15.196:3300/0 conn(0x7fa0d815aef0 0x7fa0d80b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 -- 172.21.15.99:0/2916420803 >> 172.21.15.99:0/2916420803 conn(0x7fa0d806bc10 msgr2=0x7fa0d806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 -- 172.21.15.99:0/2916420803 shutdown_connections 2024-09-10T11:32:39.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.385+0000 7fa0deaf1640 1 -- 172.21.15.99:0/2916420803 wait complete. 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.386+0000 7fa0deaf1640 1 Processor -- start 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.386+0000 7fa0deaf1640 1 -- start start 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0d80cd290 0x7fa0d81f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8071d30 con 0x7fa0d815aef0 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8071bb0 con 0x7fa0d815b2d0 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0deaf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0d8071eb0 con 0x7fa0d80cd290 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43084/0 (socket says 172.21.15.99:43084) 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 -- 172.21.15.99:0/419734241 learned_addr learned my addr 172.21.15.99:0/419734241 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 -- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0d80cd290 msgr2=0x7fa0d81f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0dd08e640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0d80cd290 0x7fa0d81f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 -- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d815b2d0 msgr2=0x7fa0d80ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 -- 172.21.15.99:0/419734241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d80af180 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.387+0000 7fa0cffff640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8035040 tx=0x7fa0c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.388+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8037040 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.388+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0c8030500 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.388+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d80af460 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.388+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c8030910 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.388+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d81f5e90 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.389+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d80cd790 con 0x7fa0d815aef0 2024-09-10T11:32:39.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.389+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa0c8044020 con 0x7fa0d815aef0 2024-09-10T11:32:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.389+0000 7fa0cd7fa640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 0x7fa0c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.390+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fa0c80ab040 con 0x7fa0d815aef0 2024-09-10T11:32:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.390+0000 7fa0dc88d640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 0x7fa0c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.390+0000 7fa0dc88d640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 0x7fa0c406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4006fd0 tx=0x7fa0d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:39.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.394+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa0c8079a80 con 0x7fa0d815aef0 2024-09-10T11:32:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.591+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa0d80afb40 con 0x7fa0d815aef0 2024-09-10T11:32:39.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.591+0000 7fa0cd7fa640 1 -- 172.21.15.99:0/419734241 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7fa0c8079380 con 0x7fa0d815aef0 2024-09-10T11:32:39.593 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:32:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 msgr2=0x7fa0c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 0x7fa0c406e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4006fd0 tx=0x7fa0d4008040 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 msgr2=0x7fa0d8157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8035040 tx=0x7fa0c80046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 shutdown_connections 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa0c406be30 0x7fa0c406e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0d815b2d0 0x7fa0d80ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0d815aef0 0x7fa0d8157360 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 --2- 172.21.15.99:0/419734241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0d80cd290 0x7fa0d81f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 >> 172.21.15.99:0/419734241 conn(0x7fa0d806bc10 msgr2=0x7fa0d81597d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.593+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 shutdown_connections 2024-09-10T11:32:39.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:39.594+0000 7fa0deaf1640 1 -- 172.21.15.99:0/419734241 wait complete. 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:39.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:32:39.607 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:32:39.607 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:32:39.607 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:32:39.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.607 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:32:39.627 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:39.628 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.629 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:32:39.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.631 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:32:39.696 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:32:39.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.698 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:32:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:32:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:32:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:32:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:32:39.757 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:32:39.758 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:32:39.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:32:39.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:32:39.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.890 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:32:39.905 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:32:39.906 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:32:39.906 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:32:39.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:39.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:32:39.973 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:32:39.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:32:39.973 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:40.028 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:32:40.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:32:40.028 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:40.083 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:32:40.149 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:32:40.205 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:32:40.206 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:32:40.207 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:32:40.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:40.208 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:32:40.261 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:32:40.261 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:32:40.304 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:32:40.328 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:32:40.385 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:32:40.469 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:32:40.470 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:32:40.471 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:32:40.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:40.472 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:32:40.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:32:40.384+0000 7fb953d47480 -1 init, newargv = 0x555e8ad002f0 newargc=15 2024-09-10T11:32:40.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[35462]: starting ceph client 2024-09-10T11:32:40.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[35462]: starting fuse 2024-09-10T11:32:40.488 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:32:40.488 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:32:40.489 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:32:40.490 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:32:40.605 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.35462.asok 2024-09-10T11:32:40.608 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:32:40.616 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35462.asok 2024-09-10T11:32:40.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:40.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35462.asok status 2024-09-10T11:32:40.749 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "35462", 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:32:40.750 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5117, 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 5117 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3912321895 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:32:40.751 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3912321895 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.5117 192.168.0.1:0/3912321895", 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/3912321895", 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 29, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 66, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:32:40.752 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:32:40.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:40.758 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:32:40.758 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:40.774 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:32:40.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:40.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:32:40.774 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:32:40.839 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:32:40.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.947+0000 7fcf340e6640 1 Processor -- start 2024-09-10T11:32:40.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.947+0000 7fcf340e6640 1 -- start start 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf340e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcf2c0d1200 0x7fcf2c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf340e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 0x7fcf2c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf340e6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c152e60 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf340e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcf2c06ef80 con 0x7fcf2c0d1200 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf340e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c0d15e0 con 0x7fcf2c15c770 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf31681640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf2c06e5d0 0x7fcf2c15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47080/0 (socket says 172.21.15.99:47080) 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf31681640 1 -- 172.21.15.99:0/4236255717 learned_addr learned my addr 172.21.15.99:0/4236255717 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf31e82640 1 --2- 172.21.15.99:0/4236255717 >> v2:172.21.15.196:3300/0 conn(0x7fcf2c0d1200 0x7fcf2c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.948+0000 7fcf32683640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 0x7fcf2c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775173648 0 0) 0x7fcf2c152e60 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf18003520 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3022201385 0 0) 0x7fcf18003520 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf31e82640 1 -- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 msgr2=0x7fcf2c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf31e82640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 0x7fcf2c06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf31e82640 1 -- 172.21.15.99:0/4236255717 >> v1:172.21.15.99:6789/0 conn(0x7fcf2c06e5d0 legacy=0x7fcf2c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf31e82640 1 -- 172.21.15.99:0/4236255717 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c072680 con 0x7fcf2c0d1200 2024-09-10T11:32:40.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.949+0000 7fcf32683640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 0x7fcf2c06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:32:40.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf31e82640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c0d1200 0x7fcf2c073440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097a0 tx=0x7fcf2802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5bab1db110e2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:40.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28037070 con 0x7fcf2c0d1200 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2802ab10 con 0x7fcf2c0d1200 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf30e80640 1 -- 172.21.15.99:0/4236255717 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c001670 con 0x7fcf2c0d1200 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 -- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c0d1200 msgr2=0x7fcf2c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c0d1200 0x7fcf2c073440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097a0 tx=0x7fcf2802a080 comp rx=0 tx=0).stop 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 -- 172.21.15.99:0/4236255717 shutdown_connections 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c15c770 0x7fcf2c06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 --2- 172.21.15.99:0/4236255717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c0d1200 0x7fcf2c073440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.950+0000 7fcf340e6640 1 -- 172.21.15.99:0/4236255717 >> 172.21.15.99:0/4236255717 conn(0x7fcf2c06bc10 msgr2=0x7fcf2c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.951+0000 7fcf340e6640 1 -- 172.21.15.99:0/4236255717 shutdown_connections 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.951+0000 7fcf340e6640 1 -- 172.21.15.99:0/4236255717 wait complete. 2024-09-10T11:32:40.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.952+0000 7fcf340e6640 1 Processor -- start 2024-09-10T11:32:40.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.952+0000 7fcf340e6640 1 -- start start 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.952+0000 7fcf340e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.952+0000 7fcf340e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c0d1200 0x7fcf2c159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.952+0000 7fcf340e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 0x7fcf2c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf340e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c072f20 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf340e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c072c40 con 0x7fcf2c15c770 2024-09-10T11:32:40.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf340e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c15cd90 con 0x7fcf2c0d1200 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43092/0 (socket says 172.21.15.99:43092) 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 -- 172.21.15.99:0/3554298572 learned_addr learned my addr 172.21.15.99:0/3554298572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31681640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c0d1200 0x7fcf2c159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf32683640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 0x7fcf2c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 -- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c0d1200 msgr2=0x7fcf2c159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c0d1200 0x7fcf2c159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 -- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 msgr2=0x7fcf2c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 0x7fcf2c0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 -- 172.21.15.99:0/3554298572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c159760 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf32683640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 0x7fcf2c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf31e82640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcf28035040 tx=0x7fcf280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28041020 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2803d620 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c159a20 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28037040 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.953+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c159fe0 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.954+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c073440 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.955+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf28044020 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.955+0000 7fcf227fc640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 0x7fcf1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.955+0000 7fcf31681640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 0x7fcf1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.955+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fcf280ab2f0 con 0x7fcf2c06e5d0 2024-09-10T11:32:40.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.956+0000 7fcf31681640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 0x7fcf1006e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c15e4b0 tx=0x7fcf1c0057f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:40.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:40.960+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf2804b030 con 0x7fcf2c06e5d0 2024-09-10T11:32:41.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.156+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf2c15bb90 con 0x7fcf2c06e5d0 2024-09-10T11:32:41.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.157+0000 7fcf227fc640 1 -- 172.21.15.99:0/3554298572 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7fcf28079d30 con 0x7fcf2c06e5d0 2024-09-10T11:32:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:32:41.159 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:32:41.159 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 msgr2=0x7fcf1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 0x7fcf1006e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c15e4b0 tx=0x7fcf1c0057f0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 msgr2=0x7fcf2c15aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcf28035040 tx=0x7fcf280046c0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 shutdown_connections 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf1006be30 0x7fcf1006e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf2c15c770 0x7fcf2c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf2c0d1200 0x7fcf2c159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 --2- 172.21.15.99:0/3554298572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf2c06e5d0 0x7fcf2c15aac0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 >> 172.21.15.99:0/3554298572 conn(0x7fcf2c06bc10 msgr2=0x7fcf2c15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:41.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 shutdown_connections 2024-09-10T11:32:41.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:32:41.159+0000 7fcf340e6640 1 -- 172.21.15.99:0/3554298572 wait complete. 2024-09-10T11:32:41.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:32:41.289 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":5592,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.373279671000001,"mdsmap_epoch":29,"osdmap_epoch":66,"osdmap_epoch_barrier":66,"uptime":17.376671720000001} 2024-09-10T11:32:41.289 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 5592, "mdsmap_epoch": 29, "osdmap_epoch": 66, "osdmap_epoch_barrier": 66, "rank_uptime": 12.373279671, "state": "up:active", "uptime": 17.37667172, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:32:41.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:32:41.399 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:32:41.400 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:32:41.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-09-10T11:32:41.504 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2024-09-10T11:32:41.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-09-10T11:32:41.504 INFO:tasks.cephfs.test_full:pool capacity 557078080, 531MB should be enough to fill it 2024-09-10T11:32:41.504 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:32:41.504 DEBUG:teuthology.orchestra.run.smithi099:> import time 2024-09-10T11:32:41.504 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.505 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 0.9)): 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> print("done writing {0} bytes".format(bytes)) 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # OK, now we should sneak in under the full condition 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # due to the time it takes the OSDs to report to the 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> # mons, and get a successful fsync on our full-making data 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:32:41.506 DEBUG:teuthology.orchestra.run.smithi099:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> # buffered write, add more dirty data to the buffer 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> print("starting buffered write") 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 0.2)): 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> pass; 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> # Wait long enough for a background flush that should fail 2024-09-10T11:32:41.507 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(30) 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> # ...and check that the failed background flush is reflected in fclose 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> print("close() returned an error as expected") 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("close() failed to raise error") 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> # The kernel cephfs client does not raise errors on fclose 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-09-10T11:32:41.508 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:32:44.208 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-09-10T11:32:44.209 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-09-10T11:32:44.210 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-09-10T11:32:44.211 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-09-10T11:32:44.212 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-09-10T11:32:44.213 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-09-10T11:32:44.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-09-10T11:32:44.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-09-10T11:32:44.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-09-10T11:32:44.217 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-09-10T11:32:44.218 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-09-10T11:32:44.219 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-09-10T11:32:44.220 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-09-10T11:32:44.446 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-09-10T11:32:44.450 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-09-10T11:32:44.453 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-09-10T11:32:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-09-10T11:32:44.455 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-09-10T11:32:44.456 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-09-10T11:32:44.457 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-09-10T11:32:44.458 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-09-10T11:32:44.459 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-09-10T11:32:44.668 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-09-10T11:32:44.669 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-09-10T11:32:44.670 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-09-10T11:32:44.671 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-09-10T11:32:44.672 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-09-10T11:32:44.673 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-09-10T11:32:44.674 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-09-10T11:32:44.675 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-09-10T11:32:44.676 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-09-10T11:32:44.677 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-09-10T11:32:44.678 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-09-10T11:32:44.679 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-09-10T11:32:44.680 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-09-10T11:32:44.680 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-09-10T11:32:46.973 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:32:46.971+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-09-10T11:32:55.656 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:32:55.654+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-09-10T11:32:58.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:32:58.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:33:28.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:33:28.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-09-10T11:33:29.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:done writing 500695040 bytes 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:successfully fsync'ed prior to getting full state reported 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:starting buffered write 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-09-10T11:33:29.776 INFO:teuthology.orchestra.run.smithi099.stdout:close() returned an error as expected 2024-09-10T11:33:29.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:33:29.787 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:33:29.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:33:29.827 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:33:29.884 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:33:29.886 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:33:29.886 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:33:29.886 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:29.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:33:30.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.048+0000 7eff606f8640 1 Processor -- start 2024-09-10T11:33:30.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.048+0000 7eff606f8640 1 -- start start 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5815ee00 0x7eff580b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7eff5815f1e0 0x7eff580b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff580b0090 0x7eff580b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7eff580b40e0 con 0x7eff580b0090 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7eff580b5ab0 con 0x7eff5815f1e0 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff606f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7eff58073c50 con 0x7eff5815ee00 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff580b0090 0x7eff580b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff580b0090 0x7eff580b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42806/0 (socket says 172.21.15.99:42806) 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 -- 172.21.15.99:0/407690217 learned_addr learned my addr 172.21.15.99:0/407690217 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5e494640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5815ee00 0x7eff580b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5dc93640 1 --2- 172.21.15.99:0/407690217 >> v2:172.21.15.196:3300/0 conn(0x7eff5815f1e0 0x7eff580b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 -- 172.21.15.99:0/407690217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5815ee00 msgr2=0x7eff580b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5815ee00 0x7eff580b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 -- 172.21.15.99:0/407690217 >> v2:172.21.15.196:3300/0 conn(0x7eff5815f1e0 msgr2=0x7eff580b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 --2- 172.21.15.99:0/407690217 >> v2:172.21.15.196:3300/0 conn(0x7eff5815f1e0 0x7eff580b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.049+0000 7eff5ec95640 1 -- 172.21.15.99:0/407690217 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff580b4380 con 0x7eff580b0090 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.050+0000 7eff5ec95640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff580b0090 0x7eff580b1aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff580cd5f0 tx=0x7eff5002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29458f0a8681f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.050+0000 7eff5d492640 1 -- 172.21.15.99:0/407690217 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50036070 con 0x7eff580b0090 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.050+0000 7eff5d492640 1 -- 172.21.15.99:0/407690217 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff50031600 con 0x7eff580b0090 2024-09-10T11:33:30.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.050+0000 7eff5d492640 1 -- 172.21.15.99:0/407690217 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff500319d0 con 0x7eff580b0090 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 -- 172.21.15.99:0/407690217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff580b0090 msgr2=0x7eff580b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff580b0090 0x7eff580b1aa0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff580cd5f0 tx=0x7eff5002a6a0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 -- 172.21.15.99:0/407690217 shutdown_connections 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff580b0090 0x7eff580b1aa0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 --2- 172.21.15.99:0/407690217 >> v2:172.21.15.196:3300/0 conn(0x7eff5815f1e0 0x7eff580b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 --2- 172.21.15.99:0/407690217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5815ee00 0x7eff580b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 -- 172.21.15.99:0/407690217 >> 172.21.15.99:0/407690217 conn(0x7eff580730c0 msgr2=0x7eff580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 -- 172.21.15.99:0/407690217 shutdown_connections 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.051+0000 7eff606f8640 1 -- 172.21.15.99:0/407690217 wait complete. 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.052+0000 7eff606f8640 1 Processor -- start 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.052+0000 7eff606f8640 1 -- start start 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff580b0090 0x7eff5815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5815ee00 0x7eff581e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff580b1fe0 con 0x7eff5815f1e0 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff580afd60 con 0x7eff5815ee00 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7eff580b2160 con 0x7eff580b0090 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42816/0 (socket says 172.21.15.99:42816) 2024-09-10T11:33:30.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 -- 172.21.15.99:0/4233979012 learned_addr learned my addr 172.21.15.99:0/4233979012 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5e494640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff580b0090 0x7eff5815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5dc93640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5815ee00 0x7eff581e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 -- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff580b0090 msgr2=0x7eff5815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff580b0090 0x7eff5815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 -- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5815ee00 msgr2=0x7eff581e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5815ee00 0x7eff581e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff580ca930 con 0x7eff5815f1e0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff5ec95640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff50034040 tx=0x7eff50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50041020 con 0x7eff5815f1e0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff500077a0 con 0x7eff5815f1e0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff580cac10 con 0x7eff5815f1e0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50036040 con 0x7eff5815f1e0 2024-09-10T11:33:30.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.053+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff580cb1d0 con 0x7eff5815f1e0 2024-09-10T11:33:30.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.054+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff580b0600 con 0x7eff5815f1e0 2024-09-10T11:33:30.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.055+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff50044020 con 0x7eff5815f1e0 2024-09-10T11:33:30.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.055+0000 7eff4effd640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 0x7eff3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.055+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7eff500abaf0 con 0x7eff5815f1e0 2024-09-10T11:33:30.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.055+0000 7eff5e494640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 0x7eff3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.055+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7eff3c073a20 con 0x7eff5815f1e0 2024-09-10T11:33:30.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.056+0000 7eff5e494640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 0x7eff3c06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff540079a0 tx=0x7eff54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.060+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff5004b030 con 0x7eff5815f1e0 2024-09-10T11:33:30.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.238+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff58068ef0 con 0x7eff5815f1e0 2024-09-10T11:33:30.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.238+0000 7eff4effd640 1 -- 172.21.15.99:0/4233979012 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7eff50079d80 con 0x7eff5815f1e0 2024-09-10T11:33:30.241 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:33:30.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 msgr2=0x7eff3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 0x7eff3c06e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7eff540079a0 tx=0x7eff54008040 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 msgr2=0x7eff580cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff50034040 tx=0x7eff50004a40 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 shutdown_connections 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff3c06be30 0x7eff3c06e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5815f1e0 0x7eff580cce00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5815ee00 0x7eff581e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 --2- 172.21.15.99:0/4233979012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff580b0090 0x7eff5815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 >> 172.21.15.99:0/4233979012 conn(0x7eff580730c0 msgr2=0x7eff5806f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.240+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 shutdown_connections 2024-09-10T11:33:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.241+0000 7eff606f8640 1 -- 172.21.15.99:0/4233979012 wait complete. 2024-09-10T11:33:30.253 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:33:30.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:30.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[44575]: fuse finished with error 0 and tester_r 0 2024-09-10T11:33:30.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:33:30.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.278 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:33:30.334 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:33:30.335 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:33:30.335 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:30.355 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:33:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.472+0000 7f76cdf4b640 1 Processor -- start 2024-09-10T11:33:30.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.472+0000 7f76cdf4b640 1 -- start start 2024-09-10T11:33:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76cdf4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76cdf4b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76c80b0090 0x7f76c80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76cdf4b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76c80b40e0 con 0x7f76c815f1e0 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76cdf4b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f76c80b5ab0 con 0x7f76c80b0090 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76cdf4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76c8073c50 con 0x7f76c815ee00 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76c77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76c7fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76c80b0090 0x7f76c80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76c77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:53480/0 (socket says 172.21.15.99:53480) 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76c6ffd640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f76c815f1e0 0x7f76c80b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41712/0 (socket says 172.21.15.99:41712) 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.473+0000 7f76c77fe640 1 -- 172.21.15.99:0/3816172507 learned_addr learned my addr 172.21.15.99:0/3816172507 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.474+0000 7f76c7fff640 1 -- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 msgr2=0x7f76c80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.474+0000 7f76c7fff640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.474+0000 7f76c7fff640 1 -- 172.21.15.99:0/3816172507 >> v1:172.21.15.196:6789/0 conn(0x7f76c815f1e0 legacy=0x7f76c80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.474+0000 7f76c7fff640 1 -- 172.21.15.99:0/3816172507 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c80b4380 con 0x7f76c80b0090 2024-09-10T11:33:30.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.474+0000 7f76c77fe640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76c7fff640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c80b0090 0x7f76c80b1c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb398ee012c2b9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76c67fc640 1 -- 172.21.15.99:0/3816172507 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0007740 con 0x7f76c80b0090 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76c67fc640 1 -- 172.21.15.99:0/3816172507 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c00078e0 con 0x7f76c80b0090 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76c67fc640 1 -- 172.21.15.99:0/3816172507 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c0007cb0 con 0x7f76c80b0090 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 -- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c80b0090 msgr2=0x7f76c80b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c80b0090 0x7f76c80b1c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009d90 tx=0x7f76c002a660 comp rx=0 tx=0).stop 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 -- 172.21.15.99:0/3816172507 shutdown_connections 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c80b0090 0x7f76c80b1c70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/3816172507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c815ee00 0x7f76c80b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 -- 172.21.15.99:0/3816172507 >> 172.21.15.99:0/3816172507 conn(0x7f76c80730c0 msgr2=0x7f76c80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 -- 172.21.15.99:0/3816172507 shutdown_connections 2024-09-10T11:33:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.475+0000 7f76cdf4b640 1 -- 172.21.15.99:0/3816172507 wait complete. 2024-09-10T11:33:30.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.476+0000 7f76cdf4b640 1 Processor -- start 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 -- start start 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c80b0090 0x7f76c81f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c815f1e0 0x7f76c8159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80b21b0 con 0x7f76c815ee00 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76c80afd60 con 0x7f76c815f1e0 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76cdf4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76c80b0600 con 0x7f76c80b0090 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76c6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42818/0 (socket says 172.21.15.99:42818) 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.477+0000 7f76c6ffd640 1 -- 172.21.15.99:0/2464257836 learned_addr learned my addr 172.21.15.99:0/2464257836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:30.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c77fe640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c80b0090 0x7f76c81f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c7fff640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c815f1e0 0x7f76c8159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 -- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c80b0090 msgr2=0x7f76c81f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c80b0090 0x7f76c81f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 -- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c815f1e0 msgr2=0x7f76c8159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c815f1e0 0x7f76c8159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c81583e0 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76c6ffd640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c3c0 tx=0x7f76bc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0074e0 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76bc007680 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c8160a70 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc004a00 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.478+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c81f54e0 con 0x7f76c815ee00 2024-09-10T11:33:30.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.479+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c80b5f90 con 0x7f76c815ee00 2024-09-10T11:33:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76bc01a020 con 0x7f76c815ee00 2024-09-10T11:33:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76affff640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 0x7f76b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:30.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f76bc08a530 con 0x7f76c815ee00 2024-09-10T11:33:30.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76c77fe640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 0x7f76b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:30.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f76b4073a80 con 0x7f76c815ee00 2024-09-10T11:33:30.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.480+0000 7f76c77fe640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 0x7f76b406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76b8006ca0 tx=0x7f76b8009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:30.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.484+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76bc058850 con 0x7f76c815ee00 2024-09-10T11:33:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.660+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76c815c620 con 0x7f76c815ee00 2024-09-10T11:33:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.660+0000 7f76affff640 1 -- 172.21.15.99:0/2464257836 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f76bc05cb50 con 0x7f76c815ee00 2024-09-10T11:33:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.662+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 msgr2=0x7f76b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.662+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 0x7f76b406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76b8006ca0 tx=0x7f76b8009240 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.662+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 msgr2=0x7f76c8157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.662+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00c3c0 tx=0x7f76bc00c7c0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 shutdown_connections 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76b406be30 0x7f76b406e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76c815f1e0 0x7f76c8159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76c815ee00 0x7f76c8157ea0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 --2- 172.21.15.99:0/2464257836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76c80b0090 0x7f76c81f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 >> 172.21.15.99:0/2464257836 conn(0x7f76c80730c0 msgr2=0x7f76c80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 shutdown_connections 2024-09-10T11:33:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:30.663+0000 7f76cdf4b640 1 -- 172.21.15.99:0/2464257836 wait complete. 2024-09-10T11:33:30.675 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:33:30.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:30.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[35462]: fuse finished with error 0 and tester_r 0 2024-09-10T11:33:30.706 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:33:30.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.706 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:33:30.845 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:33:30.846 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:33:30.846 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:30.862 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:33:30.862 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:33:30.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.862 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:33:30.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:33:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:33:30.918 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:33:30.918 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:33:30.919 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:30.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:33:30.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:33:30.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:30.939 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:33:31.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:33:31.004 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:33:31.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:31.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.100+0000 7f0998311640 1 Processor -- start 2024-09-10T11:33:31.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.101+0000 7f0998311640 1 -- start start 2024-09-10T11:33:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.101+0000 7f0998311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.101+0000 7f0998311640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f099015c770 0x7f099006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.101+0000 7f0998311640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0990152e60 con 0x7f099006e5d0 2024-09-10T11:33:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f0998311640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f099006ef50 con 0x7f099015c770 2024-09-10T11:33:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f0998311640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f09900d15e0 con 0x7f09900d1200 2024-09-10T11:33:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f099015c770 0x7f099006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09960ad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f099015c770 0x7f099006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:46618/0 (socket says 172.21.15.99:46618) 2024-09-10T11:33:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 -- 172.21.15.99:0/3779238489 learned_addr learned my addr 172.21.15.99:0/3779238489 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09958ac640 1 --1- 172.21.15.99:0/3779238489 >> v1:172.21.15.196:6789/0 conn(0x7f099006e5d0 0x7f099015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41724/0 (socket says 172.21.15.99:41724) 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 -- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 msgr2=0x7f0990073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 -- 172.21.15.99:0/3779238489 >> v1:172.21.15.196:6789/0 conn(0x7f099006e5d0 legacy=0x7f099015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09968ae640 1 -- 172.21.15.99:0/3779238489 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0990072680 con 0x7f099015c770 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09950ab640 1 -- 172.21.15.99:0/3779238489 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222467068 0 0) 0x7f0990152e60 con 0x7f099006e5d0 2024-09-10T11:33:31.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.102+0000 7f09960ad640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.103+0000 7f09968ae640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f099006ea10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0980009e50 tx=0x7f098002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f5fd4a17b837333 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.103+0000 7f09950ab640 1 -- 172.21.15.99:0/3779238489 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980036070 con 0x7f099015c770 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.103+0000 7f09950ab640 1 -- 172.21.15.99:0/3779238489 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0980031600 con 0x7f099015c770 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.103+0000 7f09950ab640 1 -- 172.21.15.99:0/3779238489 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800319d0 con 0x7f099015c770 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 -- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 msgr2=0x7f099006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f099006ea10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0980009e50 tx=0x7f098002a680 comp rx=0 tx=0).stop 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 -- 172.21.15.99:0/3779238489 shutdown_connections 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f099006ea10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 --2- 172.21.15.99:0/3779238489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 -- 172.21.15.99:0/3779238489 >> 172.21.15.99:0/3779238489 conn(0x7f099006bc10 msgr2=0x7f099006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 -- 172.21.15.99:0/3779238489 shutdown_connections 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.104+0000 7f0998311640 1 -- 172.21.15.99:0/3779238489 wait complete. 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.105+0000 7f0998311640 1 Processor -- start 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.105+0000 7f0998311640 1 -- start start 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f0990157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0990072f20 con 0x7f099006e5d0 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0990072c40 con 0x7f099015c770 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f099015cd90 con 0x7f09900d1200 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09960ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09960ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42830/0 (socket says 172.21.15.99:42830) 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09960ad640 1 -- 172.21.15.99:0/1811355827 learned_addr learned my addr 172.21.15.99:0/1811355827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09968ae640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f0990157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 -- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 msgr2=0x7f0990157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f0990157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 -- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 msgr2=0x7f09901565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0990156b10 con 0x7f09900d1200 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09960ad640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f09958ac640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990154400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f098400c410 tx=0x7f098400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09840074e0 con 0x7f09900d1200 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09840047e0 con 0x7f09900d1200 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.106+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0990154940 con 0x7f09900d1200 2024-09-10T11:33:31.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.107+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0984004bd0 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.107+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0990157070 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.107+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09900b5780 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.108+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0984011070 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.109+0000 7f098effd640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 0x7f097c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.109+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f098408a990 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.109+0000 7f09960ad640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 0x7f097c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.109+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=72}) -- 0x7f097c073a20 con 0x7f09900d1200 2024-09-10T11:33:31.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.109+0000 7f09960ad640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 0x7f097c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09880095e0 tx=0x7f0988009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.113+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f09900b5780 con 0x7f09900d1200 2024-09-10T11:33:31.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.311+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f099006e9f0 con 0x7f09900d1200 2024-09-10T11:33:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.312+0000 7f098effd640 1 -- 172.21.15.99:0/1811355827 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f0990156b10 con 0x7f09900d1200 2024-09-10T11:33:31.314 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:31.314 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:33:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 msgr2=0x7f097c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 0x7f097c06e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f09880095e0 tx=0x7f0988009190 comp rx=0 tx=0).stop 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 msgr2=0x7f0990154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990154400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f098400c410 tx=0x7f098400c8d0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 shutdown_connections 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f097c06be30 0x7f097c06e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f099015c770 0x7f0990157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09900d1200 0x7f0990154400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 --2- 172.21.15.99:0/1811355827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f099006e5d0 0x7f09901565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 >> 172.21.15.99:0/1811355827 conn(0x7f099006bc10 msgr2=0x7f099015d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 shutdown_connections 2024-09-10T11:33:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.314+0000 7f0998311640 1 -- 172.21.15.99:0/1811355827 wait complete. 2024-09-10T11:33:31.327 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:31.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.423+0000 7f4c44447640 1 Processor -- start 2024-09-10T11:33:31.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.423+0000 7f4c44447640 1 -- start start 2024-09-10T11:33:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c44447640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c3c073440 0x7f4c3c15c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c44447640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3c152e60 con 0x7f4c3c0d1200 2024-09-10T11:33:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c44447640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3c0d15e0 con 0x7f4c3c06ece0 2024-09-10T11:33:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c44447640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4c3c072680 con 0x7f4c3c073440 2024-09-10T11:33:31.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c421e3640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4c3c0d1200 0x7f4c3c06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35700/0 (socket says 172.21.15.99:35700) 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c421e3640 1 -- 172.21.15.99:0/3551735758 learned_addr learned my addr 172.21.15.99:0/3551735758 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94854798 0 0) 0x7f4c3c152e60 con 0x7f4c3c0d1200 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c28003520 con 0x7f4c3c0d1200 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c429e4640 1 --2- 172.21.15.99:0/3551735758 >> v2:172.21.15.196:3300/0 conn(0x7f4c3c073440 0x7f4c3c15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4112541068 0 0) 0x7f4c28003520 con 0x7f4c3c0d1200 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 >> v2:172.21.15.196:3300/0 conn(0x7f4c3c073440 msgr2=0x7f4c3c15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 --2- 172.21.15.99:0/3551735758 >> v2:172.21.15.196:3300/0 conn(0x7f4c3c073440 0x7f4c3c15c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 >> v1:172.21.15.196:6789/0 conn(0x7f4c3c06ece0 legacy=0x7f4c3c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c072960 con 0x7f4c3c0d1200 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4c38001670 con 0x7f4c3c0d1200 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 >> v1:172.21.15.99:6789/0 conn(0x7f4c3c0d1200 legacy=0x7f4c3c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.424+0000 7f4c411e1640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c280052e0 0x7f4c2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c072680 con 0x7f4c280039b0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2800c1b0 con 0x7f4c280052e0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2800c3b0 con 0x7f4c28004800 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c429e4640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c280052e0 0x7f4c2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c419e2640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 -- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c28004800 msgr2=0x7f4c28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 -- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c280052e0 msgr2=0x7f4c2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c280052e0 0x7f4c2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c421e3640 1 -- 172.21.15.99:0/3551735758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c28003520 con 0x7f4c280039b0 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c429e4640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c280052e0 0x7f4c2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c419e2640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c28004800 0x7f4c28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:31.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.425+0000 7f4c429e4640 1 -- 172.21.15.99:0/3551735758 reap_dead start 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c421e3640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001eb0 tx=0x7f4c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ddfe47cd4c16e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38007d30 con 0x7f4c280039b0 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c380058d0 con 0x7f4c280039b0 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c411e1640 1 -- 172.21.15.99:0/3551735758 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38005d20 con 0x7f4c280039b0 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 -- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 msgr2=0x7f4c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001eb0 tx=0x7f4c28005830 comp rx=0 tx=0).stop 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 -- 172.21.15.99:0/3551735758 shutdown_connections 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 --2- 172.21.15.99:0/3551735758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c280039b0 0x7f4c28003dd0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 -- 172.21.15.99:0/3551735758 >> 172.21.15.99:0/3551735758 conn(0x7f4c3c06bc10 msgr2=0x7f4c3c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 -- 172.21.15.99:0/3551735758 shutdown_connections 2024-09-10T11:33:31.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.426+0000 7f4c44447640 1 -- 172.21.15.99:0/3551735758 wait complete. 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 Processor -- start 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 -- start start 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c280039b0 0x7f4c3c145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c3c154e30 0x7f4c3c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c15cd90 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c072f20 con 0x7f4c3c154e30 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c44447640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c15c060 con 0x7f4c280039b0 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c429e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c429e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42858/0 (socket says 172.21.15.99:42858) 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.428+0000 7f4c429e4640 1 -- 172.21.15.99:0/518786158 learned_addr learned my addr 172.21.15.99:0/518786158 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 -- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c280039b0 msgr2=0x7f4c3c145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c421e3640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c280039b0 0x7f4c3c145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c280039b0 0x7f4c3c145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 -- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c3c154e30 msgr2=0x7f4c3c1552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c3c154e30 0x7f4c3c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c1457b0 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c429e4640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0b5400 tx=0x7f4c34009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34008a20 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c34008bc0 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c1e5400 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34003990 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.429+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c1e5980 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.430+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c15c3f0 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.430+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c3401c020 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.431+0000 7f4c32ffd640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 0x7f4c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.431+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f4c3408ab60 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.431+0000 7f4c421e3640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 0x7f4c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.431+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f4c20073a20 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.431+0000 7f4c421e3640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 0x7f4c2006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c38033000 tx=0x7f4c3802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.435+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c34012070 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.633+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c3c06d4c0 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.633+0000 7f4c32ffd640 1 -- 172.21.15.99:0/518786158 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f4c340594d0 con 0x7f4c3c1e7d10 2024-09-10T11:33:31.636 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:31.636 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:33:31.636 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:33:31.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 msgr2=0x7f4c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 0x7f4c2006e2f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c38033000 tx=0x7f4c3802e000 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 msgr2=0x7f4c3c1e8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0b5400 tx=0x7f4c34009f70 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 shutdown_connections 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.635+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c2006be30 0x7f4c2006e2f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c3c1e7d10 0x7f4c3c1e8190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c3c154e30 0x7f4c3c1552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 --2- 172.21.15.99:0/518786158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c280039b0 0x7f4c3c145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 >> 172.21.15.99:0/518786158 conn(0x7f4c3c06bc10 msgr2=0x7f4c3c15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 shutdown_connections 2024-09-10T11:33:31.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.636+0000 7f4c44447640 1 -- 172.21.15.99:0/518786158 wait complete. 2024-09-10T11:33:31.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:31.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.744+0000 7f417418d640 1 Processor -- start 2024-09-10T11:33:31.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.744+0000 7f417418d640 1 -- start start 2024-09-10T11:33:31.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f417418d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f416c160030 0x7f416c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f417418d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c0b5140 con 0x7f416c163620 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f417418d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f416c162b40 con 0x7f416c06bc10 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f417418d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f416c0b5420 con 0x7f416c160030 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f4171728640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f416c163620 0x7f416c0b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35716/0 (socket says 172.21.15.99:35716) 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f4171728640 1 -- 172.21.15.99:0/3516575691 learned_addr learned my addr 172.21.15.99:0/3516575691 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 457411564 0 0) 0x7f416c0b5140 con 0x7f416c163620 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4158003520 con 0x7f416c163620 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.745+0000 7f417272a640 1 --2- 172.21.15.99:0/3516575691 >> v2:172.21.15.196:3300/0 conn(0x7f416c160030 0x7f416c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2876520592 0 0) 0x7f4158003520 con 0x7f416c163620 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 >> v2:172.21.15.196:3300/0 conn(0x7f416c160030 msgr2=0x7f416c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 --2- 172.21.15.99:0/3516575691 >> v2:172.21.15.196:3300/0 conn(0x7f416c160030 0x7f416c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 >> v1:172.21.15.196:6789/0 conn(0x7f416c06bc10 legacy=0x7f416c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c162e20 con 0x7f416c163620 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f415c001670 con 0x7f416c163620 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 >> v1:172.21.15.99:6789/0 conn(0x7f416c163620 legacy=0x7f416c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41580039b0 0x7f4158003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4158004800 0x7f4158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f415800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0b5420 con 0x7f41580051c0 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f415800c160 con 0x7f41580039b0 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f415800c360 con 0x7f4158004800 2024-09-10T11:33:31.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4171f29640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41580039b0 0x7f4158003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f415800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f4171728640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4158004800 0x7f4158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 -- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4158004800 msgr2=0x7f4158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4158004800 0x7f4158004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 -- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41580039b0 msgr2=0x7f4158003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41580039b0 0x7f4158003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.746+0000 7f417272a640 1 -- 172.21.15.99:0/3516575691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4158003520 con 0x7f41580051c0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f417272a640 1 -- 172.21.15.99:0/3516575691 reap_dead start 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f417272a640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f415800b9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4164009fc0 tx=0x7f4158005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f679ac2882484c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164035020 con 0x7f41580051c0 2024-09-10T11:33:31.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4164003bd0 con 0x7f41580051c0 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f4170f27640 1 -- 172.21.15.99:0/3516575691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164009c10 con 0x7f41580051c0 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f417418d640 1 -- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 msgr2=0x7f415800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.747+0000 7f417418d640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f415800b9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4164009fc0 tx=0x7f4158005830 comp rx=0 tx=0).stop 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.748+0000 7f417418d640 1 -- 172.21.15.99:0/3516575691 shutdown_connections 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.748+0000 7f417418d640 1 --2- 172.21.15.99:0/3516575691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f415800b9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.748+0000 7f417418d640 1 -- 172.21.15.99:0/3516575691 >> 172.21.15.99:0/3516575691 conn(0x7f416c0b3630 msgr2=0x7f416c0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.748+0000 7f417418d640 1 -- 172.21.15.99:0/3516575691 shutdown_connections 2024-09-10T11:33:31.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.748+0000 7f417418d640 1 -- 172.21.15.99:0/3516575691 wait complete. 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.749+0000 7f417418d640 1 Processor -- start 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.749+0000 7f417418d640 1 -- start start 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.749+0000 7f417418d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.749+0000 7f417418d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f416c142540 0x7f416c142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f416c140fc0 0x7f416c141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0d16e0 con 0x7f41580051c0 2024-09-10T11:33:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0d1560 con 0x7f416c142540 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f416c06bff0 con 0x7f416c140fc0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42868/0 (socket says 172.21.15.99:42868) 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 -- 172.21.15.99:0/1523893198 learned_addr learned my addr 172.21.15.99:0/1523893198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417272a640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f416c140fc0 0x7f416c141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171728640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f416c142540 0x7f416c142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 -- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f416c140fc0 msgr2=0x7f416c141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f416c140fc0 0x7f416c141420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 -- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f416c142540 msgr2=0x7f416c142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f416c142540 0x7f416c142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c1434b0 con 0x7f41580051c0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f4171f29640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f416800f5f0 tx=0x7f416800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41680052e0 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4168018070 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168003810 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c143790 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.750+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c143d50 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.751+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c0b5140 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.752+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41680039b0 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.752+0000 7f41627fc640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 0x7f415006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.752+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f416808acb0 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.752+0000 7f4171728640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 0x7f415006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.752+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f4150073a20 con 0x7f41580051c0 2024-09-10T11:33:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.753+0000 7f4171728640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 0x7f415006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f416c142af0 tx=0x7f415c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.756+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4168059620 con 0x7f41580051c0 2024-09-10T11:33:31.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.924+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f416c0c68a0 con 0x7f41580051c0 2024-09-10T11:33:31.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.925+0000 7f41627fc640 1 -- 172.21.15.99:0/1523893198 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16872 (secure 0 0 0) 0x7f4168058f20 con 0x7f41580051c0 2024-09-10T11:33:31.928 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:31.928 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":71,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:16.134336+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:32:26.850541+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T11:32:27.213215+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.28","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 msgr2=0x7f415006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 0x7f415006e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f416c142af0 tx=0x7f415c009040 comp rx=0 tx=0).stop 2024-09-10T11:33:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 msgr2=0x7f416c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f416800f5f0 tx=0x7f416800fab0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 shutdown_connections 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f415006be30 0x7f415006e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f416c140fc0 0x7f416c141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f416c142540 0x7f416c142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 --2- 172.21.15.99:0/1523893198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41580051c0 0x7f416c06e840 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 >> 172.21.15.99:0/1523893198 conn(0x7f416c0b3630 msgr2=0x7f416c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.927+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 shutdown_connections 2024-09-10T11:33:31.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:31.928+0000 7f417418d640 1 -- 172.21.15.99:0/1523893198 wait complete. 2024-09-10T11:33:31.941 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:33:31.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:32.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.048+0000 7f4f8501e640 1 Processor -- start 2024-09-10T11:33:32.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.048+0000 7f4f8501e640 1 -- start start 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f8501e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f800b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f8501e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f800b0090 0x7f4f800b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f8501e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f800b40e0 con 0x7f4f8015f1e0 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f8501e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4f800b5ab0 con 0x7f4f800b0090 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f8501e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f80073c50 con 0x7f4f8015ee00 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7e59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4f8015f1e0 0x7f4f800b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35724/0 (socket says 172.21.15.99:35724) 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7e59c640 1 -- 172.21.15.99:0/3007632906 learned_addr learned my addr 172.21.15.99:0/3007632906 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1981918337 0 0) 0x7f4f800b40e0 con 0x7f4f8015f1e0 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7ed9d640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f800b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f68003520 con 0x7f4f8015f1e0 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> v2:172.21.15.196:3300/0 conn(0x7f4f800b0090 0x7f4f800b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 76633446 0 0) 0x7f4f68003520 con 0x7f4f8015f1e0 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 msgr2=0x7f4f800b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f800b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 >> v2:172.21.15.196:3300/0 conn(0x7f4f800b0090 msgr2=0x7f4f800b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 --2- 172.21.15.99:0/3007632906 >> v2:172.21.15.196:3300/0 conn(0x7f4f800b0090 0x7f4f800b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f800b4380 con 0x7f4f8015f1e0 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4f74001670 con 0x7f4f8015f1e0 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 >> v1:172.21.15.99:6789/0 conn(0x7f4f8015f1e0 legacy=0x7f4f800b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68003f30 0x7f4f68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800b5ab0 con 0x7f4f68004d40 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f80073c50 con 0x7f4f68003f30 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6800ba10 con 0x7f4f8015ee00 2024-09-10T11:33:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> v2:172.21.15.196:3300/0 conn(0x7f4f800b0090 0x7f4f800b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7e59c640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68003f30 0x7f4f68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.049+0000 7f4f7ed9d640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 -- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 msgr2=0x7f4f680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015ee00 0x7f4f680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 -- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68003f30 msgr2=0x7f4f68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f68003f30 0x7f4f68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 -- 172.21.15.99:0/3007632906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f68003520 con 0x7f4f68004d40 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f680051c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4f780073e0 tx=0x7f4f68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93729d1d4589e5d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.050+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78009c70 con 0x7f4f68004d40 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7802c680 con 0x7f4f68004d40 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f7dd9b640 1 -- 172.21.15.99:0/3007632906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7802ca30 con 0x7f4f68004d40 2024-09-10T11:33:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 -- 172.21.15.99:0/3007632906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 msgr2=0x7f4f680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 --2- 172.21.15.99:0/3007632906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f680051c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4f780073e0 tx=0x7f4f68005830 comp rx=0 tx=0).stop 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f7f59e640 1 -- 172.21.15.99:0/3007632906 reap_dead start 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 -- 172.21.15.99:0/3007632906 shutdown_connections 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 -- 172.21.15.99:0/3007632906 >> 172.21.15.99:0/3007632906 conn(0x7f4f800730c0 msgr2=0x7f4f800734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 -- 172.21.15.99:0/3007632906 shutdown_connections 2024-09-10T11:33:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.051+0000 7f4f8501e640 1 -- 172.21.15.99:0/3007632906 wait complete. 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.052+0000 7f4f8501e640 1 Processor -- start 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 -- start start 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f801538e0 0x7f4f8009eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015afa0 0x7f4f8015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800b21b0 con 0x7f4f68004d40 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f800afd60 con 0x7f4f801538e0 2024-09-10T11:33:32.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f8501e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f800b0600 con 0x7f4f8015afa0 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f7ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f7ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42892/0 (socket says 172.21.15.99:42892) 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.053+0000 7f4f7ed9d640 1 -- 172.21.15.99:0/326164239 learned_addr learned my addr 172.21.15.99:0/326164239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7e59c640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f801538e0 0x7f4f8009eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7f59e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015afa0 0x7f4f8015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 -- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015afa0 msgr2=0x7f4f8015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015afa0 0x7f4f8015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 -- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f801538e0 msgr2=0x7f4f8009eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f801538e0 0x7f4f8009eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f80156670 con 0x7f4f68004d40 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f7ed9d640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f700119a0 tx=0x7f4f70011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000fa90 con 0x7f4f68004d40 2024-09-10T11:33:32.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7000fc30 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f801560b0 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70014440 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.054+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f800ca460 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.055+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f800b5f90 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.055+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f70012040 con 0x7f4f68004d40 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.056+0000 7f4f677fe640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 0x7f4f6c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.056+0000 7f4f7e59c640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 0x7f4f6c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.056+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f4f7008b4d0 con 0x7f4f68004d40 2024-09-10T11:33:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.056+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f4f6c073a50 con 0x7f4f68004d40 2024-09-10T11:33:32.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.056+0000 7f4f7e59c640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 0x7f4f6c06e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f800b1850 tx=0x7f4f74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.060+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f70019030 con 0x7f4f68004d40 2024-09-10T11:33:32.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.263+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f8006eac0 con 0x7f4f68004d40 2024-09-10T11:33:32.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.263+0000 7f4f677fe640 1 -- 172.21.15.99:0/326164239 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f4f700597f0 con 0x7f4f68004d40 2024-09-10T11:33:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:32.266 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:33:32.266 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:33:32.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 msgr2=0x7f4f6c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 0x7f4f6c06e320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4f800b1850 tx=0x7f4f74009040 comp rx=0 tx=0).stop 2024-09-10T11:33:32.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 msgr2=0x7f4f801f5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4f700119a0 tx=0x7f4f70011e60 comp rx=0 tx=0).stop 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 shutdown_connections 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4f6c06be60 0x7f4f6c06e320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f8015afa0 0x7f4f8015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f801538e0 0x7f4f8009eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 --2- 172.21.15.99:0/326164239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f68004d40 0x7f4f801f5a40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 >> 172.21.15.99:0/326164239 conn(0x7f4f800730c0 msgr2=0x7f4f8015c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 shutdown_connections 2024-09-10T11:33:32.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.265+0000 7f4f8501e640 1 -- 172.21.15.99:0/326164239 wait complete. 2024-09-10T11:33:32.278 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:32.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.375+0000 7f8fb796d640 1 Processor -- start 2024-09-10T11:33:32.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.375+0000 7f8fb796d640 1 -- start start 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb796d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8fb00d1200 0x7f8fb0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb796d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8fb006e5d0 0x7f8fb006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb796d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb0152e60 con 0x7f8fb015c060 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb796d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb006ef80 con 0x7f8fb006e5d0 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb796d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb00d15e0 con 0x7f8fb00d1200 2024-09-10T11:33:32.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8fb006e5d0 0x7f8fb006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8fb006e5d0 0x7f8fb006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42898/0 (socket says 172.21.15.99:42898) 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 -- 172.21.15.99:0/4059458996 learned_addr learned my addr 172.21.15.99:0/4059458996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb5709640 1 --2- 172.21.15.99:0/4059458996 >> v2:172.21.15.196:3300/0 conn(0x7f8fb00d1200 0x7f8fb0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 -- 172.21.15.99:0/4059458996 >> v2:172.21.15.196:3300/0 conn(0x7f8fb00d1200 msgr2=0x7f8fb0073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 --2- 172.21.15.99:0/4059458996 >> v2:172.21.15.196:3300/0 conn(0x7f8fb00d1200 0x7f8fb0073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 -- 172.21.15.99:0/4059458996 >> v1:172.21.15.196:6789/0 conn(0x7f8fb015c060 legacy=0x7f8fb015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.376+0000 7f8fb4f08640 1 -- 172.21.15.99:0/4059458996 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb0072680 con 0x7f8fb006e5d0 2024-09-10T11:33:32.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.377+0000 7f8fb4f08640 1 --2- 172.21.15.99:0/4059458996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb006e5d0 0x7f8fb006ea40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00d1850 tx=0x7f8fa002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c330a9e45624bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.377+0000 7f8fa7fff640 1 -- 172.21.15.99:0/4059458996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0037070 con 0x7f8fb006e5d0 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.377+0000 7f8fa7fff640 1 -- 172.21.15.99:0/4059458996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa0032440 con 0x7f8fb006e5d0 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.377+0000 7f8fa7fff640 1 -- 172.21.15.99:0/4059458996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0032810 con 0x7f8fb006e5d0 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.377+0000 7f8fb796d640 1 -- 172.21.15.99:0/4059458996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb006e5d0 msgr2=0x7f8fb006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 --2- 172.21.15.99:0/4059458996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb006e5d0 0x7f8fb006ea40 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00d1850 tx=0x7f8fa002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 -- 172.21.15.99:0/4059458996 shutdown_connections 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 --2- 172.21.15.99:0/4059458996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb006e5d0 0x7f8fb006ea40 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 --2- 172.21.15.99:0/4059458996 >> v2:172.21.15.196:3300/0 conn(0x7f8fb00d1200 0x7f8fb0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 -- 172.21.15.99:0/4059458996 >> 172.21.15.99:0/4059458996 conn(0x7f8fb006bc10 msgr2=0x7f8fb006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 -- 172.21.15.99:0/4059458996 shutdown_connections 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.378+0000 7f8fb796d640 1 -- 172.21.15.99:0/4059458996 wait complete. 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.379+0000 7f8fb796d640 1 Processor -- start 2024-09-10T11:33:32.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.379+0000 7f8fb796d640 1 -- start start 2024-09-10T11:33:32.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.379+0000 7f8fb796d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8fb006e5d0 0x7f8fb0141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 0x7f8fb01584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0072f20 con 0x7f8fb015c060 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0072c40 con 0x7f8fb00d1200 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb015cd90 con 0x7f8fb006e5d0 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:42906/0 (socket says 172.21.15.99:42906) 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 -- 172.21.15.99:0/1165744827 learned_addr learned my addr 172.21.15.99:0/1165744827 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5709640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8fb006e5d0 0x7f8fb0141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb4f08640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 0x7f8fb01584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 -- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8fb006e5d0 msgr2=0x7f8fb0141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8fb006e5d0 0x7f8fb0141150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 -- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 msgr2=0x7f8fb01584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 0x7f8fb01584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb01546f0 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb4f08640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 0x7f8fb01584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb5f0a640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800eab0 tx=0x7f8fa80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa800cde0 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa8003970 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01549d0 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.380+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8003d40 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.381+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb0154f90 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.381+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb006ea40 con 0x7f8fb015c060 2024-09-10T11:33:32.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.382+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fa80050d0 con 0x7f8fb015c060 2024-09-10T11:33:32.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.382+0000 7f8fa5ffb640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.382+0000 7f8fb5709640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.382+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f8fa808d470 con 0x7f8fb015c060 2024-09-10T11:33:32.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.383+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f8f94073a50 con 0x7f8fb015c060 2024-09-10T11:33:32.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.383+0000 7f8fb5709640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 0x7f8f9406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0079a0 tx=0x7f8fac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.387+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8fa8016020 con 0x7f8fb015c060 2024-09-10T11:33:32.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.587+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb00ce490 con 0x7f8fb015c060 2024-09-10T11:33:32.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.587+0000 7f8fa5ffb640 1 -- 172.21.15.99:0/1165744827 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f8fa805b790 con 0x7f8fb015c060 2024-09-10T11:33:32.589 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:32.589 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":29,"btime":"2024-09-10T11:32:30:928630+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:32:28.253730+0000","modified":"2024-09-10T11:32:30.276228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5592},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5592":{"gid":5592,"name":"a","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2576628819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2576628819},{"type":"v1","addr":"172.21.15.99:6837","nonce":2576628819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5592,"qdb_cluster":[5592]},"id":4}]} 2024-09-10T11:33:32.589 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 29 2024-09-10T11:33:32.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 msgr2=0x7f8f9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 0x7f8f9406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0079a0 tx=0x7f8fac008040 comp rx=0 tx=0).stop 2024-09-10T11:33:32.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 msgr2=0x7f8fb01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8fa800eab0 tx=0x7f8fa80021d0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 shutdown_connections 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f9406be60 0x7f8f9406e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fb015c060 0x7f8fb01565d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb00d1200 0x7f8fb01584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 --2- 172.21.15.99:0/1165744827 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8fb006e5d0 0x7f8fb0141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 >> 172.21.15.99:0/1165744827 conn(0x7f8fb006bc10 msgr2=0x7f8fb00ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 shutdown_connections 2024-09-10T11:33:32.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.589+0000 7f8fb796d640 1 -- 172.21.15.99:0/1165744827 wait complete. 2024-09-10T11:33:32.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:32.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 Processor -- start 2024-09-10T11:33:32.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 -- start start 2024-09-10T11:33:32.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b9006ece0 0x7f4b90073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b9015c060 0x7f4b9015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b90152e60 con 0x7f4b900d1200 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4b900d15e0 con 0x7f4b9006ece0 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.699+0000 7f4b9514b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b90072680 con 0x7f4b9015c060 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b9006ece0 0x7f4b90073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b9006ece0 0x7f4b90073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36292/0 (socket says 172.21.15.99:36292) 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 -- 172.21.15.99:0/29783105 learned_addr learned my addr 172.21.15.99:0/29783105 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8ed9d640 1 --1- 172.21.15.99:0/29783105 >> v1:172.21.15.99:6789/0 conn(0x7f4b900d1200 0x7f4b9006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37304/0 (socket says 172.21.15.99:37304) 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8f59e640 1 --2- 172.21.15.99:0/29783105 >> v2:172.21.15.196:3300/0 conn(0x7f4b9015c060 0x7f4b9015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 -- 172.21.15.99:0/29783105 >> v2:172.21.15.196:3300/0 conn(0x7f4b9015c060 msgr2=0x7f4b9015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 --2- 172.21.15.99:0/29783105 >> v2:172.21.15.196:3300/0 conn(0x7f4b9015c060 0x7f4b9015c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 -- 172.21.15.99:0/29783105 >> v1:172.21.15.99:6789/0 conn(0x7f4b900d1200 legacy=0x7f4b9006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8e59c640 1 -- 172.21.15.99:0/29783105 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2696566076 0 0) 0x7f4b90152e60 con 0x7f4b900d1200 2024-09-10T11:33:32.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.700+0000 7f4b8659c640 1 -- 172.21.15.99:0/29783105 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90072960 con 0x7f4b9006ece0 2024-09-10T11:33:32.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b8659c640 1 --2- 172.21.15.99:0/29783105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9006ece0 0x7f4b90073440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4b900cd570 tx=0x7f4b7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7534d77feb8e8b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b8e59c640 1 -- 172.21.15.99:0/29783105 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78037070 con 0x7f4b9006ece0 2024-09-10T11:33:32.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b8e59c640 1 -- 172.21.15.99:0/29783105 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b78032440 con 0x7f4b9006ece0 2024-09-10T11:33:32.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b8e59c640 1 -- 172.21.15.99:0/29783105 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78032810 con 0x7f4b9006ece0 2024-09-10T11:33:32.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 -- 172.21.15.99:0/29783105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9006ece0 msgr2=0x7f4b90073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 --2- 172.21.15.99:0/29783105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9006ece0 0x7f4b90073440 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4b900cd570 tx=0x7f4b7802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 -- 172.21.15.99:0/29783105 shutdown_connections 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 --2- 172.21.15.99:0/29783105 >> v2:172.21.15.196:3300/0 conn(0x7f4b9015c060 0x7f4b9015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 --2- 172.21.15.99:0/29783105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9006ece0 0x7f4b90073440 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 -- 172.21.15.99:0/29783105 >> 172.21.15.99:0/29783105 conn(0x7f4b9006bc10 msgr2=0x7f4b9006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 -- 172.21.15.99:0/29783105 shutdown_connections 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.701+0000 7f4b9514b640 1 -- 172.21.15.99:0/29783105 wait complete. 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.703+0000 7f4b9514b640 1 Processor -- start 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.703+0000 7f4b9514b640 1 -- start start 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.703+0000 7f4b9514b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b9006ece0 0x7f4b90140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b900d1200 0x7f4b90151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9015cd90 con 0x7f4b9015c060 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90072f20 con 0x7f4b9006ece0 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9015ca30 con 0x7f4b900d1200 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36296/0 (socket says 172.21.15.99:36296) 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 -- 172.21.15.99:0/4067093914 learned_addr learned my addr 172.21.15.99:0/4067093914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8ed9d640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b9006ece0 0x7f4b90140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 -- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b900d1200 msgr2=0x7f4b90151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b900d1200 0x7f4b90151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 -- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b9006ece0 msgr2=0x7f4b90140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b9006ece0 0x7f4b90140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b90141480 con 0x7f4b9015c060 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b8f59e640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800cda0 tx=0x7f4b8800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003b90 con 0x7f4b9015c060 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b88003d30 con 0x7f4b9015c060 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88004240 con 0x7f4b9015c060 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b900c7190 con 0x7f4b9015c060 2024-09-10T11:33:32.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.704+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b90141b80 con 0x7f4b9015c060 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.705+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90073440 con 0x7f4b9015c060 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.706+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b880043e0 con 0x7f4b9015c060 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.706+0000 7f4b87fff640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 0x7f4b7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.706+0000 7f4b8ed9d640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 0x7f4b7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.706+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f4b8808b930 con 0x7f4b9015c060 2024-09-10T11:33:32.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.706+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f4b70073a80 con 0x7f4b9015c060 2024-09-10T11:33:32.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.707+0000 7f4b8ed9d640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 0x7f4b7006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000f60 tx=0x7f4b80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:32.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.710+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b8801d340 con 0x7f4b9015c060 2024-09-10T11:33:32.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.877+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b9006c190 con 0x7f4b9015c060 2024-09-10T11:33:32.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.878+0000 7f4b87fff640 1 -- 172.21.15.99:0/4067093914 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16872 (secure 0 0 0) 0x7f4b8805a330 con 0x7f4b9015c060 2024-09-10T11:33:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:32.880 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":71,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:16.134336+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:32:26.850541+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T11:32:27.213215+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.28","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:32.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 msgr2=0x7f4b7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 0x7f4b7006e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b80000f60 tx=0x7f4b80009040 comp rx=0 tx=0).stop 2024-09-10T11:33:32.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 msgr2=0x7f4b90151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800cda0 tx=0x7f4b8800a660 comp rx=0 tx=0).stop 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 shutdown_connections 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b7006be30 0x7f4b7006e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b9015c060 0x7f4b90151e50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b900d1200 0x7f4b90151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 --2- 172.21.15.99:0/4067093914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b9006ece0 0x7f4b90140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 >> 172.21.15.99:0/4067093914 conn(0x7f4b9006bc10 msgr2=0x7f4b900cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 shutdown_connections 2024-09-10T11:33:32.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.880+0000 7f4b9514b640 1 -- 172.21.15.99:0/4067093914 wait complete. 2024-09-10T11:33:32.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:33:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.993+0000 7f03e4ffd640 1 Processor -- start 2024-09-10T11:33:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.993+0000 7f03e4ffd640 1 -- start start 2024-09-10T11:33:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03e4ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f03e00cfb90 0x7f03e00b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03e4ffd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e006df80 con 0x7f03e00d1410 2024-09-10T11:33:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03e4ffd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03e00d1ad0 con 0x7f03e00b3660 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03e4ffd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f03e00d18a0 con 0x7f03e00cfb90 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03ded9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f03e00d1410 0x7f03e00cde00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37318/0 (socket says 172.21.15.99:37318) 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.994+0000 7f03ded9d640 1 -- 172.21.15.99:0/3564157203 learned_addr learned my addr 172.21.15.99:0/3564157203 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310326919 0 0) 0x7f03e006df80 con 0x7f03e00d1410 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03c8003520 con 0x7f03e00d1410 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 --2- 172.21.15.99:0/3564157203 >> v2:172.21.15.196:3300/0 conn(0x7f03e00cfb90 0x7f03e00b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3501277083 0 0) 0x7f03c8003520 con 0x7f03e00d1410 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 >> v2:172.21.15.196:3300/0 conn(0x7f03e00cfb90 msgr2=0x7f03e00b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 --2- 172.21.15.99:0/3564157203 >> v2:172.21.15.196:3300/0 conn(0x7f03e00cfb90 0x7f03e00b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 >> v1:172.21.15.196:6789/0 conn(0x7f03e00b3660 legacy=0x7f03e00d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e00d1db0 con 0x7f03e00d1410 2024-09-10T11:33:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f03d8001670 con 0x7f03e00d1410 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 >> v1:172.21.15.99:6789/0 conn(0x7f03e00d1410 legacy=0x7f03e00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 0x7f03c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03c8004800 0x7f03c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80051c0 0x7f03c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03c800c130 con 0x7f03c80039b0 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03c800c2e0 con 0x7f03c80051c0 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03c800c4e0 con 0x7f03c8004800 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 0x7f03c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03ded9d640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80051c0 0x7f03c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03ddd9b640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03c8004800 0x7f03c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 -- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03c8004800 msgr2=0x7f03c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03c8004800 0x7f03c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 -- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80051c0 msgr2=0x7f03c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.995+0000 7f03de59c640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80051c0 0x7f03c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03de59c640 1 -- 172.21.15.99:0/3564157203 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c8003520 con 0x7f03c80039b0 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03ded9d640 1 -- 172.21.15.99:0/3564157203 reap_dead start 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03ddd9b640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03c8004800 0x7f03c8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03de59c640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 0x7f03c8003dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fc0 tx=0x7f03c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d53a1ae05e98880 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0009c10 con 0x7f03c80039b0 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d0036070 con 0x7f03c80039b0 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.996+0000 7f03dd59a640 1 -- 172.21.15.99:0/3564157203 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0031880 con 0x7f03c80039b0 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 msgr2=0x7f03c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 0x7f03c8003dd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fc0 tx=0x7f03c8005830 comp rx=0 tx=0).stop 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3564157203 shutdown_connections 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3564157203 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c80039b0 0x7f03c8003dd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3564157203 >> 172.21.15.99:0/3564157203 conn(0x7f03e00b4650 msgr2=0x7f03e00b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:33.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3564157203 shutdown_connections 2024-09-10T11:33:33.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.997+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3564157203 wait complete. 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.998+0000 7f03e4ffd640 1 Processor -- start 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.998+0000 7f03e4ffd640 1 -- start start 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80039b0 0x7f03e00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 0x7f03e00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03e006e260 con 0x7f03e0154f80 2024-09-10T11:33:33.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03e00d2120 con 0x7f03c80039b0 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03e4ffd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f03e00b5170 con 0x7f03e00cbd60 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36310/0 (socket says 172.21.15.99:36310) 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 -- 172.21.15.99:0/3659617434 learned_addr learned my addr 172.21.15.99:0/3659617434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03de59c640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80039b0 0x7f03e00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ded9d640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 0x7f03e00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 -- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 msgr2=0x7f03e00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 0x7f03e00cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 -- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80039b0 msgr2=0x7f03e00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80039b0 0x7f03e00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ddd9b640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e01f8db0 con 0x7f03e0154f80 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:32.999+0000 7f03ded9d640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 0x7f03e00cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03ddd9b640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e4b0 tx=0x7f03d400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d40052e0 con 0x7f03e0154f80 2024-09-10T11:33:33.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d4007020 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e01f8230 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4007210 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.000+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e01f87f0 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.001+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03e006df80 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.001+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f03d401a030 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.002+0000 7f03ceffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 0x7f03c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.002+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5767+0+0 (secure 0 0 0) 0x7f03d408a330 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.002+0000 7f03de59c640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 0x7f03c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.002+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f03c0073a50 con 0x7f03e0154f80 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.002+0000 7f03de59c640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 0x7f03c006e320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f03d0035000 tx=0x7f03d002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.006+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f03d4021020 con 0x7f03e0154f80 2024-09-10T11:33:33.174 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:33:33.173+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-09-10T11:33:33.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.181+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 7 ==== osd_map(72..72 src has 1..72) ==== 296+0+0 (secure 0 0 0) 0x7f03d40bed50 con 0x7f03e0154f80 2024-09-10T11:33:33.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.181+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=73}) -- 0x7f03c0074340 con 0x7f03e0154f80 2024-09-10T11:33:33.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.210+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f03e00b3420 con 0x7f03e0154f80 2024-09-10T11:33:33.212 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:33:33.210+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:33:33.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.212+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f03d4058d30 con 0x7f03e0154f80 2024-09-10T11:33:33.213 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:33:33.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.212+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 <== mon.0 v2:172.21.15.99:3300/0 9 ==== osd_map(73..73 src has 1..73) ==== 374+0+0 (secure 0 0 0) 0x7f03d4088930 con 0x7f03e0154f80 2024-09-10T11:33:33.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.213+0000 7f03ceffd640 1 -- 172.21.15.99:0/3659617434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7f03c0074490 con 0x7f03e0154f80 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.215+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 msgr2=0x7f03c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.215+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 0x7f03c006e320 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f03d0035000 tx=0x7f03d002f000 comp rx=0 tx=0).stop 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 msgr2=0x7f03e00c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e4b0 tx=0x7f03d400e970 comp rx=0 tx=0).stop 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 shutdown_connections 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f03c006be60 0x7f03c006e320 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03e00cbd60 0x7f03e00cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03e0154f80 0x7f03e00c65f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 --2- 172.21.15.99:0/3659617434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03c80039b0 0x7f03e00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 >> 172.21.15.99:0/3659617434 conn(0x7f03e00b4650 msgr2=0x7f03e00b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:33.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 shutdown_connections 2024-09-10T11:33:33.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.216+0000 7f03e4ffd640 1 -- 172.21.15.99:0/3659617434 wait complete. 2024-09-10T11:33:33.220 INFO:tasks.ceph.mds.a.smithi099.stderr: -7459> 2024-09-10T11:32:34.888+0000 7f88a58e1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:33:33.230 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:33:33.251 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:33:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.328+0000 7f6643ff8640 1 Processor -- start 2024-09-10T11:33:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.328+0000 7f6643ff8640 1 -- start start 2024-09-10T11:33:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6643ff8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c15efe0 0x7f663c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6643ff8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f663c0d1cd0 0x7f663c0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6643ff8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f663c0d29b0 con 0x7f663c0d3a50 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6643ff8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f663c0d2c90 con 0x7f663c0d1cd0 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6643ff8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0d2f70 con 0x7f663c15efe0 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6641593640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f663c0d3a50 0x7f663c0d15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37344/0 (socket says 172.21.15.99:37344) 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6641593640 1 -- 172.21.15.99:0/3304650259 learned_addr learned my addr 172.21.15.99:0/3304650259 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6642595640 1 --2- 172.21.15.99:0/3304650259 >> v2:172.21.15.196:3300/0 conn(0x7f663c0d1cd0 0x7f663c0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6641d94640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c15efe0 0x7f663c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823329021 0 0) 0x7f663c0d29b0 con 0x7f663c0d3a50 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6628003520 con 0x7f663c0d3a50 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6642595640 1 -- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c15efe0 msgr2=0x7f663c0d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6642595640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c15efe0 0x7f663c0d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 212603036 0 0) 0x7f6628003520 con 0x7f663c0d3a50 2024-09-10T11:33:33.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6642595640 1 -- 172.21.15.99:0/3304650259 >> v1:172.21.15.99:6789/0 conn(0x7f663c0d3a50 legacy=0x7f663c0d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.329+0000 7f6642595640 1 -- 172.21.15.99:0/3304650259 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c0d3250 con 0x7f663c0d1cd0 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.330+0000 7f6642595640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c0d1cd0 0x7f663c0d0950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6634009dc0 tx=0x7f663402a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4815fd16171b3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.330+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c001690 con 0x7f663c0d1cd0 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.330+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6634007540 con 0x7f663c0d1cd0 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.330+0000 7f6640d92640 1 -- 172.21.15.99:0/3304650259 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340078f0 con 0x7f663c0d1cd0 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 -- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c0d1cd0 msgr2=0x7f663c0d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c0d1cd0 0x7f663c0d0950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6634009dc0 tx=0x7f663402a610 comp rx=0 tx=0).stop 2024-09-10T11:33:33.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 -- 172.21.15.99:0/3304650259 shutdown_connections 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c0d1cd0 0x7f663c0d0950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3304650259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c15efe0 0x7f663c0d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 -- 172.21.15.99:0/3304650259 >> 172.21.15.99:0/3304650259 conn(0x7f663c0d4c40 msgr2=0x7f663c0d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 -- 172.21.15.99:0/3304650259 shutdown_connections 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.331+0000 7f6643ff8640 1 -- 172.21.15.99:0/3304650259 wait complete. 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.332+0000 7f6643ff8640 1 Processor -- start 2024-09-10T11:33:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.332+0000 7f6643ff8640 1 -- start start 2024-09-10T11:33:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c0d3a50 0x7f663c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c15efe0 0x7f663c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f663c0b4910 con 0x7f663c0d1cd0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f663c0b4790 con 0x7f663c15efe0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6643ff8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f663c0b4a90 con 0x7f663c0d3a50 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36328/0 (socket says 172.21.15.99:36328) 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 -- 172.21.15.99:0/3526378997 learned_addr learned my addr 172.21.15.99:0/3526378997 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641593640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c0d3a50 0x7f663c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6642595640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c15efe0 0x7f663c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 -- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c0d3a50 msgr2=0x7f663c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c0d3a50 0x7f663c153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 -- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c15efe0 msgr2=0x7f663c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c15efe0 0x7f663c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c0c9770 con 0x7f663c0d1cd0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f6641d94640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f663800e8a0 tx=0x7f663800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638017070 con 0x7f663c0d1cd0 2024-09-10T11:33:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.333+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663800c940 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.334+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c0ca1c0 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.334+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800cd30 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.334+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c0c8a30 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.335+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0d15c0 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.335+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f663801a070 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.335+0000 7f66327fc640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 0x7f662006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.336+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f663808ac20 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.336+0000 7f6641593640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 0x7f662006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.336+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=74}) -- 0x7f6620073b60 con 0x7f663c0d1cd0 2024-09-10T11:33:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.336+0000 7f6641593640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 0x7f662006e320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f663c0bab00 tx=0x7f662c001de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:33.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.340+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6638058f80 con 0x7f663c0d1cd0 2024-09-10T11:33:33.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:33.538+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f663c08ec90 con 0x7f663c0d1cd0 2024-09-10T11:33:34.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.213+0000 7f66327fc640 1 -- 172.21.15.99:0/3526378997 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f663805d280 con 0x7f663c0d1cd0 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 msgr2=0x7f662006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 0x7f662006e320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f663c0bab00 tx=0x7f662c001de0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 msgr2=0x7f663c0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f663800e8a0 tx=0x7f663800ed60 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 shutdown_connections 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f662006be60 0x7f662006e320 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f663c15efe0 0x7f663c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f663c0d3a50 0x7f663c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 --2- 172.21.15.99:0/3526378997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f663c0d1cd0 0x7f663c0cdaa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 >> 172.21.15.99:0/3526378997 conn(0x7f663c0d4c40 msgr2=0x7f663c15e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:34.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 shutdown_connections 2024-09-10T11:33:34.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.217+0000 7f6643ff8640 1 -- 172.21.15.99:0/3526378997 wait complete. 2024-09-10T11:33:34.233 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:33:34.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.377+0000 7ff09ba1e640 1 Processor -- start 2024-09-10T11:33:34.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.377+0000 7ff09ba1e640 1 -- start start 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff09ba1e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff0941485b0 0x7ff0940b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff09ba1e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff094162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff09ba1e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09406d8c0 con 0x7ff094162910 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff09ba1e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff09406dbc0 con 0x7ff0941485b0 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff09ba1e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff094163290 con 0x7ff09406f060 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff098fb9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff094162910 0x7ff0940cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37358/0 (socket says 172.21.15.99:37358) 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.378+0000 7ff098fb9640 1 -- 172.21.15.99:0/1609574075 learned_addr learned my addr 172.21.15.99:0/1609574075 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187167894 0 0) 0x7ff09406d8c0 con 0x7ff094162910 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff099fbb640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff094162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff0997ba640 1 --2- 172.21.15.99:0/1609574075 >> v2:172.21.15.196:3300/0 conn(0x7ff0941485b0 0x7ff0940b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff080003520 con 0x7ff094162910 2024-09-10T11:33:34.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2075890619 0 0) 0x7ff080003520 con 0x7ff094162910 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 msgr2=0x7ff094162d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff094162d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 >> v2:172.21.15.196:3300/0 conn(0x7ff0941485b0 msgr2=0x7ff0940b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 --2- 172.21.15.99:0/1609574075 >> v2:172.21.15.196:3300/0 conn(0x7ff0941485b0 0x7ff0940b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0940b3870 con 0x7ff094162910 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7ff084001670 con 0x7ff094162910 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 >> v1:172.21.15.99:6789/0 conn(0x7ff094162910 legacy=0x7ff0940cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff0800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff080003f30 0x7ff080004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff09406dbc0 con 0x7ff080003f30 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff094163290 con 0x7ff080004d40 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff08000ba10 con 0x7ff09406f060 2024-09-10T11:33:34.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff080003f30 0x7ff080004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff0997ba640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff0800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff099fbb640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 -- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 msgr2=0x7ff0800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff09406f060 0x7ff0800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 -- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff080004d40 msgr2=0x7ff0800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff080004d40 0x7ff0800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.379+0000 7ff098fb9640 1 -- 172.21.15.99:0/1609574075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff080003520 con 0x7ff080003f30 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.380+0000 7ff098fb9640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff080003f30 0x7ff080004800 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff084001050 tx=0x7ff080005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2029d26ab3915f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.380+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08402e070 con 0x7ff080003f30 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.380+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084005af0 con 0x7ff080003f30 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.380+0000 7ff08bfff640 1 -- 172.21.15.99:0/1609574075 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084007d70 con 0x7ff080003f30 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 -- 172.21.15.99:0/1609574075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff080003f30 msgr2=0x7ff080004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/1609574075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff080003f30 0x7ff080004800 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff084001050 tx=0x7ff080005830 comp rx=0 tx=0).stop 2024-09-10T11:33:34.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff099fbb640 1 -- 172.21.15.99:0/1609574075 reap_dead start 2024-09-10T11:33:34.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 -- 172.21.15.99:0/1609574075 shutdown_connections 2024-09-10T11:33:34.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 -- 172.21.15.99:0/1609574075 >> 172.21.15.99:0/1609574075 conn(0x7ff0940b2a80 msgr2=0x7ff0940b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:34.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 -- 172.21.15.99:0/1609574075 shutdown_connections 2024-09-10T11:33:34.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.381+0000 7ff09ba1e640 1 -- 172.21.15.99:0/1609574075 wait complete. 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.382+0000 7ff09ba1e640 1 Processor -- start 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.382+0000 7ff09ba1e640 1 -- start start 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff080003f30 0x7ff0940cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941456a0 0x7ff094145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0940d00e0 con 0x7ff09415f410 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff094148990 con 0x7ff0941456a0 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff0940d03c0 con 0x7ff080003f30 2024-09-10T11:33:34.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36356/0 (socket says 172.21.15.99:36356) 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 -- 172.21.15.99:0/2155033435 learned_addr learned my addr 172.21.15.99:0/2155033435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff0997ba640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff080003f30 0x7ff0940cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff099fbb640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941456a0 0x7ff094145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 -- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff080003f30 msgr2=0x7ff0940cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff080003f30 0x7ff0940cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 -- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941456a0 msgr2=0x7ff094145b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941456a0 0x7ff094145b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff094146060 con 0x7ff09415f410 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff098fb9640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084042070 con 0x7ff09415f410 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084007480 con 0x7ff09415f410 2024-09-10T11:33:34.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0941619c0 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.383+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff084007d50 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.384+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff094160240 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.385+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0940d06a0 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.385+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff084045070 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.385+0000 7ff089ffb640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 0x7ff07806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.386+0000 7ff0997ba640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 0x7ff07806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.386+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5853+0+0 (secure 0 0 0) 0x7ff0840ab610 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.386+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=75}) -- 0x7ff078073b30 con 0x7ff09415f410 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.386+0000 7ff0997ba640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 0x7ff07806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff090006fd0 tx=0x7ff090008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:34.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.390+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff094146060 con 0x7ff09415f410 2024-09-10T11:33:34.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:34.559+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0940d0950 con 0x7ff09415f410 2024-09-10T11:33:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.219+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7ff084005ac0 con 0x7ff09415f410 2024-09-10T11:33:35.221 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:33:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.219+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(75..75 src has 1..75) ==== 304+0+0 (secure 0 0 0) 0x7ff0840dfd50 con 0x7ff09415f410 2024-09-10T11:33:35.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.219+0000 7ff089ffb640 1 -- 172.21.15.99:0/2155033435 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=76}) -- 0x7ff078071f60 con 0x7ff09415f410 2024-09-10T11:33:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.225+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 msgr2=0x7ff07806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.225+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 0x7ff07806e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff090006fd0 tx=0x7ff090008040 comp rx=0 tx=0).stop 2024-09-10T11:33:35.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.225+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 msgr2=0x7ff09415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.225+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084000fe0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 shutdown_connections 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff07806be30 0x7ff07806e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0941456a0 0x7ff094145b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff09415f410 0x7ff09415f890 secure :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c00 tx=0x7ff084000fe0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 --2- 172.21.15.99:0/2155033435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff080003f30 0x7ff0940cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 >> 172.21.15.99:0/2155033435 conn(0x7ff0940b2a80 msgr2=0x7ff0941468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.226+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 shutdown_connections 2024-09-10T11:33:35.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.227+0000 7ff09ba1e640 1 -- 172.21.15.99:0/2155033435 wait complete. 2024-09-10T11:33:35.242 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:33:35.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.348+0000 7fcbda4e4640 1 Processor -- start 2024-09-10T11:33:35.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.348+0000 7fcbda4e4640 1 -- start start 2024-09-10T11:33:35.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbda4e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcbd415ee10 0x7fcbd40af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbda4e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd41635c0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbda4e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd40b7b50 con 0x7fcbd4162ef0 2024-09-10T11:33:35.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbda4e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd406d550 con 0x7fcbd415ee10 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbd3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcbd415ee10 0x7fcbd40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbcbfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcbd40b6fe0 0x7fcbd41627e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37364/0 (socket says 172.21.15.99:37364) 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbcbfff640 1 -- 172.21.15.99:0/827801772 learned_addr learned my addr 172.21.15.99:0/827801772 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbd3fff640 1 -- 172.21.15.99:0/827801772 >> v1:172.21.15.196:6789/0 conn(0x7fcbd4162ef0 legacy=0x7fcbd406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbd3fff640 1 -- 172.21.15.99:0/827801772 >> v1:172.21.15.99:6789/0 conn(0x7fcbd40b6fe0 legacy=0x7fcbd41627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.349+0000 7fcbd3fff640 1 -- 172.21.15.99:0/827801772 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd406cf00 con 0x7fcbd415ee10 2024-09-10T11:33:35.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbd3fff640 1 --2- 172.21.15.99:0/827801772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd415ee10 0x7fcbd40af930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9a8fe0e64f831f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbd37fe640 1 -- 172.21.15.99:0/827801772 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4037070 con 0x7fcbd415ee10 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbd37fe640 1 -- 172.21.15.99:0/827801772 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc4032440 con 0x7fcbd415ee10 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbd37fe640 1 -- 172.21.15.99:0/827801772 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4032810 con 0x7fcbd415ee10 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbda4e4640 1 -- 172.21.15.99:0/827801772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd415ee10 msgr2=0x7fcbd40af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.350+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/827801772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd415ee10 0x7fcbd40af930 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402a1e0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.351+0000 7fcbda4e4640 1 -- 172.21.15.99:0/827801772 shutdown_connections 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.351+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/827801772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd415ee10 0x7fcbd40af930 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.351+0000 7fcbda4e4640 1 -- 172.21.15.99:0/827801772 >> 172.21.15.99:0/827801772 conn(0x7fcbd4073c60 msgr2=0x7fcbd409ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.351+0000 7fcbda4e4640 1 -- 172.21.15.99:0/827801772 shutdown_connections 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.351+0000 7fcbda4e4640 1 -- 172.21.15.99:0/827801772 wait complete. 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.352+0000 7fcbda4e4640 1 Processor -- start 2024-09-10T11:33:35.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.352+0000 7fcbda4e4640 1 -- start start 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.352+0000 7fcbda4e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 0x7fcbd40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 0x7fcbd406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40b75f0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd40b7470 con 0x7fcbd415ee10 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd40b7770 con 0x7fcbd4162ef0 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36378/0 (socket says 172.21.15.99:36378) 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 -- 172.21.15.99:0/1815120906 learned_addr learned my addr 172.21.15.99:0/1815120906 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbcbfff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 0x7fcbd40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 -- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 msgr2=0x7fcbd406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd8a81640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 0x7fcbd406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 0x7fcbd406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 -- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 msgr2=0x7fcbd40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 0x7fcbd40b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4152530 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbcbfff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 0x7fcbd40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd8a81640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 0x7fcbd406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd3fff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40098c0 tx=0x7fcbc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4041070 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc403c930 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4037040 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd4152810 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.353+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd4152dd0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.354+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd40af930 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.355+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbc4044070 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.355+0000 7fcbd17fa640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 0x7fcbb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.355+0000 7fcbcbfff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 0x7fcbb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.355+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5415+0+0 (secure 0 0 0) 0x7fcbc40abaf0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.356+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=76}) -- 0x7fcbb4072cc0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.356+0000 7fcbcbfff640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 0x7fcbb406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4159f60 tx=0x7fcbbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.360+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcbc407a6e0 con 0x7fcbd40b6fe0 2024-09-10T11:33:35.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:35.525+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcbd40d3790 con 0x7fcbd40b6fe0 2024-09-10T11:33:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.228+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7fcbc4079fe0 con 0x7fcbd40b6fe0 2024-09-10T11:33:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:33:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.228+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(76..76 src has 1..76) ==== 708+0+0 (secure 0 0 0) 0x7fcbc40aa250 con 0x7fcbd40b6fe0 2024-09-10T11:33:36.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.228+0000 7fcbd17fa640 1 -- 172.21.15.99:0/1815120906 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=77}) -- 0x7fcbb4071a50 con 0x7fcbd40b6fe0 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.236+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 msgr2=0x7fcbb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.236+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 0x7fcbb406e2f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4159f60 tx=0x7fcbbc009040 comp rx=0 tx=0).stop 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.236+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 msgr2=0x7fcbd414f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.236+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40098c0 tx=0x7fcbc40046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 shutdown_connections 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb406be30 0x7fcbb406e2f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbd4162ef0 0x7fcbd406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbd415ee10 0x7fcbd40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 --2- 172.21.15.99:0/1815120906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbd40b6fe0 0x7fcbd414f7b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 >> 172.21.15.99:0/1815120906 conn(0x7fcbd4073c60 msgr2=0x7fcbd4160390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 shutdown_connections 2024-09-10T11:33:36.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.237+0000 7fcbda4e4640 1 -- 172.21.15.99:0/1815120906 wait complete. 2024-09-10T11:33:36.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:33:36.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.353+0000 7f2402559640 1 Processor -- start 2024-09-10T11:33:36.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.353+0000 7f2402559640 1 -- start start 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f2402559640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f23fc163620 0x7f23fc160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f2402559640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23fc0b5140 con 0x7f23fc06bc10 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f2402559640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23fc162b40 con 0x7f23fc0b4930 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f2402559640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f23fc0b5420 con 0x7f23fc163620 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f23fbfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f23fc06bc10 0x7f23fc06bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37370/0 (socket says 172.21.15.99:37370) 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.354+0000 7f23fbfff640 1 -- 172.21.15.99:0/415872902 learned_addr learned my addr 172.21.15.99:0/415872902 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941906647 0 0) 0x7f23fc0b5140 con 0x7f23fc06bc10 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23e8003520 con 0x7f23fc06bc10 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/415872902 >> v2:172.21.15.196:3300/0 conn(0x7f23fc163620 0x7f23fc160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 341661146 0 0) 0x7f23e8003520 con 0x7f23fc06bc10 2024-09-10T11:33:36.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 >> v2:172.21.15.196:3300/0 conn(0x7f23fc163620 msgr2=0x7f23fc160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 --2- 172.21.15.99:0/415872902 >> v2:172.21.15.196:3300/0 conn(0x7f23fc163620 0x7f23fc160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 >> v1:172.21.15.196:6789/0 conn(0x7f23fc0b4930 legacy=0x7f23fc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc162e20 con 0x7f23fc06bc10 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f23ec001670 con 0x7f23fc06bc10 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 >> v1:172.21.15.99:6789/0 conn(0x7f23fc06bc10 legacy=0x7f23fc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc0b5420 con 0x7f23e80052e0 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23e800c1b0 con 0x7f23e8004800 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f23e800c3b0 con 0x7f23e80039b0 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23fbfff640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 -- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23e80039b0 msgr2=0x7f23e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23e80039b0 0x7f23e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 -- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23e8004800 msgr2=0x7f23e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f2400af6640 1 -- 172.21.15.99:0/415872902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e8003520 con 0x7f23e80052e0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.355+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23e8004800 0x7f23e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.356+0000 7f2400af6640 1 -- 172.21.15.99:0/415872902 reap_dead start 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.356+0000 7f2400af6640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23e800b9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001e20 tx=0x7f23e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87d058badfdbd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.356+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4001a70 con 0x7f23e80052e0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.356+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f402ae40 con 0x7f23e80052e0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.356+0000 7f23faffd640 1 -- 172.21.15.99:0/415872902 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4003bb0 con 0x7f23e80052e0 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 -- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 msgr2=0x7f23e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23e800b9c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001e20 tx=0x7f23e8005830 comp rx=0 tx=0).stop 2024-09-10T11:33:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 -- 172.21.15.99:0/415872902 shutdown_connections 2024-09-10T11:33:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 --2- 172.21.15.99:0/415872902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23e800b9c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 -- 172.21.15.99:0/415872902 >> 172.21.15.99:0/415872902 conn(0x7f23fc0b3630 msgr2=0x7f23fc0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 -- 172.21.15.99:0/415872902 shutdown_connections 2024-09-10T11:33:36.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.357+0000 7f2402559640 1 -- 172.21.15.99:0/415872902 wait complete. 2024-09-10T11:33:36.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.358+0000 7f2402559640 1 Processor -- start 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.358+0000 7f2402559640 1 -- start start 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23fc1f4b70 0x7f23fc143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23fc15f950 0x7f23fc15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc0d1560 con 0x7f23e80052e0 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc0b4dc0 con 0x7f23fc15f950 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f23fc0d16e0 con 0x7f23fc1f4b70 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36398/0 (socket says 172.21.15.99:36398) 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 -- 172.21.15.99:0/708383709 learned_addr learned my addr 172.21.15.99:0/708383709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23fc1f4b70 0x7f23fc143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2400af6640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23fc15f950 0x7f23fc15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 -- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23fc1f4b70 msgr2=0x7f23fc143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23fc1f4b70 0x7f23fc143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 -- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23fc15f950 msgr2=0x7f23fc15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23fc15f950 0x7f23fc15fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 -- 172.21.15.99:0/708383709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc140c80 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23fbfff640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00e780 tx=0x7f23ec00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00c660 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23ec018070 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00ca40 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.359+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc140f60 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.360+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc141520 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.360+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc0b5140 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.361+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23ec00cbe0 con 0x7f23e80052e0 2024-09-10T11:33:36.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.361+0000 7f23f8ff9640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 0x7f23dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.361+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7f23ec08a490 con 0x7f23e80052e0 2024-09-10T11:33:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.362+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 0x7f23dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.362+0000 7f23fb7fe640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 0x7f23dc06e320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0b1720 tx=0x7f23e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.366+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23ec058af0 con 0x7f23e80052e0 2024-09-10T11:33:36.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.530+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f23fc1e8f00 con 0x7f23e80052e0 2024-09-10T11:33:36.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.534+0000 7f23f8ff9640 1 -- 172.21.15.99:0/708383709 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) ==== 104+0+0 (secure 0 0 0) 0x7f23ec05cdf0 con 0x7f23e80052e0 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 msgr2=0x7f23dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 0x7f23dc06e320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0b1720 tx=0x7f23e4008040 comp rx=0 tx=0).stop 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 msgr2=0x7f23fc143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00e780 tx=0x7f23ec00ec40 comp rx=0 tx=0).stop 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 shutdown_connections 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f23dc06be60 0x7f23dc06e320 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23fc15f950 0x7f23fc15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23fc1f4b70 0x7f23fc143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 --2- 172.21.15.99:0/708383709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e80052e0 0x7f23fc143780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 >> 172.21.15.99:0/708383709 conn(0x7f23fc0b3630 msgr2=0x7f23fc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 shutdown_connections 2024-09-10T11:33:36.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.536+0000 7f2402559640 1 -- 172.21.15.99:0/708383709 wait complete. 2024-09-10T11:33:36.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.646+0000 7f163d98d640 1 Processor -- start 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.646+0000 7f163d98d640 1 -- start start 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f163d98d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16380d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f163d98d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f16380d1ae0 0x7f16380b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f163d98d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16380d2990 con 0x7f163815efc0 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f163d98d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f16380d2c70 con 0x7f16380d1ae0 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f163d98d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f16380d2f50 con 0x7f16380d3a30 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f1636ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f163815efc0 0x7f16380d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37374/0 (socket says 172.21.15.99:37374) 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f1636ffd640 1 -- 172.21.15.99:0/2967626937 learned_addr learned my addr 172.21.15.99:0/2967626937 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f16377fe640 1 --2- 172.21.15.99:0/2967626937 >> v2:172.21.15.196:3300/0 conn(0x7f16380d1ae0 0x7f16380b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713150864 0 0) 0x7f16380d2990 con 0x7f163815efc0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f16367fc640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16380d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.647+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1620003520 con 0x7f163815efc0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4198669002 0 0) 0x7f1620003520 con 0x7f163815efc0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 msgr2=0x7f16380d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16380d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 >> v2:172.21.15.196:3300/0 conn(0x7f16380d1ae0 msgr2=0x7f16380b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 --2- 172.21.15.99:0/2967626937 >> v2:172.21.15.196:3300/0 conn(0x7f16380d1ae0 0x7f16380b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16380d3230 con 0x7f163815efc0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1628001670 con 0x7f163815efc0 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 >> v1:172.21.15.99:6789/0 conn(0x7f163815efc0 legacy=0x7f16380d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16200039b0 0x7f1620003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1620004800 0x7f1620004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16380d2c70 con 0x7f16200039b0 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16380d2f50 con 0x7f1620004800 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f162000b9c0 con 0x7f16380d3a30 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16200039b0 0x7f1620003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f16367fc640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1620004800 0x7f1620004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f16377fe640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 -- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 msgr2=0x7f16200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 -- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1620004800 msgr2=0x7f1620004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1620004800 0x7f1620004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f1636ffd640 1 -- 172.21.15.99:0/2967626937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1620003520 con 0x7f16200039b0 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f16367fc640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1620004800 0x7f1620004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.648+0000 7f16377fe640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16380d3a30 0x7f16200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.649+0000 7f1636ffd640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16200039b0 0x7f1620003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f162800c190 tx=0x7f1620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14db61e54a8313e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.649+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628007c60 con 0x7f16200039b0 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.649+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628005af0 con 0x7f16200039b0 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.649+0000 7f1635ffb640 1 -- 172.21.15.99:0/2967626937 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628007070 con 0x7f16200039b0 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 -- 172.21.15.99:0/2967626937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16200039b0 msgr2=0x7f1620003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 --2- 172.21.15.99:0/2967626937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16200039b0 0x7f1620003dd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f162800c190 tx=0x7f1620005830 comp rx=0 tx=0).stop 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f16377fe640 1 -- 172.21.15.99:0/2967626937 reap_dead start 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 -- 172.21.15.99:0/2967626937 shutdown_connections 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 -- 172.21.15.99:0/2967626937 >> 172.21.15.99:0/2967626937 conn(0x7f16380d4c20 msgr2=0x7f16380d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 -- 172.21.15.99:0/2967626937 shutdown_connections 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.650+0000 7f163d98d640 1 -- 172.21.15.99:0/2967626937 wait complete. 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.651+0000 7f163d98d640 1 Processor -- start 2024-09-10T11:33:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.651+0000 7f163d98d640 1 -- start start 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 0x7f16381566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16380c8fd0 0x7f16380ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16380d1720 con 0x7f1638156d40 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16380d15a0 con 0x7f16380c8fd0 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f16380d18a0 con 0x7f16200039b0 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36410/0 (socket says 172.21.15.99:36410) 2024-09-10T11:33:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 -- 172.21.15.99:0/1234558348 learned_addr learned my addr 172.21.15.99:0/1234558348 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16367fc640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16380c8fd0 0x7f16380ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f1636ffd640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 0x7f16381566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 -- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 msgr2=0x7f16381566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 0x7f16381566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 -- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16380c8fd0 msgr2=0x7f16380ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16380c8fd0 0x7f16380ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 -- 172.21.15.99:0/1234558348 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16380c94d0 con 0x7f1638156d40 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f1636ffd640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 0x7f16381566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f16377fe640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f16380d4a40 tx=0x7f1630009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163001a030 con 0x7f1638156d40 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1630010e80 con 0x7f1638156d40 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16380c97b0 con 0x7f1638156d40 2024-09-10T11:33:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630003bd0 con 0x7f1638156d40 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.652+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16380c9d70 con 0x7f1638156d40 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.653+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1638157700 con 0x7f1638156d40 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.654+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f163001d070 con 0x7f1638156d40 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.654+0000 7f161ffff640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.654+0000 7f1636ffd640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.654+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7f163008b5a0 con 0x7f1638156d40 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.655+0000 7f1636ffd640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 0x7f162406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f162800c160 tx=0x7f162802d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.659+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f163005a2e0 con 0x7f1638156d40 2024-09-10T11:33:36.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:36.889+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7f16380d03f0 con 0x7f1638156d40 2024-09-10T11:33:37.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.187+0000 7f161ffff640 1 -- 172.21.15.99:0/1234558348 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v173) ==== 134+0+0 (secure 0 0 0) 0x7f1630059be0 con 0x7f1638156d40 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 msgr2=0x7f162406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 0x7f162406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f162800c160 tx=0x7f162802d000 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 msgr2=0x7f16381571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f16380d4a40 tx=0x7f1630009f90 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 shutdown_connections 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f162406be30 0x7f162406e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1638156d40 0x7f16381571c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16380c8fd0 0x7f16380ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 --2- 172.21.15.99:0/1234558348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16200039b0 0x7f16381566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 >> 172.21.15.99:0/1234558348 conn(0x7f16380d4c20 msgr2=0x7f16380b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 shutdown_connections 2024-09-10T11:33:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.189+0000 7f163d98d640 1 -- 172.21.15.99:0/1234558348 wait complete. 2024-09-10T11:33:37.202 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-09-10T11:33:37.202 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-09-10T11:33:37.202 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-09-10T11:33:37.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:37.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.299+0000 7fb53838d640 1 Processor -- start 2024-09-10T11:33:37.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.299+0000 7fb53838d640 1 -- start start 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb53015c060 0x7fb53015c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb53006ef50 con 0x7fb5300d1200 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb530152e60 con 0x7fb53015c060 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53838d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5300d15e0 con 0x7fb53006e5d0 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36424/0 (socket says 172.21.15.99:36424) 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 -- 172.21.15.99:0/1208140749 learned_addr learned my addr 172.21.15.99:0/1208140749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 -- 172.21.15.99:0/1208140749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb53006e5d0 msgr2=0x7fb53006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:37.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb535928640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb53692a640 1 --2- 172.21.15.99:0/1208140749 >> v2:172.21.15.196:3300/0 conn(0x7fb53015c060 0x7fb53015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 -- 172.21.15.99:0/1208140749 >> v2:172.21.15.196:3300/0 conn(0x7fb53015c060 msgr2=0x7fb53015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 --2- 172.21.15.99:0/1208140749 >> v2:172.21.15.196:3300/0 conn(0x7fb53015c060 0x7fb53015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.300+0000 7fb536129640 1 -- 172.21.15.99:0/1208140749 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530072680 con 0x7fb5300d1200 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.301+0000 7fb536129640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5300d1200 0x7fb530073440 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb530071b80 tx=0x7fb52c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fd38d3514865d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.301+0000 7fb535127640 1 -- 172.21.15.99:0/1208140749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037070 con 0x7fb5300d1200 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.301+0000 7fb535127640 1 -- 172.21.15.99:0/1208140749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c032440 con 0x7fb5300d1200 2024-09-10T11:33:37.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.301+0000 7fb535127640 1 -- 172.21.15.99:0/1208140749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c032810 con 0x7fb5300d1200 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 -- 172.21.15.99:0/1208140749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5300d1200 msgr2=0x7fb530073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5300d1200 0x7fb530073440 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb530071b80 tx=0x7fb52c02a140 comp rx=0 tx=0).stop 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 -- 172.21.15.99:0/1208140749 shutdown_connections 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 --2- 172.21.15.99:0/1208140749 >> v2:172.21.15.196:3300/0 conn(0x7fb53015c060 0x7fb53015c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb53006e5d0 0x7fb53006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 --2- 172.21.15.99:0/1208140749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5300d1200 0x7fb530073440 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 -- 172.21.15.99:0/1208140749 >> 172.21.15.99:0/1208140749 conn(0x7fb53006bc10 msgr2=0x7fb53006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 -- 172.21.15.99:0/1208140749 shutdown_connections 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.302+0000 7fb53838d640 1 -- 172.21.15.99:0/1208140749 wait complete. 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.303+0000 7fb53838d640 1 Processor -- start 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.303+0000 7fb53838d640 1 -- start start 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53015c060 0x7fb53006da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb530072f20 con 0x7fb53006e5d0 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb530072c40 con 0x7fb53015c060 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb53015cd90 con 0x7fb5300d1200 2024-09-10T11:33:37.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36428/0 (socket says 172.21.15.99:36428) 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 -- 172.21.15.99:0/2993341226 learned_addr learned my addr 172.21.15.99:0/2993341226 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb535928640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53692a640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53015c060 0x7fb53006da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 -- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 msgr2=0x7fb53006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 -- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53015c060 msgr2=0x7fb53006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53015c060 0x7fb53006da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 -- 172.21.15.99:0/2993341226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb530152750 con 0x7fb53006e5d0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb536129640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb52c02a180 tx=0x7fb52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c037040 con 0x7fb53006e5d0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c042070 con 0x7fb53006e5d0 2024-09-10T11:33:37.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c030bb0 con 0x7fb53006e5d0 2024-09-10T11:33:37.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.304+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5300adf90 con 0x7fb53006e5d0 2024-09-10T11:33:37.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.305+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5300ae550 con 0x7fb53006e5d0 2024-09-10T11:33:37.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.305+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53006ea10 con 0x7fb53006e5d0 2024-09-10T11:33:37.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.306+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb52c047460 con 0x7fb53006e5d0 2024-09-10T11:33:37.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.306+0000 7fb526ffd640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 0x7fb51406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.306+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7fb52c0ab620 con 0x7fb53006e5d0 2024-09-10T11:33:37.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.306+0000 7fb535928640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 0x7fb51406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.307+0000 7fb535928640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 0x7fb51406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb53015b820 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.311+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb52c079c80 con 0x7fb53006e5d0 2024-09-10T11:33:37.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.475+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb53015ff60 con 0x7fb53006e5d0 2024-09-10T11:33:37.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.476+0000 7fb526ffd640 1 -- 172.21.15.99:0/2993341226 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13032 (secure 0 0 0) 0x7fb52c07df80 con 0x7fb53006e5d0 2024-09-10T11:33:37.478 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:37.478 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":76,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:36.228416+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6837/2576628819":"2024-09-11T11:33:33.211793+0000","172.21.15.99:6836/2576628819":"2024-09-11T11:33:33.211793+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 msgr2=0x7fb51406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 0x7fb51406e2f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb53015b820 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 msgr2=0x7fb530152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb52c02a180 tx=0x7fb52c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 shutdown_connections 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb51406be30 0x7fb51406e2f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb53015c060 0x7fb53006da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5300d1200 0x7fb53006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 --2- 172.21.15.99:0/2993341226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb53006e5d0 0x7fb530152210 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 >> 172.21.15.99:0/2993341226 conn(0x7fb53006bc10 msgr2=0x7fb5300b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 shutdown_connections 2024-09-10T11:33:37.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.478+0000 7fb53838d640 1 -- 172.21.15.99:0/2993341226 wait complete. 2024-09-10T11:33:37.491 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:33:37.491 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:33:37.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.591+0000 7ff40028d640 1 Processor -- start 2024-09-10T11:33:37.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.591+0000 7ff40028d640 1 -- start start 2024-09-10T11:33:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff40028d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 0x7ff3f8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff40028d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f806e5d0 0x7ff3f806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff40028d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3f8152e60 con 0x7ff3f815c060 2024-09-10T11:33:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff40028d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f806ef50 con 0x7ff3f806e5d0 2024-09-10T11:33:37.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff40028d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f80d15e0 con 0x7ff3f80d1200 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f806e5d0 0x7ff3f806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff3f806e5d0 0x7ff3f806ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36434/0 (socket says 172.21.15.99:36434) 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 -- 172.21.15.99:0/1932680899 learned_addr learned my addr 172.21.15.99:0/1932680899 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fe029640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 0x7ff3f8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 -- 172.21.15.99:0/1932680899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 msgr2=0x7ff3f8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 0x7ff3f8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 -- 172.21.15.99:0/1932680899 >> v1:172.21.15.196:6789/0 conn(0x7ff3f815c060 legacy=0x7ff3f815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fd828640 1 -- 172.21.15.99:0/1932680899 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f8072680 con 0x7ff3f806e5d0 2024-09-10T11:33:37.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.592+0000 7ff3fe029640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 0x7ff3f8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:37.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.593+0000 7ff3fd828640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f806e5d0 0x7ff3f806ea10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009870 tx=0x7ff3e802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0057dbc338b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.593+0000 7ff3fd027640 1 -- 172.21.15.99:0/1932680899 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8037070 con 0x7ff3f806e5d0 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.593+0000 7ff3fd027640 1 -- 172.21.15.99:0/1932680899 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8032440 con 0x7ff3f806e5d0 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.593+0000 7ff3fd027640 1 -- 172.21.15.99:0/1932680899 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8032810 con 0x7ff3f806e5d0 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 -- 172.21.15.99:0/1932680899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f806e5d0 msgr2=0x7ff3f806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f806e5d0 0x7ff3f806ea10 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009870 tx=0x7ff3e802a180 comp rx=0 tx=0).stop 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 -- 172.21.15.99:0/1932680899 shutdown_connections 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f806e5d0 0x7ff3f806ea10 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 --2- 172.21.15.99:0/1932680899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f80d1200 0x7ff3f8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 -- 172.21.15.99:0/1932680899 >> 172.21.15.99:0/1932680899 conn(0x7ff3f806bc10 msgr2=0x7ff3f806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 -- 172.21.15.99:0/1932680899 shutdown_connections 2024-09-10T11:33:37.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.594+0000 7ff40028d640 1 -- 172.21.15.99:0/1932680899 wait complete. 2024-09-10T11:33:37.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.595+0000 7ff40028d640 1 Processor -- start 2024-09-10T11:33:37.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.595+0000 7ff40028d640 1 -- start start 2024-09-10T11:33:37.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f806e5d0 0x7ff3f815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f80d1200 0x7ff3f80b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8072f20 con 0x7ff3f815c060 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f8072c40 con 0x7ff3f80d1200 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff40028d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f815cd90 con 0x7ff3f806e5d0 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36440/0 (socket says 172.21.15.99:36440) 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 -- 172.21.15.99:0/2361897470 learned_addr learned my addr 172.21.15.99:0/2361897470 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe029640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f806e5d0 0x7ff3f815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fd828640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f80d1200 0x7ff3f80b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 -- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f806e5d0 msgr2=0x7ff3f815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f806e5d0 0x7ff3f815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 -- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f80d1200 msgr2=0x7ff3f80b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f80d1200 0x7ff3f80b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 -- 172.21.15.99:0/2361897470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f8155f80 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3fe82a640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c780 tx=0x7ff3f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.596+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0004b60 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.597+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3f0004d00 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.597+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f81f24f0 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.597+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f000aa90 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.597+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f81f2ab0 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.598+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f806ea10 con 0x7ff3f815c060 2024-09-10T11:33:37.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.598+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3f000ac30 con 0x7ff3f815c060 2024-09-10T11:33:37.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.598+0000 7ff3eeffd640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 0x7ff3dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.598+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7ff3f008ac70 con 0x7ff3f815c060 2024-09-10T11:33:37.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.598+0000 7ff3fe029640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 0x7ff3dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.599+0000 7ff3fe029640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 0x7ff3dc06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400aac0 tx=0x7ff3f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.603+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3f0020070 con 0x7ff3f815c060 2024-09-10T11:33:37.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.770+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3f815b700 con 0x7ff3f815c060 2024-09-10T11:33:37.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.770+0000 7ff3eeffd640 1 -- 172.21.15.99:0/2361897470 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7ff3f00592d0 con 0x7ff3f815c060 2024-09-10T11:33:37.772 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:33:37.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 msgr2=0x7ff3dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 0x7ff3dc06e2f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400aac0 tx=0x7ff3f4008040 comp rx=0 tx=0).stop 2024-09-10T11:33:37.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 msgr2=0x7ff3f80b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c780 tx=0x7ff3f000cc40 comp rx=0 tx=0).stop 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 shutdown_connections 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3dc06be30 0x7ff3dc06e2f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3f815c060 0x7ff3f80b1d60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3f80d1200 0x7ff3f80b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 --2- 172.21.15.99:0/2361897470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3f806e5d0 0x7ff3f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 >> 172.21.15.99:0/2361897470 conn(0x7ff3f806bc10 msgr2=0x7ff3f815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 shutdown_connections 2024-09-10T11:33:37.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.772+0000 7ff40028d640 1 -- 172.21.15.99:0/2361897470 wait complete. 2024-09-10T11:33:37.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-09-10T11:33:37.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.883+0000 7f8cb071a640 1 Processor -- start 2024-09-10T11:33:37.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.883+0000 7f8cb071a640 1 -- start start 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cb071a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cb071a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80b40e0 con 0x7f8ca815f1e0 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cb071a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80b5ab0 con 0x7f8ca80b0090 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cb071a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca8073c50 con 0x7f8ca815ee00 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cadcb5640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ca815f1e0 0x7f8ca80b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37382/0 (socket says 172.21.15.99:37382) 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cadcb5640 1 -- 172.21.15.99:0/1021921084 learned_addr learned my addr 172.21.15.99:0/1021921084 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.884+0000 7f8cae4b6640 1 --2- 172.21.15.99:0/1021921084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422078835 0 0) 0x7f8ca80b40e0 con 0x7f8ca815f1e0 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c94003520 con 0x7f8ca815f1e0 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2116454752 0 0) 0x7f8ca80b5ab0 con 0x7f8ca80b0090 2024-09-10T11:33:37.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ca80b40e0 con 0x7f8ca80b0090 2024-09-10T11:33:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cae4b6640 1 -- 172.21.15.99:0/1021921084 >> v1:172.21.15.196:6789/0 conn(0x7f8ca80b0090 legacy=0x7f8ca80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cae4b6640 1 -- 172.21.15.99:0/1021921084 >> v1:172.21.15.99:6789/0 conn(0x7f8ca815f1e0 legacy=0x7f8ca80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.885+0000 7f8cae4b6640 1 -- 172.21.15.99:0/1021921084 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca80b4380 con 0x7f8ca815ee00 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cae4b6640 1 --2- 172.21.15.99:0/1021921084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca80b5f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40097d0 tx=0x7f8ca402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14458ead7450a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4037070 con 0x7f8ca815ee00 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca4032440 con 0x7f8ca815ee00 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cad4b4640 1 -- 172.21.15.99:0/1021921084 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40327f0 con 0x7f8ca815ee00 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 -- 172.21.15.99:0/1021921084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 msgr2=0x7f8ca80b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 --2- 172.21.15.99:0/1021921084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca80b5f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40097d0 tx=0x7f8ca402a210 comp rx=0 tx=0).stop 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 -- 172.21.15.99:0/1021921084 shutdown_connections 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 --2- 172.21.15.99:0/1021921084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca80b5f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 -- 172.21.15.99:0/1021921084 >> 172.21.15.99:0/1021921084 conn(0x7f8ca80730c0 msgr2=0x7f8ca80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 -- 172.21.15.99:0/1021921084 shutdown_connections 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.886+0000 7f8cb071a640 1 -- 172.21.15.99:0/1021921084 wait complete. 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 Processor -- start 2024-09-10T11:33:37.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 -- start start 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca815fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 0x7f8ca81514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80b0600 con 0x7f8ca80b0090 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca80afd60 con 0x7f8ca815f1e0 2024-09-10T11:33:37.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cb071a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca80b0780 con 0x7f8ca815ee00 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cae4b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cae4b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36448/0 (socket says 172.21.15.99:36448) 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.888+0000 7f8cae4b6640 1 -- 172.21.15.99:0/382493249 learned_addr learned my addr 172.21.15.99:0/382493249 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cadcb5640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca815fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8caecb7640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 0x7f8ca81514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 -- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 msgr2=0x7f8ca815fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca815fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 -- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 msgr2=0x7f8ca81514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 0x7f8ca81514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 -- 172.21.15.99:0/382493249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca8160840 con 0x7f8ca80b0090 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8caecb7640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 0x7f8ca81514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cae4b6640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4035040 tx=0x7f8ca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4041020 con 0x7f8ca80b0090 2024-09-10T11:33:37.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca403d6e0 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca8160b20 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4037040 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.889+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca809b320 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.890+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca80b5f90 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.890+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ca4044020 con 0x7f8ca80b0090 2024-09-10T11:33:37.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.891+0000 7f8c9effd640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.891+0000 7f8cadcb5640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.891+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7f8ca40ab2f0 con 0x7f8ca80b0090 2024-09-10T11:33:37.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.891+0000 7f8cadcb5640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cd5c0 tx=0x7f8c98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:37.896+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ca404b030 con 0x7f8ca80b0090 2024-09-10T11:33:38.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.122+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7f8ca80af7a0 con 0x7f8ca80b0090 2024-09-10T11:33:38.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.188+0000 7f8c9effd640 1 -- 172.21.15.99:0/382493249 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v174) ==== 136+0+0 (secure 0 0 0) 0x7f8ca4079fb0 con 0x7f8ca80b0090 2024-09-10T11:33:38.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 msgr2=0x7f8c8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80cd5c0 tx=0x7f8c98009040 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 msgr2=0x7f8ca809ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4035040 tx=0x7f8ca40046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 shutdown_connections 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c8c06be30 0x7f8c8c06e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ca815f1e0 0x7f8ca81514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ca815ee00 0x7f8ca815fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.189+0000 7f8cb071a640 1 --2- 172.21.15.99:0/382493249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ca80b0090 0x7f8ca809ed10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.190+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 >> 172.21.15.99:0/382493249 conn(0x7f8ca80730c0 msgr2=0x7f8ca80cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.190+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 shutdown_connections 2024-09-10T11:33:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.190+0000 7f8cb071a640 1 -- 172.21.15.99:0/382493249 wait complete. 2024-09-10T11:33:38.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-09-10T11:33:38.312 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2024-09-10T11:33:38.312 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-09-10T11:33:38.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:33:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.411+0000 7f6f7261e640 1 Processor -- start 2024-09-10T11:33:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.411+0000 7f6f7261e640 1 -- start start 2024-09-10T11:33:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f7261e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f6c0d1200 0x7f6f6c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f7261e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f7261e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6c152e60 con 0x7f6f6c06e5d0 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f7261e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c06ef80 con 0x7f6f6c0d1200 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f7261e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c0d15e0 con 0x7f6f6c15c770 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f6bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6f6c0d1200 0x7f6f6c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f6b7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6f6c06e5d0 0x7f6f6c15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37398/0 (socket says 172.21.15.99:37398) 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.412+0000 7f6f6b7fe640 1 -- 172.21.15.99:0/2367113139 learned_addr learned my addr 172.21.15.99:0/2367113139 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f70bbb640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6bfff640 1 -- 172.21.15.99:0/2367113139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 msgr2=0x7f6f6c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6bfff640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6bfff640 1 -- 172.21.15.99:0/2367113139 >> v1:172.21.15.99:6789/0 conn(0x7f6f6c06e5d0 legacy=0x7f6f6c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6affd640 1 -- 172.21.15.99:0/2367113139 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 226222639 0 0) 0x7f6f6c152e60 con 0x7f6f6c06e5d0 2024-09-10T11:33:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6bfff640 1 -- 172.21.15.99:0/2367113139 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c072680 con 0x7f6f6c0d1200 2024-09-10T11:33:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.413+0000 7f6f6bfff640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c073440 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0097a0 tx=0x7f6f5c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7db91cb214dc74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f6affd640 1 -- 172.21.15.99:0/2367113139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c037070 con 0x7f6f6c0d1200 2024-09-10T11:33:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f6affd640 1 -- 172.21.15.99:0/2367113139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c02ae50 con 0x7f6f6c0d1200 2024-09-10T11:33:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f6affd640 1 -- 172.21.15.99:0/2367113139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c032660 con 0x7f6f6c0d1200 2024-09-10T11:33:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 -- 172.21.15.99:0/2367113139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 msgr2=0x7f6f6c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c073440 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0097a0 tx=0x7f6f5c02a100 comp rx=0 tx=0).stop 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 -- 172.21.15.99:0/2367113139 shutdown_connections 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 --2- 172.21.15.99:0/2367113139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c073440 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 -- 172.21.15.99:0/2367113139 >> 172.21.15.99:0/2367113139 conn(0x7f6f6c06bc10 msgr2=0x7f6f6c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 -- 172.21.15.99:0/2367113139 shutdown_connections 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.414+0000 7f6f7261e640 1 -- 172.21.15.99:0/2367113139 wait complete. 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.415+0000 7f6f7261e640 1 Processor -- start 2024-09-10T11:33:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 -- start start 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f6c06e5d0 0x7f6f6c0ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c072f20 con 0x7f6f6c0d1200 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c072c40 con 0x7f6f6c06e5d0 2024-09-10T11:33:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f7261e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c15cd90 con 0x7f6f6c15c770 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f6b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36468/0 (socket says 172.21.15.99:36468) 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.416+0000 7f6f6b7fe640 1 -- 172.21.15.99:0/3767830788 learned_addr learned my addr 172.21.15.99:0/3767830788 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 -- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 msgr2=0x7f6f6c155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 -- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f6c06e5d0 msgr2=0x7f6f6c0ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f6c06e5d0 0x7f6f6c0ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 -- 172.21.15.99:0/3767830788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c13eb20 con 0x7f6f6c0d1200 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f6b7fe640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c3f0 tx=0x7f6f6000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600074e0 con 0x7f6f6c0d1200 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f600047e0 con 0x7f6f6c0d1200 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c13bf20 con 0x7f6f6c0d1200 2024-09-10T11:33:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60004c10 con 0x7f6f6c0d1200 2024-09-10T11:33:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.417+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c0ad950 con 0x7f6f6c0d1200 2024-09-10T11:33:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.418+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c073440 con 0x7f6f6c0d1200 2024-09-10T11:33:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.419+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f60011070 con 0x7f6f6c0d1200 2024-09-10T11:33:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.419+0000 7f6f68ff9640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 0x7f6f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.419+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7f6f6008a710 con 0x7f6f6c0d1200 2024-09-10T11:33:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.419+0000 7f6f6bfff640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 0x7f6f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.419+0000 7f6f6bfff640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 0x7f6f5806e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.423+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f6001a2b0 con 0x7f6f6c0d1200 2024-09-10T11:33:38.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.588+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6f6c156130 con 0x7f6f6c0d1200 2024-09-10T11:33:38.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.594+0000 7f6f68ff9640 1 -- 172.21.15.99:0/3767830788 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f6f60059450 con 0x7f6f6c0d1200 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 msgr2=0x7f6f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 0x7f6f5806e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c034000 tx=0x7f6f5c02c000 comp rx=0 tx=0).stop 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 msgr2=0x7f6f6c154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000c3f0 tx=0x7f6f6000c8b0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 shutdown_connections 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6f5806be30 0x7f6f5806e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f6c15c770 0x7f6f6c155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f6c0d1200 0x7f6f6c154e00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 --2- 172.21.15.99:0/3767830788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f6c06e5d0 0x7f6f6c0ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 >> 172.21.15.99:0/3767830788 conn(0x7f6f6c06bc10 msgr2=0x7f6f6c0d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 shutdown_connections 2024-09-10T11:33:38.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.596+0000 7f6f7261e640 1 -- 172.21.15.99:0/3767830788 wait complete. 2024-09-10T11:33:38.609 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:33:38.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:33:38.650 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:33:38.695 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:33:38.696 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:33:38.696 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:38.711 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:33:38.712 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:33:38.712 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:38.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:33:38.730 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:33:38.730 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:38.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:33:38.767 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:33:38.767 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:33:38.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:33:38.787 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:33:38.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.904+0000 7fd664caf640 1 Processor -- start 2024-09-10T11:33:38.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.904+0000 7fd664caf640 1 -- start start 2024-09-10T11:33:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd664caf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd660073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd664caf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd66015c060 0x7fd66015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd664caf640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd660152e60 con 0x7fd6600d1200 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd664caf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd6600d15e0 con 0x7fd66015c060 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd664caf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd660072680 con 0x7fd66006ece0 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65e59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd6600d1200 0x7fd66006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37414/0 (socket says 172.21.15.99:37414) 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65e59c640 1 -- 172.21.15.99:0/3575288518 learned_addr learned my addr 172.21.15.99:0/3575288518 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 558935601 0 0) 0x7fd660152e60 con 0x7fd6600d1200 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd648003520 con 0x7fd6600d1200 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65dd9b640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd660073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/3575288518 >> v2:172.21.15.196:3300/0 conn(0x7fd66015c060 0x7fd66015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3283693820 0 0) 0x7fd648003520 con 0x7fd6600d1200 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 msgr2=0x7fd660073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd660073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 >> v2:172.21.15.196:3300/0 conn(0x7fd66015c060 msgr2=0x7fd66015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:38.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 --2- 172.21.15.99:0/3575288518 >> v2:172.21.15.196:3300/0 conn(0x7fd66015c060 0x7fd66015c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd660072960 con 0x7fd6600d1200 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fd650001670 con 0x7fd6600d1200 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 >> v1:172.21.15.99:6789/0 conn(0x7fd6600d1200 legacy=0x7fd66006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6480039b0 0x7fd648003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd648004800 0x7fd648004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd6480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6600d15e0 con 0x7fd6480039b0 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd660072680 con 0x7fd648004800 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.905+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd64800bb50 con 0x7fd66006ece0 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6480039b0 0x7fd648003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65dd9b640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd648004800 0x7fd648004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd6480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 -- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 msgr2=0x7fd6480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:38.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd6480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 -- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd648004800 msgr2=0x7fd648004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd648004800 0x7fd648004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 -- 172.21.15.99:0/3575288518 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd648003520 con 0x7fd6480039b0 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65dd9b640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd648004800 0x7fd648004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd66006ece0 0x7fd6480051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.906+0000 7fd65e59c640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6480039b0 0x7fd648003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd650001eb0 tx=0x7fd648005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a781527318956740 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650007d70 con 0x7fd6480039b0 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6500058d0 con 0x7fd6480039b0 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd65d59a640 1 -- 172.21.15.99:0/3575288518 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650005ca0 con 0x7fd6480039b0 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 -- 172.21.15.99:0/3575288518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6480039b0 msgr2=0x7fd648003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 --2- 172.21.15.99:0/3575288518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6480039b0 0x7fd648003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd650001eb0 tx=0x7fd648005830 comp rx=0 tx=0).stop 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd65ed9d640 1 -- 172.21.15.99:0/3575288518 reap_dead start 2024-09-10T11:33:38.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 -- 172.21.15.99:0/3575288518 shutdown_connections 2024-09-10T11:33:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 -- 172.21.15.99:0/3575288518 >> 172.21.15.99:0/3575288518 conn(0x7fd66006bc10 msgr2=0x7fd66006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 -- 172.21.15.99:0/3575288518 shutdown_connections 2024-09-10T11:33:38.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.907+0000 7fd664caf640 1 -- 172.21.15.99:0/3575288518 wait complete. 2024-09-10T11:33:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 Processor -- start 2024-09-10T11:33:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 -- start start 2024-09-10T11:33:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480039b0 0x7fd6601462c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd660169090 0x7fd6601694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd66015cd90 con 0x7fd660151a80 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd660072f20 con 0x7fd6480039b0 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd664caf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd66015c9e0 con 0x7fd660169090 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd65ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd65ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36488/0 (socket says 172.21.15.99:36488) 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.909+0000 7fd65ed9d640 1 -- 172.21.15.99:0/1060658719 learned_addr learned my addr 172.21.15.99:0/1060658719 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65dd9b640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd660169090 0x7fd6601694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 -- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd660169090 msgr2=0x7fd6601694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd660169090 0x7fd6601694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 -- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480039b0 msgr2=0x7fd6601462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480039b0 0x7fd6601462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 -- 172.21.15.99:0/1060658719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd660169b50 con 0x7fd660151a80 2024-09-10T11:33:38.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd65ed9d640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd648004780 tx=0x7fd658004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658007d00 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd658019070 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd660169e30 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658014650 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.910+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66016a3f0 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.911+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd660152440 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.911+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd6580147f0 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.912+0000 7fd64effd640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.912+0000 7fd65e59c640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.912+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7fd65808b070 con 0x7fd660151a80 2024-09-10T11:33:38.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.912+0000 7fd65e59c640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd650033000 tx=0x7fd650007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:38.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:38.916+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd658059d20 con 0x7fd660151a80 2024-09-10T11:33:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.118+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd66006c300 con 0x7fd660151a80 2024-09-10T11:33:39.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.119+0000 7fd64effd640 1 -- 172.21.15.99:0/1060658719 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+3917 (secure 0 0 0) 0x7fd658059620 con 0x7fd660151a80 2024-09-10T11:33:39.121 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:39.121 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":31,"btime":"2024-09-10T11:33:34:213608+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5368,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1312481471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1312481471},{"type":"v1","addr":"172.21.15.196:6834","nonce":1312481471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5371,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2391372585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2391372585},{"type":"v1","addr":"172.21.15.196:6835","nonce":2391372585}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5574,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1796253606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1796253606},{"type":"v1","addr":"172.21.15.99:6835","nonce":1796253606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":5829,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3644719535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3644719535},{"type":"v1","addr":"172.21.15.99:6837","nonce":3644719535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-09-10T11:33:39.121 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 31 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 msgr2=0x7fd64006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 0x7fd64006e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd650033000 tx=0x7fd650007000 comp rx=0 tx=0).stop 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 msgr2=0x7fd660151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd648004780 tx=0x7fd658004840 comp rx=0 tx=0).stop 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 shutdown_connections 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd64006be30 0x7fd64006e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd660151a80 0x7fd660151f00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd660169090 0x7fd6601694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 --2- 172.21.15.99:0/1060658719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6480039b0 0x7fd6601462c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 >> 172.21.15.99:0/1060658719 conn(0x7fd66006bc10 msgr2=0x7fd66015e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 shutdown_connections 2024-09-10T11:33:39.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.121+0000 7fd664caf640 1 -- 172.21.15.99:0/1060658719 wait complete. 2024-09-10T11:33:39.134 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:33:39.134 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:33:39.135 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:33:39.135 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:33:39.135 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:33:39.135 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:33:39.135 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:33:39.135 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:33:39.135 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:33:39.135 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:33:39.136 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:33:39.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:33:39.136 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:33:39.136 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:33:39.136 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:33:39.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:33:39.144 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:33:39.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:33:39.145 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:33:39.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:33:39.148 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:33:39.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:33:39.149 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:33:39.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:33:39.187 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:33:39.187 DEBUG:teuthology.parallel:result is None 2024-09-10T11:33:39.188 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:33:39.188 DEBUG:teuthology.parallel:result is None 2024-09-10T11:33:39.190 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:33:39.190 DEBUG:teuthology.parallel:result is None 2024-09-10T11:33:39.191 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:33:39.191 DEBUG:teuthology.parallel:result is None 2024-09-10T11:33:39.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:33:39.262 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:33:39.262 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:33:39.274 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:33:39.275 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:33:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.295+0000 7f53ec18d640 1 Processor -- start 2024-09-10T11:33:39.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.295+0000 7f53ec18d640 1 -- start start 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.296+0000 7f53ec18d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.296+0000 7f53ec18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 0x7f53e40703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.296+0000 7f53ec18d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40ce6c0 con 0x7f53e40d0ab0 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.296+0000 7f53ec18d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53e40ce9a0 con 0x7f53e41603c0 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.296+0000 7f53ec18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53e406f910 con 0x7f53e40d1600 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53e9f29640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53ea72a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 0x7f53e40703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53e9f29640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:53448/0 (socket says 172.21.15.99:53448) 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53e9f29640 1 -- 172.21.15.99:0/673872304 learned_addr learned my addr 172.21.15.99:0/673872304 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53ea72a640 1 -- 172.21.15.99:0/673872304 >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 msgr2=0x7f53e40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53ea72a640 1 --2- 172.21.15.99:0/673872304 >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53ea72a640 1 -- 172.21.15.99:0/673872304 >> v1:172.21.15.196:6789/0 conn(0x7f53e40d0ab0 legacy=0x7f53e40d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53e8f27640 1 -- 172.21.15.99:0/673872304 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463740982 0 0) 0x7f53e40ce6c0 con 0x7f53e40d0ab0 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53ea72a640 1 -- 172.21.15.99:0/673872304 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e406fbf0 con 0x7f53e40d1600 2024-09-10T11:33:39.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.297+0000 7f53e9f29640 1 --2- 172.21.15.99:0/673872304 >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53ea72a640 1 --2- 172.21.15.99:0/673872304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 0x7f53e40703f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cbc15d8eda7144 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53e8f27640 1 -- 172.21.15.99:0/673872304 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007740 con 0x7f53e40d1600 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53e8f27640 1 -- 172.21.15.99:0/673872304 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc0078e0 con 0x7f53e40d1600 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53e8f27640 1 -- 172.21.15.99:0/673872304 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007cb0 con 0x7f53e40d1600 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53ec18d640 1 -- 172.21.15.99:0/673872304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 msgr2=0x7f53e40703f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.298+0000 7f53ec18d640 1 --2- 172.21.15.99:0/673872304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 0x7f53e40703f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc02a530 comp rx=0 tx=0).stop 2024-09-10T11:33:39.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 -- 172.21.15.99:0/673872304 shutdown_connections 2024-09-10T11:33:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 --2- 172.21.15.99:0/673872304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d1600 0x7f53e40703f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 --2- 172.21.15.99:0/673872304 >> v2:172.21.15.196:3300/0 conn(0x7f53e41603c0 0x7f53e40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 -- 172.21.15.99:0/673872304 >> 172.21.15.99:0/673872304 conn(0x7f53e40b45c0 msgr2=0x7f53e40b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 -- 172.21.15.99:0/673872304 shutdown_connections 2024-09-10T11:33:39.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.299+0000 7f53ec18d640 1 -- 172.21.15.99:0/673872304 wait complete. 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.300+0000 7f53ec18d640 1 Processor -- start 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.300+0000 7f53ec18d640 1 -- start start 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 0x7f53e40c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e40d1600 0x7f53e40b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4070ab0 con 0x7f53e41603c0 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4070930 con 0x7f53e40d1600 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ec18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53e4070c30 con 0x7f53e40d0ab0 2024-09-10T11:33:39.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53e9f29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 0x7f53e40c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36550/0 (socket says 172.21.15.99:36550) 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 -- 172.21.15.99:0/3348700522 learned_addr learned my addr 172.21.15.99:0/3348700522 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53e9f29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 0x7f53e40c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:48612/0 (socket says 172.21.15.99:48612) 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53e9728640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e40d1600 0x7f53e40b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 -- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 msgr2=0x7f53e40c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 0x7f53e40c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 -- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e40d1600 msgr2=0x7f53e40b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e40d1600 0x7f53e40b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 -- 172.21.15.99:0/3348700522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e40ca040 con 0x7f53e41603c0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53ea72a640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.301+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc040070 con 0x7f53e41603c0 2024-09-10T11:33:39.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.302+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc031c40 con 0x7f53e41603c0 2024-09-10T11:33:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.302+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e40c8ca0 con 0x7f53e41603c0 2024-09-10T11:33:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.302+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc009450 con 0x7f53e41603c0 2024-09-10T11:33:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.302+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e40c9260 con 0x7f53e41603c0 2024-09-10T11:33:39.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.302+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e40761a0 con 0x7f53e41603c0 2024-09-10T11:33:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.303+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53dc039040 con 0x7f53e41603c0 2024-09-10T11:33:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.303+0000 7f53e27fc640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.303+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 4943+0+0 (secure 0 0 0) 0x7f53dc0ab9a0 con 0x7f53e41603c0 2024-09-10T11:33:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.303+0000 7f53e9f29640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:39.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.304+0000 7f53e9f29640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 0x7f53d006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006ca0 tx=0x7f53d4009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:39.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.308+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53dc079f80 con 0x7f53e41603c0 2024-09-10T11:33:39.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:39.484+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f53e40703f0 con 0x7f53e41603c0 2024-09-10T11:33:40.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.608+0000 7f53e27fc640 1 -- 172.21.15.99:0/3348700522 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) ==== 97+0+0 (secure 0 0 0) 0x7f53dc07e280 con 0x7f53e41603c0 2024-09-10T11:33:40.609 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:33:40.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 msgr2=0x7f53d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 0x7f53d006e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f53d4006ca0 tx=0x7f53d4009240 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 msgr2=0x7f53e40c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009d90 tx=0x7f53dc004a40 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 shutdown_connections 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d006be30 0x7f53d006e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e41603c0 0x7f53e40c6040 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e40d1600 0x7f53e40b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 --2- 172.21.15.99:0/3348700522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e40d0ab0 0x7f53e40c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 >> 172.21.15.99:0/3348700522 conn(0x7f53e40b45c0 msgr2=0x7f53e40d1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 shutdown_connections 2024-09-10T11:33:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.611+0000 7f53ec18d640 1 -- 172.21.15.99:0/3348700522 wait complete. 2024-09-10T11:33:40.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:33:40.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.728+0000 7f0e07ce6640 1 Processor -- start 2024-09-10T11:33:40.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.729+0000 7f0e07ce6640 1 -- start start 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e07ce6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e00073440 0x7f0e0015c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e07ce6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e00152e60 con 0x7f0e000d1200 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e07ce6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e000d15e0 con 0x7f0e0006ece0 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e07ce6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0e00072680 con 0x7f0e00073440 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e00073440 0x7f0e0015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e00073440 0x7f0e0015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36552/0 (socket says 172.21.15.99:36552) 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e05a82640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0e000d1200 0x7f0e0006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37430/0 (socket says 172.21.15.99:37430) 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 -- 172.21.15.99:0/3296984283 learned_addr learned my addr 172.21.15.99:0/3296984283 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 -- 172.21.15.99:0/3296984283 >> v1:172.21.15.196:6789/0 conn(0x7f0e0006ece0 legacy=0x7f0e0015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 -- 172.21.15.99:0/3296984283 >> v1:172.21.15.99:6789/0 conn(0x7f0e000d1200 legacy=0x7f0e0006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.730+0000 7f0e06283640 1 -- 172.21.15.99:0/3296984283 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e00072960 con 0x7f0e00073440 2024-09-10T11:33:40.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e06283640 1 --2- 172.21.15.99:0/3296984283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e0015c770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009e50 tx=0x7f0df802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f17c142a4c48e82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e04a80640 1 -- 172.21.15.99:0/3296984283 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8036070 con 0x7f0e00073440 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e04a80640 1 -- 172.21.15.99:0/3296984283 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8031600 con 0x7f0e00073440 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e04a80640 1 -- 172.21.15.99:0/3296984283 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80319d0 con 0x7f0e00073440 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 -- 172.21.15.99:0/3296984283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 msgr2=0x7f0e0015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/3296984283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e0015c770 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009e50 tx=0x7f0df802a6d0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 -- 172.21.15.99:0/3296984283 shutdown_connections 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/3296984283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e0015c770 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 -- 172.21.15.99:0/3296984283 >> 172.21.15.99:0/3296984283 conn(0x7f0e0006bc10 msgr2=0x7f0e0006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 -- 172.21.15.99:0/3296984283 shutdown_connections 2024-09-10T11:33:40.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.731+0000 7f0e07ce6640 1 -- 172.21.15.99:0/3296984283 wait complete. 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 Processor -- start 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 -- start start 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0006ece0 0x7f0e00155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e000d1200 0x7f0e001565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:40.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0015cd90 con 0x7f0e00073440 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00072f20 con 0x7f0e0006ece0 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e07ce6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0e00159b50 con 0x7f0e000d1200 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e05281640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e05281640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36566/0 (socket says 172.21.15.99:36566) 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.733+0000 7f0e05281640 1 -- 172.21.15.99:0/2397702153 learned_addr learned my addr 172.21.15.99:0/2397702153 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 -- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e000d1200 msgr2=0x7f0e001565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e000d1200 0x7f0e001565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 -- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0006ece0 msgr2=0x7f0e00155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0006ece0 0x7f0e00155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 -- 172.21.15.99:0/2397702153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e001525f0 con 0x7f0e00073440 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e05281640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0df400c400 tx=0x7f0df400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4017070 con 0x7f0e00073440 2024-09-10T11:33:40.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df40035f0 con 0x7f0e00073440 2024-09-10T11:33:40.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e001528d0 con 0x7f0e00073440 2024-09-10T11:33:40.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df40039e0 con 0x7f0e00073440 2024-09-10T11:33:40.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.734+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0014e240 con 0x7f0e00073440 2024-09-10T11:33:40.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.735+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0015a670 con 0x7f0e00073440 2024-09-10T11:33:40.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.735+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0df401a070 con 0x7f0e00073440 2024-09-10T11:33:40.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.736+0000 7f0dfe7fc640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:40.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.736+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f0df4089dd0 con 0x7f0e00073440 2024-09-10T11:33:40.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.736+0000 7f0e05a82640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:40.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.736+0000 7f0e05a82640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 0x7f0dec06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000f60 tx=0x7f0df0007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:40.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.740+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0df4058510 con 0x7f0e00073440 2024-09-10T11:33:40.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.970+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0e0006e780 con 0x7f0e00073440 2024-09-10T11:33:40.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.970+0000 7f0dfe7fc640 1 -- 172.21.15.99:0/2397702153 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f0df408b020 con 0x7f0e00073440 2024-09-10T11:33:40.971 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.973 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.974 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.976 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.982 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:33:40.984 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:33:40.985 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:33:40.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 msgr2=0x7f0dec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 0x7f0dec06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000f60 tx=0x7f0df0007a00 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 msgr2=0x7f0e00169d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0df400c400 tx=0x7f0df400c8c0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 shutdown_connections 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0dec06be30 0x7f0dec06e2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e000d1200 0x7f0e001565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e00073440 0x7f0e00169d60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 --2- 172.21.15.99:0/2397702153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e0006ece0 0x7f0e00155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 >> 172.21.15.99:0/2397702153 conn(0x7f0e0006bc10 msgr2=0x7f0e0015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 shutdown_connections 2024-09-10T11:33:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:40.973+0000 7f0e07ce6640 1 -- 172.21.15.99:0/2397702153 wait complete. 2024-09-10T11:33:40.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:33:41.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.085+0000 7fc80e5c4640 1 Processor -- start 2024-09-10T11:33:41.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.085+0000 7fc80e5c4640 1 -- start start 2024-09-10T11:33:41.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc80e5c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc8080b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc80e5c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8080b3620 con 0x7fc8080b3960 2024-09-10T11:33:41.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc80e5c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80806d960 con 0x7fc8080b3160 2024-09-10T11:33:41.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc80e5c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc80806dc40 con 0x7fc808163580 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc8077fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc8080b3960 0x7fc8080d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37444/0 (socket says 172.21.15.99:37444) 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.086+0000 7fc8077fe640 1 -- 172.21.15.99:0/3673649150 learned_addr learned my addr 172.21.15.99:0/3673649150 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc807fff640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc8080b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088691387 0 0) 0x7fc8080b3620 con 0x7fc8080b3960 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f0003520 con 0x7fc8080b3960 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795176519 0 0) 0x7fc80806d960 con 0x7fc8080b3160 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8080b3620 con 0x7fc8080b3160 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3594333568 0 0) 0x7fc7f0003520 con 0x7fc8080b3960 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 msgr2=0x7fc8080b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc8080b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 >> v1:172.21.15.196:6789/0 conn(0x7fc8080b3160 legacy=0x7fc80806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80806df20 con 0x7fc8080b3960 2024-09-10T11:33:41.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fc7fc001670 con 0x7fc8080b3960 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 >> v1:172.21.15.99:6789/0 conn(0x7fc8080b3960 legacy=0x7fc8080d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f0003b70 0x7fc7f0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc7f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f0004d40 0x7fc7f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc80806dc40 con 0x7fc7f0004d40 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f000bbd0 con 0x7fc7f0003b70 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f000bdd0 con 0x7fc808163580 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f0004d40 0x7fc7f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc807fff640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f0003b70 0x7fc7f0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc8077fe640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc7f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 -- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 msgr2=0x7fc7f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc808163580 0x7fc7f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 -- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f0003b70 msgr2=0x7fc7f0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:41.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f0003b70 0x7fc7f0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.087+0000 7fc80cb61640 1 -- 172.21.15.99:0/3673649150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f0003520 con 0x7fc7f0004d40 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.088+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f0004d40 0x7fc7f00051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc800001fa0 tx=0x7fc7f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8c2fc7d0fcff0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.088+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800001bf0 con 0x7fc7f0004d40 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.088+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800003b70 con 0x7fc7f0004d40 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.088+0000 7fc806ffd640 1 -- 172.21.15.99:0/3673649150 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000050d0 con 0x7fc7f0004d40 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3673649150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f0004d40 msgr2=0x7fc7f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3673649150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc7f0004d40 0x7fc7f00051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc800001fa0 tx=0x7fc7f0005830 comp rx=0 tx=0).stop 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80cb61640 1 -- 172.21.15.99:0/3673649150 reap_dead start 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3673649150 shutdown_connections 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3673649150 >> 172.21.15.99:0/3673649150 conn(0x7fc8080b4570 msgr2=0x7fc8080b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3673649150 shutdown_connections 2024-09-10T11:33:41.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.089+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3673649150 wait complete. 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.090+0000 7fc80e5c4640 1 Processor -- start 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.090+0000 7fc80e5c4640 1 -- start start 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7f0004d40 0x7fc80815e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 0x7fc8080c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc80806d5b0 con 0x7fc8080c6a90 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc80806d430 con 0x7fc80815e050 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80e5c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc80815e4b0 con 0x7fc7f0004d40 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36578/0 (socket says 172.21.15.99:36578) 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 -- 172.21.15.99:0/3376740603 learned_addr learned my addr 172.21.15.99:0/3376740603 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc807fff640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7f0004d40 0x7fc80815e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc8077fe640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 0x7fc8080c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 -- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7f0004d40 msgr2=0x7fc80815e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7f0004d40 0x7fc80815e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 -- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 msgr2=0x7fc8080c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 0x7fc8080c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc80cb61640 1 -- 172.21.15.99:0/3376740603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080c9880 con 0x7fc8080c6a90 2024-09-10T11:33:41.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.091+0000 7fc8077fe640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 0x7fc8080c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc80cb61640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc80002aa70 tx=0x7fc800001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800001bf0 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800043070 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8080c70a0 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800005610 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.092+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8080b6340 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.093+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8080761a0 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.093+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc80003a040 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.093+0000 7fc804ff9640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.094+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7fc8000aaec0 con 0x7fc8080c6a90 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.094+0000 7fc807fff640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.094+0000 7fc807fff640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 0x7fc7f406e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009bc0 tx=0x7fc7f8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.098+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc800079c50 con 0x7fc8080c6a90 2024-09-10T11:33:41.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.265+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc8081ea350 con 0x7fc8080c6a90 2024-09-10T11:33:41.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.265+0000 7fc804ff9640 1 -- 172.21.15.99:0/3376740603 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fc800079550 con 0x7fc8080c6a90 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:41.267 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:33:41.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 msgr2=0x7fc7f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 0x7fc7f406e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009bc0 tx=0x7fc7f8009190 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 msgr2=0x7fc808161fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc80002aa70 tx=0x7fc800001650 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 shutdown_connections 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc7f406be30 0x7fc7f406e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8080c6a90 0x7fc808161fd0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc80815e050 0x7fc8080c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.267+0000 7fc80e5c4640 1 --2- 172.21.15.99:0/3376740603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7f0004d40 0x7fc80815e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.268+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 >> 172.21.15.99:0/3376740603 conn(0x7fc8080b4570 msgr2=0x7fc8080cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.268+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 shutdown_connections 2024-09-10T11:33:41.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.268+0000 7fc80e5c4640 1 -- 172.21.15.99:0/3376740603 wait complete. 2024-09-10T11:33:41.279 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:33:41.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:33:41.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.380+0000 7f9734bcf640 1 Processor -- start 2024-09-10T11:33:41.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.380+0000 7f9734bcf640 1 -- start start 2024-09-10T11:33:41.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f9734bcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97300d1200 0x7f9730073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f9734bcf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f973006e5d0 0x7f973006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f9734bcf640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9730152e60 con 0x7f973015c060 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f9734bcf640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f973006ef50 con 0x7f973006e5d0 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f9734bcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97300d15e0 con 0x7f97300d1200 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f973006e5d0 0x7f973006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972ed9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f973015c060 0x7f973015c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37446/0 (socket says 172.21.15.99:37446) 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f973006e5d0 0x7f973006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36586/0 (socket says 172.21.15.99:36586) 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972ed9d640 1 -- 172.21.15.99:0/2233791624 learned_addr learned my addr 172.21.15.99:0/2233791624 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972e59c640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97300d1200 0x7f9730073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 -- 172.21.15.99:0/2233791624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97300d1200 msgr2=0x7f9730073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97300d1200 0x7f9730073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 -- 172.21.15.99:0/2233791624 >> v1:172.21.15.99:6789/0 conn(0x7f973015c060 legacy=0x7f973015c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972d59a640 1 -- 172.21.15.99:0/2233791624 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128849183 0 0) 0x7f9730152e60 con 0x7f973015c060 2024-09-10T11:33:41.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.381+0000 7f972dd9b640 1 -- 172.21.15.99:0/2233791624 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9730072680 con 0x7f973006e5d0 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.382+0000 7f972dd9b640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973006e5d0 0x7f973006ea10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34b8e7eddc4a0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.382+0000 7f972d59a640 1 -- 172.21.15.99:0/2233791624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724037070 con 0x7f973006e5d0 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.382+0000 7f972d59a640 1 -- 172.21.15.99:0/2233791624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972402ae50 con 0x7f973006e5d0 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.382+0000 7f972d59a640 1 -- 172.21.15.99:0/2233791624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724032660 con 0x7f973006e5d0 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 -- 172.21.15.99:0/2233791624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973006e5d0 msgr2=0x7f973006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973006e5d0 0x7f973006ea10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402a100 comp rx=0 tx=0).stop 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 -- 172.21.15.99:0/2233791624 shutdown_connections 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973006e5d0 0x7f973006ea10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2233791624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97300d1200 0x7f9730073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 -- 172.21.15.99:0/2233791624 >> 172.21.15.99:0/2233791624 conn(0x7f973006bc10 msgr2=0x7f973006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:41.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 -- 172.21.15.99:0/2233791624 shutdown_connections 2024-09-10T11:33:41.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.383+0000 7f9734bcf640 1 -- 172.21.15.99:0/2233791624 wait complete. 2024-09-10T11:33:41.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.384+0000 7f9734bcf640 1 Processor -- start 2024-09-10T11:33:41.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.384+0000 7f9734bcf640 1 -- start start 2024-09-10T11:33:41.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f973006e5d0 0x7f97301f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 0x7f973006d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9730072f20 con 0x7f973015c060 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9730072c40 con 0x7f97300d1200 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f9734bcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f973015cd90 con 0x7f973006e5d0 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36588/0 (socket says 172.21.15.99:36588) 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 -- 172.21.15.99:0/2604969180 learned_addr learned my addr 172.21.15.99:0/2604969180 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972e59c640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f973006e5d0 0x7f97301f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972dd9b640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 0x7f973006d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 -- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f973006e5d0 msgr2=0x7f97301f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f973006e5d0 0x7f97301f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 -- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 msgr2=0x7f973006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 0x7f973006d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972ed9d640 1 -- 172.21.15.99:0/2604969180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301f8b50 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.385+0000 7f972dd9b640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 0x7f973006d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f972ed9d640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f972800c480 tx=0x7f972800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728018070 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728007500 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728004a20 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973015b8c0 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.386+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97301574d0 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.387+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f973015bc90 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.387+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f972801b070 con 0x7f973015c060 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.387+0000 7f971effd640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 0x7f971006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:41.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.388+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f9728089d50 con 0x7f973015c060 2024-09-10T11:33:41.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.388+0000 7f972e59c640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 0x7f971006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:41.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.388+0000 7f972e59c640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 0x7f971006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97300b5280 tx=0x7f97200039b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:41.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.392+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f973015bc90 con 0x7f973015c060 2024-09-10T11:33:41.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:41.557+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f973006ea10 con 0x7f973015c060 2024-09-10T11:33:42.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.198+0000 7f971effd640 1 -- 172.21.15.99:0/2604969180 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f972808d030 con 0x7f973015c060 2024-09-10T11:33:42.199 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:33:42.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.202+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 msgr2=0x7f971006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.202+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 0x7f971006e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f97300b5280 tx=0x7f97200039b0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.202+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 msgr2=0x7f9730151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.202+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f972800c480 tx=0x7f972800c940 comp rx=0 tx=0).stop 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 shutdown_connections 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f971006be30 0x7f971006e2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973015c060 0x7f9730151e30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97300d1200 0x7f973006d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 --2- 172.21.15.99:0/2604969180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f973006e5d0 0x7f97301f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.203+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 >> 172.21.15.99:0/2604969180 conn(0x7f973006bc10 msgr2=0x7f97300cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:42.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.204+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 shutdown_connections 2024-09-10T11:33:42.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.204+0000 7f9734bcf640 1 -- 172.21.15.99:0/2604969180 wait complete. 2024-09-10T11:33:42.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:33:42.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.325+0000 7fc0c95e4640 1 Processor -- start 2024-09-10T11:33:42.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.325+0000 7fc0c95e4640 1 -- start start 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0c95e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc0c406c320 0x7fc0c4070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0c95e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc0c4070ab0 0x7fc0c40cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0c95e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c4162760 con 0x7fc0c4148400 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0c95e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c4162a40 con 0x7fc0c406c320 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0c95e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c4162d20 con 0x7fc0c4070ab0 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0ba7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc0c406c320 0x7fc0c4070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0ba7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc0c406c320 0x7fc0c4070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36600/0 (socket says 172.21.15.99:36600) 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.326+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/2150618643 learned_addr learned my addr 172.21.15.99:0/2150618643 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:42.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0c37fe640 1 --2- 172.21.15.99:0/2150618643 >> v2:172.21.15.196:3300/0 conn(0x7fc0c4070ab0 0x7fc0c40cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/2150618643 >> v2:172.21.15.196:3300/0 conn(0x7fc0c4070ab0 msgr2=0x7fc0c40cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:42.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0ba7fc640 1 --2- 172.21.15.99:0/2150618643 >> v2:172.21.15.196:3300/0 conn(0x7fc0c4070ab0 0x7fc0c40cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/2150618643 >> v1:172.21.15.196:6789/0 conn(0x7fc0c4148400 legacy=0x7fc0c406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/2150618643 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c4163000 con 0x7fc0c406c320 2024-09-10T11:33:42.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.327+0000 7fc0ba7fc640 1 --2- 172.21.15.99:0/2150618643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c406c320 0x7fc0c4070570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b35c0 tx=0x7fc0ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75042204f0d2c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c27fc640 1 -- 172.21.15.99:0/2150618643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac037070 con 0x7fc0c406c320 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c27fc640 1 -- 172.21.15.99:0/2150618643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac032440 con 0x7fc0c406c320 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c27fc640 1 -- 172.21.15.99:0/2150618643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac032810 con 0x7fc0c406c320 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 -- 172.21.15.99:0/2150618643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c406c320 msgr2=0x7fc0c4070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/2150618643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c406c320 0x7fc0c4070570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40b35c0 tx=0x7fc0ac02a270 comp rx=0 tx=0).stop 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 -- 172.21.15.99:0/2150618643 shutdown_connections 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/2150618643 >> v2:172.21.15.196:3300/0 conn(0x7fc0c4070ab0 0x7fc0c40cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/2150618643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c406c320 0x7fc0c4070570 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 -- 172.21.15.99:0/2150618643 >> 172.21.15.99:0/2150618643 conn(0x7fc0c40d1900 msgr2=0x7fc0c40d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 -- 172.21.15.99:0/2150618643 shutdown_connections 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.328+0000 7fc0c95e4640 1 -- 172.21.15.99:0/2150618643 wait complete. 2024-09-10T11:33:42.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.329+0000 7fc0c95e4640 1 Processor -- start 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 -- start start 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0c406c320 0x7fc0c413b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c4148400 0x7fc0c41e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c40cfdb0 con 0x7fc0c4070ab0 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c40cfc30 con 0x7fc0c4148400 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c95e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c40cff30 con 0x7fc0c406c320 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0ba7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36604/0 (socket says 172.21.15.99:36604) 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/228936628 learned_addr learned my addr 172.21.15.99:0/228936628 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.330+0000 7fc0c2ffd640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0c406c320 0x7fc0c413b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0c37fe640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c4148400 0x7fc0c41e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0c406c320 msgr2=0x7fc0c413b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0c406c320 0x7fc0c413b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c4148400 msgr2=0x7fc0c41e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c4148400 0x7fc0c41e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 -- 172.21.15.99:0/228936628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41f4990 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0ba7fc640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac02a2b0 tx=0x7fc0ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac041020 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0306c0 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c41f8860 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac037040 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.331+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c4160a30 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.332+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c4070570 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.333+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0ac044020 con 0x7fc0c4070ab0 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.333+0000 7fc0bbfff640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 0x7fc0a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:42.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.333+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5227+0+0 (secure 0 0 0) 0x7fc0ac0ab2d0 con 0x7fc0c4070ab0 2024-09-10T11:33:42.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.333+0000 7fc0c2ffd640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 0x7fc0a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:42.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.333+0000 7fc0c2ffd640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 0x7fc0a406e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0c41e9e30 tx=0x7fc0b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:42.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.337+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0ac04b030 con 0x7fc0c4070ab0 2024-09-10T11:33:42.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:42.503+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc0c40cf4d0 con 0x7fc0c4070ab0 2024-09-10T11:33:43.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.200+0000 7fc0bbfff640 1 -- 172.21.15.99:0/228936628 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7fc0ac0798a0 con 0x7fc0c4070ab0 2024-09-10T11:33:43.201 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:33:43.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.203+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 msgr2=0x7fc0a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.204+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 0x7fc0a406e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0c41e9e30 tx=0x7fc0b4009040 comp rx=0 tx=0).stop 2024-09-10T11:33:43.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.204+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 msgr2=0x7fc0c41f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.204+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac02a2b0 tx=0x7fc0ac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 shutdown_connections 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc0a406be30 0x7fc0a406e2f0 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc0c41e9e30 tx=0x7fc0b4009040 comp rx=0 tx=0).stop 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0c4148400 0x7fc0c41e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c4070ab0 0x7fc0c41f7f80 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac02a2b0 tx=0x7fc0ac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 --2- 172.21.15.99:0/228936628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0c406c320 0x7fc0c413b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.205+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 >> 172.21.15.99:0/228936628 conn(0x7fc0c40d1900 msgr2=0x7fc0c40ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:43.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.210+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 shutdown_connections 2024-09-10T11:33:43.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.212+0000 7fc0c95e4640 1 -- 172.21.15.99:0/228936628 wait complete. 2024-09-10T11:33:43.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:33:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.330+0000 7f9a5711f640 1 Processor -- start 2024-09-10T11:33:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.330+0000 7f9a5711f640 1 -- start start 2024-09-10T11:33:43.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a5711f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a500cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a5711f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a500ce430 con 0x7f9a50160030 2024-09-10T11:33:43.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a5711f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a500ce710 con 0x7f9a500d0130 2024-09-10T11:33:43.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a5711f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a500ce200 con 0x7f9a500d1760 2024-09-10T11:33:43.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a54ebb640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9a50160030 0x7f9a500b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54744/0 (socket says 172.21.15.99:54744) 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.331+0000 7f9a54ebb640 1 -- 172.21.15.99:0/2117761661 learned_addr learned my addr 172.21.15.99:0/2117761661 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4ffff640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a500cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337582077 0 0) 0x7f9a500ce430 con 0x7f9a50160030 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a38003520 con 0x7f9a50160030 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001536971 0 0) 0x7f9a500ce710 con 0x7f9a500d0130 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a500ce430 con 0x7f9a500d0130 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3132114891 0 0) 0x7f9a38003520 con 0x7f9a50160030 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 msgr2=0x7f9a500cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a500cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 >> v1:172.21.15.196:6789/0 conn(0x7f9a500d0130 legacy=0x7f9a500cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a500d1240 con 0x7f9a50160030 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9a40001670 con 0x7f9a50160030 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 >> v1:172.21.15.99:6789/0 conn(0x7f9a50160030 legacy=0x7f9a500b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a38003b70 0x7f9a38003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38004d40 0x7f9a380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800ba70 con 0x7f9a38003b70 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800bbf0 con 0x7f9a38004d40 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3800bdf0 con 0x7f9a500d1760 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a38003b70 0x7f9a38003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a4ffff640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a556bc640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38004d40 0x7f9a380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 -- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 msgr2=0x7f9a38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500d1760 0x7f9a38004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 -- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38004d40 msgr2=0x7f9a380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38004d40 0x7f9a380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a54ebb640 1 -- 172.21.15.99:0/2117761661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38003520 con 0x7f9a38003b70 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.332+0000 7f9a556bc640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38004d40 0x7f9a380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.333+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a38003b70 0x7f9a38003f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c190 tx=0x7f9a38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb1caf3e641c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.333+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007ce0 con 0x7f9a38003b70 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.333+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40005ad0 con 0x7f9a38003b70 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.333+0000 7f9a4f7fe640 1 -- 172.21.15.99:0/2117761661 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a40007070 con 0x7f9a38003b70 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 -- 172.21.15.99:0/2117761661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a38003b70 msgr2=0x7f9a38003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 --2- 172.21.15.99:0/2117761661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a38003b70 0x7f9a38003f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000c190 tx=0x7f9a38005830 comp rx=0 tx=0).stop 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a556bc640 1 -- 172.21.15.99:0/2117761661 reap_dead start 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 -- 172.21.15.99:0/2117761661 shutdown_connections 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 -- 172.21.15.99:0/2117761661 >> 172.21.15.99:0/2117761661 conn(0x7f9a500b4650 msgr2=0x7f9a500b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 -- 172.21.15.99:0/2117761661 shutdown_connections 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.334+0000 7f9a5711f640 1 -- 172.21.15.99:0/2117761661 wait complete. 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.335+0000 7f9a5711f640 1 Processor -- start 2024-09-10T11:33:43.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.335+0000 7f9a5711f640 1 -- start start 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 0x7f9a5015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 0x7f9a501f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a500cd5b0 con 0x7f9a500b8310 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a50160660 con 0x7f9a38003b70 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a5711f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a500cd730 con 0x7f9a500b8790 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43596/0 (socket says 172.21.15.99:43596) 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 -- 172.21.15.99:0/3465435653 learned_addr learned my addr 172.21.15.99:0/3465435653 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 0x7f9a5015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a556bc640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 0x7f9a501f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 -- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 msgr2=0x7f9a501f8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 0x7f9a501f8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 -- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 msgr2=0x7f9a5015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 0x7f9a5015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 -- 172.21.15.99:0/3465435653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a500b6b80 con 0x7f9a500b8310 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a556bc640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 0x7f9a501f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 0x7f9a5015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.336+0000 7f9a4ffff640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a44011970 tx=0x7f9a44011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44019070 con 0x7f9a500b8310 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a44004240 con 0x7f9a500b8310 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400f8e0 con 0x7f9a500b8310 2024-09-10T11:33:43.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a500b6e60 con 0x7f9a500b8310 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a500b7420 con 0x7f9a500b8310 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.337+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5006c800 con 0x7f9a500b8310 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.338+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a4401c070 con 0x7f9a500b8310 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.338+0000 7f9a4d7fa640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.339+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5609+0+0 (secure 0 0 0) 0x7f9a4408b230 con 0x7f9a500b8310 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.339+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.339+0000 7f9a54ebb640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a40032000 tx=0x7f9a40009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:43.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.343+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a5006c800 con 0x7f9a500b8310 2024-09-10T11:33:43.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:43.542+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9a500ce710 con 0x7f9a500b8310 2024-09-10T11:33:44.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.233+0000 7f9a4d7fa640 1 -- 172.21.15.99:0/3465435653 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v33) ==== 174+0+0 (secure 0 0 0) 0x7f9a44059cd0 con 0x7f9a500b8310 2024-09-10T11:33:44.235 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T11:33:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.243+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 msgr2=0x7f9a3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.243+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9a40032000 tx=0x7f9a40009030 comp rx=0 tx=0).stop 2024-09-10T11:33:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.243+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 msgr2=0x7f9a501f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.243+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9a44011970 tx=0x7f9a44011e30 comp rx=0 tx=0).stop 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 shutdown_connections 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a3c06be30 0x7f9a3c06e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a500b8790 0x7f9a501f8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a500b8310 0x7f9a501f81f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 --2- 172.21.15.99:0/3465435653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a38003b70 0x7f9a5015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.245+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 >> 172.21.15.99:0/3465435653 conn(0x7f9a500b4650 msgr2=0x7f9a500d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.247+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 shutdown_connections 2024-09-10T11:33:44.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.247+0000 7f9a5711f640 1 -- 172.21.15.99:0/3465435653 wait complete. 2024-09-10T11:33:44.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:44.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.362+0000 7f59b2453640 1 Processor -- start 2024-09-10T11:33:44.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.362+0000 7f59b2453640 1 -- start start 2024-09-10T11:33:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59b2453640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0b02a0 0x7f59ac0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59b2453640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f59ac0af3d0 0x7f59ac0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59b2453640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59ac073d70 con 0x7f59ac15ba90 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59b2453640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac0b5360 con 0x7f59ac0b02a0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59b2453640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f59ac0b55f0 con 0x7f59ac0af3d0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59ab7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0b02a0 0x7f59ac0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59ab7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0b02a0 0x7f59ac0aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43606/0 (socket says 172.21.15.99:43606) 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59ab7fe640 1 -- 172.21.15.99:0/3349387536 learned_addr learned my addr 172.21.15.99:0/3349387536 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.363+0000 7f59abfff640 1 --1- 172.21.15.99:0/3349387536 >> v1:172.21.15.99:6789/0 conn(0x7f59ac15ba90 0x7f59ac0ae780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54760/0 (socket says 172.21.15.99:54760) 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.364+0000 7f59ab7fe640 1 -- 172.21.15.99:0/3349387536 >> v2:172.21.15.196:3300/0 conn(0x7f59ac0af3d0 msgr2=0x7f59ac0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.364+0000 7f59ab7fe640 1 --2- 172.21.15.99:0/3349387536 >> v2:172.21.15.196:3300/0 conn(0x7f59ac0af3d0 0x7f59ac0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.364+0000 7f59ab7fe640 1 -- 172.21.15.99:0/3349387536 >> v1:172.21.15.99:6789/0 conn(0x7f59ac15ba90 legacy=0x7f59ac0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.364+0000 7f59ab7fe640 1 -- 172.21.15.99:0/3349387536 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac0b0970 con 0x7f59ac0b02a0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.364+0000 7f59ab7fe640 1 --2- 172.21.15.99:0/3349387536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b02a0 0x7f59ac0aee90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f59ac072fb0 tx=0x7f59a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1450b7a330ffd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59aaffd640 1 -- 172.21.15.99:0/3349387536 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0037070 con 0x7f59ac0b02a0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59aaffd640 1 -- 172.21.15.99:0/3349387536 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a0032440 con 0x7f59ac0b02a0 2024-09-10T11:33:44.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59aaffd640 1 -- 172.21.15.99:0/3349387536 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0032810 con 0x7f59ac0b02a0 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 -- 172.21.15.99:0/3349387536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b02a0 msgr2=0x7f59ac0aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 --2- 172.21.15.99:0/3349387536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b02a0 0x7f59ac0aee90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f59ac072fb0 tx=0x7f59a002a270 comp rx=0 tx=0).stop 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 -- 172.21.15.99:0/3349387536 shutdown_connections 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 --2- 172.21.15.99:0/3349387536 >> v2:172.21.15.196:3300/0 conn(0x7f59ac0af3d0 0x7f59ac0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 --2- 172.21.15.99:0/3349387536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b02a0 0x7f59ac0aee90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 -- 172.21.15.99:0/3349387536 >> 172.21.15.99:0/3349387536 conn(0x7f59ac0730c0 msgr2=0x7f59ac0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 -- 172.21.15.99:0/3349387536 shutdown_connections 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.365+0000 7f59b2453640 1 -- 172.21.15.99:0/3349387536 wait complete. 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.366+0000 7f59b2453640 1 Processor -- start 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 -- start start 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59ac0af3d0 0x7f59ac14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59ac0b02a0 0x7f59ac151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac15be70 con 0x7f59ac15ba90 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f59ac0b0f30 con 0x7f59ac0af3d0 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b2453640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f59ac15bff0 con 0x7f59ac0b02a0 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b09f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b09f0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43622/0 (socket says 172.21.15.99:43622) 2024-09-10T11:33:44.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.367+0000 7f59b09f0640 1 -- 172.21.15.99:0/2112372095 learned_addr learned my addr 172.21.15.99:0/2112372095 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 -- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59ac0b02a0 msgr2=0x7f59ac151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59ac0b02a0 0x7f59ac151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 -- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59ac0af3d0 msgr2=0x7f59ac14f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59ac0af3d0 0x7f59ac14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 -- 172.21.15.99:0/2112372095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59ac154480 con 0x7f59ac15ba90 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b09f0640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f59a400b970 tx=0x7f59a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4019070 con 0x7f59ac15ba90 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a400cb00 con 0x7f59ac15ba90 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59ac15d450 con 0x7f59ac15ba90 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a4014440 con 0x7f59ac15ba90 2024-09-10T11:33:44.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.368+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59ac15da10 con 0x7f59ac15ba90 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.369+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59ac0ae780 con 0x7f59ac15ba90 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.369+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59a401c070 con 0x7f59ac15ba90 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.370+0000 7f59a8ff9640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 0x7f599806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.370+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f59a408ab50 con 0x7f59ac15ba90 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.370+0000 7f59abfff640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 0x7f599806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.370+0000 7f59abfff640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 0x7f599806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f599c000f60 tx=0x7f599c007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.374+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59a4059510 con 0x7f59ac15ba90 2024-09-10T11:33:44.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.541+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59ac154820 con 0x7f59ac15ba90 2024-09-10T11:33:44.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.542+0000 7f59a8ff9640 1 -- 172.21.15.99:0/2112372095 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16681 (secure 0 0 0) 0x7f59a4058e10 con 0x7f59ac15ba90 2024-09-10T11:33:44.544 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:44.544 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:43.543794+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:33:41.576087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T11:33:42.522066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 msgr2=0x7f599806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 0x7f599806e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f599c000f60 tx=0x7f599c007a00 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 msgr2=0x7f59ac151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f59a400b970 tx=0x7f59a400be30 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 shutdown_connections 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f599806be30 0x7f599806e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac15ba90 0x7f59ac151ad0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59ac0b02a0 0x7f59ac151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 --2- 172.21.15.99:0/2112372095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59ac0af3d0 0x7f59ac14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 >> 172.21.15.99:0/2112372095 conn(0x7f59ac0730c0 msgr2=0x7f59ac06e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 shutdown_connections 2024-09-10T11:33:44.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.545+0000 7f59b2453640 1 -- 172.21.15.99:0/2112372095 wait complete. 2024-09-10T11:33:44.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 Processor -- start 2024-09-10T11:33:44.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 -- start start 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12a815c7b0 0x7f12a80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a80ceb10 con 0x7f12a806f000 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12a80b3a90 con 0x7f12a80d0fb0 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.655+0000 7f12ae6e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f12a80d1420 con 0x7f12a815c7b0 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12a815c7b0 0x7f12a80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12a815c7b0 0x7f12a80d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43626/0 (socket says 172.21.15.99:43626) 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12acc81640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f12a806f000 0x7f12a806d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54772/0 (socket says 172.21.15.99:54772) 2024-09-10T11:33:44.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 -- 172.21.15.99:0/46569624 learned_addr learned my addr 172.21.15.99:0/46569624 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 -- 172.21.15.99:0/46569624 >> v1:172.21.15.196:6789/0 conn(0x7f12a80d0fb0 legacy=0x7f12a806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:44.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 -- 172.21.15.99:0/46569624 >> v1:172.21.15.99:6789/0 conn(0x7f12a806f000 legacy=0x7f12a806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.656+0000 7f12a7fff640 1 -- 172.21.15.99:0/46569624 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a80b3d70 con 0x7f12a815c7b0 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12a7fff640 1 --2- 172.21.15.99:0/46569624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a815c7b0 0x7f12a80d0a70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1298009870 tx=0x7f129802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ca013bb507d170 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12a6ffd640 1 -- 172.21.15.99:0/46569624 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298037070 con 0x7f12a815c7b0 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12a6ffd640 1 -- 172.21.15.99:0/46569624 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1298032440 con 0x7f12a815c7b0 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12a6ffd640 1 -- 172.21.15.99:0/46569624 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298032810 con 0x7f12a815c7b0 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 -- 172.21.15.99:0/46569624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a815c7b0 msgr2=0x7f12a80d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/46569624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a815c7b0 0x7f12a80d0a70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1298009870 tx=0x7f129802a2a0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 -- 172.21.15.99:0/46569624 shutdown_connections 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/46569624 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a815c7b0 0x7f12a80d0a70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 -- 172.21.15.99:0/46569624 >> 172.21.15.99:0/46569624 conn(0x7f12a806bc10 msgr2=0x7f12a806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 -- 172.21.15.99:0/46569624 shutdown_connections 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.657+0000 7f12ae6e4640 1 -- 172.21.15.99:0/46569624 wait complete. 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.658+0000 7f12ae6e4640 1 Processor -- start 2024-09-10T11:33:44.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 -- start start 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a806f000 0x7f12a814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 0x7f12a8152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80b4330 con 0x7f12a80d0fb0 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80b3840 con 0x7f12a815c7b0 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12ae6e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12a80b4610 con 0x7f12a806f000 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43642/0 (socket says 172.21.15.99:43642) 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12a77fe640 1 -- 172.21.15.99:0/2509435665 learned_addr learned my addr 172.21.15.99:0/2509435665 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.659+0000 7f12a7fff640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a806f000 0x7f12a814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12acc81640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 0x7f12a8152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 -- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a806f000 msgr2=0x7f12a814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a806f000 0x7f12a814d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 -- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 msgr2=0x7f12a8152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 0x7f12a8152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 -- 172.21.15.99:0/2509435665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a8153b30 con 0x7f12a80d0fb0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12acc81640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 0x7f12a8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a77fe640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f129000c460 tx=0x7f129000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290016020 con 0x7f12a80d0fb0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1290012870 con 0x7f12a80d0fb0 2024-09-10T11:33:44.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a8153e10 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290012c60 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.660+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a81543d0 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.661+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a815ccf0 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.662+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1290019020 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.662+0000 7f12a4ff9640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 0x7f128806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.662+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f129008ade0 con 0x7f12a80d0fb0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.662+0000 7f12a7fff640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 0x7f128806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.662+0000 7f12a7fff640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 0x7f128806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1298034000 tx=0x7f129802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.666+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1290020020 con 0x7f12a80d0fb0 2024-09-10T11:33:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.831+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12a815bfd0 con 0x7f12a80d0fb0 2024-09-10T11:33:44.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.832+0000 7f12a4ff9640 1 -- 172.21.15.99:0/2509435665 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16681 (secure 0 0 0) 0x7f1290059820 con 0x7f12a80d0fb0 2024-09-10T11:33:44.833 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:44.833 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":84,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:43.543794+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:33:41.576087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T11:33:42.522066+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 msgr2=0x7f128806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 0x7f128806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1298034000 tx=0x7f129802e000 comp rx=0 tx=0).stop 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 msgr2=0x7f12a814d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f129000c460 tx=0x7f129000c920 comp rx=0 tx=0).stop 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 shutdown_connections 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f128806be30 0x7f128806e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a815c7b0 0x7f12a8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80d0fb0 0x7f12a814d680 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 --2- 172.21.15.99:0/2509435665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a806f000 0x7f12a814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 >> 172.21.15.99:0/2509435665 conn(0x7f12a806bc10 msgr2=0x7f12a8157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 shutdown_connections 2024-09-10T11:33:44.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.834+0000 7f12ae6e4640 1 -- 172.21.15.99:0/2509435665 wait complete. 2024-09-10T11:33:44.846 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:33:44.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.946+0000 7f62ce45e640 1 Processor -- start 2024-09-10T11:33:44.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.946+0000 7f62ce45e640 1 -- start start 2024-09-10T11:33:44.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c815cb70 0x7f62c8072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62c80b4150 0x7f62c80b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f62c80b0df0 0x7f62c815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f62c80b1260 con 0x7f62c80b4150 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f62c8072650 con 0x7f62c80b0df0 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62ce45e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62c815db70 con 0x7f62c815cb70 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62bffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62c80b4150 0x7f62c80b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62bffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f62c80b4150 0x7f62c80b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43654/0 (socket says 172.21.15.99:43654) 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.947+0000 7f62bffff640 1 -- 172.21.15.99:0/834193094 learned_addr learned my addr 172.21.15.99:0/834193094 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 -- 172.21.15.99:0/834193094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c815cb70 msgr2=0x7f62c8072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 --2- 172.21.15.99:0/834193094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c815cb70 0x7f62c8072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 -- 172.21.15.99:0/834193094 >> v2:172.21.15.196:3300/0 conn(0x7f62c80b0df0 msgr2=0x7f62c815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 --2- 172.21.15.99:0/834193094 >> v2:172.21.15.196:3300/0 conn(0x7f62c80b0df0 0x7f62c815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 -- 172.21.15.99:0/834193094 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c815de50 con 0x7f62c80b4150 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.948+0000 7f62bffff640 1 --2- 172.21.15.99:0/834193094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c80b4150 0x7f62c80b08b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f62c80705c0 tx=0x7f62b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90905b2da89addb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62c77fe640 1 -- 172.21.15.99:0/834193094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0037070 con 0x7f62c80b4150 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62c77fe640 1 -- 172.21.15.99:0/834193094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0032440 con 0x7f62c80b4150 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62c77fe640 1 -- 172.21.15.99:0/834193094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0032810 con 0x7f62c80b4150 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 -- 172.21.15.99:0/834193094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c80b4150 msgr2=0x7f62c80b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 --2- 172.21.15.99:0/834193094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c80b4150 0x7f62c80b08b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f62c80705c0 tx=0x7f62b002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 -- 172.21.15.99:0/834193094 shutdown_connections 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 --2- 172.21.15.99:0/834193094 >> v2:172.21.15.196:3300/0 conn(0x7f62c80b0df0 0x7f62c815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 --2- 172.21.15.99:0/834193094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c80b4150 0x7f62c80b08b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 --2- 172.21.15.99:0/834193094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c815cb70 0x7f62c8072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 -- 172.21.15.99:0/834193094 >> 172.21.15.99:0/834193094 conn(0x7f62c80b3630 msgr2=0x7f62c80b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:44.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 -- 172.21.15.99:0/834193094 shutdown_connections 2024-09-10T11:33:44.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.949+0000 7f62ce45e640 1 -- 172.21.15.99:0/834193094 wait complete. 2024-09-10T11:33:44.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 Processor -- start 2024-09-10T11:33:44.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 -- start start 2024-09-10T11:33:44.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 0x7f62c814bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 0x7f62c81f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f62c8073320 con 0x7f62c815cb70 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f62c815e130 con 0x7f62c80b4150 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.951+0000 7f62ce45e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f62c80734a0 con 0x7f62c80b0df0 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 0x7f62c814bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62bffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 0x7f62c81f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 0x7f62c814bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:41212/0 (socket says 172.21.15.99:41212) 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43656/0 (socket says 172.21.15.99:43656) 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62bffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 0x7f62c81f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:56336/0 (socket says 172.21.15.99:56336) 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c7fff640 1 -- 172.21.15.99:0/114956938 learned_addr learned my addr 172.21.15.99:0/114956938 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:44.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 -- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 msgr2=0x7f62c814bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 0x7f62c814bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 -- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 msgr2=0x7f62c81f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 0x7f62c81f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 -- 172.21.15.99:0/114956938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c81f6ad0 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62cc9fb640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f62c000be50 tx=0x7f62c000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0019070 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c0014440 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0014830 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c81f4010 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.952+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81f6d90 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.953+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c8069640 con 0x7f62c815cb70 2024-09-10T11:33:44.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.954+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f62c001c070 con 0x7f62c815cb70 2024-09-10T11:33:44.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.954+0000 7f62c57fa640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 0x7f62a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:44.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.954+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f62c008acd0 con 0x7f62c815cb70 2024-09-10T11:33:44.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.954+0000 7f62c7fff640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 0x7f62a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:44.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.955+0000 7f62c7fff640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 0x7f62a806e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009a40 tx=0x7f62b80091c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:44.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:44.959+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f62c8069640 con 0x7f62c815cb70 2024-09-10T11:33:45.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.155+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f62c814f920 con 0x7f62c815cb70 2024-09-10T11:33:45.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.235+0000 7f62c57fa640 1 -- 172.21.15.99:0/114956938 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f62c00590c0 con 0x7f62c815cb70 2024-09-10T11:33:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 msgr2=0x7f62a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 0x7f62a806e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009a40 tx=0x7f62b80091c0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 msgr2=0x7f62c81f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f62c000be50 tx=0x7f62c000bf40 comp rx=0 tx=0).stop 2024-09-10T11:33:45.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 shutdown_connections 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f62a806be30 0x7f62a806e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f62c815cb70 0x7f62c81f34e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62c80b4150 0x7f62c81f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 --2- 172.21.15.99:0/114956938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62c80b0df0 0x7f62c814bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 >> 172.21.15.99:0/114956938 conn(0x7f62c80b3630 msgr2=0x7f62c806f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 shutdown_connections 2024-09-10T11:33:45.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.237+0000 7f62ce45e640 1 -- 172.21.15.99:0/114956938 wait complete. 2024-09-10T11:33:45.249 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:33:45.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:33:45.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.351+0000 7f151cb6d640 1 Processor -- start 2024-09-10T11:33:45.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.351+0000 7f151cb6d640 1 -- start start 2024-09-10T11:33:45.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151cb6d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f151806bc10 0x7f1518163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151cb6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1518160030 0x7f151806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151cb6d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151806c530 con 0x7f15180b4930 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151cb6d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f15180b5420 con 0x7f151806bc10 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151cb6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15180b5140 con 0x7f1518160030 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f151806bc10 0x7f1518163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f151806bc10 0x7f1518163620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43658/0 (socket says 172.21.15.99:43658) 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f1516d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15180b4930 0x7f151806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54780/0 (socket says 172.21.15.99:54780) 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.352+0000 7f151659c640 1 -- 172.21.15.99:0/852442942 learned_addr learned my addr 172.21.15.99:0/852442942 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151659c640 1 -- 172.21.15.99:0/852442942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1518160030 msgr2=0x7f151806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151659c640 1 --2- 172.21.15.99:0/852442942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1518160030 0x7f151806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151659c640 1 -- 172.21.15.99:0/852442942 >> v1:172.21.15.99:6789/0 conn(0x7f15180b4930 legacy=0x7f151806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151659c640 1 -- 172.21.15.99:0/852442942 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1518162b40 con 0x7f151806bc10 2024-09-10T11:33:45.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151559a640 1 -- 172.21.15.99:0/852442942 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407868792 0 0) 0x7f151806c530 con 0x7f15180b4930 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151659c640 1 --2- 172.21.15.99:0/852442942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f1518163620 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1508009870 tx=0x7f150802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675fdeabf0dc158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151559a640 1 -- 172.21.15.99:0/852442942 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508037070 con 0x7f151806bc10 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151559a640 1 -- 172.21.15.99:0/852442942 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508032440 con 0x7f151806bc10 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.353+0000 7f151559a640 1 -- 172.21.15.99:0/852442942 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508032810 con 0x7f151806bc10 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 -- 172.21.15.99:0/852442942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 msgr2=0x7f1518163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 --2- 172.21.15.99:0/852442942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f1518163620 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1508009870 tx=0x7f150802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 -- 172.21.15.99:0/852442942 shutdown_connections 2024-09-10T11:33:45.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 --2- 172.21.15.99:0/852442942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1518160030 0x7f151806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 --2- 172.21.15.99:0/852442942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f1518163620 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 -- 172.21.15.99:0/852442942 >> 172.21.15.99:0/852442942 conn(0x7f15180b3630 msgr2=0x7f15180b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 -- 172.21.15.99:0/852442942 shutdown_connections 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.354+0000 7f151cb6d640 1 -- 172.21.15.99:0/852442942 wait complete. 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.355+0000 7f151cb6d640 1 Processor -- start 2024-09-10T11:33:45.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.355+0000 7f151cb6d640 1 -- start start 2024-09-10T11:33:45.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15180b4930 0x7f15180cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518160030 0x7f15180d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15180b4d90 con 0x7f151806bc10 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1518163100 con 0x7f1518160030 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151cb6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f151815c590 con 0x7f15180b4930 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43672/0 (socket says 172.21.15.99:43672) 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 -- 172.21.15.99:0/3384382823 learned_addr learned my addr 172.21.15.99:0/3384382823 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 -- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15180b4930 msgr2=0x7f15180cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15180b4930 0x7f15180cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 -- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518160030 msgr2=0x7f15180d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:45.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518160030 0x7f15180d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.356+0000 7f151659c640 1 -- 172.21.15.99:0/3384382823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151815c830 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f151659c640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1508009870 tx=0x7f15080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508037040 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508004250 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151815cb10 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150803c670 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.357+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151815d0d0 con 0x7f151806bc10 2024-09-10T11:33:45.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.358+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15180d1560 con 0x7f151806bc10 2024-09-10T11:33:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.358+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f150803a070 con 0x7f151806bc10 2024-09-10T11:33:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.359+0000 7f1506ffd640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 0x7f14f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.359+0000 7f1515d9b640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 0x7f14f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.359+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f15080ab740 con 0x7f151806bc10 2024-09-10T11:33:45.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.359+0000 7f1515d9b640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 0x7f14f806e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1518068250 tx=0x7f150c006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:45.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.363+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1508041020 con 0x7f151806bc10 2024-09-10T11:33:45.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:45.564+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f151815d310 con 0x7f151806bc10 2024-09-10T11:33:46.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.236+0000 7f1506ffd640 1 -- 172.21.15.99:0/3384382823 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7f151815d310 con 0x7f151806bc10 2024-09-10T11:33:46.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 msgr2=0x7f14f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 0x7f14f806e2f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1518068250 tx=0x7f150c006bf0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 msgr2=0x7f15180c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1508009870 tx=0x7f15080046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 shutdown_connections 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14f806be30 0x7f14f806e2f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1518160030 0x7f15180d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15180b4930 0x7f15180cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 --2- 172.21.15.99:0/3384382823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f151806bc10 0x7f15180c6b10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 >> 172.21.15.99:0/3384382823 conn(0x7f15180b3630 msgr2=0x7f1518148370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 shutdown_connections 2024-09-10T11:33:46.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.238+0000 7f151cb6d640 1 -- 172.21.15.99:0/3384382823 wait complete. 2024-09-10T11:33:46.251 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:33:46.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:46.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.352+0000 7f20403f6640 1 Processor -- start 2024-09-10T11:33:46.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.352+0000 7f20403f6640 1 -- start start 2024-09-10T11:33:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.353+0000 7f20403f6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2038163580 0x7f20380b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.353+0000 7f20403f6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20380b3960 0x7f20380d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.353+0000 7f20403f6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20380b3620 con 0x7f203806cd20 2024-09-10T11:33:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.353+0000 7f20403f6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f203806d960 con 0x7f2038163580 2024-09-10T11:33:46.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.353+0000 7f20403f6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f203806dc40 con 0x7f20380b3960 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2038163580 0x7f20380b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203d991640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20380b3960 0x7f20380d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2038163580 0x7f20380b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:56340/0 (socket says 172.21.15.99:56340) 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 -- 172.21.15.99:0/3463936084 learned_addr learned my addr 172.21.15.99:0/3463936084 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e993640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f203806cd20 0x7f203806d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:52906/0 (socket says 172.21.15.99:52906) 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 -- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20380b3960 msgr2=0x7f20380d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 --2- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20380b3960 0x7f20380d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 -- 172.21.15.99:0/3463936084 >> v1:172.21.15.196:6789/0 conn(0x7f203806cd20 legacy=0x7f203806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203d190640 1 -- 172.21.15.99:0/3463936084 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476114567 0 0) 0x7f20380b3620 con 0x7f203806cd20 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.354+0000 7f203e192640 1 -- 172.21.15.99:0/3463936084 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f203806df20 con 0x7f2038163580 2024-09-10T11:33:46.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.355+0000 7f203e192640 1 --2- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038163580 0x7f20380b5090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2028009e50 tx=0x7f202802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4498ae11f785ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.355+0000 7f203d190640 1 -- 172.21.15.99:0/3463936084 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028036070 con 0x7f2038163580 2024-09-10T11:33:46.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.355+0000 7f203d190640 1 -- 172.21.15.99:0/3463936084 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2028031600 con 0x7f2038163580 2024-09-10T11:33:46.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.355+0000 7f203d190640 1 -- 172.21.15.99:0/3463936084 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20280319d0 con 0x7f2038163580 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 -- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038163580 msgr2=0x7f20380b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 --2- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038163580 0x7f20380b5090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2028009e50 tx=0x7f202802a680 comp rx=0 tx=0).stop 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 -- 172.21.15.99:0/3463936084 shutdown_connections 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 --2- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20380b3960 0x7f20380d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 --2- 172.21.15.99:0/3463936084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2038163580 0x7f20380b5090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 -- 172.21.15.99:0/3463936084 >> 172.21.15.99:0/3463936084 conn(0x7f20380b4570 msgr2=0x7f20380b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 -- 172.21.15.99:0/3463936084 shutdown_connections 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.356+0000 7f20403f6640 1 -- 172.21.15.99:0/3463936084 wait complete. 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.357+0000 7f20403f6640 1 Processor -- start 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.357+0000 7f20403f6640 1 -- start start 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203806cd20 0x7f20380cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2038163580 0x7f20381f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20380b3160 con 0x7f20380b3960 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20380d2120 con 0x7f203806cd20 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20380b32e0 con 0x7f2038163580 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43682/0 (socket says 172.21.15.99:43682) 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 -- 172.21.15.99:0/4250886482 learned_addr learned my addr 172.21.15.99:0/4250886482 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203e192640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203806cd20 0x7f20380cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203e993640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2038163580 0x7f20381f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 -- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2038163580 msgr2=0x7f20381f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2038163580 0x7f20381f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 -- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203806cd20 msgr2=0x7f20380cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203806cd20 0x7f20380cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 -- 172.21.15.99:0/4250886482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380b7070 con 0x7f20380b3960 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f203d991640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9b0 tx=0x7f202c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c017070 con 0x7f20380b3960 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c00c9a0 con 0x7f20380b3960 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c00cd90 con 0x7f20380b3960 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20380b7350 con 0x7f20380b3960 2024-09-10T11:33:46.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.358+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20380b7910 con 0x7f20380b3960 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.359+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20380d1be0 con 0x7f20380b3960 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.360+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f202c01a070 con 0x7f20380b3960 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.360+0000 7f2036ffd640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 0x7f202406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.360+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f202c08b1b0 con 0x7f20380b3960 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.360+0000 7f203e192640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 0x7f202406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.361+0000 7f203e192640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 0x7f202406e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2028033000 tx=0x7f2028009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.365+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f202c059510 con 0x7f20380b3960 2024-09-10T11:33:46.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.565+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2038071140 con 0x7f20380b3960 2024-09-10T11:33:46.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.566+0000 7f2036ffd640 1 -- 172.21.15.99:0/4250886482 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7f202c05d810 con 0x7f20380b3960 2024-09-10T11:33:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:46.567 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:33:46.567 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:33:46.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 msgr2=0x7f202406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 0x7f202406e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2028033000 tx=0x7f2028009000 comp rx=0 tx=0).stop 2024-09-10T11:33:46.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 msgr2=0x7f2038157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f202c00e9b0 tx=0x7f202c00ee70 comp rx=0 tx=0).stop 2024-09-10T11:33:46.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 shutdown_connections 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f202406be30 0x7f202406e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2038163580 0x7f20381f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20380b3960 0x7f2038157d50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 --2- 172.21.15.99:0/4250886482 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f203806cd20 0x7f20380cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 >> 172.21.15.99:0/4250886482 conn(0x7f20380b4570 msgr2=0x7f203815e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 shutdown_connections 2024-09-10T11:33:46.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.568+0000 7f20403f6640 1 -- 172.21.15.99:0/4250886482 wait complete. 2024-09-10T11:33:46.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:33:46.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.678+0000 7fb52fda6640 1 Processor -- start 2024-09-10T11:33:46.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 -- start start 2024-09-10T11:33:46.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 0x7fb5280b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5280d1410 0x7fb5280cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52806df80 con 0x7fb5280cfb90 2024-09-10T11:33:46.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb5280d2120 con 0x7fb5280d1410 2024-09-10T11:33:46.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.679+0000 7fb52fda6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb52806e260 con 0x7fb5280b3660 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5280d1410 0x7fb5280cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5280d1410 0x7fb5280cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43696/0 (socket says 172.21.15.99:43696) 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 -- 172.21.15.99:0/118817343 learned_addr learned my addr 172.21.15.99:0/118817343 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52d341640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 0x7fb5280b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 -- 172.21.15.99:0/118817343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 msgr2=0x7fb5280b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 0x7fb5280b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 -- 172.21.15.99:0/118817343 >> v1:172.21.15.196:6789/0 conn(0x7fb5280cfb90 legacy=0x7fb5280d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52e343640 1 -- 172.21.15.99:0/118817343 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280d1ad0 con 0x7fb5280d1410 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.680+0000 7fb52d341640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 0x7fb5280b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52e343640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280d1410 0x7fb5280cde00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb524009e50 tx=0x7fb52402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720feb4d1e042880 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52cb40640 1 -- 172.21.15.99:0/118817343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524036070 con 0x7fb5280d1410 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52cb40640 1 -- 172.21.15.99:0/118817343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb524031600 con 0x7fb5280d1410 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52cb40640 1 -- 172.21.15.99:0/118817343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5240319d0 con 0x7fb5280d1410 2024-09-10T11:33:46.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 -- 172.21.15.99:0/118817343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280d1410 msgr2=0x7fb5280cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280d1410 0x7fb5280cde00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb524009e50 tx=0x7fb52402a680 comp rx=0 tx=0).stop 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 -- 172.21.15.99:0/118817343 shutdown_connections 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280d1410 0x7fb5280cde00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 --2- 172.21.15.99:0/118817343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280b3660 0x7fb5280b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 -- 172.21.15.99:0/118817343 >> 172.21.15.99:0/118817343 conn(0x7fb5280b4650 msgr2=0x7fb5280b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 -- 172.21.15.99:0/118817343 shutdown_connections 2024-09-10T11:33:46.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.681+0000 7fb52fda6640 1 -- 172.21.15.99:0/118817343 wait complete. 2024-09-10T11:33:46.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.682+0000 7fb52fda6640 1 Processor -- start 2024-09-10T11:33:46.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 -- start start 2024-09-10T11:33:46.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5280cfb90 0x7fb5281605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280d1410 0x7fb52815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280cff70 con 0x7fb5280b3660 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb52806dd30 con 0x7fb5280cfb90 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52fda6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5280d00f0 con 0x7fb5280d1410 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52db42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52db42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43702/0 (socket says 172.21.15.99:43702) 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52db42640 1 -- 172.21.15.99:0/837863772 learned_addr learned my addr 172.21.15.99:0/837863772 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52d341640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5280cfb90 0x7fb5281605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.683+0000 7fb52e343640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280d1410 0x7fb52815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 -- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280d1410 msgr2=0x7fb52815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280d1410 0x7fb52815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 -- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5280cfb90 msgr2=0x7fb5281605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5280cfb90 0x7fb5281605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 -- 172.21.15.99:0/837863772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280c7fa0 con 0x7fb5280b3660 2024-09-10T11:33:46.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52db42640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c3c0 tx=0x7fb52000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200052d0 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb520005490 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52815de00 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520007260 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.684+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5280c8540 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.685+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5280b5170 con 0x7fb5280b3660 2024-09-10T11:33:46.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.685+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb520007400 con 0x7fb5280b3660 2024-09-10T11:33:46.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.686+0000 7fb51e7fc640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 0x7fb50c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.686+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fb52008a2d0 con 0x7fb5280b3660 2024-09-10T11:33:46.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.686+0000 7fb52d341640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 0x7fb50c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.686+0000 7fb52d341640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 0x7fb50c06e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5281f7ff0 tx=0x7fb518008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.690+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb520058630 con 0x7fb5280b3660 2024-09-10T11:33:46.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.854+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5280cccf0 con 0x7fb5280b3660 2024-09-10T11:33:46.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.856+0000 7fb51e7fc640 1 -- 172.21.15.99:0/837863772 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16668 (secure 0 0 0) 0x7fb52005c930 con 0x7fb5280b3660 2024-09-10T11:33:46.857 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:46.857 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":85,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:33:45.224660+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:33:41.576087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T11:33:42.522066+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:33:46.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.857+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 msgr2=0x7fb50c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.857+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 0x7fb50c06e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb5281f7ff0 tx=0x7fb518008040 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.857+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 msgr2=0x7fb52815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.857+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb52000c3c0 tx=0x7fb52000c880 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 shutdown_connections 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb50c06be30 0x7fb50c06e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5280d1410 0x7fb52815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5280cfb90 0x7fb5281605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 --2- 172.21.15.99:0/837863772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5280b3660 0x7fb52815d820 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 >> 172.21.15.99:0/837863772 conn(0x7fb5280b4650 msgr2=0x7fb5280ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 shutdown_connections 2024-09-10T11:33:46.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.858+0000 7fb52fda6640 1 -- 172.21.15.99:0/837863772 wait complete. 2024-09-10T11:33:46.870 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:33:46.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:33:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 Processor -- start 2024-09-10T11:33:46.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 -- start start 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 0x7f78f40b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78f415b2d0 0x7f78f40afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f40cd290 0x7f78f40714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f78f406ee10 con 0x7f78f415b2d0 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f78f40b0290 con 0x7f78f415aef0 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.968+0000 7f78fa903640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78f406e570 con 0x7f78f40cd290 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78f415b2d0 0x7f78f40afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f78f415b2d0 0x7f78f40afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43710/0 (socket says 172.21.15.99:43710) 2024-09-10T11:33:46.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 -- 172.21.15.99:0/1996282450 learned_addr learned my addr 172.21.15.99:0/1996282450 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f3fff640 1 --2- 172.21.15.99:0/1996282450 >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 0x7f78f40b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 -- 172.21.15.99:0/1996282450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f40cd290 msgr2=0x7f78f40714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 --2- 172.21.15.99:0/1996282450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f40cd290 0x7f78f40714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 -- 172.21.15.99:0/1996282450 >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 msgr2=0x7f78f40b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 --2- 172.21.15.99:0/1996282450 >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 0x7f78f40b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f37fe640 1 -- 172.21.15.99:0/1996282450 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f406e850 con 0x7f78f415b2d0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.969+0000 7f78f3fff640 1 --2- 172.21.15.99:0/1996282450 >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 0x7f78f40b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78f37fe640 1 --2- 172.21.15.99:0/1996282450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415b2d0 0x7f78f40afbb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f78f406f620 tx=0x7f78e802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aa21a75e69eaef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78f2ffd640 1 -- 172.21.15.99:0/1996282450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e803a070 con 0x7f78f415b2d0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78f2ffd640 1 -- 172.21.15.99:0/1996282450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e8032480 con 0x7f78f415b2d0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78f2ffd640 1 -- 172.21.15.99:0/1996282450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8032850 con 0x7f78f415b2d0 2024-09-10T11:33:46.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 -- 172.21.15.99:0/1996282450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415b2d0 msgr2=0x7f78f40afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 --2- 172.21.15.99:0/1996282450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415b2d0 0x7f78f40afbb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f78f406f620 tx=0x7f78e802c670 comp rx=0 tx=0).stop 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 -- 172.21.15.99:0/1996282450 shutdown_connections 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 --2- 172.21.15.99:0/1996282450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f40cd290 0x7f78f40714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 --2- 172.21.15.99:0/1996282450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415b2d0 0x7f78f40afbb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 --2- 172.21.15.99:0/1996282450 >> v2:172.21.15.196:3300/0 conn(0x7f78f415aef0 0x7f78f40b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 -- 172.21.15.99:0/1996282450 >> 172.21.15.99:0/1996282450 conn(0x7f78f406bc10 msgr2=0x7f78f406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 -- 172.21.15.99:0/1996282450 shutdown_connections 2024-09-10T11:33:46.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.970+0000 7f78fa903640 1 -- 172.21.15.99:0/1996282450 wait complete. 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.971+0000 7f78fa903640 1 Processor -- start 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 -- start start 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 0x7f78f40cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 0x7f78f40ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f78f40cd880 con 0x7f78f415aef0 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f78f40cd700 con 0x7f78f40cd290 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78fa903640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78f40cda00 con 0x7f78f415b2d0 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78f37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43714/0 (socket says 172.21.15.99:43714) 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.972+0000 7f78f37fe640 1 -- 172.21.15.99:0/3297038212 learned_addr learned my addr 172.21.15.99:0/3297038212 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f8ea0640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 0x7f78f40ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f3fff640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 0x7f78f40cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 -- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 msgr2=0x7f78f40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 0x7f78f40ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 -- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 msgr2=0x7f78f40cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 0x7f78f40cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 -- 172.21.15.99:0/3297038212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f41568e0 con 0x7f78f415aef0 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f8ea0640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 0x7f78f40ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f3fff640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 0x7f78f40cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f37fe640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f78e8038040 tx=0x7f78e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8044020 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e8035a50 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e803a070 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f4156bc0 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.973+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f4157180 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.974+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f40b4510 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.975+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78e8040440 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.975+0000 7f78f0ff9640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 0x7f78e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.975+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f78e80abb80 con 0x7f78f415aef0 2024-09-10T11:33:46.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.975+0000 7f78f3fff640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 0x7f78e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:46.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.976+0000 7f78f3fff640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 0x7f78e006e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78e400aa30 tx=0x7f78e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:46.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:46.979+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f78e804e060 con 0x7f78f415aef0 2024-09-10T11:33:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.141+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78f40afcf0 con 0x7f78e006be30 2024-09-10T11:33:47.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.147+0000 7f78f0ff9640 1 -- 172.21.15.99:0/3297038212 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247590 (secure 0 0 0) 0x7f78f40afcf0 con 0x7f78e006be30 2024-09-10T11:33:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:47.152 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:33:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 msgr2=0x7f78e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 0x7f78e006e2f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f78e400aa30 tx=0x7f78e4008040 comp rx=0 tx=0).stop 2024-09-10T11:33:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 msgr2=0x7f78f4155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f78e8038040 tx=0x7f78e80046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 shutdown_connections 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f78e006be30 0x7f78e006e2f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78f415b2d0 0x7f78f40ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78f415aef0 0x7f78f4155f70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 --2- 172.21.15.99:0/3297038212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78f40cd290 0x7f78f40cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 >> 172.21.15.99:0/3297038212 conn(0x7f78f406bc10 msgr2=0x7f78f4072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 shutdown_connections 2024-09-10T11:33:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.150+0000 7f78fa903640 1 -- 172.21.15.99:0/3297038212 wait complete. 2024-09-10T11:33:47.161 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-10T11:33:46.950778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4677,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557822,"statfs":{"total":1600000000,"available":1595210628,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.749173"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238775+0000","last_change":"2024-09-10T11:33:44.242903+0000","last_active":"2024-09-10T11:33:45.238775+0000","last_peered":"2024-09-10T11:33:45.238775+0000","last_clean":"2024-09-10T11:33:45.238775+0000","last_became_active":"2024-09-10T11:33:44.242268+0000","last_became_peered":"2024-09-10T11:33:44.242268+0000","last_unstale":"2024-09-10T11:33:45.238775+0000","last_undegraded":"2024-09-10T11:33:45.238775+0000","last_fullsized":"2024-09-10T11:33:45.238775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:18.258245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290478+0000","last_change":"2024-09-10T11:33:44.233956+0000","last_active":"2024-09-10T11:33:45.290478+0000","last_peered":"2024-09-10T11:33:45.290478+0000","last_clean":"2024-09-10T11:33:45.290478+0000","last_became_active":"2024-09-10T11:33:44.233728+0000","last_became_peered":"2024-09-10T11:33:44.233728+0000","last_unstale":"2024-09-10T11:33:45.290478+0000","last_undegraded":"2024-09-10T11:33:45.290478+0000","last_fullsized":"2024-09-10T11:33:45.290478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:34.016351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290629+0000","last_change":"2024-09-10T11:33:44.234335+0000","last_active":"2024-09-10T11:33:45.290629+0000","last_peered":"2024-09-10T11:33:45.290629+0000","last_clean":"2024-09-10T11:33:45.290629+0000","last_became_active":"2024-09-10T11:33:44.234101+0000","last_became_peered":"2024-09-10T11:33:44.234101+0000","last_unstale":"2024-09-10T11:33:45.290629+0000","last_undegraded":"2024-09-10T11:33:45.290629+0000","last_fullsized":"2024-09-10T11:33:45.290629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:53.013805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290606+0000","last_change":"2024-09-10T11:33:44.232883+0000","last_active":"2024-09-10T11:33:45.290606+0000","last_peered":"2024-09-10T11:33:45.290606+0000","last_clean":"2024-09-10T11:33:45.290606+0000","last_became_active":"2024-09-10T11:33:44.232651+0000","last_became_peered":"2024-09-10T11:33:44.232651+0000","last_unstale":"2024-09-10T11:33:45.290606+0000","last_undegraded":"2024-09-10T11:33:45.290606+0000","last_fullsized":"2024-09-10T11:33:45.290606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:57.683258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291003+0000","last_change":"2024-09-10T11:33:44.242800+0000","last_active":"2024-09-10T11:33:45.291003+0000","last_peered":"2024-09-10T11:33:45.291003+0000","last_clean":"2024-09-10T11:33:45.291003+0000","last_became_active":"2024-09-10T11:33:44.242547+0000","last_became_peered":"2024-09-10T11:33:44.242547+0000","last_unstale":"2024-09-10T11:33:45.291003+0000","last_undegraded":"2024-09-10T11:33:45.291003+0000","last_fullsized":"2024-09-10T11:33:45.291003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:36.669550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290533+0000","last_change":"2024-09-10T11:33:44.243899+0000","last_active":"2024-09-10T11:33:45.290533+0000","last_peered":"2024-09-10T11:33:45.290533+0000","last_clean":"2024-09-10T11:33:45.290533+0000","last_became_active":"2024-09-10T11:33:44.243454+0000","last_became_peered":"2024-09-10T11:33:44.243454+0000","last_unstale":"2024-09-10T11:33:45.290533+0000","last_undegraded":"2024-09-10T11:33:45.290533+0000","last_fullsized":"2024-09-10T11:33:45.290533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:23.514699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227632+0000","last_change":"2024-09-10T11:33:44.241886+0000","last_active":"2024-09-10T11:33:45.227632+0000","last_peered":"2024-09-10T11:33:45.227632+0000","last_clean":"2024-09-10T11:33:45.227632+0000","last_became_active":"2024-09-10T11:33:44.241626+0000","last_became_peered":"2024-09-10T11:33:44.241626+0000","last_unstale":"2024-09-10T11:33:45.227632+0000","last_undegraded":"2024-09-10T11:33:45.227632+0000","last_fullsized":"2024-09-10T11:33:45.227632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:07.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238873+0000","last_change":"2024-09-10T11:33:44.244007+0000","last_active":"2024-09-10T11:33:45.238873+0000","last_peered":"2024-09-10T11:33:45.238873+0000","last_clean":"2024-09-10T11:33:45.238873+0000","last_became_active":"2024-09-10T11:33:44.243401+0000","last_became_peered":"2024-09-10T11:33:44.243401+0000","last_unstale":"2024-09-10T11:33:45.238873+0000","last_undegraded":"2024-09-10T11:33:45.238873+0000","last_fullsized":"2024-09-10T11:33:45.238873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:26.125164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291107+0000","last_change":"2024-09-10T11:33:44.240028+0000","last_active":"2024-09-10T11:33:45.291107+0000","last_peered":"2024-09-10T11:33:45.291107+0000","last_clean":"2024-09-10T11:33:45.291107+0000","last_became_active":"2024-09-10T11:33:44.239783+0000","last_became_peered":"2024-09-10T11:33:44.239783+0000","last_unstale":"2024-09-10T11:33:45.291107+0000","last_undegraded":"2024-09-10T11:33:45.291107+0000","last_fullsized":"2024-09-10T11:33:45.291107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:09.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227927+0000","last_change":"2024-09-10T11:33:44.241780+0000","last_active":"2024-09-10T11:33:45.227927+0000","last_peered":"2024-09-10T11:33:45.227927+0000","last_clean":"2024-09-10T11:33:45.227927+0000","last_became_active":"2024-09-10T11:33:44.241549+0000","last_became_peered":"2024-09-10T11:33:44.241549+0000","last_unstale":"2024-09-10T11:33:45.227927+0000","last_undegraded":"2024-09-10T11:33:45.227927+0000","last_fullsized":"2024-09-10T11:33:45.227927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:46.368268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238974+0000","last_change":"2024-09-10T11:33:44.243102+0000","last_active":"2024-09-10T11:33:45.238974+0000","last_peered":"2024-09-10T11:33:45.238974+0000","last_clean":"2024-09-10T11:33:45.238974+0000","last_became_active":"2024-09-10T11:33:44.242606+0000","last_became_peered":"2024-09-10T11:33:44.242606+0000","last_unstale":"2024-09-10T11:33:45.238974+0000","last_undegraded":"2024-09-10T11:33:45.238974+0000","last_fullsized":"2024-09-10T11:33:45.238974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:30:27.411597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228638+0000","last_change":"2024-09-10T11:33:44.243432+0000","last_active":"2024-09-10T11:33:45.228638+0000","last_peered":"2024-09-10T11:33:45.228638+0000","last_clean":"2024-09-10T11:33:45.228638+0000","last_became_active":"2024-09-10T11:33:44.242750+0000","last_became_peered":"2024-09-10T11:33:44.242750+0000","last_unstale":"2024-09-10T11:33:45.228638+0000","last_undegraded":"2024-09-10T11:33:45.228638+0000","last_fullsized":"2024-09-10T11:33:45.228638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:05.622221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290897+0000","last_change":"2024-09-10T11:33:44.235298+0000","last_active":"2024-09-10T11:33:45.290897+0000","last_peered":"2024-09-10T11:33:45.290897+0000","last_clean":"2024-09-10T11:33:45.290897+0000","last_became_active":"2024-09-10T11:33:44.235037+0000","last_became_peered":"2024-09-10T11:33:44.235037+0000","last_unstale":"2024-09-10T11:33:45.290897+0000","last_undegraded":"2024-09-10T11:33:45.290897+0000","last_fullsized":"2024-09-10T11:33:45.290897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:33.258933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227972+0000","last_change":"2024-09-10T11:33:44.246907+0000","last_active":"2024-09-10T11:33:45.227972+0000","last_peered":"2024-09-10T11:33:45.227972+0000","last_clean":"2024-09-10T11:33:45.227972+0000","last_became_active":"2024-09-10T11:33:44.246484+0000","last_became_peered":"2024-09-10T11:33:44.246484+0000","last_unstale":"2024-09-10T11:33:45.227972+0000","last_undegraded":"2024-09-10T11:33:45.227972+0000","last_fullsized":"2024-09-10T11:33:45.227972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:07.902652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228112+0000","last_change":"2024-09-10T11:33:44.244538+0000","last_active":"2024-09-10T11:33:45.228112+0000","last_peered":"2024-09-10T11:33:45.228112+0000","last_clean":"2024-09-10T11:33:45.228112+0000","last_became_active":"2024-09-10T11:33:44.244273+0000","last_became_peered":"2024-09-10T11:33:44.244273+0000","last_unstale":"2024-09-10T11:33:45.228112+0000","last_undegraded":"2024-09-10T11:33:45.228112+0000","last_fullsized":"2024-09-10T11:33:45.228112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:32.520810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239097+0000","last_change":"2024-09-10T11:33:44.241906+0000","last_active":"2024-09-10T11:33:45.239097+0000","last_peered":"2024-09-10T11:33:45.239097+0000","last_clean":"2024-09-10T11:33:45.239097+0000","last_became_active":"2024-09-10T11:33:44.241221+0000","last_became_peered":"2024-09-10T11:33:44.241221+0000","last_unstale":"2024-09-10T11:33:45.239097+0000","last_undegraded":"2024-09-10T11:33:45.239097+0000","last_fullsized":"2024-09-10T11:33:45.239097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291084+0000","last_change":"2024-09-10T11:33:44.239346+0000","last_active":"2024-09-10T11:33:45.291084+0000","last_peered":"2024-09-10T11:33:45.291084+0000","last_clean":"2024-09-10T11:33:45.291084+0000","last_became_active":"2024-09-10T11:33:44.238914+0000","last_became_peered":"2024-09-10T11:33:44.238914+0000","last_unstale":"2024-09-10T11:33:45.291084+0000","last_undegraded":"2024-09-10T11:33:45.291084+0000","last_fullsized":"2024-09-10T11:33:45.291084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:04.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291150+0000","last_change":"2024-09-10T11:33:44.239233+0000","last_active":"2024-09-10T11:33:45.291150+0000","last_peered":"2024-09-10T11:33:45.291150+0000","last_clean":"2024-09-10T11:33:45.291150+0000","last_became_active":"2024-09-10T11:33:44.237759+0000","last_became_peered":"2024-09-10T11:33:44.237759+0000","last_unstale":"2024-09-10T11:33:45.291150+0000","last_undegraded":"2024-09-10T11:33:45.291150+0000","last_fullsized":"2024-09-10T11:33:45.291150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:28.475163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238798+0000","last_change":"2024-09-10T11:33:44.241109+0000","last_active":"2024-09-10T11:33:45.238798+0000","last_peered":"2024-09-10T11:33:45.238798+0000","last_clean":"2024-09-10T11:33:45.238798+0000","last_became_active":"2024-09-10T11:33:44.240613+0000","last_became_peered":"2024-09-10T11:33:44.240613+0000","last_unstale":"2024-09-10T11:33:45.238798+0000","last_undegraded":"2024-09-10T11:33:45.238798+0000","last_fullsized":"2024-09-10T11:33:45.238798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:20.962771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291167+0000","last_change":"2024-09-10T11:33:44.241747+0000","last_active":"2024-09-10T11:33:45.291167+0000","last_peered":"2024-09-10T11:33:45.291167+0000","last_clean":"2024-09-10T11:33:45.291167+0000","last_became_active":"2024-09-10T11:33:44.241469+0000","last_became_peered":"2024-09-10T11:33:44.241469+0000","last_unstale":"2024-09-10T11:33:45.291167+0000","last_undegraded":"2024-09-10T11:33:45.291167+0000","last_fullsized":"2024-09-10T11:33:45.291167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:11.567000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290856+0000","last_change":"2024-09-10T11:33:44.231985+0000","last_active":"2024-09-10T11:33:45.290856+0000","last_peered":"2024-09-10T11:33:45.290856+0000","last_clean":"2024-09-10T11:33:45.290856+0000","last_became_active":"2024-09-10T11:33:44.231422+0000","last_became_peered":"2024-09-10T11:33:44.231422+0000","last_unstale":"2024-09-10T11:33:45.290856+0000","last_undegraded":"2024-09-10T11:33:45.290856+0000","last_fullsized":"2024-09-10T11:33:45.290856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:22.485023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290971+0000","last_change":"2024-09-10T11:33:44.239371+0000","last_active":"2024-09-10T11:33:45.290971+0000","last_peered":"2024-09-10T11:33:45.290971+0000","last_clean":"2024-09-10T11:33:45.290971+0000","last_became_active":"2024-09-10T11:33:44.238917+0000","last_became_peered":"2024-09-10T11:33:44.238917+0000","last_unstale":"2024-09-10T11:33:45.290971+0000","last_undegraded":"2024-09-10T11:33:45.290971+0000","last_fullsized":"2024-09-10T11:33:45.290971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:01.020188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227800+0000","last_change":"2024-09-10T11:33:44.235195+0000","last_active":"2024-09-10T11:33:45.227800+0000","last_peered":"2024-09-10T11:33:45.227800+0000","last_clean":"2024-09-10T11:33:45.227800+0000","last_became_active":"2024-09-10T11:33:44.234816+0000","last_became_peered":"2024-09-10T11:33:44.234816+0000","last_unstale":"2024-09-10T11:33:45.227800+0000","last_undegraded":"2024-09-10T11:33:45.227800+0000","last_fullsized":"2024-09-10T11:33:45.227800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:02.905660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238933+0000","last_change":"2024-09-10T11:33:44.240850+0000","last_active":"2024-09-10T11:33:45.238933+0000","last_peered":"2024-09-10T11:33:45.238933+0000","last_clean":"2024-09-10T11:33:45.238933+0000","last_became_active":"2024-09-10T11:33:44.240139+0000","last_became_peered":"2024-09-10T11:33:44.240139+0000","last_unstale":"2024-09-10T11:33:45.238933+0000","last_undegraded":"2024-09-10T11:33:45.238933+0000","last_fullsized":"2024-09-10T11:33:45.238933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:23.630606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227832+0000","last_change":"2024-09-10T11:33:44.237162+0000","last_active":"2024-09-10T11:33:45.227832+0000","last_peered":"2024-09-10T11:33:45.227832+0000","last_clean":"2024-09-10T11:33:45.227832+0000","last_became_active":"2024-09-10T11:33:44.236924+0000","last_became_peered":"2024-09-10T11:33:44.236924+0000","last_unstale":"2024-09-10T11:33:45.227832+0000","last_undegraded":"2024-09-10T11:33:45.227832+0000","last_fullsized":"2024-09-10T11:33:45.227832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:51.374696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239219+0000","last_change":"2024-09-10T11:33:44.241475+0000","last_active":"2024-09-10T11:33:45.239219+0000","last_peered":"2024-09-10T11:33:45.239219+0000","last_clean":"2024-09-10T11:33:45.239219+0000","last_became_active":"2024-09-10T11:33:44.240581+0000","last_became_peered":"2024-09-10T11:33:44.240581+0000","last_unstale":"2024-09-10T11:33:45.239219+0000","last_undegraded":"2024-09-10T11:33:45.239219+0000","last_fullsized":"2024-09-10T11:33:45.239219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:17.782292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291251+0000","last_change":"2024-09-10T11:33:44.239152+0000","last_active":"2024-09-10T11:33:45.291251+0000","last_peered":"2024-09-10T11:33:45.291251+0000","last_clean":"2024-09-10T11:33:45.291251+0000","last_became_active":"2024-09-10T11:33:44.238377+0000","last_became_peered":"2024-09-10T11:33:44.238377+0000","last_unstale":"2024-09-10T11:33:45.291251+0000","last_undegraded":"2024-09-10T11:33:45.291251+0000","last_fullsized":"2024-09-10T11:33:45.291251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:49.060965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290407+0000","last_change":"2024-09-10T11:33:44.229762+0000","last_active":"2024-09-10T11:33:45.290407+0000","last_peered":"2024-09-10T11:33:45.290407+0000","last_clean":"2024-09-10T11:33:45.290407+0000","last_became_active":"2024-09-10T11:33:44.229500+0000","last_became_peered":"2024-09-10T11:33:44.229500+0000","last_unstale":"2024-09-10T11:33:45.290407+0000","last_undegraded":"2024-09-10T11:33:45.290407+0000","last_fullsized":"2024-09-10T11:33:45.290407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:45.119789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291256+0000","last_change":"2024-09-10T11:33:44.238133+0000","last_active":"2024-09-10T11:33:45.291256+0000","last_peered":"2024-09-10T11:33:45.291256+0000","last_clean":"2024-09-10T11:33:45.291256+0000","last_became_active":"2024-09-10T11:33:44.237838+0000","last_became_peered":"2024-09-10T11:33:44.237838+0000","last_unstale":"2024-09-10T11:33:45.291256+0000","last_undegraded":"2024-09-10T11:33:45.291256+0000","last_fullsized":"2024-09-10T11:33:45.291256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:07.818150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238975+0000","last_change":"2024-09-10T11:33:44.241252+0000","last_active":"2024-09-10T11:33:45.238975+0000","last_peered":"2024-09-10T11:33:45.238975+0000","last_clean":"2024-09-10T11:33:45.238975+0000","last_became_active":"2024-09-10T11:33:44.241010+0000","last_became_peered":"2024-09-10T11:33:44.241010+0000","last_unstale":"2024-09-10T11:33:45.238975+0000","last_undegraded":"2024-09-10T11:33:45.238975+0000","last_fullsized":"2024-09-10T11:33:45.238975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:49.157541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290991+0000","last_change":"2024-09-10T11:33:44.230187+0000","last_active":"2024-09-10T11:33:45.290991+0000","last_peered":"2024-09-10T11:33:45.290991+0000","last_clean":"2024-09-10T11:33:45.290991+0000","last_became_active":"2024-09-10T11:33:44.229949+0000","last_became_peered":"2024-09-10T11:33:44.229949+0000","last_unstale":"2024-09-10T11:33:45.290991+0000","last_undegraded":"2024-09-10T11:33:45.290991+0000","last_fullsized":"2024-09-10T11:33:45.290991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:38.786009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290610+0000","last_change":"2024-09-10T11:33:44.229723+0000","last_active":"2024-09-10T11:33:45.290610+0000","last_peered":"2024-09-10T11:33:45.290610+0000","last_clean":"2024-09-10T11:33:45.290610+0000","last_became_active":"2024-09-10T11:33:44.229432+0000","last_became_peered":"2024-09-10T11:33:44.229432+0000","last_unstale":"2024-09-10T11:33:45.290610+0000","last_undegraded":"2024-09-10T11:33:45.290610+0000","last_fullsized":"2024-09-10T11:33:45.290610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:41.547433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239065+0000","last_change":"2024-09-10T11:33:44.242808+0000","last_active":"2024-09-10T11:33:45.239065+0000","last_peered":"2024-09-10T11:33:45.239065+0000","last_clean":"2024-09-10T11:33:45.239065+0000","last_became_active":"2024-09-10T11:33:44.242532+0000","last_became_peered":"2024-09-10T11:33:44.242532+0000","last_unstale":"2024-09-10T11:33:45.239065+0000","last_undegraded":"2024-09-10T11:33:45.239065+0000","last_fullsized":"2024-09-10T11:33:45.239065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:50.880848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290770+0000","last_change":"2024-09-10T11:33:43.214560+0000","last_active":"2024-09-10T11:33:45.290770+0000","last_peered":"2024-09-10T11:33:45.290770+0000","last_clean":"2024-09-10T11:33:45.290770+0000","last_became_active":"2024-09-10T11:33:43.214277+0000","last_became_peered":"2024-09-10T11:33:43.214277+0000","last_unstale":"2024-09-10T11:33:45.290770+0000","last_undegraded":"2024-09-10T11:33:45.290770+0000","last_fullsized":"2024-09-10T11:33:45.290770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:33.121927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291070+0000","last_change":"2024-09-10T11:33:44.241702+0000","last_active":"2024-09-10T11:33:45.291070+0000","last_peered":"2024-09-10T11:33:45.291070+0000","last_clean":"2024-09-10T11:33:45.291070+0000","last_became_active":"2024-09-10T11:33:44.241365+0000","last_became_peered":"2024-09-10T11:33:44.241365+0000","last_unstale":"2024-09-10T11:33:45.291070+0000","last_undegraded":"2024-09-10T11:33:45.291070+0000","last_fullsized":"2024-09-10T11:33:45.291070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:46.461857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290601+0000","last_change":"2024-09-10T11:33:43.211331+0000","last_active":"2024-09-10T11:33:45.290601+0000","last_peered":"2024-09-10T11:33:45.290601+0000","last_clean":"2024-09-10T11:33:45.290601+0000","last_became_active":"2024-09-10T11:33:43.211080+0000","last_became_peered":"2024-09-10T11:33:43.211080+0000","last_unstale":"2024-09-10T11:33:45.290601+0000","last_undegraded":"2024-09-10T11:33:45.290601+0000","last_fullsized":"2024-09-10T11:33:45.290601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:00.236266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290678+0000","last_change":"2024-09-10T11:33:44.243692+0000","last_active":"2024-09-10T11:33:45.290678+0000","last_peered":"2024-09-10T11:33:45.290678+0000","last_clean":"2024-09-10T11:33:45.290678+0000","last_became_active":"2024-09-10T11:33:44.243144+0000","last_became_peered":"2024-09-10T11:33:44.243144+0000","last_unstale":"2024-09-10T11:33:45.290678+0000","last_undegraded":"2024-09-10T11:33:45.290678+0000","last_fullsized":"2024-09-10T11:33:45.290678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:09.299856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239067+0000","last_change":"2024-09-10T11:33:43.215222+0000","last_active":"2024-09-10T11:33:45.239067+0000","last_peered":"2024-09-10T11:33:45.239067+0000","last_clean":"2024-09-10T11:33:45.239067+0000","last_became_active":"2024-09-10T11:33:43.214983+0000","last_became_peered":"2024-09-10T11:33:43.214983+0000","last_unstale":"2024-09-10T11:33:45.239067+0000","last_undegraded":"2024-09-10T11:33:45.239067+0000","last_fullsized":"2024-09-10T11:33:45.239067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:50.688965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228008+0000","last_change":"2024-09-10T11:33:44.241195+0000","last_active":"2024-09-10T11:33:45.228008+0000","last_peered":"2024-09-10T11:33:45.228008+0000","last_clean":"2024-09-10T11:33:45.228008+0000","last_became_active":"2024-09-10T11:33:44.240917+0000","last_became_peered":"2024-09-10T11:33:44.240917+0000","last_unstale":"2024-09-10T11:33:45.228008+0000","last_undegraded":"2024-09-10T11:33:45.228008+0000","last_fullsized":"2024-09-10T11:33:45.228008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:27.576400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290975+0000","last_change":"2024-09-10T11:33:43.216249+0000","last_active":"2024-09-10T11:33:45.290975+0000","last_peered":"2024-09-10T11:33:45.290975+0000","last_clean":"2024-09-10T11:33:45.290975+0000","last_became_active":"2024-09-10T11:33:43.216029+0000","last_became_peered":"2024-09-10T11:33:43.216029+0000","last_unstale":"2024-09-10T11:33:45.290975+0000","last_undegraded":"2024-09-10T11:33:45.290975+0000","last_fullsized":"2024-09-10T11:33:45.290975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.458052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238739+0000","last_change":"2024-09-10T11:33:44.244203+0000","last_active":"2024-09-10T11:33:45.238739+0000","last_peered":"2024-09-10T11:33:45.238739+0000","last_clean":"2024-09-10T11:33:45.238739+0000","last_became_active":"2024-09-10T11:33:44.243781+0000","last_became_peered":"2024-09-10T11:33:44.243781+0000","last_unstale":"2024-09-10T11:33:45.238739+0000","last_undegraded":"2024-09-10T11:33:45.238739+0000","last_fullsized":"2024-09-10T11:33:45.238739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:31.825081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228739+0000","last_change":"2024-09-10T11:33:43.214491+0000","last_active":"2024-09-10T11:33:45.228739+0000","last_peered":"2024-09-10T11:33:45.228739+0000","last_clean":"2024-09-10T11:33:45.228739+0000","last_became_active":"2024-09-10T11:33:43.214224+0000","last_became_peered":"2024-09-10T11:33:43.214224+0000","last_unstale":"2024-09-10T11:33:45.228739+0000","last_undegraded":"2024-09-10T11:33:45.228739+0000","last_fullsized":"2024-09-10T11:33:45.228739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291098+0000","last_change":"2024-09-10T11:33:44.239174+0000","last_active":"2024-09-10T11:33:45.291098+0000","last_peered":"2024-09-10T11:33:45.291098+0000","last_clean":"2024-09-10T11:33:45.291098+0000","last_became_active":"2024-09-10T11:33:44.238615+0000","last_became_peered":"2024-09-10T11:33:44.238615+0000","last_unstale":"2024-09-10T11:33:45.291098+0000","last_undegraded":"2024-09-10T11:33:45.291098+0000","last_fullsized":"2024-09-10T11:33:45.291098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:49.193572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227891+0000","last_change":"2024-09-10T11:33:43.215090+0000","last_active":"2024-09-10T11:33:45.227891+0000","last_peered":"2024-09-10T11:33:45.227891+0000","last_clean":"2024-09-10T11:33:45.227891+0000","last_became_active":"2024-09-10T11:33:43.214670+0000","last_became_peered":"2024-09-10T11:33:43.214670+0000","last_unstale":"2024-09-10T11:33:45.227891+0000","last_undegraded":"2024-09-10T11:33:45.227891+0000","last_fullsized":"2024-09-10T11:33:45.227891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.903660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227649+0000","last_change":"2024-09-10T11:33:44.237216+0000","last_active":"2024-09-10T11:33:45.227649+0000","last_peered":"2024-09-10T11:33:45.227649+0000","last_clean":"2024-09-10T11:33:45.227649+0000","last_became_active":"2024-09-10T11:33:44.236963+0000","last_became_peered":"2024-09-10T11:33:44.236963+0000","last_unstale":"2024-09-10T11:33:45.227649+0000","last_undegraded":"2024-09-10T11:33:45.227649+0000","last_fullsized":"2024-09-10T11:33:45.227649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:27.799466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228474+0000","last_change":"2024-09-10T11:33:43.215568+0000","last_active":"2024-09-10T11:33:45.228474+0000","last_peered":"2024-09-10T11:33:45.228474+0000","last_clean":"2024-09-10T11:33:45.228474+0000","last_became_active":"2024-09-10T11:33:43.214417+0000","last_became_peered":"2024-09-10T11:33:43.214417+0000","last_unstale":"2024-09-10T11:33:45.228474+0000","last_undegraded":"2024-09-10T11:33:45.228474+0000","last_fullsized":"2024-09-10T11:33:45.228474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:51.065071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228126+0000","last_change":"2024-09-10T11:33:44.246719+0000","last_active":"2024-09-10T11:33:45.228126+0000","last_peered":"2024-09-10T11:33:45.228126+0000","last_clean":"2024-09-10T11:33:45.228126+0000","last_became_active":"2024-09-10T11:33:44.245977+0000","last_became_peered":"2024-09-10T11:33:44.245977+0000","last_unstale":"2024-09-10T11:33:45.228126+0000","last_undegraded":"2024-09-10T11:33:45.228126+0000","last_fullsized":"2024-09-10T11:33:45.228126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:08.809966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228408+0000","last_change":"2024-09-10T11:33:43.208420+0000","last_active":"2024-09-10T11:33:45.228408+0000","last_peered":"2024-09-10T11:33:45.228408+0000","last_clean":"2024-09-10T11:33:45.228408+0000","last_became_active":"2024-09-10T11:33:43.207842+0000","last_became_peered":"2024-09-10T11:33:43.207842+0000","last_unstale":"2024-09-10T11:33:45.228408+0000","last_undegraded":"2024-09-10T11:33:45.228408+0000","last_fullsized":"2024-09-10T11:33:45.228408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:30.741183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:44.234980+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:44.234730+0000","last_became_peered":"2024-09-10T11:33:44.234730+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:14.462937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239363+0000","last_change":"2024-09-10T11:33:43.210937+0000","last_active":"2024-09-10T11:33:45.239363+0000","last_peered":"2024-09-10T11:33:45.239363+0000","last_clean":"2024-09-10T11:33:45.239363+0000","last_became_active":"2024-09-10T11:33:43.210245+0000","last_became_peered":"2024-09-10T11:33:43.210245+0000","last_unstale":"2024-09-10T11:33:45.239363+0000","last_undegraded":"2024-09-10T11:33:45.239363+0000","last_fullsized":"2024-09-10T11:33:45.239363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:37.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291170+0000","last_change":"2024-09-10T11:33:44.232525+0000","last_active":"2024-09-10T11:33:45.291170+0000","last_peered":"2024-09-10T11:33:45.291170+0000","last_clean":"2024-09-10T11:33:45.291170+0000","last_became_active":"2024-09-10T11:33:44.232277+0000","last_became_peered":"2024-09-10T11:33:44.232277+0000","last_unstale":"2024-09-10T11:33:45.291170+0000","last_undegraded":"2024-09-10T11:33:45.291170+0000","last_fullsized":"2024-09-10T11:33:45.291170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:42.486387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290695+0000","last_change":"2024-09-10T11:33:43.214840+0000","last_active":"2024-09-10T11:33:45.290695+0000","last_peered":"2024-09-10T11:33:45.290695+0000","last_clean":"2024-09-10T11:33:45.290695+0000","last_became_active":"2024-09-10T11:33:43.214617+0000","last_became_peered":"2024-09-10T11:33:43.214617+0000","last_unstale":"2024-09-10T11:33:45.290695+0000","last_undegraded":"2024-09-10T11:33:45.290695+0000","last_fullsized":"2024-09-10T11:33:45.290695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:22.268583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239165+0000","last_change":"2024-09-10T11:33:44.245910+0000","last_active":"2024-09-10T11:33:45.239165+0000","last_peered":"2024-09-10T11:33:45.239165+0000","last_clean":"2024-09-10T11:33:45.239165+0000","last_became_active":"2024-09-10T11:33:44.245601+0000","last_became_peered":"2024-09-10T11:33:44.245601+0000","last_unstale":"2024-09-10T11:33:45.239165+0000","last_undegraded":"2024-09-10T11:33:45.239165+0000","last_fullsized":"2024-09-10T11:33:45.239165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:47.170771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228571+0000","last_change":"2024-09-10T11:33:43.209367+0000","last_active":"2024-09-10T11:33:45.228571+0000","last_peered":"2024-09-10T11:33:45.228571+0000","last_clean":"2024-09-10T11:33:45.228571+0000","last_became_active":"2024-09-10T11:33:43.209057+0000","last_became_peered":"2024-09-10T11:33:43.209057+0000","last_unstale":"2024-09-10T11:33:45.228571+0000","last_undegraded":"2024-09-10T11:33:45.228571+0000","last_fullsized":"2024-09-10T11:33:45.228571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:59.128934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291260+0000","last_change":"2024-09-10T11:33:44.240524+0000","last_active":"2024-09-10T11:33:45.291260+0000","last_peered":"2024-09-10T11:33:45.291260+0000","last_clean":"2024-09-10T11:33:45.291260+0000","last_became_active":"2024-09-10T11:33:44.238970+0000","last_became_peered":"2024-09-10T11:33:44.238970+0000","last_unstale":"2024-09-10T11:33:45.291260+0000","last_undegraded":"2024-09-10T11:33:45.291260+0000","last_fullsized":"2024-09-10T11:33:45.291260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:18.018206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228609+0000","last_change":"2024-09-10T11:33:43.208981+0000","last_active":"2024-09-10T11:33:45.228609+0000","last_peered":"2024-09-10T11:33:45.228609+0000","last_clean":"2024-09-10T11:33:45.228609+0000","last_became_active":"2024-09-10T11:33:43.208559+0000","last_became_peered":"2024-09-10T11:33:43.208559+0000","last_unstale":"2024-09-10T11:33:45.228609+0000","last_undegraded":"2024-09-10T11:33:45.228609+0000","last_fullsized":"2024-09-10T11:33:45.228609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:03.592321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290819+0000","last_change":"2024-09-10T11:33:44.239690+0000","last_active":"2024-09-10T11:33:45.290819+0000","last_peered":"2024-09-10T11:33:45.290819+0000","last_clean":"2024-09-10T11:33:45.290819+0000","last_became_active":"2024-09-10T11:33:44.239401+0000","last_became_peered":"2024-09-10T11:33:44.239401+0000","last_unstale":"2024-09-10T11:33:45.290819+0000","last_undegraded":"2024-09-10T11:33:45.290819+0000","last_fullsized":"2024-09-10T11:33:45.290819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.699528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227762+0000","last_change":"2024-09-10T11:33:43.209064+0000","last_active":"2024-09-10T11:33:45.227762+0000","last_peered":"2024-09-10T11:33:45.227762+0000","last_clean":"2024-09-10T11:33:45.227762+0000","last_became_active":"2024-09-10T11:33:43.208043+0000","last_became_peered":"2024-09-10T11:33:43.208043+0000","last_unstale":"2024-09-10T11:33:45.227762+0000","last_undegraded":"2024-09-10T11:33:45.227762+0000","last_fullsized":"2024-09-10T11:33:45.227762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:45.382171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228074+0000","last_change":"2024-09-10T11:33:44.237793+0000","last_active":"2024-09-10T11:33:45.228074+0000","last_peered":"2024-09-10T11:33:45.228074+0000","last_clean":"2024-09-10T11:33:45.228074+0000","last_became_active":"2024-09-10T11:33:44.237536+0000","last_became_peered":"2024-09-10T11:33:44.237536+0000","last_unstale":"2024-09-10T11:33:45.228074+0000","last_undegraded":"2024-09-10T11:33:45.228074+0000","last_fullsized":"2024-09-10T11:33:45.228074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:14.390264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:43.209065+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:43.208821+0000","last_became_peered":"2024-09-10T11:33:43.208821+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:15.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291354+0000","last_change":"2024-09-10T11:33:44.235181+0000","last_active":"2024-09-10T11:33:45.291354+0000","last_peered":"2024-09-10T11:33:45.291354+0000","last_clean":"2024-09-10T11:33:45.291354+0000","last_became_active":"2024-09-10T11:33:44.234939+0000","last_became_peered":"2024-09-10T11:33:44.234939+0000","last_unstale":"2024-09-10T11:33:45.291354+0000","last_undegraded":"2024-09-10T11:33:45.291354+0000","last_fullsized":"2024-09-10T11:33:45.291354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:54.993335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228038+0000","last_change":"2024-09-10T11:33:43.208219+0000","last_active":"2024-09-10T11:33:45.228038+0000","last_peered":"2024-09-10T11:33:45.228038+0000","last_clean":"2024-09-10T11:33:45.228038+0000","last_became_active":"2024-09-10T11:33:43.207992+0000","last_became_peered":"2024-09-10T11:33:43.207992+0000","last_unstale":"2024-09-10T11:33:45.228038+0000","last_undegraded":"2024-09-10T11:33:45.228038+0000","last_fullsized":"2024-09-10T11:33:45.228038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:45.628443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290880+0000","last_change":"2024-09-10T11:33:44.236199+0000","last_active":"2024-09-10T11:33:45.290880+0000","last_peered":"2024-09-10T11:33:45.290880+0000","last_clean":"2024-09-10T11:33:45.290880+0000","last_became_active":"2024-09-10T11:33:44.235961+0000","last_became_peered":"2024-09-10T11:33:44.235961+0000","last_unstale":"2024-09-10T11:33:45.290880+0000","last_undegraded":"2024-09-10T11:33:45.290880+0000","last_fullsized":"2024-09-10T11:33:45.290880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.797536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227944+0000","last_change":"2024-09-10T11:33:43.214900+0000","last_active":"2024-09-10T11:33:45.227944+0000","last_peered":"2024-09-10T11:33:45.227944+0000","last_clean":"2024-09-10T11:33:45.227944+0000","last_became_active":"2024-09-10T11:33:43.214372+0000","last_became_peered":"2024-09-10T11:33:43.214372+0000","last_unstale":"2024-09-10T11:33:45.227944+0000","last_undegraded":"2024-09-10T11:33:45.227944+0000","last_fullsized":"2024-09-10T11:33:45.227944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.940181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290859+0000","last_change":"2024-09-10T11:33:44.242982+0000","last_active":"2024-09-10T11:33:45.290859+0000","last_peered":"2024-09-10T11:33:45.290859+0000","last_clean":"2024-09-10T11:33:45.290859+0000","last_became_active":"2024-09-10T11:33:44.242310+0000","last_became_peered":"2024-09-10T11:33:44.242310+0000","last_unstale":"2024-09-10T11:33:45.290859+0000","last_undegraded":"2024-09-10T11:33:45.290859+0000","last_fullsized":"2024-09-10T11:33:45.290859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:38.992677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239252+0000","last_change":"2024-09-10T11:33:43.207140+0000","last_active":"2024-09-10T11:33:45.239252+0000","last_peered":"2024-09-10T11:33:45.239252+0000","last_clean":"2024-09-10T11:33:45.239252+0000","last_became_active":"2024-09-10T11:33:43.206926+0000","last_became_peered":"2024-09-10T11:33:43.206926+0000","last_unstale":"2024-09-10T11:33:45.239252+0000","last_undegraded":"2024-09-10T11:33:45.239252+0000","last_fullsized":"2024-09-10T11:33:45.239252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:16.821363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290467+0000","last_change":"2024-09-10T11:33:44.236919+0000","last_active":"2024-09-10T11:33:45.290467+0000","last_peered":"2024-09-10T11:33:45.290467+0000","last_clean":"2024-09-10T11:33:45.290467+0000","last_became_active":"2024-09-10T11:33:44.236683+0000","last_became_peered":"2024-09-10T11:33:44.236683+0000","last_unstale":"2024-09-10T11:33:45.290467+0000","last_undegraded":"2024-09-10T11:33:45.290467+0000","last_fullsized":"2024-09-10T11:33:45.290467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:48.375628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291086+0000","last_change":"2024-09-10T11:33:43.212385+0000","last_active":"2024-09-10T11:33:45.291086+0000","last_peered":"2024-09-10T11:33:45.291086+0000","last_clean":"2024-09-10T11:33:45.291086+0000","last_became_active":"2024-09-10T11:33:43.212163+0000","last_became_peered":"2024-09-10T11:33:43.212163+0000","last_unstale":"2024-09-10T11:33:45.291086+0000","last_undegraded":"2024-09-10T11:33:45.291086+0000","last_fullsized":"2024-09-10T11:33:45.291086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:58.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291084+0000","last_change":"2024-09-10T11:33:44.244452+0000","last_active":"2024-09-10T11:33:45.291084+0000","last_peered":"2024-09-10T11:33:45.291084+0000","last_clean":"2024-09-10T11:33:45.291084+0000","last_became_active":"2024-09-10T11:33:44.244228+0000","last_became_peered":"2024-09-10T11:33:44.244228+0000","last_unstale":"2024-09-10T11:33:45.291084+0000","last_undegraded":"2024-09-10T11:33:45.291084+0000","last_fullsized":"2024-09-10T11:33:45.291084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:03.316534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291265+0000","last_change":"2024-09-10T11:33:43.213782+0000","last_active":"2024-09-10T11:33:45.291265+0000","last_peered":"2024-09-10T11:33:45.291265+0000","last_clean":"2024-09-10T11:33:45.291265+0000","last_became_active":"2024-09-10T11:33:43.213512+0000","last_became_peered":"2024-09-10T11:33:43.213512+0000","last_unstale":"2024-09-10T11:33:45.291265+0000","last_undegraded":"2024-09-10T11:33:45.291265+0000","last_fullsized":"2024-09-10T11:33:45.291265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:54.809490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290922+0000","last_change":"2024-09-10T11:33:44.243474+0000","last_active":"2024-09-10T11:33:45.290922+0000","last_peered":"2024-09-10T11:33:45.290922+0000","last_clean":"2024-09-10T11:33:45.290922+0000","last_became_active":"2024-09-10T11:33:44.243212+0000","last_became_peered":"2024-09-10T11:33:44.243212+0000","last_unstale":"2024-09-10T11:33:45.290922+0000","last_undegraded":"2024-09-10T11:33:45.290922+0000","last_fullsized":"2024-09-10T11:33:45.290922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:06.229999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291286+0000","last_change":"2024-09-10T11:33:43.214433+0000","last_active":"2024-09-10T11:33:45.291286+0000","last_peered":"2024-09-10T11:33:45.291286+0000","last_clean":"2024-09-10T11:33:45.291286+0000","last_became_active":"2024-09-10T11:33:43.214060+0000","last_became_peered":"2024-09-10T11:33:43.214060+0000","last_unstale":"2024-09-10T11:33:45.291286+0000","last_undegraded":"2024-09-10T11:33:45.291286+0000","last_fullsized":"2024-09-10T11:33:45.291286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:29.435158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227694+0000","last_change":"2024-09-10T11:33:44.236573+0000","last_active":"2024-09-10T11:33:45.227694+0000","last_peered":"2024-09-10T11:33:45.227694+0000","last_clean":"2024-09-10T11:33:45.227694+0000","last_became_active":"2024-09-10T11:33:44.236279+0000","last_became_peered":"2024-09-10T11:33:44.236279+0000","last_unstale":"2024-09-10T11:33:45.227694+0000","last_undegraded":"2024-09-10T11:33:45.227694+0000","last_fullsized":"2024-09-10T11:33:45.227694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:16.865227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291043+0000","last_change":"2024-09-10T11:33:43.212318+0000","last_active":"2024-09-10T11:33:45.291043+0000","last_peered":"2024-09-10T11:33:45.291043+0000","last_clean":"2024-09-10T11:33:45.291043+0000","last_became_active":"2024-09-10T11:33:43.212105+0000","last_became_peered":"2024-09-10T11:33:43.212105+0000","last_unstale":"2024-09-10T11:33:45.291043+0000","last_undegraded":"2024-09-10T11:33:45.291043+0000","last_fullsized":"2024-09-10T11:33:45.291043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:32.124944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227542+0000","last_change":"2024-09-10T11:33:44.241022+0000","last_active":"2024-09-10T11:33:45.227542+0000","last_peered":"2024-09-10T11:33:45.227542+0000","last_clean":"2024-09-10T11:33:45.227542+0000","last_became_active":"2024-09-10T11:33:44.240784+0000","last_became_peered":"2024-09-10T11:33:44.240784+0000","last_unstale":"2024-09-10T11:33:45.227542+0000","last_undegraded":"2024-09-10T11:33:45.227542+0000","last_fullsized":"2024-09-10T11:33:45.227542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:07.362654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291165+0000","last_change":"2024-09-10T11:33:43.209703+0000","last_active":"2024-09-10T11:33:45.291165+0000","last_peered":"2024-09-10T11:33:45.291165+0000","last_clean":"2024-09-10T11:33:45.291165+0000","last_became_active":"2024-09-10T11:33:43.209476+0000","last_became_peered":"2024-09-10T11:33:43.209476+0000","last_unstale":"2024-09-10T11:33:45.291165+0000","last_undegraded":"2024-09-10T11:33:45.291165+0000","last_fullsized":"2024-09-10T11:33:45.291165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:17.804493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228263+0000","last_change":"2024-09-10T11:33:44.242827+0000","last_active":"2024-09-10T11:33:45.228263+0000","last_peered":"2024-09-10T11:33:45.228263+0000","last_clean":"2024-09-10T11:33:45.228263+0000","last_became_active":"2024-09-10T11:33:44.242441+0000","last_became_peered":"2024-09-10T11:33:44.242441+0000","last_unstale":"2024-09-10T11:33:45.228263+0000","last_undegraded":"2024-09-10T11:33:45.228263+0000","last_fullsized":"2024-09-10T11:33:45.228263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:16.886176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290823+0000","last_change":"2024-09-10T11:33:43.211590+0000","last_active":"2024-09-10T11:33:45.290823+0000","last_peered":"2024-09-10T11:33:45.290823+0000","last_clean":"2024-09-10T11:33:45.290823+0000","last_became_active":"2024-09-10T11:33:43.211064+0000","last_became_peered":"2024-09-10T11:33:43.211064+0000","last_unstale":"2024-09-10T11:33:45.290823+0000","last_undegraded":"2024-09-10T11:33:45.290823+0000","last_fullsized":"2024-09-10T11:33:45.290823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:54.069449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291445+0000","last_change":"2024-09-10T11:33:44.233490+0000","last_active":"2024-09-10T11:33:45.291445+0000","last_peered":"2024-09-10T11:33:45.291445+0000","last_clean":"2024-09-10T11:33:45.291445+0000","last_became_active":"2024-09-10T11:33:44.233246+0000","last_became_peered":"2024-09-10T11:33:44.233246+0000","last_unstale":"2024-09-10T11:33:45.291445+0000","last_undegraded":"2024-09-10T11:33:45.291445+0000","last_fullsized":"2024-09-10T11:33:45.291445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:09.713804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238831+0000","last_change":"2024-09-10T11:33:43.208823+0000","last_active":"2024-09-10T11:33:45.238831+0000","last_peered":"2024-09-10T11:33:45.238831+0000","last_clean":"2024-09-10T11:33:45.238831+0000","last_became_active":"2024-09-10T11:33:43.207920+0000","last_became_peered":"2024-09-10T11:33:43.207920+0000","last_unstale":"2024-09-10T11:33:45.238831+0000","last_undegraded":"2024-09-10T11:33:45.238831+0000","last_fullsized":"2024-09-10T11:33:45.238831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:01.523833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227887+0000","last_change":"2024-09-10T11:33:44.236550+0000","last_active":"2024-09-10T11:33:45.227887+0000","last_peered":"2024-09-10T11:33:45.227887+0000","last_clean":"2024-09-10T11:33:45.227887+0000","last_became_active":"2024-09-10T11:33:44.236263+0000","last_became_peered":"2024-09-10T11:33:44.236263+0000","last_unstale":"2024-09-10T11:33:45.227887+0000","last_undegraded":"2024-09-10T11:33:45.227887+0000","last_fullsized":"2024-09-10T11:33:45.227887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:42.113034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239705+0000","last_change":"2024-09-10T11:33:43.211106+0000","last_active":"2024-09-10T11:33:45.239705+0000","last_peered":"2024-09-10T11:33:45.239705+0000","last_clean":"2024-09-10T11:33:45.239705+0000","last_became_active":"2024-09-10T11:33:43.210871+0000","last_became_peered":"2024-09-10T11:33:43.210871+0000","last_unstale":"2024-09-10T11:33:45.239705+0000","last_undegraded":"2024-09-10T11:33:45.239705+0000","last_fullsized":"2024-09-10T11:33:45.239705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:24.490287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290990+0000","last_change":"2024-09-10T11:33:44.229784+0000","last_active":"2024-09-10T11:33:45.290990+0000","last_peered":"2024-09-10T11:33:45.290990+0000","last_clean":"2024-09-10T11:33:45.290990+0000","last_became_active":"2024-09-10T11:33:44.229536+0000","last_became_peered":"2024-09-10T11:33:44.229536+0000","last_unstale":"2024-09-10T11:33:45.290990+0000","last_undegraded":"2024-09-10T11:33:45.290990+0000","last_fullsized":"2024-09-10T11:33:45.290990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:21.289170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228318+0000","last_change":"2024-09-10T11:33:43.207910+0000","last_active":"2024-09-10T11:33:45.228318+0000","last_peered":"2024-09-10T11:33:45.228318+0000","last_clean":"2024-09-10T11:33:45.228318+0000","last_became_active":"2024-09-10T11:33:43.207656+0000","last_became_peered":"2024-09-10T11:33:43.207656+0000","last_unstale":"2024-09-10T11:33:45.228318+0000","last_undegraded":"2024-09-10T11:33:45.228318+0000","last_fullsized":"2024-09-10T11:33:45.228318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:05.578247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:44.233670+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:44.233434+0000","last_became_peered":"2024-09-10T11:33:44.233434+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:22.761180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228522+0000","last_change":"2024-09-10T11:33:43.208706+0000","last_active":"2024-09-10T11:33:45.228522+0000","last_peered":"2024-09-10T11:33:45.228522+0000","last_clean":"2024-09-10T11:33:45.228522+0000","last_became_active":"2024-09-10T11:33:43.208169+0000","last_became_peered":"2024-09-10T11:33:43.208169+0000","last_unstale":"2024-09-10T11:33:45.228522+0000","last_undegraded":"2024-09-10T11:33:45.228522+0000","last_fullsized":"2024-09-10T11:33:45.228522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:08.272400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291342+0000","last_change":"2024-09-10T11:33:44.238629+0000","last_active":"2024-09-10T11:33:45.291342+0000","last_peered":"2024-09-10T11:33:45.291342+0000","last_clean":"2024-09-10T11:33:45.291342+0000","last_became_active":"2024-09-10T11:33:44.238034+0000","last_became_peered":"2024-09-10T11:33:44.238034+0000","last_unstale":"2024-09-10T11:33:45.291342+0000","last_undegraded":"2024-09-10T11:33:45.291342+0000","last_fullsized":"2024-09-10T11:33:45.291342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:32.096230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227989+0000","last_change":"2024-09-10T11:33:43.211173+0000","last_active":"2024-09-10T11:33:45.227989+0000","last_peered":"2024-09-10T11:33:45.227989+0000","last_clean":"2024-09-10T11:33:45.227989+0000","last_became_active":"2024-09-10T11:33:43.210941+0000","last_became_peered":"2024-09-10T11:33:43.210941+0000","last_unstale":"2024-09-10T11:33:45.227989+0000","last_undegraded":"2024-09-10T11:33:45.227989+0000","last_fullsized":"2024-09-10T11:33:45.227989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:43.320066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239297+0000","last_change":"2024-09-10T11:33:44.241685+0000","last_active":"2024-09-10T11:33:45.239297+0000","last_peered":"2024-09-10T11:33:45.239297+0000","last_clean":"2024-09-10T11:33:45.239297+0000","last_became_active":"2024-09-10T11:33:44.240898+0000","last_became_peered":"2024-09-10T11:33:44.240898+0000","last_unstale":"2024-09-10T11:33:45.239297+0000","last_undegraded":"2024-09-10T11:33:45.239297+0000","last_fullsized":"2024-09-10T11:33:45.239297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:27.807759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291403+0000","last_change":"2024-09-10T11:33:43.214265+0000","last_active":"2024-09-10T11:33:45.291403+0000","last_peered":"2024-09-10T11:33:45.291403+0000","last_clean":"2024-09-10T11:33:45.291403+0000","last_became_active":"2024-09-10T11:33:43.214039+0000","last_became_peered":"2024-09-10T11:33:43.214039+0000","last_unstale":"2024-09-10T11:33:45.291403+0000","last_undegraded":"2024-09-10T11:33:45.291403+0000","last_fullsized":"2024-09-10T11:33:45.291403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:02.134330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227541+0000","last_change":"2024-09-10T11:33:44.229449+0000","last_active":"2024-09-10T11:33:45.227541+0000","last_peered":"2024-09-10T11:33:45.227541+0000","last_clean":"2024-09-10T11:33:45.227541+0000","last_became_active":"2024-09-10T11:33:44.229187+0000","last_became_peered":"2024-09-10T11:33:44.229187+0000","last_unstale":"2024-09-10T11:33:45.227541+0000","last_undegraded":"2024-09-10T11:33:45.227541+0000","last_fullsized":"2024-09-10T11:33:45.227541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:34.466978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290649+0000","last_change":"2024-09-10T11:33:43.216018+0000","last_active":"2024-09-10T11:33:45.290649+0000","last_peered":"2024-09-10T11:33:45.290649+0000","last_clean":"2024-09-10T11:33:45.290649+0000","last_became_active":"2024-09-10T11:33:43.215806+0000","last_became_peered":"2024-09-10T11:33:43.215806+0000","last_unstale":"2024-09-10T11:33:45.290649+0000","last_undegraded":"2024-09-10T11:33:45.290649+0000","last_fullsized":"2024-09-10T11:33:45.290649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290554+0000","last_change":"2024-09-10T11:33:44.233598+0000","last_active":"2024-09-10T11:33:45.290554+0000","last_peered":"2024-09-10T11:33:45.290554+0000","last_clean":"2024-09-10T11:33:45.290554+0000","last_became_active":"2024-09-10T11:33:44.233341+0000","last_became_peered":"2024-09-10T11:33:44.233341+0000","last_unstale":"2024-09-10T11:33:45.290554+0000","last_undegraded":"2024-09-10T11:33:45.290554+0000","last_fullsized":"2024-09-10T11:33:45.290554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:18.365927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291135+0000","last_change":"2024-09-10T11:33:43.215353+0000","last_active":"2024-09-10T11:33:45.291135+0000","last_peered":"2024-09-10T11:33:45.291135+0000","last_clean":"2024-09-10T11:33:45.291135+0000","last_became_active":"2024-09-10T11:33:43.215133+0000","last_became_peered":"2024-09-10T11:33:43.215133+0000","last_unstale":"2024-09-10T11:33:45.291135+0000","last_undegraded":"2024-09-10T11:33:45.291135+0000","last_fullsized":"2024-09-10T11:33:45.291135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:29.332573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290774+0000","last_change":"2024-09-10T11:33:44.236979+0000","last_active":"2024-09-10T11:33:45.290774+0000","last_peered":"2024-09-10T11:33:45.290774+0000","last_clean":"2024-09-10T11:33:45.290774+0000","last_became_active":"2024-09-10T11:33:44.236759+0000","last_became_peered":"2024-09-10T11:33:44.236759+0000","last_unstale":"2024-09-10T11:33:45.290774+0000","last_undegraded":"2024-09-10T11:33:45.290774+0000","last_fullsized":"2024-09-10T11:33:45.290774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:41.737701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227994+0000","last_change":"2024-09-10T11:33:43.208935+0000","last_active":"2024-09-10T11:33:45.227994+0000","last_peered":"2024-09-10T11:33:45.227994+0000","last_clean":"2024-09-10T11:33:45.227994+0000","last_became_active":"2024-09-10T11:33:43.208470+0000","last_became_peered":"2024-09-10T11:33:43.208470+0000","last_unstale":"2024-09-10T11:33:45.227994+0000","last_undegraded":"2024-09-10T11:33:45.227994+0000","last_fullsized":"2024-09-10T11:33:45.227994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:51.574030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":300,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227809+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:33:45.227809+0000","last_peered":"2024-09-10T11:33:45.227809+0000","last_clean":"2024-09-10T11:33:45.227809+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:33:45.227809+0000","last_undegraded":"2024-09-10T11:33:45.227809+0000","last_fullsized":"2024-09-10T11:33:45.227809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:27.070388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199547911,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194870,"statfs":{"total":200000000,"available":199547877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1016,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194295,"statfs":{"total":200000000,"available":198959065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548829,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549397,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199548941,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1016,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194296,"statfs":{"total":200000000,"available":198959155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":439,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549453,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:33:47.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:33:47.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.299+0000 7f46c9db9640 1 Processor -- start 2024-09-10T11:33:47.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.299+0000 7f46c9db9640 1 -- start start 2024-09-10T11:33:47.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c9db9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46c4148620 0x7f46c40b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c9db9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c406f0d0 0x7f46c4162d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c9db9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c40b38a0 con 0x7f46c4162910 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c9db9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f46c41632c0 con 0x7f46c4148620 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c9db9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46c406dc30 con 0x7f46c406f0d0 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46c4148620 0x7f46c40b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f46c4148620 0x7f46c40b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43718/0 (socket says 172.21.15.99:43718) 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 -- 172.21.15.99:0/4143258189 learned_addr learned my addr 172.21.15.99:0/4143258189 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 -- 172.21.15.99:0/4143258189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c406f0d0 msgr2=0x7f46c4162d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 --2- 172.21.15.99:0/4143258189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c406f0d0 0x7f46c4162d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 -- 172.21.15.99:0/4143258189 >> v1:172.21.15.196:6789/0 conn(0x7f46c4162910 legacy=0x7f46c40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.300+0000 7f46c37fe640 1 -- 172.21.15.99:0/4143258189 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c40d00c0 con 0x7f46c4148620 2024-09-10T11:33:47.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c37fe640 1 --2- 172.21.15.99:0/4143258189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4148620 0x7f46c40b5ac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009870 tx=0x7f46b402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58146de6ea265ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c27fc640 1 -- 172.21.15.99:0/4143258189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4037070 con 0x7f46c4148620 2024-09-10T11:33:47.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c27fc640 1 -- 172.21.15.99:0/4143258189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b4032440 con 0x7f46c4148620 2024-09-10T11:33:47.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c27fc640 1 -- 172.21.15.99:0/4143258189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4032810 con 0x7f46c4148620 2024-09-10T11:33:47.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 -- 172.21.15.99:0/4143258189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4148620 msgr2=0x7f46c40b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 --2- 172.21.15.99:0/4143258189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4148620 0x7f46c40b5ac0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009870 tx=0x7f46b402a210 comp rx=0 tx=0).stop 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 -- 172.21.15.99:0/4143258189 shutdown_connections 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 --2- 172.21.15.99:0/4143258189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c406f0d0 0x7f46c4162d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 --2- 172.21.15.99:0/4143258189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4148620 0x7f46c40b5ac0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 -- 172.21.15.99:0/4143258189 >> 172.21.15.99:0/4143258189 conn(0x7f46c40b2a80 msgr2=0x7f46c40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 -- 172.21.15.99:0/4143258189 shutdown_connections 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.301+0000 7f46c9db9640 1 -- 172.21.15.99:0/4143258189 wait complete. 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 Processor -- start 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 -- start start 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 0x7f46c409cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 0x7f46c409d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46c406d8c0 con 0x7f46c4162910 2024-09-10T11:33:47.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f46c40d0680 con 0x7f46c406f0d0 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.303+0000 7f46c9db9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f46c40b6000 con 0x7f46c4148620 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43734/0 (socket says 172.21.15.99:43734) 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 -- 172.21.15.99:0/3226540582 learned_addr learned my addr 172.21.15.99:0/3226540582 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c37fe640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 0x7f46c409cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c2ffd640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 0x7f46c409d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 -- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 msgr2=0x7f46c409d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 0x7f46c409d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 -- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 msgr2=0x7f46c409cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 0x7f46c409cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 -- 172.21.15.99:0/3226540582 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c409bb60 con 0x7f46c4162910 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c37fe640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 0x7f46c409cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:47.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c2ffd640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 0x7f46c409d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c3fff640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b9e0 tx=0x7f46bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc019070 con 0x7f46c4162910 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc00cb20 con 0x7f46c4162910 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc014440 con 0x7f46c4162910 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c409be40 con 0x7f46c4162910 2024-09-10T11:33:47.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.304+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c409c400 con 0x7f46c4162910 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.305+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c4148a00 con 0x7f46c4162910 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.306+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46bc01c070 con 0x7f46c4162910 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.306+0000 7f46abfff640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.306+0000 7f46c37fe640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.306+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f46bc08ab80 con 0x7f46c4162910 2024-09-10T11:33:47.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.306+0000 7f46c37fe640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 0x7f46b006e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.310+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46bc059520 con 0x7f46c4162910 2024-09-10T11:33:47.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.471+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46c4071980 con 0x7f46b006be30 2024-09-10T11:33:47.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.477+0000 7f46abfff640 1 -- 172.21.15.99:0/3226540582 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247590 (secure 0 0 0) 0x7f46c4071980 con 0x7f46b006be30 2024-09-10T11:33:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:47.482 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:33:47.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 msgr2=0x7f46b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 0x7f46b006e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46b4034000 tx=0x7f46b402e000 comp rx=0 tx=0).stop 2024-09-10T11:33:47.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 msgr2=0x7f46c409d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00b9e0 tx=0x7f46bc00bea0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 shutdown_connections 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46c4162910 0x7f46c409d680 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46c4148620 0x7f46c409d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 --2- 172.21.15.99:0/3226540582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46c406f0d0 0x7f46c409cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 >> 172.21.15.99:0/3226540582 conn(0x7f46c40b2a80 msgr2=0x7f46c406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 shutdown_connections 2024-09-10T11:33:47.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.480+0000 7f46c9db9640 1 -- 172.21.15.99:0/3226540582 wait complete. 2024-09-10T11:33:47.492 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-10T11:33:46.950778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4677,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557822,"statfs":{"total":1600000000,"available":1595210628,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.749173"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238775+0000","last_change":"2024-09-10T11:33:44.242903+0000","last_active":"2024-09-10T11:33:45.238775+0000","last_peered":"2024-09-10T11:33:45.238775+0000","last_clean":"2024-09-10T11:33:45.238775+0000","last_became_active":"2024-09-10T11:33:44.242268+0000","last_became_peered":"2024-09-10T11:33:44.242268+0000","last_unstale":"2024-09-10T11:33:45.238775+0000","last_undegraded":"2024-09-10T11:33:45.238775+0000","last_fullsized":"2024-09-10T11:33:45.238775+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:18.258245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290478+0000","last_change":"2024-09-10T11:33:44.233956+0000","last_active":"2024-09-10T11:33:45.290478+0000","last_peered":"2024-09-10T11:33:45.290478+0000","last_clean":"2024-09-10T11:33:45.290478+0000","last_became_active":"2024-09-10T11:33:44.233728+0000","last_became_peered":"2024-09-10T11:33:44.233728+0000","last_unstale":"2024-09-10T11:33:45.290478+0000","last_undegraded":"2024-09-10T11:33:45.290478+0000","last_fullsized":"2024-09-10T11:33:45.290478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:34.016351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290629+0000","last_change":"2024-09-10T11:33:44.234335+0000","last_active":"2024-09-10T11:33:45.290629+0000","last_peered":"2024-09-10T11:33:45.290629+0000","last_clean":"2024-09-10T11:33:45.290629+0000","last_became_active":"2024-09-10T11:33:44.234101+0000","last_became_peered":"2024-09-10T11:33:44.234101+0000","last_unstale":"2024-09-10T11:33:45.290629+0000","last_undegraded":"2024-09-10T11:33:45.290629+0000","last_fullsized":"2024-09-10T11:33:45.290629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:53.013805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290606+0000","last_change":"2024-09-10T11:33:44.232883+0000","last_active":"2024-09-10T11:33:45.290606+0000","last_peered":"2024-09-10T11:33:45.290606+0000","last_clean":"2024-09-10T11:33:45.290606+0000","last_became_active":"2024-09-10T11:33:44.232651+0000","last_became_peered":"2024-09-10T11:33:44.232651+0000","last_unstale":"2024-09-10T11:33:45.290606+0000","last_undegraded":"2024-09-10T11:33:45.290606+0000","last_fullsized":"2024-09-10T11:33:45.290606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:57.683258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291003+0000","last_change":"2024-09-10T11:33:44.242800+0000","last_active":"2024-09-10T11:33:45.291003+0000","last_peered":"2024-09-10T11:33:45.291003+0000","last_clean":"2024-09-10T11:33:45.291003+0000","last_became_active":"2024-09-10T11:33:44.242547+0000","last_became_peered":"2024-09-10T11:33:44.242547+0000","last_unstale":"2024-09-10T11:33:45.291003+0000","last_undegraded":"2024-09-10T11:33:45.291003+0000","last_fullsized":"2024-09-10T11:33:45.291003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:36.669550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290533+0000","last_change":"2024-09-10T11:33:44.243899+0000","last_active":"2024-09-10T11:33:45.290533+0000","last_peered":"2024-09-10T11:33:45.290533+0000","last_clean":"2024-09-10T11:33:45.290533+0000","last_became_active":"2024-09-10T11:33:44.243454+0000","last_became_peered":"2024-09-10T11:33:44.243454+0000","last_unstale":"2024-09-10T11:33:45.290533+0000","last_undegraded":"2024-09-10T11:33:45.290533+0000","last_fullsized":"2024-09-10T11:33:45.290533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:23.514699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227632+0000","last_change":"2024-09-10T11:33:44.241886+0000","last_active":"2024-09-10T11:33:45.227632+0000","last_peered":"2024-09-10T11:33:45.227632+0000","last_clean":"2024-09-10T11:33:45.227632+0000","last_became_active":"2024-09-10T11:33:44.241626+0000","last_became_peered":"2024-09-10T11:33:44.241626+0000","last_unstale":"2024-09-10T11:33:45.227632+0000","last_undegraded":"2024-09-10T11:33:45.227632+0000","last_fullsized":"2024-09-10T11:33:45.227632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:07.198111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238873+0000","last_change":"2024-09-10T11:33:44.244007+0000","last_active":"2024-09-10T11:33:45.238873+0000","last_peered":"2024-09-10T11:33:45.238873+0000","last_clean":"2024-09-10T11:33:45.238873+0000","last_became_active":"2024-09-10T11:33:44.243401+0000","last_became_peered":"2024-09-10T11:33:44.243401+0000","last_unstale":"2024-09-10T11:33:45.238873+0000","last_undegraded":"2024-09-10T11:33:45.238873+0000","last_fullsized":"2024-09-10T11:33:45.238873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:28:26.125164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291107+0000","last_change":"2024-09-10T11:33:44.240028+0000","last_active":"2024-09-10T11:33:45.291107+0000","last_peered":"2024-09-10T11:33:45.291107+0000","last_clean":"2024-09-10T11:33:45.291107+0000","last_became_active":"2024-09-10T11:33:44.239783+0000","last_became_peered":"2024-09-10T11:33:44.239783+0000","last_unstale":"2024-09-10T11:33:45.291107+0000","last_undegraded":"2024-09-10T11:33:45.291107+0000","last_fullsized":"2024-09-10T11:33:45.291107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:09.311640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227927+0000","last_change":"2024-09-10T11:33:44.241780+0000","last_active":"2024-09-10T11:33:45.227927+0000","last_peered":"2024-09-10T11:33:45.227927+0000","last_clean":"2024-09-10T11:33:45.227927+0000","last_became_active":"2024-09-10T11:33:44.241549+0000","last_became_peered":"2024-09-10T11:33:44.241549+0000","last_unstale":"2024-09-10T11:33:45.227927+0000","last_undegraded":"2024-09-10T11:33:45.227927+0000","last_fullsized":"2024-09-10T11:33:45.227927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:46.368268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238974+0000","last_change":"2024-09-10T11:33:44.243102+0000","last_active":"2024-09-10T11:33:45.238974+0000","last_peered":"2024-09-10T11:33:45.238974+0000","last_clean":"2024-09-10T11:33:45.238974+0000","last_became_active":"2024-09-10T11:33:44.242606+0000","last_became_peered":"2024-09-10T11:33:44.242606+0000","last_unstale":"2024-09-10T11:33:45.238974+0000","last_undegraded":"2024-09-10T11:33:45.238974+0000","last_fullsized":"2024-09-10T11:33:45.238974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:30:27.411597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228638+0000","last_change":"2024-09-10T11:33:44.243432+0000","last_active":"2024-09-10T11:33:45.228638+0000","last_peered":"2024-09-10T11:33:45.228638+0000","last_clean":"2024-09-10T11:33:45.228638+0000","last_became_active":"2024-09-10T11:33:44.242750+0000","last_became_peered":"2024-09-10T11:33:44.242750+0000","last_unstale":"2024-09-10T11:33:45.228638+0000","last_undegraded":"2024-09-10T11:33:45.228638+0000","last_fullsized":"2024-09-10T11:33:45.228638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:05.622221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290897+0000","last_change":"2024-09-10T11:33:44.235298+0000","last_active":"2024-09-10T11:33:45.290897+0000","last_peered":"2024-09-10T11:33:45.290897+0000","last_clean":"2024-09-10T11:33:45.290897+0000","last_became_active":"2024-09-10T11:33:44.235037+0000","last_became_peered":"2024-09-10T11:33:44.235037+0000","last_unstale":"2024-09-10T11:33:45.290897+0000","last_undegraded":"2024-09-10T11:33:45.290897+0000","last_fullsized":"2024-09-10T11:33:45.290897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:33.258933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227972+0000","last_change":"2024-09-10T11:33:44.246907+0000","last_active":"2024-09-10T11:33:45.227972+0000","last_peered":"2024-09-10T11:33:45.227972+0000","last_clean":"2024-09-10T11:33:45.227972+0000","last_became_active":"2024-09-10T11:33:44.246484+0000","last_became_peered":"2024-09-10T11:33:44.246484+0000","last_unstale":"2024-09-10T11:33:45.227972+0000","last_undegraded":"2024-09-10T11:33:45.227972+0000","last_fullsized":"2024-09-10T11:33:45.227972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:07.902652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228112+0000","last_change":"2024-09-10T11:33:44.244538+0000","last_active":"2024-09-10T11:33:45.228112+0000","last_peered":"2024-09-10T11:33:45.228112+0000","last_clean":"2024-09-10T11:33:45.228112+0000","last_became_active":"2024-09-10T11:33:44.244273+0000","last_became_peered":"2024-09-10T11:33:44.244273+0000","last_unstale":"2024-09-10T11:33:45.228112+0000","last_undegraded":"2024-09-10T11:33:45.228112+0000","last_fullsized":"2024-09-10T11:33:45.228112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:32.520810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239097+0000","last_change":"2024-09-10T11:33:44.241906+0000","last_active":"2024-09-10T11:33:45.239097+0000","last_peered":"2024-09-10T11:33:45.239097+0000","last_clean":"2024-09-10T11:33:45.239097+0000","last_became_active":"2024-09-10T11:33:44.241221+0000","last_became_peered":"2024-09-10T11:33:44.241221+0000","last_unstale":"2024-09-10T11:33:45.239097+0000","last_undegraded":"2024-09-10T11:33:45.239097+0000","last_fullsized":"2024-09-10T11:33:45.239097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:22.782755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291084+0000","last_change":"2024-09-10T11:33:44.239346+0000","last_active":"2024-09-10T11:33:45.291084+0000","last_peered":"2024-09-10T11:33:45.291084+0000","last_clean":"2024-09-10T11:33:45.291084+0000","last_became_active":"2024-09-10T11:33:44.238914+0000","last_became_peered":"2024-09-10T11:33:44.238914+0000","last_unstale":"2024-09-10T11:33:45.291084+0000","last_undegraded":"2024-09-10T11:33:45.291084+0000","last_fullsized":"2024-09-10T11:33:45.291084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:04.512195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291150+0000","last_change":"2024-09-10T11:33:44.239233+0000","last_active":"2024-09-10T11:33:45.291150+0000","last_peered":"2024-09-10T11:33:45.291150+0000","last_clean":"2024-09-10T11:33:45.291150+0000","last_became_active":"2024-09-10T11:33:44.237759+0000","last_became_peered":"2024-09-10T11:33:44.237759+0000","last_unstale":"2024-09-10T11:33:45.291150+0000","last_undegraded":"2024-09-10T11:33:45.291150+0000","last_fullsized":"2024-09-10T11:33:45.291150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:28.475163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238798+0000","last_change":"2024-09-10T11:33:44.241109+0000","last_active":"2024-09-10T11:33:45.238798+0000","last_peered":"2024-09-10T11:33:45.238798+0000","last_clean":"2024-09-10T11:33:45.238798+0000","last_became_active":"2024-09-10T11:33:44.240613+0000","last_became_peered":"2024-09-10T11:33:44.240613+0000","last_unstale":"2024-09-10T11:33:45.238798+0000","last_undegraded":"2024-09-10T11:33:45.238798+0000","last_fullsized":"2024-09-10T11:33:45.238798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:20.962771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291167+0000","last_change":"2024-09-10T11:33:44.241747+0000","last_active":"2024-09-10T11:33:45.291167+0000","last_peered":"2024-09-10T11:33:45.291167+0000","last_clean":"2024-09-10T11:33:45.291167+0000","last_became_active":"2024-09-10T11:33:44.241469+0000","last_became_peered":"2024-09-10T11:33:44.241469+0000","last_unstale":"2024-09-10T11:33:45.291167+0000","last_undegraded":"2024-09-10T11:33:45.291167+0000","last_fullsized":"2024-09-10T11:33:45.291167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:11.567000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290856+0000","last_change":"2024-09-10T11:33:44.231985+0000","last_active":"2024-09-10T11:33:45.290856+0000","last_peered":"2024-09-10T11:33:45.290856+0000","last_clean":"2024-09-10T11:33:45.290856+0000","last_became_active":"2024-09-10T11:33:44.231422+0000","last_became_peered":"2024-09-10T11:33:44.231422+0000","last_unstale":"2024-09-10T11:33:45.290856+0000","last_undegraded":"2024-09-10T11:33:45.290856+0000","last_fullsized":"2024-09-10T11:33:45.290856+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:22.485023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290971+0000","last_change":"2024-09-10T11:33:44.239371+0000","last_active":"2024-09-10T11:33:45.290971+0000","last_peered":"2024-09-10T11:33:45.290971+0000","last_clean":"2024-09-10T11:33:45.290971+0000","last_became_active":"2024-09-10T11:33:44.238917+0000","last_became_peered":"2024-09-10T11:33:44.238917+0000","last_unstale":"2024-09-10T11:33:45.290971+0000","last_undegraded":"2024-09-10T11:33:45.290971+0000","last_fullsized":"2024-09-10T11:33:45.290971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:01.020188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227800+0000","last_change":"2024-09-10T11:33:44.235195+0000","last_active":"2024-09-10T11:33:45.227800+0000","last_peered":"2024-09-10T11:33:45.227800+0000","last_clean":"2024-09-10T11:33:45.227800+0000","last_became_active":"2024-09-10T11:33:44.234816+0000","last_became_peered":"2024-09-10T11:33:44.234816+0000","last_unstale":"2024-09-10T11:33:45.227800+0000","last_undegraded":"2024-09-10T11:33:45.227800+0000","last_fullsized":"2024-09-10T11:33:45.227800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:02.905660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238933+0000","last_change":"2024-09-10T11:33:44.240850+0000","last_active":"2024-09-10T11:33:45.238933+0000","last_peered":"2024-09-10T11:33:45.238933+0000","last_clean":"2024-09-10T11:33:45.238933+0000","last_became_active":"2024-09-10T11:33:44.240139+0000","last_became_peered":"2024-09-10T11:33:44.240139+0000","last_unstale":"2024-09-10T11:33:45.238933+0000","last_undegraded":"2024-09-10T11:33:45.238933+0000","last_fullsized":"2024-09-10T11:33:45.238933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:23.630606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227832+0000","last_change":"2024-09-10T11:33:44.237162+0000","last_active":"2024-09-10T11:33:45.227832+0000","last_peered":"2024-09-10T11:33:45.227832+0000","last_clean":"2024-09-10T11:33:45.227832+0000","last_became_active":"2024-09-10T11:33:44.236924+0000","last_became_peered":"2024-09-10T11:33:44.236924+0000","last_unstale":"2024-09-10T11:33:45.227832+0000","last_undegraded":"2024-09-10T11:33:45.227832+0000","last_fullsized":"2024-09-10T11:33:45.227832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:51.374696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239219+0000","last_change":"2024-09-10T11:33:44.241475+0000","last_active":"2024-09-10T11:33:45.239219+0000","last_peered":"2024-09-10T11:33:45.239219+0000","last_clean":"2024-09-10T11:33:45.239219+0000","last_became_active":"2024-09-10T11:33:44.240581+0000","last_became_peered":"2024-09-10T11:33:44.240581+0000","last_unstale":"2024-09-10T11:33:45.239219+0000","last_undegraded":"2024-09-10T11:33:45.239219+0000","last_fullsized":"2024-09-10T11:33:45.239219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:17.782292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291251+0000","last_change":"2024-09-10T11:33:44.239152+0000","last_active":"2024-09-10T11:33:45.291251+0000","last_peered":"2024-09-10T11:33:45.291251+0000","last_clean":"2024-09-10T11:33:45.291251+0000","last_became_active":"2024-09-10T11:33:44.238377+0000","last_became_peered":"2024-09-10T11:33:44.238377+0000","last_unstale":"2024-09-10T11:33:45.291251+0000","last_undegraded":"2024-09-10T11:33:45.291251+0000","last_fullsized":"2024-09-10T11:33:45.291251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:49.060965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290407+0000","last_change":"2024-09-10T11:33:44.229762+0000","last_active":"2024-09-10T11:33:45.290407+0000","last_peered":"2024-09-10T11:33:45.290407+0000","last_clean":"2024-09-10T11:33:45.290407+0000","last_became_active":"2024-09-10T11:33:44.229500+0000","last_became_peered":"2024-09-10T11:33:44.229500+0000","last_unstale":"2024-09-10T11:33:45.290407+0000","last_undegraded":"2024-09-10T11:33:45.290407+0000","last_fullsized":"2024-09-10T11:33:45.290407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:45.119789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291256+0000","last_change":"2024-09-10T11:33:44.238133+0000","last_active":"2024-09-10T11:33:45.291256+0000","last_peered":"2024-09-10T11:33:45.291256+0000","last_clean":"2024-09-10T11:33:45.291256+0000","last_became_active":"2024-09-10T11:33:44.237838+0000","last_became_peered":"2024-09-10T11:33:44.237838+0000","last_unstale":"2024-09-10T11:33:45.291256+0000","last_undegraded":"2024-09-10T11:33:45.291256+0000","last_fullsized":"2024-09-10T11:33:45.291256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:07.818150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238975+0000","last_change":"2024-09-10T11:33:44.241252+0000","last_active":"2024-09-10T11:33:45.238975+0000","last_peered":"2024-09-10T11:33:45.238975+0000","last_clean":"2024-09-10T11:33:45.238975+0000","last_became_active":"2024-09-10T11:33:44.241010+0000","last_became_peered":"2024-09-10T11:33:44.241010+0000","last_unstale":"2024-09-10T11:33:45.238975+0000","last_undegraded":"2024-09-10T11:33:45.238975+0000","last_fullsized":"2024-09-10T11:33:45.238975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:44:49.157541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290991+0000","last_change":"2024-09-10T11:33:44.230187+0000","last_active":"2024-09-10T11:33:45.290991+0000","last_peered":"2024-09-10T11:33:45.290991+0000","last_clean":"2024-09-10T11:33:45.290991+0000","last_became_active":"2024-09-10T11:33:44.229949+0000","last_became_peered":"2024-09-10T11:33:44.229949+0000","last_unstale":"2024-09-10T11:33:45.290991+0000","last_undegraded":"2024-09-10T11:33:45.290991+0000","last_fullsized":"2024-09-10T11:33:45.290991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:38.786009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290610+0000","last_change":"2024-09-10T11:33:44.229723+0000","last_active":"2024-09-10T11:33:45.290610+0000","last_peered":"2024-09-10T11:33:45.290610+0000","last_clean":"2024-09-10T11:33:45.290610+0000","last_became_active":"2024-09-10T11:33:44.229432+0000","last_became_peered":"2024-09-10T11:33:44.229432+0000","last_unstale":"2024-09-10T11:33:45.290610+0000","last_undegraded":"2024-09-10T11:33:45.290610+0000","last_fullsized":"2024-09-10T11:33:45.290610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:41.547433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239065+0000","last_change":"2024-09-10T11:33:44.242808+0000","last_active":"2024-09-10T11:33:45.239065+0000","last_peered":"2024-09-10T11:33:45.239065+0000","last_clean":"2024-09-10T11:33:45.239065+0000","last_became_active":"2024-09-10T11:33:44.242532+0000","last_became_peered":"2024-09-10T11:33:44.242532+0000","last_unstale":"2024-09-10T11:33:45.239065+0000","last_undegraded":"2024-09-10T11:33:45.239065+0000","last_fullsized":"2024-09-10T11:33:45.239065+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:50.880848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290770+0000","last_change":"2024-09-10T11:33:43.214560+0000","last_active":"2024-09-10T11:33:45.290770+0000","last_peered":"2024-09-10T11:33:45.290770+0000","last_clean":"2024-09-10T11:33:45.290770+0000","last_became_active":"2024-09-10T11:33:43.214277+0000","last_became_peered":"2024-09-10T11:33:43.214277+0000","last_unstale":"2024-09-10T11:33:45.290770+0000","last_undegraded":"2024-09-10T11:33:45.290770+0000","last_fullsized":"2024-09-10T11:33:45.290770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:33.121927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291070+0000","last_change":"2024-09-10T11:33:44.241702+0000","last_active":"2024-09-10T11:33:45.291070+0000","last_peered":"2024-09-10T11:33:45.291070+0000","last_clean":"2024-09-10T11:33:45.291070+0000","last_became_active":"2024-09-10T11:33:44.241365+0000","last_became_peered":"2024-09-10T11:33:44.241365+0000","last_unstale":"2024-09-10T11:33:45.291070+0000","last_undegraded":"2024-09-10T11:33:45.291070+0000","last_fullsized":"2024-09-10T11:33:45.291070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:46.461857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290601+0000","last_change":"2024-09-10T11:33:43.211331+0000","last_active":"2024-09-10T11:33:45.290601+0000","last_peered":"2024-09-10T11:33:45.290601+0000","last_clean":"2024-09-10T11:33:45.290601+0000","last_became_active":"2024-09-10T11:33:43.211080+0000","last_became_peered":"2024-09-10T11:33:43.211080+0000","last_unstale":"2024-09-10T11:33:45.290601+0000","last_undegraded":"2024-09-10T11:33:45.290601+0000","last_fullsized":"2024-09-10T11:33:45.290601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:00.236266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290678+0000","last_change":"2024-09-10T11:33:44.243692+0000","last_active":"2024-09-10T11:33:45.290678+0000","last_peered":"2024-09-10T11:33:45.290678+0000","last_clean":"2024-09-10T11:33:45.290678+0000","last_became_active":"2024-09-10T11:33:44.243144+0000","last_became_peered":"2024-09-10T11:33:44.243144+0000","last_unstale":"2024-09-10T11:33:45.290678+0000","last_undegraded":"2024-09-10T11:33:45.290678+0000","last_fullsized":"2024-09-10T11:33:45.290678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:09.299856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239067+0000","last_change":"2024-09-10T11:33:43.215222+0000","last_active":"2024-09-10T11:33:45.239067+0000","last_peered":"2024-09-10T11:33:45.239067+0000","last_clean":"2024-09-10T11:33:45.239067+0000","last_became_active":"2024-09-10T11:33:43.214983+0000","last_became_peered":"2024-09-10T11:33:43.214983+0000","last_unstale":"2024-09-10T11:33:45.239067+0000","last_undegraded":"2024-09-10T11:33:45.239067+0000","last_fullsized":"2024-09-10T11:33:45.239067+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:50.688965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228008+0000","last_change":"2024-09-10T11:33:44.241195+0000","last_active":"2024-09-10T11:33:45.228008+0000","last_peered":"2024-09-10T11:33:45.228008+0000","last_clean":"2024-09-10T11:33:45.228008+0000","last_became_active":"2024-09-10T11:33:44.240917+0000","last_became_peered":"2024-09-10T11:33:44.240917+0000","last_unstale":"2024-09-10T11:33:45.228008+0000","last_undegraded":"2024-09-10T11:33:45.228008+0000","last_fullsized":"2024-09-10T11:33:45.228008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:27.576400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290975+0000","last_change":"2024-09-10T11:33:43.216249+0000","last_active":"2024-09-10T11:33:45.290975+0000","last_peered":"2024-09-10T11:33:45.290975+0000","last_clean":"2024-09-10T11:33:45.290975+0000","last_became_active":"2024-09-10T11:33:43.216029+0000","last_became_peered":"2024-09-10T11:33:43.216029+0000","last_unstale":"2024-09-10T11:33:45.290975+0000","last_undegraded":"2024-09-10T11:33:45.290975+0000","last_fullsized":"2024-09-10T11:33:45.290975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.458052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238739+0000","last_change":"2024-09-10T11:33:44.244203+0000","last_active":"2024-09-10T11:33:45.238739+0000","last_peered":"2024-09-10T11:33:45.238739+0000","last_clean":"2024-09-10T11:33:45.238739+0000","last_became_active":"2024-09-10T11:33:44.243781+0000","last_became_peered":"2024-09-10T11:33:44.243781+0000","last_unstale":"2024-09-10T11:33:45.238739+0000","last_undegraded":"2024-09-10T11:33:45.238739+0000","last_fullsized":"2024-09-10T11:33:45.238739+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:31.825081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228739+0000","last_change":"2024-09-10T11:33:43.214491+0000","last_active":"2024-09-10T11:33:45.228739+0000","last_peered":"2024-09-10T11:33:45.228739+0000","last_clean":"2024-09-10T11:33:45.228739+0000","last_became_active":"2024-09-10T11:33:43.214224+0000","last_became_peered":"2024-09-10T11:33:43.214224+0000","last_unstale":"2024-09-10T11:33:45.228739+0000","last_undegraded":"2024-09-10T11:33:45.228739+0000","last_fullsized":"2024-09-10T11:33:45.228739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.615027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291098+0000","last_change":"2024-09-10T11:33:44.239174+0000","last_active":"2024-09-10T11:33:45.291098+0000","last_peered":"2024-09-10T11:33:45.291098+0000","last_clean":"2024-09-10T11:33:45.291098+0000","last_became_active":"2024-09-10T11:33:44.238615+0000","last_became_peered":"2024-09-10T11:33:44.238615+0000","last_unstale":"2024-09-10T11:33:45.291098+0000","last_undegraded":"2024-09-10T11:33:45.291098+0000","last_fullsized":"2024-09-10T11:33:45.291098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:49.193572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227891+0000","last_change":"2024-09-10T11:33:43.215090+0000","last_active":"2024-09-10T11:33:45.227891+0000","last_peered":"2024-09-10T11:33:45.227891+0000","last_clean":"2024-09-10T11:33:45.227891+0000","last_became_active":"2024-09-10T11:33:43.214670+0000","last_became_peered":"2024-09-10T11:33:43.214670+0000","last_unstale":"2024-09-10T11:33:45.227891+0000","last_undegraded":"2024-09-10T11:33:45.227891+0000","last_fullsized":"2024-09-10T11:33:45.227891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.903660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227649+0000","last_change":"2024-09-10T11:33:44.237216+0000","last_active":"2024-09-10T11:33:45.227649+0000","last_peered":"2024-09-10T11:33:45.227649+0000","last_clean":"2024-09-10T11:33:45.227649+0000","last_became_active":"2024-09-10T11:33:44.236963+0000","last_became_peered":"2024-09-10T11:33:44.236963+0000","last_unstale":"2024-09-10T11:33:45.227649+0000","last_undegraded":"2024-09-10T11:33:45.227649+0000","last_fullsized":"2024-09-10T11:33:45.227649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:27.799466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228474+0000","last_change":"2024-09-10T11:33:43.215568+0000","last_active":"2024-09-10T11:33:45.228474+0000","last_peered":"2024-09-10T11:33:45.228474+0000","last_clean":"2024-09-10T11:33:45.228474+0000","last_became_active":"2024-09-10T11:33:43.214417+0000","last_became_peered":"2024-09-10T11:33:43.214417+0000","last_unstale":"2024-09-10T11:33:45.228474+0000","last_undegraded":"2024-09-10T11:33:45.228474+0000","last_fullsized":"2024-09-10T11:33:45.228474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:51.065071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228126+0000","last_change":"2024-09-10T11:33:44.246719+0000","last_active":"2024-09-10T11:33:45.228126+0000","last_peered":"2024-09-10T11:33:45.228126+0000","last_clean":"2024-09-10T11:33:45.228126+0000","last_became_active":"2024-09-10T11:33:44.245977+0000","last_became_peered":"2024-09-10T11:33:44.245977+0000","last_unstale":"2024-09-10T11:33:45.228126+0000","last_undegraded":"2024-09-10T11:33:45.228126+0000","last_fullsized":"2024-09-10T11:33:45.228126+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:08.809966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228408+0000","last_change":"2024-09-10T11:33:43.208420+0000","last_active":"2024-09-10T11:33:45.228408+0000","last_peered":"2024-09-10T11:33:45.228408+0000","last_clean":"2024-09-10T11:33:45.228408+0000","last_became_active":"2024-09-10T11:33:43.207842+0000","last_became_peered":"2024-09-10T11:33:43.207842+0000","last_unstale":"2024-09-10T11:33:45.228408+0000","last_undegraded":"2024-09-10T11:33:45.228408+0000","last_fullsized":"2024-09-10T11:33:45.228408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:30.741183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:44.234980+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:44.234730+0000","last_became_peered":"2024-09-10T11:33:44.234730+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:14.462937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239363+0000","last_change":"2024-09-10T11:33:43.210937+0000","last_active":"2024-09-10T11:33:45.239363+0000","last_peered":"2024-09-10T11:33:45.239363+0000","last_clean":"2024-09-10T11:33:45.239363+0000","last_became_active":"2024-09-10T11:33:43.210245+0000","last_became_peered":"2024-09-10T11:33:43.210245+0000","last_unstale":"2024-09-10T11:33:45.239363+0000","last_undegraded":"2024-09-10T11:33:45.239363+0000","last_fullsized":"2024-09-10T11:33:45.239363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:37.405065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291170+0000","last_change":"2024-09-10T11:33:44.232525+0000","last_active":"2024-09-10T11:33:45.291170+0000","last_peered":"2024-09-10T11:33:45.291170+0000","last_clean":"2024-09-10T11:33:45.291170+0000","last_became_active":"2024-09-10T11:33:44.232277+0000","last_became_peered":"2024-09-10T11:33:44.232277+0000","last_unstale":"2024-09-10T11:33:45.291170+0000","last_undegraded":"2024-09-10T11:33:45.291170+0000","last_fullsized":"2024-09-10T11:33:45.291170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:42.486387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290695+0000","last_change":"2024-09-10T11:33:43.214840+0000","last_active":"2024-09-10T11:33:45.290695+0000","last_peered":"2024-09-10T11:33:45.290695+0000","last_clean":"2024-09-10T11:33:45.290695+0000","last_became_active":"2024-09-10T11:33:43.214617+0000","last_became_peered":"2024-09-10T11:33:43.214617+0000","last_unstale":"2024-09-10T11:33:45.290695+0000","last_undegraded":"2024-09-10T11:33:45.290695+0000","last_fullsized":"2024-09-10T11:33:45.290695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:22.268583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239165+0000","last_change":"2024-09-10T11:33:44.245910+0000","last_active":"2024-09-10T11:33:45.239165+0000","last_peered":"2024-09-10T11:33:45.239165+0000","last_clean":"2024-09-10T11:33:45.239165+0000","last_became_active":"2024-09-10T11:33:44.245601+0000","last_became_peered":"2024-09-10T11:33:44.245601+0000","last_unstale":"2024-09-10T11:33:45.239165+0000","last_undegraded":"2024-09-10T11:33:45.239165+0000","last_fullsized":"2024-09-10T11:33:45.239165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:47.170771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228571+0000","last_change":"2024-09-10T11:33:43.209367+0000","last_active":"2024-09-10T11:33:45.228571+0000","last_peered":"2024-09-10T11:33:45.228571+0000","last_clean":"2024-09-10T11:33:45.228571+0000","last_became_active":"2024-09-10T11:33:43.209057+0000","last_became_peered":"2024-09-10T11:33:43.209057+0000","last_unstale":"2024-09-10T11:33:45.228571+0000","last_undegraded":"2024-09-10T11:33:45.228571+0000","last_fullsized":"2024-09-10T11:33:45.228571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:59.128934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291260+0000","last_change":"2024-09-10T11:33:44.240524+0000","last_active":"2024-09-10T11:33:45.291260+0000","last_peered":"2024-09-10T11:33:45.291260+0000","last_clean":"2024-09-10T11:33:45.291260+0000","last_became_active":"2024-09-10T11:33:44.238970+0000","last_became_peered":"2024-09-10T11:33:44.238970+0000","last_unstale":"2024-09-10T11:33:45.291260+0000","last_undegraded":"2024-09-10T11:33:45.291260+0000","last_fullsized":"2024-09-10T11:33:45.291260+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:18.018206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228609+0000","last_change":"2024-09-10T11:33:43.208981+0000","last_active":"2024-09-10T11:33:45.228609+0000","last_peered":"2024-09-10T11:33:45.228609+0000","last_clean":"2024-09-10T11:33:45.228609+0000","last_became_active":"2024-09-10T11:33:43.208559+0000","last_became_peered":"2024-09-10T11:33:43.208559+0000","last_unstale":"2024-09-10T11:33:45.228609+0000","last_undegraded":"2024-09-10T11:33:45.228609+0000","last_fullsized":"2024-09-10T11:33:45.228609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:03.592321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290819+0000","last_change":"2024-09-10T11:33:44.239690+0000","last_active":"2024-09-10T11:33:45.290819+0000","last_peered":"2024-09-10T11:33:45.290819+0000","last_clean":"2024-09-10T11:33:45.290819+0000","last_became_active":"2024-09-10T11:33:44.239401+0000","last_became_peered":"2024-09-10T11:33:44.239401+0000","last_unstale":"2024-09-10T11:33:45.290819+0000","last_undegraded":"2024-09-10T11:33:45.290819+0000","last_fullsized":"2024-09-10T11:33:45.290819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:23:05.699528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227762+0000","last_change":"2024-09-10T11:33:43.209064+0000","last_active":"2024-09-10T11:33:45.227762+0000","last_peered":"2024-09-10T11:33:45.227762+0000","last_clean":"2024-09-10T11:33:45.227762+0000","last_became_active":"2024-09-10T11:33:43.208043+0000","last_became_peered":"2024-09-10T11:33:43.208043+0000","last_unstale":"2024-09-10T11:33:45.227762+0000","last_undegraded":"2024-09-10T11:33:45.227762+0000","last_fullsized":"2024-09-10T11:33:45.227762+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:45.382171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228074+0000","last_change":"2024-09-10T11:33:44.237793+0000","last_active":"2024-09-10T11:33:45.228074+0000","last_peered":"2024-09-10T11:33:45.228074+0000","last_clean":"2024-09-10T11:33:45.228074+0000","last_became_active":"2024-09-10T11:33:44.237536+0000","last_became_peered":"2024-09-10T11:33:44.237536+0000","last_unstale":"2024-09-10T11:33:45.228074+0000","last_undegraded":"2024-09-10T11:33:45.228074+0000","last_fullsized":"2024-09-10T11:33:45.228074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:14.390264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:43.209065+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:43.208821+0000","last_became_peered":"2024-09-10T11:33:43.208821+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:15.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291354+0000","last_change":"2024-09-10T11:33:44.235181+0000","last_active":"2024-09-10T11:33:45.291354+0000","last_peered":"2024-09-10T11:33:45.291354+0000","last_clean":"2024-09-10T11:33:45.291354+0000","last_became_active":"2024-09-10T11:33:44.234939+0000","last_became_peered":"2024-09-10T11:33:44.234939+0000","last_unstale":"2024-09-10T11:33:45.291354+0000","last_undegraded":"2024-09-10T11:33:45.291354+0000","last_fullsized":"2024-09-10T11:33:45.291354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:54.993335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228038+0000","last_change":"2024-09-10T11:33:43.208219+0000","last_active":"2024-09-10T11:33:45.228038+0000","last_peered":"2024-09-10T11:33:45.228038+0000","last_clean":"2024-09-10T11:33:45.228038+0000","last_became_active":"2024-09-10T11:33:43.207992+0000","last_became_peered":"2024-09-10T11:33:43.207992+0000","last_unstale":"2024-09-10T11:33:45.228038+0000","last_undegraded":"2024-09-10T11:33:45.228038+0000","last_fullsized":"2024-09-10T11:33:45.228038+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:45.628443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290880+0000","last_change":"2024-09-10T11:33:44.236199+0000","last_active":"2024-09-10T11:33:45.290880+0000","last_peered":"2024-09-10T11:33:45.290880+0000","last_clean":"2024-09-10T11:33:45.290880+0000","last_became_active":"2024-09-10T11:33:44.235961+0000","last_became_peered":"2024-09-10T11:33:44.235961+0000","last_unstale":"2024-09-10T11:33:45.290880+0000","last_undegraded":"2024-09-10T11:33:45.290880+0000","last_fullsized":"2024-09-10T11:33:45.290880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.797536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227944+0000","last_change":"2024-09-10T11:33:43.214900+0000","last_active":"2024-09-10T11:33:45.227944+0000","last_peered":"2024-09-10T11:33:45.227944+0000","last_clean":"2024-09-10T11:33:45.227944+0000","last_became_active":"2024-09-10T11:33:43.214372+0000","last_became_peered":"2024-09-10T11:33:43.214372+0000","last_unstale":"2024-09-10T11:33:45.227944+0000","last_undegraded":"2024-09-10T11:33:45.227944+0000","last_fullsized":"2024-09-10T11:33:45.227944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:10.940181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290859+0000","last_change":"2024-09-10T11:33:44.242982+0000","last_active":"2024-09-10T11:33:45.290859+0000","last_peered":"2024-09-10T11:33:45.290859+0000","last_clean":"2024-09-10T11:33:45.290859+0000","last_became_active":"2024-09-10T11:33:44.242310+0000","last_became_peered":"2024-09-10T11:33:44.242310+0000","last_unstale":"2024-09-10T11:33:45.290859+0000","last_undegraded":"2024-09-10T11:33:45.290859+0000","last_fullsized":"2024-09-10T11:33:45.290859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:38.992677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239252+0000","last_change":"2024-09-10T11:33:43.207140+0000","last_active":"2024-09-10T11:33:45.239252+0000","last_peered":"2024-09-10T11:33:45.239252+0000","last_clean":"2024-09-10T11:33:45.239252+0000","last_became_active":"2024-09-10T11:33:43.206926+0000","last_became_peered":"2024-09-10T11:33:43.206926+0000","last_unstale":"2024-09-10T11:33:45.239252+0000","last_undegraded":"2024-09-10T11:33:45.239252+0000","last_fullsized":"2024-09-10T11:33:45.239252+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:16.821363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290467+0000","last_change":"2024-09-10T11:33:44.236919+0000","last_active":"2024-09-10T11:33:45.290467+0000","last_peered":"2024-09-10T11:33:45.290467+0000","last_clean":"2024-09-10T11:33:45.290467+0000","last_became_active":"2024-09-10T11:33:44.236683+0000","last_became_peered":"2024-09-10T11:33:44.236683+0000","last_unstale":"2024-09-10T11:33:45.290467+0000","last_undegraded":"2024-09-10T11:33:45.290467+0000","last_fullsized":"2024-09-10T11:33:45.290467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:48.375628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291086+0000","last_change":"2024-09-10T11:33:43.212385+0000","last_active":"2024-09-10T11:33:45.291086+0000","last_peered":"2024-09-10T11:33:45.291086+0000","last_clean":"2024-09-10T11:33:45.291086+0000","last_became_active":"2024-09-10T11:33:43.212163+0000","last_became_peered":"2024-09-10T11:33:43.212163+0000","last_unstale":"2024-09-10T11:33:45.291086+0000","last_undegraded":"2024-09-10T11:33:45.291086+0000","last_fullsized":"2024-09-10T11:33:45.291086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:58.221936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291084+0000","last_change":"2024-09-10T11:33:44.244452+0000","last_active":"2024-09-10T11:33:45.291084+0000","last_peered":"2024-09-10T11:33:45.291084+0000","last_clean":"2024-09-10T11:33:45.291084+0000","last_became_active":"2024-09-10T11:33:44.244228+0000","last_became_peered":"2024-09-10T11:33:44.244228+0000","last_unstale":"2024-09-10T11:33:45.291084+0000","last_undegraded":"2024-09-10T11:33:45.291084+0000","last_fullsized":"2024-09-10T11:33:45.291084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:03.316534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291265+0000","last_change":"2024-09-10T11:33:43.213782+0000","last_active":"2024-09-10T11:33:45.291265+0000","last_peered":"2024-09-10T11:33:45.291265+0000","last_clean":"2024-09-10T11:33:45.291265+0000","last_became_active":"2024-09-10T11:33:43.213512+0000","last_became_peered":"2024-09-10T11:33:43.213512+0000","last_unstale":"2024-09-10T11:33:45.291265+0000","last_undegraded":"2024-09-10T11:33:45.291265+0000","last_fullsized":"2024-09-10T11:33:45.291265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:54.809490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290922+0000","last_change":"2024-09-10T11:33:44.243474+0000","last_active":"2024-09-10T11:33:45.290922+0000","last_peered":"2024-09-10T11:33:45.290922+0000","last_clean":"2024-09-10T11:33:45.290922+0000","last_became_active":"2024-09-10T11:33:44.243212+0000","last_became_peered":"2024-09-10T11:33:44.243212+0000","last_unstale":"2024-09-10T11:33:45.290922+0000","last_undegraded":"2024-09-10T11:33:45.290922+0000","last_fullsized":"2024-09-10T11:33:45.290922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:58:06.229999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291286+0000","last_change":"2024-09-10T11:33:43.214433+0000","last_active":"2024-09-10T11:33:45.291286+0000","last_peered":"2024-09-10T11:33:45.291286+0000","last_clean":"2024-09-10T11:33:45.291286+0000","last_became_active":"2024-09-10T11:33:43.214060+0000","last_became_peered":"2024-09-10T11:33:43.214060+0000","last_unstale":"2024-09-10T11:33:45.291286+0000","last_undegraded":"2024-09-10T11:33:45.291286+0000","last_fullsized":"2024-09-10T11:33:45.291286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:29.435158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227694+0000","last_change":"2024-09-10T11:33:44.236573+0000","last_active":"2024-09-10T11:33:45.227694+0000","last_peered":"2024-09-10T11:33:45.227694+0000","last_clean":"2024-09-10T11:33:45.227694+0000","last_became_active":"2024-09-10T11:33:44.236279+0000","last_became_peered":"2024-09-10T11:33:44.236279+0000","last_unstale":"2024-09-10T11:33:45.227694+0000","last_undegraded":"2024-09-10T11:33:45.227694+0000","last_fullsized":"2024-09-10T11:33:45.227694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:16.865227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291043+0000","last_change":"2024-09-10T11:33:43.212318+0000","last_active":"2024-09-10T11:33:45.291043+0000","last_peered":"2024-09-10T11:33:45.291043+0000","last_clean":"2024-09-10T11:33:45.291043+0000","last_became_active":"2024-09-10T11:33:43.212105+0000","last_became_peered":"2024-09-10T11:33:43.212105+0000","last_unstale":"2024-09-10T11:33:45.291043+0000","last_undegraded":"2024-09-10T11:33:45.291043+0000","last_fullsized":"2024-09-10T11:33:45.291043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:32.124944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227542+0000","last_change":"2024-09-10T11:33:44.241022+0000","last_active":"2024-09-10T11:33:45.227542+0000","last_peered":"2024-09-10T11:33:45.227542+0000","last_clean":"2024-09-10T11:33:45.227542+0000","last_became_active":"2024-09-10T11:33:44.240784+0000","last_became_peered":"2024-09-10T11:33:44.240784+0000","last_unstale":"2024-09-10T11:33:45.227542+0000","last_undegraded":"2024-09-10T11:33:45.227542+0000","last_fullsized":"2024-09-10T11:33:45.227542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:12:07.362654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291165+0000","last_change":"2024-09-10T11:33:43.209703+0000","last_active":"2024-09-10T11:33:45.291165+0000","last_peered":"2024-09-10T11:33:45.291165+0000","last_clean":"2024-09-10T11:33:45.291165+0000","last_became_active":"2024-09-10T11:33:43.209476+0000","last_became_peered":"2024-09-10T11:33:43.209476+0000","last_unstale":"2024-09-10T11:33:45.291165+0000","last_undegraded":"2024-09-10T11:33:45.291165+0000","last_fullsized":"2024-09-10T11:33:45.291165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:17.804493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228263+0000","last_change":"2024-09-10T11:33:44.242827+0000","last_active":"2024-09-10T11:33:45.228263+0000","last_peered":"2024-09-10T11:33:45.228263+0000","last_clean":"2024-09-10T11:33:45.228263+0000","last_became_active":"2024-09-10T11:33:44.242441+0000","last_became_peered":"2024-09-10T11:33:44.242441+0000","last_unstale":"2024-09-10T11:33:45.228263+0000","last_undegraded":"2024-09-10T11:33:45.228263+0000","last_fullsized":"2024-09-10T11:33:45.228263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:16.886176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290823+0000","last_change":"2024-09-10T11:33:43.211590+0000","last_active":"2024-09-10T11:33:45.290823+0000","last_peered":"2024-09-10T11:33:45.290823+0000","last_clean":"2024-09-10T11:33:45.290823+0000","last_became_active":"2024-09-10T11:33:43.211064+0000","last_became_peered":"2024-09-10T11:33:43.211064+0000","last_unstale":"2024-09-10T11:33:45.290823+0000","last_undegraded":"2024-09-10T11:33:45.290823+0000","last_fullsized":"2024-09-10T11:33:45.290823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:54.069449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291445+0000","last_change":"2024-09-10T11:33:44.233490+0000","last_active":"2024-09-10T11:33:45.291445+0000","last_peered":"2024-09-10T11:33:45.291445+0000","last_clean":"2024-09-10T11:33:45.291445+0000","last_became_active":"2024-09-10T11:33:44.233246+0000","last_became_peered":"2024-09-10T11:33:44.233246+0000","last_unstale":"2024-09-10T11:33:45.291445+0000","last_undegraded":"2024-09-10T11:33:45.291445+0000","last_fullsized":"2024-09-10T11:33:45.291445+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:09.713804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.238831+0000","last_change":"2024-09-10T11:33:43.208823+0000","last_active":"2024-09-10T11:33:45.238831+0000","last_peered":"2024-09-10T11:33:45.238831+0000","last_clean":"2024-09-10T11:33:45.238831+0000","last_became_active":"2024-09-10T11:33:43.207920+0000","last_became_peered":"2024-09-10T11:33:43.207920+0000","last_unstale":"2024-09-10T11:33:45.238831+0000","last_undegraded":"2024-09-10T11:33:45.238831+0000","last_fullsized":"2024-09-10T11:33:45.238831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:01.523833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227887+0000","last_change":"2024-09-10T11:33:44.236550+0000","last_active":"2024-09-10T11:33:45.227887+0000","last_peered":"2024-09-10T11:33:45.227887+0000","last_clean":"2024-09-10T11:33:45.227887+0000","last_became_active":"2024-09-10T11:33:44.236263+0000","last_became_peered":"2024-09-10T11:33:44.236263+0000","last_unstale":"2024-09-10T11:33:45.227887+0000","last_undegraded":"2024-09-10T11:33:45.227887+0000","last_fullsized":"2024-09-10T11:33:45.227887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:42.113034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239705+0000","last_change":"2024-09-10T11:33:43.211106+0000","last_active":"2024-09-10T11:33:45.239705+0000","last_peered":"2024-09-10T11:33:45.239705+0000","last_clean":"2024-09-10T11:33:45.239705+0000","last_became_active":"2024-09-10T11:33:43.210871+0000","last_became_peered":"2024-09-10T11:33:43.210871+0000","last_unstale":"2024-09-10T11:33:45.239705+0000","last_undegraded":"2024-09-10T11:33:45.239705+0000","last_fullsized":"2024-09-10T11:33:45.239705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:24.490287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290990+0000","last_change":"2024-09-10T11:33:44.229784+0000","last_active":"2024-09-10T11:33:45.290990+0000","last_peered":"2024-09-10T11:33:45.290990+0000","last_clean":"2024-09-10T11:33:45.290990+0000","last_became_active":"2024-09-10T11:33:44.229536+0000","last_became_peered":"2024-09-10T11:33:44.229536+0000","last_unstale":"2024-09-10T11:33:45.290990+0000","last_undegraded":"2024-09-10T11:33:45.290990+0000","last_fullsized":"2024-09-10T11:33:45.290990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:21.289170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228318+0000","last_change":"2024-09-10T11:33:43.207910+0000","last_active":"2024-09-10T11:33:45.228318+0000","last_peered":"2024-09-10T11:33:45.228318+0000","last_clean":"2024-09-10T11:33:45.228318+0000","last_became_active":"2024-09-10T11:33:43.207656+0000","last_became_peered":"2024-09-10T11:33:43.207656+0000","last_unstale":"2024-09-10T11:33:45.228318+0000","last_undegraded":"2024-09-10T11:33:45.228318+0000","last_fullsized":"2024-09-10T11:33:45.228318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:05.578247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227759+0000","last_change":"2024-09-10T11:33:44.233670+0000","last_active":"2024-09-10T11:33:45.227759+0000","last_peered":"2024-09-10T11:33:45.227759+0000","last_clean":"2024-09-10T11:33:45.227759+0000","last_became_active":"2024-09-10T11:33:44.233434+0000","last_became_peered":"2024-09-10T11:33:44.233434+0000","last_unstale":"2024-09-10T11:33:45.227759+0000","last_undegraded":"2024-09-10T11:33:45.227759+0000","last_fullsized":"2024-09-10T11:33:45.227759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:22.761180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.228522+0000","last_change":"2024-09-10T11:33:43.208706+0000","last_active":"2024-09-10T11:33:45.228522+0000","last_peered":"2024-09-10T11:33:45.228522+0000","last_clean":"2024-09-10T11:33:45.228522+0000","last_became_active":"2024-09-10T11:33:43.208169+0000","last_became_peered":"2024-09-10T11:33:43.208169+0000","last_unstale":"2024-09-10T11:33:45.228522+0000","last_undegraded":"2024-09-10T11:33:45.228522+0000","last_fullsized":"2024-09-10T11:33:45.228522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:08.272400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291342+0000","last_change":"2024-09-10T11:33:44.238629+0000","last_active":"2024-09-10T11:33:45.291342+0000","last_peered":"2024-09-10T11:33:45.291342+0000","last_clean":"2024-09-10T11:33:45.291342+0000","last_became_active":"2024-09-10T11:33:44.238034+0000","last_became_peered":"2024-09-10T11:33:44.238034+0000","last_unstale":"2024-09-10T11:33:45.291342+0000","last_undegraded":"2024-09-10T11:33:45.291342+0000","last_fullsized":"2024-09-10T11:33:45.291342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:32.096230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227989+0000","last_change":"2024-09-10T11:33:43.211173+0000","last_active":"2024-09-10T11:33:45.227989+0000","last_peered":"2024-09-10T11:33:45.227989+0000","last_clean":"2024-09-10T11:33:45.227989+0000","last_became_active":"2024-09-10T11:33:43.210941+0000","last_became_peered":"2024-09-10T11:33:43.210941+0000","last_unstale":"2024-09-10T11:33:45.227989+0000","last_undegraded":"2024-09-10T11:33:45.227989+0000","last_fullsized":"2024-09-10T11:33:45.227989+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:02:43.320066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.239297+0000","last_change":"2024-09-10T11:33:44.241685+0000","last_active":"2024-09-10T11:33:45.239297+0000","last_peered":"2024-09-10T11:33:45.239297+0000","last_clean":"2024-09-10T11:33:45.239297+0000","last_became_active":"2024-09-10T11:33:44.240898+0000","last_became_peered":"2024-09-10T11:33:44.240898+0000","last_unstale":"2024-09-10T11:33:45.239297+0000","last_undegraded":"2024-09-10T11:33:45.239297+0000","last_fullsized":"2024-09-10T11:33:45.239297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:27.807759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291403+0000","last_change":"2024-09-10T11:33:43.214265+0000","last_active":"2024-09-10T11:33:45.291403+0000","last_peered":"2024-09-10T11:33:45.291403+0000","last_clean":"2024-09-10T11:33:45.291403+0000","last_became_active":"2024-09-10T11:33:43.214039+0000","last_became_peered":"2024-09-10T11:33:43.214039+0000","last_unstale":"2024-09-10T11:33:45.291403+0000","last_undegraded":"2024-09-10T11:33:45.291403+0000","last_fullsized":"2024-09-10T11:33:45.291403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:02.134330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227541+0000","last_change":"2024-09-10T11:33:44.229449+0000","last_active":"2024-09-10T11:33:45.227541+0000","last_peered":"2024-09-10T11:33:45.227541+0000","last_clean":"2024-09-10T11:33:45.227541+0000","last_became_active":"2024-09-10T11:33:44.229187+0000","last_became_peered":"2024-09-10T11:33:44.229187+0000","last_unstale":"2024-09-10T11:33:45.227541+0000","last_undegraded":"2024-09-10T11:33:45.227541+0000","last_fullsized":"2024-09-10T11:33:45.227541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:34.466978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290649+0000","last_change":"2024-09-10T11:33:43.216018+0000","last_active":"2024-09-10T11:33:45.290649+0000","last_peered":"2024-09-10T11:33:45.290649+0000","last_clean":"2024-09-10T11:33:45.290649+0000","last_became_active":"2024-09-10T11:33:43.215806+0000","last_became_peered":"2024-09-10T11:33:43.215806+0000","last_unstale":"2024-09-10T11:33:45.290649+0000","last_undegraded":"2024-09-10T11:33:45.290649+0000","last_fullsized":"2024-09-10T11:33:45.290649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:38.025842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290554+0000","last_change":"2024-09-10T11:33:44.233598+0000","last_active":"2024-09-10T11:33:45.290554+0000","last_peered":"2024-09-10T11:33:45.290554+0000","last_clean":"2024-09-10T11:33:45.290554+0000","last_became_active":"2024-09-10T11:33:44.233341+0000","last_became_peered":"2024-09-10T11:33:44.233341+0000","last_unstale":"2024-09-10T11:33:45.290554+0000","last_undegraded":"2024-09-10T11:33:45.290554+0000","last_fullsized":"2024-09-10T11:33:45.290554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:18.365927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.291135+0000","last_change":"2024-09-10T11:33:43.215353+0000","last_active":"2024-09-10T11:33:45.291135+0000","last_peered":"2024-09-10T11:33:45.291135+0000","last_clean":"2024-09-10T11:33:45.291135+0000","last_became_active":"2024-09-10T11:33:43.215133+0000","last_became_peered":"2024-09-10T11:33:43.215133+0000","last_unstale":"2024-09-10T11:33:45.291135+0000","last_undegraded":"2024-09-10T11:33:45.291135+0000","last_fullsized":"2024-09-10T11:33:45.291135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:05:29.332573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.290774+0000","last_change":"2024-09-10T11:33:44.236979+0000","last_active":"2024-09-10T11:33:45.290774+0000","last_peered":"2024-09-10T11:33:45.290774+0000","last_clean":"2024-09-10T11:33:45.290774+0000","last_became_active":"2024-09-10T11:33:44.236759+0000","last_became_peered":"2024-09-10T11:33:44.236759+0000","last_unstale":"2024-09-10T11:33:45.290774+0000","last_undegraded":"2024-09-10T11:33:45.290774+0000","last_fullsized":"2024-09-10T11:33:45.290774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:43.198874+0000","last_clean_scrub_stamp":"2024-09-10T11:33:43.198874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:41.737701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227994+0000","last_change":"2024-09-10T11:33:43.208935+0000","last_active":"2024-09-10T11:33:45.227994+0000","last_peered":"2024-09-10T11:33:45.227994+0000","last_clean":"2024-09-10T11:33:45.227994+0000","last_became_active":"2024-09-10T11:33:43.208470+0000","last_became_peered":"2024-09-10T11:33:43.208470+0000","last_unstale":"2024-09-10T11:33:45.227994+0000","last_undegraded":"2024-09-10T11:33:45.227994+0000","last_fullsized":"2024-09-10T11:33:45.227994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:33:42.196999+0000","last_clean_scrub_stamp":"2024-09-10T11:33:42.196999+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:51.574030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":300,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T11:33:45.227809+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:33:45.227809+0000","last_peered":"2024-09-10T11:33:45.227809+0000","last_clean":"2024-09-10T11:33:45.227809+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:33:45.227809+0000","last_undegraded":"2024-09-10T11:33:45.227809+0000","last_fullsized":"2024-09-10T11:33:45.227809+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:32:27.070388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199547911,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":441,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194870,"statfs":{"total":200000000,"available":199547877,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1016,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194295,"statfs":{"total":200000000,"available":198959065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194871,"statfs":{"total":200000000,"available":199548829,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549397,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199548941,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1016,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194296,"statfs":{"total":200000000,"available":198959155,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":439,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194872,"statfs":{"total":200000000,"available":199549453,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:33:47.550 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:33:47.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:33:47.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.655+0000 7f3820b33640 1 Processor -- start 2024-09-10T11:33:47.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.655+0000 7f3820b33640 1 -- start start 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f3820b33640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f381c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f3820b33640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381c0b40e0 con 0x7f381c15ee00 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f3820b33640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f381c0b5ab0 con 0x7f381c0b0090 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f3820b33640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f381c073c50 con 0x7f381c15f1e0 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f381a59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f381c15ee00 0x7f381c0b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54794/0 (socket says 172.21.15.99:54794) 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f381a59c640 1 -- 172.21.15.99:0/2026983250 learned_addr learned my addr 172.21.15.99:0/2026983250 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 870184623 0 0) 0x7f381c0b40e0 con 0x7f381c15ee00 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f3819d9b640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f381c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.656+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3804003520 con 0x7f381c15ee00 2024-09-10T11:33:47.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1379587506 0 0) 0x7f3804003520 con 0x7f381c15ee00 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 msgr2=0x7f381c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f381c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 >> v1:172.21.15.196:6789/0 conn(0x7f381c0b0090 legacy=0x7f381c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c0b4380 con 0x7f381c15ee00 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f380c001670 con 0x7f381c15ee00 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 >> v1:172.21.15.99:6789/0 conn(0x7f381c15ee00 legacy=0x7f381c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f38040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3804004060 0x7f3804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f38040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f381c073c50 con 0x7f3804004d40 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f380400bc00 con 0x7f3804004060 2024-09-10T11:33:47.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f380400be00 con 0x7f381c15f1e0 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f38040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381a59c640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f38040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f3819d9b640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3804004060 0x7f3804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 -- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 msgr2=0x7f38040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c15f1e0 0x7f38040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 -- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3804004060 msgr2=0x7f3804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3804004060 0x7f3804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.657+0000 7f381ad9d640 1 -- 172.21.15.99:0/2026983250 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3804003520 con 0x7f3804004d40 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f381ad9d640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f38040051c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3814001e50 tx=0x7f3804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbcf585fe433f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814001a70 con 0x7f3804004d40 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3814003970 con 0x7f3804004d40 2024-09-10T11:33:47.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f381959a640 1 -- 172.21.15.99:0/2026983250 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814003d60 con 0x7f3804004d40 2024-09-10T11:33:47.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f3820b33640 1 -- 172.21.15.99:0/2026983250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 msgr2=0x7f38040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.658+0000 7f3820b33640 1 --2- 172.21.15.99:0/2026983250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f38040051c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3814001e50 tx=0x7f3804005830 comp rx=0 tx=0).stop 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.659+0000 7f381ad9d640 1 -- 172.21.15.99:0/2026983250 reap_dead start 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.659+0000 7f3820b33640 1 -- 172.21.15.99:0/2026983250 shutdown_connections 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.659+0000 7f3820b33640 1 -- 172.21.15.99:0/2026983250 >> 172.21.15.99:0/2026983250 conn(0x7f381c0730c0 msgr2=0x7f381c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.659+0000 7f3820b33640 1 -- 172.21.15.99:0/2026983250 shutdown_connections 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.659+0000 7f3820b33640 1 -- 172.21.15.99:0/2026983250 wait complete. 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.660+0000 7f3820b33640 1 Processor -- start 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.660+0000 7f3820b33640 1 -- start start 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f381c156610 0x7f381c1f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c158fe0 0x7f381c1f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0b0600 con 0x7f3804004d40 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0afd60 con 0x7f381c156610 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f381c0b0780 con 0x7f381c158fe0 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43752/0 (socket says 172.21.15.99:43752) 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 -- 172.21.15.99:0/405820595 learned_addr learned my addr 172.21.15.99:0/405820595 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381ad9d640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c158fe0 0x7f381c1f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3819d9b640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f381c156610 0x7f381c1f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 -- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c158fe0 msgr2=0x7f381c1f83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c158fe0 0x7f381c1f83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 -- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f381c156610 msgr2=0x7f381c1f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f381c156610 0x7f381c1f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 -- 172.21.15.99:0/405820595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c0ca3c0 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f381a59c640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f380c007290 tx=0x7f380c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c00f8e0 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380c018070 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c0ca6a0 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.661+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c00fcc0 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.662+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c160ad0 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.662+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c160610 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.663+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f380c01d460 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.663+0000 7f380affd640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 0x7f37fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.663+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f380c08ad30 con 0x7f3804004d40 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.663+0000 7f3819d9b640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 0x7f37fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:47.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.664+0000 7f3819d9b640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 0x7f37fc06e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f381c1560d0 tx=0x7f3810008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:47.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.667+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f381c160610 con 0x7f3804004d40 2024-09-10T11:33:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.899+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f381c15e4f0 con 0x7f3804004d40 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.901+0000 7f380affd640 1 -- 172.21.15.99:0/405820595 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f380c00fe60 con 0x7f3804004d40 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:33:47.902 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:33:47.903 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:33:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 msgr2=0x7f37fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 0x7f37fc06e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f381c1560d0 tx=0x7f3810008040 comp rx=0 tx=0).stop 2024-09-10T11:33:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 msgr2=0x7f381c14b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:47.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f380c007290 tx=0x7f380c007750 comp rx=0 tx=0).stop 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 shutdown_connections 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f37fc06be30 0x7f37fc06e2f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f381c158fe0 0x7f381c1f83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f381c156610 0x7f381c1f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 --2- 172.21.15.99:0/405820595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3804004d40 0x7f381c14b8d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 >> 172.21.15.99:0/405820595 conn(0x7f381c0730c0 msgr2=0x7f381c15d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 shutdown_connections 2024-09-10T11:33:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:47.903+0000 7f3820b33640 1 -- 172.21.15.99:0/405820595 wait complete. 2024-09-10T11:33:47.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:48.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.012+0000 7f83cd39f640 1 Processor -- start 2024-09-10T11:33:48.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.012+0000 7f83cd39f640 1 -- start start 2024-09-10T11:33:48.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83cd39f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d3a30 0x7f83c80d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83cd39f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83c80d1ae0 0x7f83c80b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83cd39f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c80d2990 con 0x7f83c815efc0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83cd39f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f83c80d2c70 con 0x7f83c80d1ae0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83cd39f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f83c80d2f50 con 0x7f83c80d3a30 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c6ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f83c815efc0 0x7f83c80d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54808/0 (socket says 172.21.15.99:54808) 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c6ffd640 1 -- 172.21.15.99:0/2068583671 learned_addr learned my addr 172.21.15.99:0/2068583671 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c77fe640 1 --2- 172.21.15.99:0/2068583671 >> v2:172.21.15.196:3300/0 conn(0x7f83c80d1ae0 0x7f83c80b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83be7fc640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d3a30 0x7f83c80d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672293424 0 0) 0x7f83c80d2990 con 0x7f83c815efc0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83b4003520 con 0x7f83c815efc0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2338510660 0 0) 0x7f83b4003520 con 0x7f83c815efc0 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c77fe640 1 -- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d3a30 msgr2=0x7f83c80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c77fe640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d3a30 0x7f83c80d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c77fe640 1 -- 172.21.15.99:0/2068583671 >> v1:172.21.15.99:6789/0 conn(0x7f83c815efc0 legacy=0x7f83c80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.013+0000 7f83c77fe640 1 -- 172.21.15.99:0/2068583671 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c80d3230 con 0x7f83c80d1ae0 2024-09-10T11:33:48.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.014+0000 7f83c77fe640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d1ae0 0x7f83c80b4560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009dc0 tx=0x7f83c002a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880ee9c875f38490 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.014+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8001670 con 0x7f83c80d1ae0 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.014+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0007540 con 0x7f83c80d1ae0 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.014+0000 7f83c67fc640 1 -- 172.21.15.99:0/2068583671 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0007910 con 0x7f83c80d1ae0 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 -- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d1ae0 msgr2=0x7f83c80b4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d1ae0 0x7f83c80b4560 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009dc0 tx=0x7f83c002a610 comp rx=0 tx=0).stop 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 -- 172.21.15.99:0/2068583671 shutdown_connections 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d1ae0 0x7f83c80b4560 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 --2- 172.21.15.99:0/2068583671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d3a30 0x7f83c80d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 -- 172.21.15.99:0/2068583671 >> 172.21.15.99:0/2068583671 conn(0x7f83c80d4c20 msgr2=0x7f83c80d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 -- 172.21.15.99:0/2068583671 shutdown_connections 2024-09-10T11:33:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.015+0000 7f83cd39f640 1 -- 172.21.15.99:0/2068583671 wait complete. 2024-09-10T11:33:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.016+0000 7f83cd39f640 1 Processor -- start 2024-09-10T11:33:48.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.016+0000 7f83cd39f640 1 -- start start 2024-09-10T11:33:48.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d1ae0 0x7f83c8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d3a30 0x7f83c815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83c80b4aa0 con 0x7f83c815efc0 2024-09-10T11:33:48.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83c80d0d10 con 0x7f83c80d3a30 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83cd39f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f83c80b4c20 con 0x7f83c80d1ae0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43758/0 (socket says 172.21.15.99:43758) 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 -- 172.21.15.99:0/635337824 learned_addr learned my addr 172.21.15.99:0/635337824 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c6ffd640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d1ae0 0x7f83c8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83be7fc640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d3a30 0x7f83c815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 -- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d1ae0 msgr2=0x7f83c8155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d1ae0 0x7f83c8155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 -- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d3a30 msgr2=0x7f83c815f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d3a30 0x7f83c815f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.017+0000 7f83c77fe640 1 -- 172.21.15.99:0/635337824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c815fe00 con 0x7f83c815efc0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83c77fe640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f83c002ac00 tx=0x7f83c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0040070 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0031bc0 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0009450 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c81600e0 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.018+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c81606a0 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.019+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c80b4560 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.019+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f83c0039040 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.019+0000 7f83bffff640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.020+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f83c00ac400 con 0x7f83c815efc0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.020+0000 7f83c6ffd640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.020+0000 7f83c6ffd640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 0x7f83a806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005880 tx=0x7f83b80057f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.024+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83c007a760 con 0x7f83c815efc0 2024-09-10T11:33:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.223+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83c80b59f0 con 0x7f83c815efc0 2024-09-10T11:33:48.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.224+0000 7f83bffff640 1 -- 172.21.15.99:0/635337824 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7f83c007ea60 con 0x7f83c815efc0 2024-09-10T11:33:48.225 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:48.225 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:33:48.225 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 msgr2=0x7f83a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 0x7f83a806e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f83b8005880 tx=0x7f83b80057f0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 msgr2=0x7f83c8160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f83c002ac00 tx=0x7f83c0004a40 comp rx=0 tx=0).stop 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 shutdown_connections 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f83a806be30 0x7f83a806e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83c815efc0 0x7f83c8160ea0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83c80d3a30 0x7f83c815f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 --2- 172.21.15.99:0/635337824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83c80d1ae0 0x7f83c8155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 >> 172.21.15.99:0/635337824 conn(0x7f83c80d4c20 msgr2=0x7f83c80b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 shutdown_connections 2024-09-10T11:33:48.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.226+0000 7f83cd39f640 1 -- 172.21.15.99:0/635337824 wait complete. 2024-09-10T11:33:48.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:33:43.544162+0000', 'modified': '2024-09-10T11:33:45.565721+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5922': {'gid': 5922, 'name': 'b', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/1122563557', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 1122563557}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1122563557}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5922, 'qdb_cluster': [5922]} 2024-09-10T11:33:48.640 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:33:48.640 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T11:33:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.739+0000 7f4864496640 1 Processor -- start 2024-09-10T11:33:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.739+0000 7f4864496640 1 -- start start 2024-09-10T11:33:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4864496640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f485c0d1200 0x7f485c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4864496640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f485c06e5d0 0x7f485c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4864496640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f485c152e60 con 0x7f485c15c060 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4864496640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f485c06ef80 con 0x7f485c06e5d0 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4864496640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f485c0d15e0 con 0x7f485c0d1200 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861a31640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f485c06e5d0 0x7f485c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4862a33640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f485c15c060 0x7f485c15c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54810/0 (socket says 172.21.15.99:54810) 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4862a33640 1 -- 172.21.15.99:0/782848812 learned_addr learned my addr 172.21.15.99:0/782848812 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861a31640 1 -- 172.21.15.99:0/782848812 >> v2:172.21.15.196:3300/0 conn(0x7f485c0d1200 msgr2=0x7f485c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861a31640 1 --2- 172.21.15.99:0/782848812 >> v2:172.21.15.196:3300/0 conn(0x7f485c0d1200 0x7f485c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861a31640 1 -- 172.21.15.99:0/782848812 >> v1:172.21.15.99:6789/0 conn(0x7f485c15c060 legacy=0x7f485c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861a31640 1 -- 172.21.15.99:0/782848812 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c072680 con 0x7f485c06e5d0 2024-09-10T11:33:48.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.740+0000 7f4861230640 1 -- 172.21.15.99:0/782848812 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629967065 0 0) 0x7f485c152e60 con 0x7f485c15c060 2024-09-10T11:33:48.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.741+0000 7f4861a31640 1 --2- 172.21.15.99:0/782848812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c06e5d0 0x7f485c06ea40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f485c0d1850 tx=0x7f485002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c395f18b43f54c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.741+0000 7f4861230640 1 -- 172.21.15.99:0/782848812 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850037070 con 0x7f485c06e5d0 2024-09-10T11:33:48.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.741+0000 7f4861230640 1 -- 172.21.15.99:0/782848812 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4850032440 con 0x7f485c06e5d0 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.741+0000 7f4861230640 1 -- 172.21.15.99:0/782848812 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485002ae60 con 0x7f485c06e5d0 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 -- 172.21.15.99:0/782848812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c06e5d0 msgr2=0x7f485c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 --2- 172.21.15.99:0/782848812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c06e5d0 0x7f485c06ea40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f485c0d1850 tx=0x7f485002a110 comp rx=0 tx=0).stop 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 -- 172.21.15.99:0/782848812 shutdown_connections 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 --2- 172.21.15.99:0/782848812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c06e5d0 0x7f485c06ea40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 --2- 172.21.15.99:0/782848812 >> v2:172.21.15.196:3300/0 conn(0x7f485c0d1200 0x7f485c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 -- 172.21.15.99:0/782848812 >> 172.21.15.99:0/782848812 conn(0x7f485c06bc10 msgr2=0x7f485c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 -- 172.21.15.99:0/782848812 shutdown_connections 2024-09-10T11:33:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.742+0000 7f4864496640 1 -- 172.21.15.99:0/782848812 wait complete. 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.743+0000 7f4864496640 1 Processor -- start 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.743+0000 7f4864496640 1 -- start start 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c06e5d0 0x7f485c143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 0x7f485c148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f485c072f20 con 0x7f485c0d1200 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f485c072c40 con 0x7f485c06e5d0 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4864496640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f485c15cd90 con 0x7f485c15c060 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43766/0 (socket says 172.21.15.99:43766) 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 -- 172.21.15.99:0/2765158065 learned_addr learned my addr 172.21.15.99:0/2765158065 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4862232640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c06e5d0 0x7f485c143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4862a33640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 0x7f485c148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 -- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 msgr2=0x7f485c148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 0x7f485c148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 -- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c06e5d0 msgr2=0x7f485c143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c06e5d0 0x7f485c143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 -- 172.21.15.99:0/2765158065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c147d20 con 0x7f485c0d1200 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4862a33640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 0x7f485c148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.744+0000 7f4861a31640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4850000c00 tx=0x7f48500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.745+0000 7f485affd640 1 -- 172.21.15.99:0/2765158065 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850041070 con 0x7f485c0d1200 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.745+0000 7f485affd640 1 -- 172.21.15.99:0/2765158065 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485003c950 con 0x7f485c0d1200 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.745+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c148000 con 0x7f485c0d1200 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.745+0000 7f485affd640 1 -- 172.21.15.99:0/2765158065 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850037040 con 0x7f485c0d1200 2024-09-10T11:33:48.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.745+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c143960 con 0x7f485c0d1200 2024-09-10T11:33:48.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.746+0000 7f485affd640 1 -- 172.21.15.99:0/2765158065 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4850044070 con 0x7f485c0d1200 2024-09-10T11:33:48.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.746+0000 7f485affd640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 0x7f484806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.747+0000 7f485affd640 1 -- 172.21.15.99:0/2765158065 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f48500abcd0 con 0x7f485c0d1200 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.747+0000 7f4862232640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 0x7f484806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.747+0000 7f4862232640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 0x7f484806e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f484c009f20 tx=0x7f484c0094f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 Processor -- start 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 -- start start 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 0x55bc7b4056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b3ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b367520 con 0x55bc7b4052c0 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b30f3e0 con 0x55bc7b406a50 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.748+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55bc7b3634e0 con 0x55bc7b402a80 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43770/0 (socket says 172.21.15.99:43770) 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 -- 172.21.15.99:0/3918465011 learned_addr learned my addr 172.21.15.99:0/3918465011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862a33640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b3ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4861a31640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 0x55bc7b4056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 -- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 msgr2=0x55bc7b4056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 0x55bc7b4056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 -- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 msgr2=0x55bc7b3ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b3ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 -- 172.21.15.99:0/3918465011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7b32a2f0 con 0x55bc7b4052c0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4861a31640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 0x55bc7b4056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4862232640 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f484c0177d0 tx=0x7f484c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210bb00588745749 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4843fff640 1 -- 172.21.15.99:0/3918465011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c025070 con 0x55bc7b4052c0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4843fff640 1 -- 172.21.15.99:0/3918465011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c0203f0 con 0x55bc7b4052c0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.749+0000 7f4843fff640 1 -- 172.21.15.99:0/3918465011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c0205c0 con 0x55bc7b4052c0 2024-09-10T11:33:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- 172.21.15.99:0/3918465011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 msgr2=0x55bc7b44aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f484c0177d0 tx=0x7f484c011000 comp rx=0 tx=0).stop 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- 172.21.15.99:0/3918465011 shutdown_connections 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b3ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b402a80 0x55bc7b4056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 --2- 172.21.15.99:0/3918465011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b44aa60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- 172.21.15.99:0/3918465011 >> 172.21.15.99:0/3918465011 conn(0x55bc7b401a80 msgr2=0x55bc7b449f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- 172.21.15.99:0/3918465011 shutdown_connections 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- 172.21.15.99:0/3918465011 wait complete. 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.750+0000 7f486722f740 1 -- start start 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.751+0000 7f486722f740 1 Processor -- start 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.751+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b4052c0 0x55bc7b42f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.751+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b42d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.751+0000 7f486722f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b400f20 con 0x55bc7b42e9e0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b2f0450 con 0x55bc7b4052c0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55bc7b44b290 con 0x55bc7b406a50 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 10 client.0 _ll_register_callbacks cb 0x55bc7b079ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43778/0 (socket says 172.21.15.99:43778) 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 -- 172.21.15.99:0/2269749050 learned_addr learned my addr 172.21.15.99:0/2269749050 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862232640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b4052c0 0x55bc7b42f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4861a31640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b42d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 -- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 msgr2=0x55bc7b42d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b42d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 -- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b4052c0 msgr2=0x55bc7b42f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b4052c0 0x55bc7b42f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 -- 172.21.15.99:0/2269749050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7b07a2d0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f4862a33640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48540076f0 tx=0x7f48540016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597807af48a2699a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f48417fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854005e30 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f4854012560 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f484c0203f0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4854020050 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 10 client.6045 fetch_fsmap learned FSMap version 35 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55bc7b1446b0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.752+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c0203f0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f48540047e0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 10 client.6045 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 10 client.6045 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 20 client.6045 populate_metadata read hostname 'smithi099' 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b465ea0 0x55bc7b468290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 4 client.6045 mds_command: new command op to 5922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 --> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc7b374c50 con 0x55bc7b465ea0 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f4862232640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b465ea0 0x55bc7b468290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f48417fa640 10 client.6045 ms_handle_connect on v2:172.21.15.196:6833/1122563557 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.753+0000 7f4862232640 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b465ea0 0x55bc7b468290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26a619862b3b1b9 server_cookie=c71c1a326f8ac6d3 in_seq=0 out_seq=0 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f48417fa640 1 -- 172.21.15.99:0/2269749050 <== mds.0 v2:172.21.15.196:6833/1122563557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc7b374c50 con 0x55bc7b465ea0 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f48417fa640 10 client.6045 handle_command_reply: tid=0 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 client.6045 shutdown 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 msgr2=0x55bc7b42d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f48540076f0 tx=0x7f48540016f0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 shutdown_connections 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b42d760 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b465ea0 0x55bc7b468290 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b42d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 --2- 172.21.15.99:0/2269749050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b4052c0 0x55bc7b42f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 >> 172.21.15.99:0/2269749050 conn(0x55bc7b402a80 msgr2=0x55bc7b447fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.754+0000 7f48417fa640 0 client.6045 ms_handle_reset on v2:172.21.15.196:6833/1122563557 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.755+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 shutdown_connections 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.755+0000 7f486722f740 1 -- 172.21.15.99:0/2269749050 wait complete. 2024-09-10T11:33:48.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.755+0000 7f486722f740 20 client.6045 trim_cache size 0 max 16384 2024-09-10T11:33:48.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.772+0000 7f486722f740 1 Processor -- start 2024-09-10T11:33:48.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.772+0000 7f486722f740 1 -- start start 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 0x55bc7b44ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b44bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f485c1fe000 con 0x55bc7b42e9e0 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f485c144990 con 0x55bc7b406a50 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f485c149980 con 0x55bc7b4052c0 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f4862a33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f4862a33640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43786/0 (socket says 172.21.15.99:43786) 2024-09-10T11:33:48.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.773+0000 7f4862a33640 1 -- 172.21.15.99:0/2983056309 learned_addr learned my addr 172.21.15.99:0/2983056309 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 -- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 msgr2=0x55bc7b44ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4861a31640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b44bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862232640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 0x55bc7b44ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 0x55bc7b44ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 -- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 msgr2=0x55bc7b44bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b44bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 -- 172.21.15.99:0/2983056309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc7b3ff3e0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4861a31640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b44bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862232640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 0x55bc7b44ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f4862a33640 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4854007930 tx=0x7f485400d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce11c7ccaa90211 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f48437fe640 1 -- 172.21.15.99:0/2983056309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854012560 con 0x55bc7b42e9e0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f48437fe640 1 -- 172.21.15.99:0/2983056309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854018070 con 0x55bc7b42e9e0 2024-09-10T11:33:48.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f48437fe640 1 -- 172.21.15.99:0/2983056309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854012900 con 0x55bc7b42e9e0 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 -- 172.21.15.99:0/2983056309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 msgr2=0x55bc7b404030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4854007930 tx=0x7f485400d000 comp rx=0 tx=0).stop 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 -- 172.21.15.99:0/2983056309 shutdown_connections 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b42e9e0 0x55bc7b404030 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b406a50 0x55bc7b44bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 --2- 172.21.15.99:0/2983056309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b4052c0 0x55bc7b44ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 -- 172.21.15.99:0/2983056309 >> 172.21.15.99:0/2983056309 conn(0x55bc7b402a80 msgr2=0x55bc7b4476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 -- 172.21.15.99:0/2983056309 shutdown_connections 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.774+0000 7f486722f740 1 -- 172.21.15.99:0/2983056309 wait complete. 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.775+0000 7f486722f740 1 -- start start 2024-09-10T11:33:48.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.775+0000 7f486722f740 1 Processor -- start 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b45a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b42e9e0 0x55bc7b45a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b463b30 con 0x55bc7b4052c0 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55bc7b3ffef0 con 0x55bc7b42e9e0 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55bc7b464070 con 0x55bc7b406a50 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f486722f740 10 client.0 _ll_register_callbacks cb 0x7f4854007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43798/0 (socket says 172.21.15.99:43798) 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 -- 172.21.15.99:0/2477748959 learned_addr learned my addr 172.21.15.99:0/2477748959 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862a33640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b42e9e0 0x55bc7b45a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4861a31640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b45a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 -- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 msgr2=0x55bc7b45a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b45a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 -- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b42e9e0 msgr2=0x55bc7b45a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b42e9e0 0x55bc7b45a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 -- 172.21.15.99:0/2477748959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc7b42f7d0 con 0x55bc7b4052c0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f4862232640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f484c0175b0 tx=0x7f484c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c96d596730c140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.776+0000 7f48417fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c03a060 con 0x55bc7b4052c0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f484c020620 con 0x55bc7b4052c0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4854018070 con 0x55bc7b4052c0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f484c040050 con 0x55bc7b4052c0 2024-09-10T11:33:48.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 10 client.6051 fetch_fsmap learned FSMap version 35 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55bc7b403d50 con 0x55bc7b4052c0 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854018070 con 0x55bc7b4052c0 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f484c043020 con 0x55bc7b4052c0 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 10 client.6051 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 10 client.6051 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 20 client.6051 populate_metadata read hostname 'smithi099' 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b4cd580 0x55bc7b4460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 4 client.6051 mds_command: new command op to 5922 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:33:48.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 --> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] -- command(tid 0: {"prefix": "status"}) -- 0x55bc7b36efa0 con 0x55bc7b4cd580 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.777+0000 7f4861a31640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b4cd580 0x55bc7b4460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f48417fa640 10 client.6051 ms_handle_connect on v2:172.21.15.196:6833/1122563557 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f4861a31640 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b4cd580 0x55bc7b4460e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6823fdda854fe3ba server_cookie=c8ecdc2b4925041 in_seq=0 out_seq=0 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f48417fa640 1 -- 172.21.15.99:0/2477748959 <== mds.0 v2:172.21.15.196:6833/1122563557 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bc7b36efa0 con 0x55bc7b4cd580 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f48417fa640 10 client.6051 handle_command_reply: tid=0 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 client.6051 shutdown 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 msgr2=0x55bc7b4cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f484c0175b0 tx=0x7f484c004a40 comp rx=0 tx=0).stop 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 shutdown_connections 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bc7b42e9e0 0x55bc7b45a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:6833/1122563557,v1:172.21.15.196:6835/1122563557] conn(0x55bc7b4cd580 0x55bc7b4460e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bc7b406a50 0x55bc7b45a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 --2- 172.21.15.99:0/2477748959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc7b4052c0 0x55bc7b4cc3b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f48417fa640 0 client.6051 ms_handle_reset on v2:172.21.15.196:6833/1122563557 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 >> 172.21.15.99:0/2477748959 conn(0x55bc7b402a80 msgr2=0x55bc7b44ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 shutdown_connections 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 1 -- 172.21.15.99:0/2477748959 wait complete. 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.778+0000 7f486722f740 20 client.6051 trim_cache size 0 max 16384 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5922, 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:33:48.786 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5452064999999999, 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 35, 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 85, 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 9.5023465340000008 2024-09-10T11:33:48.787 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 msgr2=0x7f484806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 0x7f484806e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f484c009f20 tx=0x7f484c0094f0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 msgr2=0x7f485c1ef5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4850000c00 tx=0x7f48500046c0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 shutdown_connections 2024-09-10T11:33:48.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f484806be30 0x7f484806e2f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f485c15c060 0x7f485c148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f485c0d1200 0x7f485c1ef5a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 --2- 172.21.15.99:0/2765158065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f485c06e5d0 0x7f485c143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 >> 172.21.15.99:0/2765158065 conn(0x7f485c06bc10 msgr2=0x7f485c15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 shutdown_connections 2024-09-10T11:33:48.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.779+0000 7f4864496640 1 -- 172.21.15.99:0/2765158065 wait complete. 2024-09-10T11:33:48.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:33:48.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.887+0000 7fc37fb52640 1 Processor -- start 2024-09-10T11:33:48.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.887+0000 7fc37fb52640 1 -- start start 2024-09-10T11:33:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37fb52640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc378162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37fb52640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc378162e50 0x7fc37806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37fb52640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3780b38a0 con 0x7fc378148620 2024-09-10T11:33:48.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37fb52640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc37806dc30 con 0x7fc378162e50 2024-09-10T11:33:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37fb52640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc3780d00c0 con 0x7fc3780cfbe0 2024-09-10T11:33:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc378162e50 0x7fc37806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc378162e50 0x7fc37806e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43814/0 (socket says 172.21.15.99:43814) 2024-09-10T11:33:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 -- 172.21.15.99:0/2967801112 learned_addr learned my addr 172.21.15.99:0/2967801112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37d0ed640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc378162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 -- 172.21.15.99:0/2967801112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 msgr2=0x7fc378162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:48.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc378162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 -- 172.21.15.99:0/2967801112 >> v1:172.21.15.196:6789/0 conn(0x7fc378148620 legacy=0x7fc3780b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.888+0000 7fc37e0ef640 1 -- 172.21.15.99:0/2967801112 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3780d03a0 con 0x7fc378162e50 2024-09-10T11:33:48.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.889+0000 7fc37e0ef640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc37806e0c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc370009e50 tx=0x7fc37002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63c13376b35bca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.889+0000 7fc37c8ec640 1 -- 172.21.15.99:0/2967801112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370036070 con 0x7fc378162e50 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.889+0000 7fc37c8ec640 1 -- 172.21.15.99:0/2967801112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370031600 con 0x7fc378162e50 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.889+0000 7fc37c8ec640 1 -- 172.21.15.99:0/2967801112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3700319d0 con 0x7fc378162e50 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 -- 172.21.15.99:0/2967801112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 msgr2=0x7fc37806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc37806e0c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc370009e50 tx=0x7fc37002a710 comp rx=0 tx=0).stop 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 -- 172.21.15.99:0/2967801112 shutdown_connections 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc37806e0c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 --2- 172.21.15.99:0/2967801112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc378162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 -- 172.21.15.99:0/2967801112 >> 172.21.15.99:0/2967801112 conn(0x7fc3780b2a80 msgr2=0x7fc3780b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:48.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 -- 172.21.15.99:0/2967801112 shutdown_connections 2024-09-10T11:33:48.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.890+0000 7fc37fb52640 1 -- 172.21.15.99:0/2967801112 wait complete. 2024-09-10T11:33:48.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.891+0000 7fc37fb52640 1 Processor -- start 2024-09-10T11:33:48.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.891+0000 7fc37fb52640 1 -- start start 2024-09-10T11:33:48.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc3780ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc378148620 0x7fc378161a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc378148a00 con 0x7fc378162e50 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc37806d8c0 con 0x7fc378148620 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc378148b80 con 0x7fc3780cfbe0 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43830/0 (socket says 172.21.15.99:43830) 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 -- 172.21.15.99:0/856191683 learned_addr learned my addr 172.21.15.99:0/856191683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:48.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37d8ee640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc3780ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37d0ed640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc378148620 0x7fc378161a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 -- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 msgr2=0x7fc3780ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc3780ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 -- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc378148620 msgr2=0x7fc378161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc378148620 0x7fc378161a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:48.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 -- 172.21.15.99:0/856191683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37809d040 con 0x7fc378162e50 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37e0ef640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc370009e50 tx=0x7fc370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370041020 con 0x7fc378162e50 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3700077a0 con 0x7fc378162e50 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3780cd450 con 0x7fc378162e50 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.892+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370036040 con 0x7fc378162e50 2024-09-10T11:33:48.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.893+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37809d5e0 con 0x7fc378162e50 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.893+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3780b4e00 con 0x7fc378162e50 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.894+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc370044020 con 0x7fc378162e50 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.894+0000 7fc36a7fc640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 0x7fc35c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.894+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fc3700ab910 con 0x7fc378162e50 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.894+0000 7fc37d8ee640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 0x7fc35c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:48.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.895+0000 7fc37d8ee640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 0x7fc35c06e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc36c001780 tx=0x7fc36c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:48.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:48.899+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3780b4e00 con 0x7fc378162e50 2024-09-10T11:33:49.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.100+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc378162910 con 0x7fc378162e50 2024-09-10T11:33:49.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.100+0000 7fc36a7fc640 1 -- 172.21.15.99:0/856191683 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7fc37004b030 con 0x7fc378162e50 2024-09-10T11:33:49.102 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:33:49.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 msgr2=0x7fc35c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 0x7fc35c06e320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc36c001780 tx=0x7fc36c009040 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 msgr2=0x7fc3781f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc370009e50 tx=0x7fc370004a40 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 shutdown_connections 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc35c06be60 0x7fc35c06e320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc378162e50 0x7fc3781f4f30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc378148620 0x7fc378161a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 --2- 172.21.15.99:0/856191683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3780cfbe0 0x7fc3780ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 >> 172.21.15.99:0/856191683 conn(0x7fc3780b2a80 msgr2=0x7fc3780b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 shutdown_connections 2024-09-10T11:33:49.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49.102+0000 7fc37fb52640 1 -- 172.21.15.99:0/856191683 wait complete. 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:33:49.114 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:33:49.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.115 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:33:49.130 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:33:49.130 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:49.131 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42620sec preferred_lft 42620sec 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:33:49.132 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.133 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:33:49.133 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:33:49.133 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:33:49.133 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:49.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.133 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:33:49.194 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:33:49.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.196 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:33:49.251 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:33:49.252 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:33:49.252 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:33:49.252 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:33:49.252 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:33:49.252 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:33:49.252 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:33:49.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:33:49.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:33:49.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:33:49.253 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:33:49.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:33:49.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:33:49.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.377 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:33:49.393 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:33:49.393 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:33:49.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:33:49.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.393 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:33:49.457 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:33:49.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:33:49.458 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:49.513 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:33:49.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:33:49.514 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:49.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:33:49.634 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:33:49.691 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:33:49.692 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:33:49.693 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:33:49.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.694 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:33:49.746 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:33:49.747 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:33:49.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:33:49.813 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:33:49.870 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:33:49.871 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:33:49.872 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:33:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:33:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:33:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:33:49.873 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:33:49.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:49.873 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:33:49.874 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:33:49.869+0000 7f8ca7bc4480 -1 init, newargv = 0x56404ab1e7a0 newargc=15 2024-09-10T11:33:49.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[47006]: starting ceph client 2024-09-10T11:33:49.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[47006]: starting fuse 2024-09-10T11:33:49.888 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:33:49.889 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:33:49.889 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:33:49.890 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:33:49.891 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:33:49.891 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:33:49.891 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:49.891 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:33:49.891 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:33:50.005 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.47006.asok 2024-09-10T11:33:50.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:33:50.016 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47006.asok 2024-09-10T11:33:50.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47006.asok status 2024-09-10T11:33:50.149 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:33:50.149 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:33:50.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:33:50.149 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:33:50.149 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "47006", 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6063, 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6063 2024-09-10T11:33:50.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 6063338 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 6063338 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6063 192.168.0.1:0/6063338", 2024-09-10T11:33:50.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/6063338", 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 35, 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 85, 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:33:50.152 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:33:50.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.161 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:33:50.161 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:50.176 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:33:50.177 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:50.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:33:50.203 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-09-10T11:33:50.201+0000 7fb5498bd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:33:50.243 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:33:50.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.381+0000 7fcba71f1640 1 Processor -- start 2024-09-10T11:33:50.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.381+0000 7fcba71f1640 1 -- start start 2024-09-10T11:33:50.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba71f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba71f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcba015b2d0 0x7fcba00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba71f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcba006ee10 con 0x7fcba00cd290 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba71f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcba00b0290 con 0x7fcba015b2d0 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba71f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcba006e570 con 0x7fcba015aef0 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcba015b2d0 0x7fcba00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba4f8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba578e640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcba00cd290 0x7fcba00714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:52940/0 (socket says 172.21.15.99:52940) 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcba015b2d0 0x7fcba00afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:56508/0 (socket says 172.21.15.99:56508) 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba578e640 1 -- 172.21.15.99:0/905960831 learned_addr learned my addr 172.21.15.99:0/905960831 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba4f8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:41442/0 (socket says 172.21.15.99:41442) 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9f7fe640 1 -- 172.21.15.99:0/905960831 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749186015 0 0) 0x7fcba006ee10 con 0x7fcba00cd290 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9f7fe640 1 -- 172.21.15.99:0/905960831 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb90003520 con 0x7fcba00cd290 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 -- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 msgr2=0x7fcba00b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:50.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 -- 172.21.15.99:0/905960831 >> v1:172.21.15.196:6789/0 conn(0x7fcba00cd290 legacy=0x7fcba00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcb9ffff640 1 -- 172.21.15.99:0/905960831 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba006e850 con 0x7fcba015b2d0 2024-09-10T11:33:50.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.382+0000 7fcba4f8d640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.383+0000 7fcb9ffff640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015b2d0 0x7fcba00afbb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0097a0 tx=0x7fcb8c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7e24c98f4a03fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.383+0000 7fcb9f7fe640 1 -- 172.21.15.99:0/905960831 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c037070 con 0x7fcba015b2d0 2024-09-10T11:33:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.383+0000 7fcb9f7fe640 1 -- 172.21.15.99:0/905960831 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8c032440 con 0x7fcba015b2d0 2024-09-10T11:33:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.383+0000 7fcb9f7fe640 1 -- 172.21.15.99:0/905960831 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c032810 con 0x7fcba015b2d0 2024-09-10T11:33:50.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 -- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015b2d0 msgr2=0x7fcba00afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015b2d0 0x7fcba00afbb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0097a0 tx=0x7fcb8c02a180 comp rx=0 tx=0).stop 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 -- 172.21.15.99:0/905960831 shutdown_connections 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015b2d0 0x7fcba00afbb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 --2- 172.21.15.99:0/905960831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba015aef0 0x7fcba00b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 -- 172.21.15.99:0/905960831 >> 172.21.15.99:0/905960831 conn(0x7fcba006bc10 msgr2=0x7fcba006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 -- 172.21.15.99:0/905960831 shutdown_connections 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.384+0000 7fcba71f1640 1 -- 172.21.15.99:0/905960831 wait complete. 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.385+0000 7fcba71f1640 1 Processor -- start 2024-09-10T11:33:50.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.385+0000 7fcba71f1640 1 -- start start 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 0x7fcba0149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015aef0 0x7fcba014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 0x7fcba014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0071d30 con 0x7fcba015b2d0 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0071bb0 con 0x7fcba015aef0 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba71f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcba0071eb0 con 0x7fcba00cd290 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba4f8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 0x7fcba0149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba578e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 0x7fcba014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba4f8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 0x7fcba0149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:41444/0 (socket says 172.21.15.99:41444) 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcba4f8d640 1 -- 172.21.15.99:0/3600818545 learned_addr learned my addr 172.21.15.99:0/3600818545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.386+0000 7fcb9ffff640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015aef0 0x7fcba014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 -- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 msgr2=0x7fcba0149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 0x7fcba0149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 -- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015aef0 msgr2=0x7fcba014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015aef0 0x7fcba014e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 -- 172.21.15.99:0/3600818545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba014a650 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba578e640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 0x7fcba014ea60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800caa0 tx=0x7fcb9800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98017070 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9800a6d0 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba014a8f0 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800aac0 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.387+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba014aeb0 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.388+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba00716f0 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.388+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcb9801a070 con 0x7fcba015b2d0 2024-09-10T11:33:50.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.389+0000 7fcb9d7fa640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 0x7fcb8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.389+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fcb9808a5d0 con 0x7fcba015b2d0 2024-09-10T11:33:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.389+0000 7fcba4f8d640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 0x7fcb8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:50.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.389+0000 7fcba4f8d640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 0x7fcb8406e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb940095e0 tx=0x7fcb94009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:50.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.393+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb98058930 con 0x7fcba015b2d0 2024-09-10T11:33:50.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.591+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcba00761b0 con 0x7fcba015b2d0 2024-09-10T11:33:50.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.592+0000 7fcb9d7fa640 1 -- 172.21.15.99:0/3600818545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7fcb9805cc30 con 0x7fcba015b2d0 2024-09-10T11:33:50.593 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 msgr2=0x7fcb8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 0x7fcb8406e2f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcb940095e0 tx=0x7fcb94009190 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 msgr2=0x7fcba014ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 0x7fcba014ea60 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800caa0 tx=0x7fcb9800cf60 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 shutdown_connections 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcb8406be30 0x7fcb8406e2f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcba015b2d0 0x7fcba014ea60 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcba015aef0 0x7fcba014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 --2- 172.21.15.99:0/3600818545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcba00cd290 0x7fcba0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 >> 172.21.15.99:0/3600818545 conn(0x7fcba006bc10 msgr2=0x7fcba00b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:50.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 shutdown_connections 2024-09-10T11:33:50.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:50.594+0000 7fcba71f1640 1 -- 172.21.15.99:0/3600818545 wait complete. 2024-09-10T11:33:50.605 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:33:50.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:33:50.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:33:50.605 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:33:50.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:50.606 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:33:50.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:33:50.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:33:50.606 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:33:50.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.606 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:50.626 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42634sec preferred_lft 42634sec 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:33:50.627 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:33:50.628 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:33:50.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.629 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:33:50.695 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:33:50.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.697 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:33:50.755 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:33:50.756 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:33:50.756 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:33:50.756 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:33:50.756 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:33:50.756 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:33:50.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:33:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:33:50.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:33:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:33:50.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.889 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:33:50.904 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:33:50.905 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:33:50.905 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:33:50.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:50.905 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:33:50.970 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:33:50.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:33:50.970 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:51.026 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:33:51.027 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:33:51.027 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:51.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:33:51.147 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:33:51.203 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:33:51.204 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:33:51.205 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:33:51.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:51.206 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:33:51.258 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:33:51.258 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:33:51.301 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:33:51.326 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:33:51.383 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:33:51.384 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:33:51.385 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:33:51.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:51.386 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:33:51.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:33:51.382+0000 7ff77108d480 -1 init, newargv = 0x564d83538400 newargc=15 2024-09-10T11:33:51.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36123]: starting ceph client 2024-09-10T11:33:51.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36123]: starting fuse 2024-09-10T11:33:51.401 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:33:51.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:33:51.402 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:33:51.403 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:33:51.404 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:33:51.404 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:33:51.519 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.36123.asok 2024-09-10T11:33:51.522 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:33:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:33:51.529 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36123.asok 2024-09-10T11:33:51.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:51.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36123.asok status 2024-09-10T11:33:51.662 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "36123", 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5767, 2024-09-10T11:33:51.663 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 5767 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2070302074 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:33:51.664 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2070302074 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.5767 192.168.0.1:0/2070302074", 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/2070302074", 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 35, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 85, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:33:51.665 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:33:51.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:51.671 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:33:51.671 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:51.687 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:33:51.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:51.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:33:51.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:33:51.753 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:33:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.858+0000 7fd91a96f640 1 Processor -- start 2024-09-10T11:33:51.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.858+0000 7fd91a96f640 1 -- start start 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd91a96f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd91415f1e0 0x7fd9140b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd91a96f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9140b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd91a96f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9140b40e0 con 0x7fd91415ee00 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd91a96f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9140b5ab0 con 0x7fd91415f1e0 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd91a96f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd914073c50 con 0x7fd9140b0090 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd913fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd91415ee00 0x7fd9140b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54814/0 (socket says 172.21.15.99:54814) 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.859+0000 7fd913fff640 1 -- 172.21.15.99:0/540517215 learned_addr learned my addr 172.21.15.99:0/540517215 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:33:51.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 144232559 0 0) 0x7fd9140b40e0 con 0x7fd91415ee00 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd900003520 con 0x7fd91415ee00 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd918f0c640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9140b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 --2- 172.21.15.99:0/540517215 >> v2:172.21.15.196:3300/0 conn(0x7fd91415f1e0 0x7fd9140b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3940876481 0 0) 0x7fd900003520 con 0x7fd91415ee00 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 msgr2=0x7fd9140b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9140b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 >> v2:172.21.15.196:3300/0 conn(0x7fd91415f1e0 msgr2=0x7fd9140b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 --2- 172.21.15.99:0/540517215 >> v2:172.21.15.196:3300/0 conn(0x7fd91415f1e0 0x7fd9140b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9140b4380 con 0x7fd91415ee00 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fd904001670 con 0x7fd91415ee00 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 >> v1:172.21.15.99:6789/0 conn(0x7fd91415ee00 legacy=0x7fd9140b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900003f80 0x7fd900004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9140b5ab0 con 0x7fd900003f80 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd914073c50 con 0x7fd900004d40 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd90000b9c0 con 0x7fd9140b0090 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900003f80 0x7fd900004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd913fff640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd918f0c640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 -- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 msgr2=0x7fd9000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9140b0090 0x7fd9000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 -- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd900004d40 msgr2=0x7fd9000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.860+0000 7fd90bfff640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd900004d40 0x7fd9000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.861+0000 7fd90bfff640 1 -- 172.21.15.99:0/540517215 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd900003520 con 0x7fd900003f80 2024-09-10T11:33:51.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.861+0000 7fd90bfff640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900003f80 0x7fd900004800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009af0 tx=0x7fd900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110e32ef322b593b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:51.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.861+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc036070 con 0x7fd900003f80 2024-09-10T11:33:51.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.861+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc031400 con 0x7fd900003f80 2024-09-10T11:33:51.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.861+0000 7fd9137fe640 1 -- 172.21.15.99:0/540517215 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc009690 con 0x7fd900003f80 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 -- 172.21.15.99:0/540517215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900003f80 msgr2=0x7fd900004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 --2- 172.21.15.99:0/540517215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd900003f80 0x7fd900004800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009af0 tx=0x7fd900005830 comp rx=0 tx=0).stop 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd918f0c640 1 -- 172.21.15.99:0/540517215 reap_dead start 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 -- 172.21.15.99:0/540517215 shutdown_connections 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 -- 172.21.15.99:0/540517215 >> 172.21.15.99:0/540517215 conn(0x7fd9140730c0 msgr2=0x7fd9140734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 -- 172.21.15.99:0/540517215 shutdown_connections 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.862+0000 7fd91a96f640 1 -- 172.21.15.99:0/540517215 wait complete. 2024-09-10T11:33:51.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.863+0000 7fd91a96f640 1 Processor -- start 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.863+0000 7fd91a96f640 1 -- start start 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd900003f80 0x7fd9141505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd91409eb60 0x7fd9140cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9140b0600 con 0x7fd91409efe0 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9140afd60 con 0x7fd91409eb60 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd91a96f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9140b0780 con 0x7fd900003f80 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43868/0 (socket says 172.21.15.99:43868) 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 -- 172.21.15.99:0/1795261607 learned_addr learned my addr 172.21.15.99:0/1795261607 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd913fff640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd900003f80 0x7fd9141505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd90bfff640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd91409eb60 0x7fd9140cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 -- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd900003f80 msgr2=0x7fd9141505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd900003f80 0x7fd9141505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 -- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd91409eb60 msgr2=0x7fd9140cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd91409eb60 0x7fd9140cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.864+0000 7fd918f0c640 1 -- 172.21.15.99:0/1795261607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd914150b30 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd918f0c640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9140721a0 tx=0x7fd90c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00fdb0 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd90c01d070 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91414df00 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c015670 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.865+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91414e4c0 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.866+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9140b5f90 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.866+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd90c015810 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.867+0000 7fd9117fa640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 0x7fd8f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.867+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fd90c08b4a0 con 0x7fd91409efe0 2024-09-10T11:33:51.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.867+0000 7fd913fff640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 0x7fd8f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:51.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.867+0000 7fd913fff640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 0x7fd8f406e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd904001c60 tx=0x7fd904001e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:51.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:51.871+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd90c010040 con 0x7fd91409efe0 2024-09-10T11:33:52.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.070+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd91406d770 con 0x7fd91409efe0 2024-09-10T11:33:52.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.071+0000 7fd9117fa640 1 -- 172.21.15.99:0/1795261607 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7fd90c059800 con 0x7fd91409efe0 2024-09-10T11:33:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:33:52.072 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:33:52.073 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 msgr2=0x7fd8f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 0x7fd8f406e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd904001c60 tx=0x7fd904001e40 comp rx=0 tx=0).stop 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 msgr2=0x7fd9140cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9140721a0 tx=0x7fd90c00f580 comp rx=0 tx=0).stop 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 shutdown_connections 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be60 0x7fd8f406e320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:52.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91409efe0 0x7fd9140cda80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd91409eb60 0x7fd9140cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 --2- 172.21.15.99:0/1795261607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd900003f80 0x7fd9141505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 >> 172.21.15.99:0/1795261607 conn(0x7fd9140730c0 msgr2=0x7fd9140af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 shutdown_connections 2024-09-10T11:33:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:33:52.073+0000 7fd91a96f640 1 -- 172.21.15.99:0/1795261607 wait complete. 2024-09-10T11:33:52.085 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:33:52.200 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":5922,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9581480180000002,"mdsmap_epoch":35,"osdmap_epoch":85,"osdmap_epoch_barrier":85,"uptime":12.915288789} 2024-09-10T11:33:52.201 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 5922, "mdsmap_epoch": 35, "osdmap_epoch": 85, "osdmap_epoch_barrier": 85, "rank_uptime": 7.958148018, "state": "up:active", "uptime": 12.915288789, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:33:52.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:33:52.315 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:33:52.316 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:33:52.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-09-10T11:33:52.418 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2024-09-10T11:33:52.419 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-09-10T11:33:52.419 INFO:tasks.cephfs.test_full:pool capacity 557078080, 531MB should be enough to fill it 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> import time 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2024-09-10T11:33:52.419 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.420 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(531 * 1.1)): 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote bytes via buffered write, moving on to fsync") 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> raise 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> break 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:33:52.421 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed successfully") 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> break 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> if n > 531 * 0.9: 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> # Be cautious in the last region where we expect to hit 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> if not full: 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.422 DEBUG:teuthology.orchestra.run.smithi099:> # close() should not raise an error because we already caught it in 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> # since then because all IOs got cancelled on the full flag. 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> print("calling close") 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> print("close() did not raise error") 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-09-10T11:33:52.423 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:33:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2024-09-10T11:33:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-09-10T11:33:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.619 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 1.00 MB 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 2.00 MB 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 3.00 MB 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 4.00 MB 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.620 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 5.00 MB 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 6.00 MB 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 7.00 MB 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 8.00 MB 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 9.00 MB 2024-09-10T11:33:54.621 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 10.00 MB 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 11.00 MB 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 12.00 MB 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 13.00 MB 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.622 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 14.00 MB 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 15.00 MB 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 16.00 MB 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 17.00 MB 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 18.00 MB 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.623 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 19.00 MB 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 20.00 MB 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.624 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 21.00 MB 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 22.00 MB 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 23.00 MB 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 24.00 MB 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.625 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 25.00 MB 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 26.00 MB 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 27.00 MB 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 28.00 MB 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 29.00 MB 2024-09-10T11:33:54.626 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 30.00 MB 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 31.00 MB 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 32.00 MB 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 33.00 MB 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.627 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 34.00 MB 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 35.00 MB 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 36.00 MB 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 37.00 MB 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 38.00 MB 2024-09-10T11:33:54.628 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 39.00 MB 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 40.00 MB 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 41.00 MB 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 42.00 MB 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.629 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 43.00 MB 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 44.00 MB 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 45.00 MB 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 46.00 MB 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 47.00 MB 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.630 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 48.00 MB 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 49.00 MB 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 50.00 MB 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 51.00 MB 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.631 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 52.00 MB 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 53.00 MB 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 54.00 MB 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 55.00 MB 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.632 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 56.00 MB 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 57.00 MB 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 58.00 MB 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 59.00 MB 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 60.00 MB 2024-09-10T11:33:54.633 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 61.00 MB 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 62.00 MB 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 63.00 MB 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 64.00 MB 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.634 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 65.00 MB 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 66.00 MB 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 67.00 MB 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 68.00 MB 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 69.00 MB 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.635 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 70.00 MB 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 71.00 MB 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 72.00 MB 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 73.00 MB 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 74.00 MB 2024-09-10T11:33:54.636 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 75.00 MB 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 76.00 MB 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 77.00 MB 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.637 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 78.00 MB 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 79.00 MB 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.638 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 80.00 MB 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 81.00 MB 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 82.00 MB 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 83.00 MB 2024-09-10T11:33:54.930 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 84.00 MB 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 85.00 MB 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 86.00 MB 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 87.00 MB 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.931 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 88.00 MB 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 89.00 MB 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 90.00 MB 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 91.00 MB 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.932 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 92.00 MB 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 93.00 MB 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 94.00 MB 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 95.00 MB 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 96.00 MB 2024-09-10T11:33:54.933 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 97.00 MB 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 98.00 MB 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 99.00 MB 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 100.00 MB 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.934 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 101.00 MB 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 102.00 MB 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 103.00 MB 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 104.00 MB 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 105.00 MB 2024-09-10T11:33:54.935 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 106.00 MB 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 107.00 MB 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 108.00 MB 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 109.00 MB 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.936 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 110.00 MB 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 111.00 MB 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 112.00 MB 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 113.00 MB 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 114.00 MB 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.937 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 115.00 MB 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 116.00 MB 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 117.00 MB 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 118.00 MB 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 119.00 MB 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 120.00 MB 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 121.00 MB 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 122.00 MB 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 123.00 MB 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.939 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 124.00 MB 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 125.00 MB 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 126.00 MB 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 127.00 MB 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 128.00 MB 2024-09-10T11:33:54.940 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 129.00 MB 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 130.00 MB 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 131.00 MB 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 132.00 MB 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.941 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 133.00 MB 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 134.00 MB 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 135.00 MB 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 136.00 MB 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 137.00 MB 2024-09-10T11:33:54.942 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 138.00 MB 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 139.00 MB 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 140.00 MB 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 141.00 MB 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.943 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 142.00 MB 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 143.00 MB 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 144.00 MB 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 145.00 MB 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 146.00 MB 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.944 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 147.00 MB 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 148.00 MB 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 149.00 MB 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 150.00 MB 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.945 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 151.00 MB 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 152.00 MB 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 153.00 MB 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 154.00 MB 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 155.00 MB 2024-09-10T11:33:54.946 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 156.00 MB 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 157.00 MB 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 158.00 MB 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 159.00 MB 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:54.947 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 160.00 MB 2024-09-10T11:33:54.948 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.222 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.222 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 161.00 MB 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 162.00 MB 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 163.00 MB 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 164.00 MB 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 165.00 MB 2024-09-10T11:33:55.223 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 166.00 MB 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 167.00 MB 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 168.00 MB 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 169.00 MB 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.224 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 170.00 MB 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 171.00 MB 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 172.00 MB 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 173.00 MB 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.225 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 174.00 MB 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 175.00 MB 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 176.00 MB 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 177.00 MB 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 178.00 MB 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.226 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 179.00 MB 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 180.00 MB 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 181.00 MB 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.227 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 182.00 MB 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 183.00 MB 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 184.00 MB 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 185.00 MB 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 186.00 MB 2024-09-10T11:33:55.228 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 187.00 MB 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 188.00 MB 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 189.00 MB 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 190.00 MB 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.229 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 191.00 MB 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 192.00 MB 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 193.00 MB 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 194.00 MB 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 195.00 MB 2024-09-10T11:33:55.230 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 196.00 MB 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 197.00 MB 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 198.00 MB 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 199.00 MB 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.231 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 200.00 MB 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 201.00 MB 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 202.00 MB 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 203.00 MB 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 204.00 MB 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 205.00 MB 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 206.00 MB 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 207.00 MB 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 208.00 MB 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 209.00 MB 2024-09-10T11:33:55.233 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 210.00 MB 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 211.00 MB 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 212.00 MB 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 213.00 MB 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 214.00 MB 2024-09-10T11:33:55.234 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 215.00 MB 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 216.00 MB 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 217.00 MB 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 218.00 MB 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 219.00 MB 2024-09-10T11:33:55.235 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 220.00 MB 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 221.00 MB 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 222.00 MB 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 223.00 MB 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.236 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 224.00 MB 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 225.00 MB 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 226.00 MB 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 227.00 MB 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.237 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 228.00 MB 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 229.00 MB 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 230.00 MB 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 231.00 MB 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 232.00 MB 2024-09-10T11:33:55.238 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 233.00 MB 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 234.00 MB 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 235.00 MB 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 236.00 MB 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.239 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 237.00 MB 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 238.00 MB 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 239.00 MB 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 240.00 MB 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.240 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 241.00 MB 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 242.00 MB 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.503 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 243.00 MB 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 244.00 MB 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 245.00 MB 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 246.00 MB 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.504 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 247.00 MB 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 248.00 MB 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 249.00 MB 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 250.00 MB 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 251.00 MB 2024-09-10T11:33:55.505 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 252.00 MB 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 253.00 MB 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 254.00 MB 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 255.00 MB 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.506 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 256.00 MB 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 257.00 MB 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 258.00 MB 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 259.00 MB 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 260.00 MB 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.507 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 261.00 MB 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 262.00 MB 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 263.00 MB 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 264.00 MB 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.508 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 265.00 MB 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 266.00 MB 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 267.00 MB 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 268.00 MB 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 269.00 MB 2024-09-10T11:33:55.509 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 270.00 MB 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 271.00 MB 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 272.00 MB 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 273.00 MB 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.510 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 274.00 MB 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 275.00 MB 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 276.00 MB 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 277.00 MB 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 278.00 MB 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.511 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 279.00 MB 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 280.00 MB 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 281.00 MB 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 282.00 MB 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 283.00 MB 2024-09-10T11:33:55.512 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 284.00 MB 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 285.00 MB 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 286.00 MB 2024-09-10T11:33:55.513 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 287.00 MB 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 288.00 MB 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 289.00 MB 2024-09-10T11:33:55.514 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 290.00 MB 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 291.00 MB 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 292.00 MB 2024-09-10T11:33:55.515 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 293.00 MB 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 294.00 MB 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.516 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 295.00 MB 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 296.00 MB 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 297.00 MB 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.517 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 298.00 MB 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 299.00 MB 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 300.00 MB 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 301.00 MB 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 302.00 MB 2024-09-10T11:33:55.518 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 303.00 MB 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 304.00 MB 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 305.00 MB 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 306.00 MB 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.519 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 307.00 MB 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 308.00 MB 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 309.00 MB 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 310.00 MB 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 311.00 MB 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.520 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 312.00 MB 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 313.00 MB 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 314.00 MB 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 315.00 MB 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.521 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 316.00 MB 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 317.00 MB 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 318.00 MB 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 319.00 MB 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 320.00 MB 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.522 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 321.00 MB 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 322.00 MB 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 323.00 MB 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 324.00 MB 2024-09-10T11:33:55.772 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 325.00 MB 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 326.00 MB 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 327.00 MB 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 328.00 MB 2024-09-10T11:33:55.773 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 329.00 MB 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 330.00 MB 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 331.00 MB 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 332.00 MB 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.774 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 333.00 MB 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 334.00 MB 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 335.00 MB 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 336.00 MB 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 337.00 MB 2024-09-10T11:33:55.775 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 338.00 MB 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 339.00 MB 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 340.00 MB 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 341.00 MB 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.776 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 342.00 MB 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 343.00 MB 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 344.00 MB 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 345.00 MB 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.777 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 346.00 MB 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 347.00 MB 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 348.00 MB 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 349.00 MB 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 350.00 MB 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 351.00 MB 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 352.00 MB 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 353.00 MB 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 354.00 MB 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 355.00 MB 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 356.00 MB 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 357.00 MB 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 358.00 MB 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 359.00 MB 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 360.00 MB 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 361.00 MB 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 362.00 MB 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 363.00 MB 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 364.00 MB 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 365.00 MB 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 366.00 MB 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 367.00 MB 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 368.00 MB 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 369.00 MB 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 370.00 MB 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 371.00 MB 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 372.00 MB 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 373.00 MB 2024-09-10T11:33:55.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 374.00 MB 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 375.00 MB 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 376.00 MB 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 377.00 MB 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 378.00 MB 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 379.00 MB 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 380.00 MB 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 381.00 MB 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 382.00 MB 2024-09-10T11:33:55.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 383.00 MB 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 384.00 MB 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 385.00 MB 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 386.00 MB 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.786 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 387.00 MB 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 388.00 MB 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 389.00 MB 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 390.00 MB 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 391.00 MB 2024-09-10T11:33:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 392.00 MB 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 393.00 MB 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 394.00 MB 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 395.00 MB 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.788 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 396.00 MB 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 397.00 MB 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 398.00 MB 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 399.00 MB 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 400.00 MB 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:33:55.789 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:33:55.790 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 401.00 MB 2024-09-10T11:33:58.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:33:58.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:33:59.274 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:33:59.273+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-09-10T11:34:05.665 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:05.663+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 402.00 MB 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 403.00 MB 2024-09-10T11:34:11.108 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 404.00 MB 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 405.00 MB 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 406.00 MB 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 407.00 MB 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.109 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 408.00 MB 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 409.00 MB 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 410.00 MB 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 411.00 MB 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 412.00 MB 2024-09-10T11:34:11.110 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 413.00 MB 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 414.00 MB 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 415.00 MB 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 416.00 MB 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.111 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 417.00 MB 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 418.00 MB 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 419.00 MB 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 420.00 MB 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.112 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 421.00 MB 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 422.00 MB 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 423.00 MB 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.113 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 424.00 MB 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 425.00 MB 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 426.00 MB 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 427.00 MB 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 428.00 MB 2024-09-10T11:34:11.114 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 429.00 MB 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 430.00 MB 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 431.00 MB 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 432.00 MB 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.115 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 433.00 MB 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 434.00 MB 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 435.00 MB 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 436.00 MB 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.116 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 437.00 MB 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 438.00 MB 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 439.00 MB 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 440.00 MB 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 441.00 MB 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.117 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 442.00 MB 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 443.00 MB 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 444.00 MB 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 445.00 MB 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 446.00 MB 2024-09-10T11:34:11.118 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 447.00 MB 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 448.00 MB 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 449.00 MB 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 450.00 MB 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.119 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 451.00 MB 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 452.00 MB 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 453.00 MB 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 454.00 MB 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.120 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 455.00 MB 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 456.00 MB 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 457.00 MB 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 458.00 MB 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 459.00 MB 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.121 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 460.00 MB 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 461.00 MB 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 462.00 MB 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 463.00 MB 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 464.00 MB 2024-09-10T11:34:11.122 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 465.00 MB 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 466.00 MB 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 467.00 MB 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 468.00 MB 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.123 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 469.00 MB 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 470.00 MB 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 471.00 MB 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 472.00 MB 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.124 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 473.00 MB 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 474.00 MB 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 475.00 MB 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 476.00 MB 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 477.00 MB 2024-09-10T11:34:11.125 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 478.00 MB 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 479.00 MB 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:Reached fullness after 480.00 MB 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:calling close 2024-09-10T11:34:11.126 INFO:teuthology.orchestra.run.smithi099.stdout:close() did not raise error 2024-09-10T11:34:11.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:34:11.128 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:34:11.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:11.186 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:11.241 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:34:11.242 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:11.242 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:11.242 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:11.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:34:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.405+0000 7fa1ffde4640 1 Processor -- start 2024-09-10T11:34:11.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.405+0000 7fa1ffde4640 1 -- start start 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1ffde4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa1f80d1200 0x7fa1f8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1ffde4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f8152e60 con 0x7fa1f806e5d0 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1ffde4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f80d15e0 con 0x7fa1f815c770 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1ffde4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f8072680 con 0x7fa1f80d1200 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1fd37f640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa1f806e5d0 0x7fa1f815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33934/0 (socket says 172.21.15.99:33934) 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.406+0000 7fa1fd37f640 1 -- 172.21.15.99:0/870497074 learned_addr learned my addr 172.21.15.99:0/870497074 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4137614900 0 0) 0x7fa1f8152e60 con 0x7fa1f806e5d0 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/870497074 >> v2:172.21.15.196:3300/0 conn(0x7fa1f80d1200 0x7fa1f8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e4003520 con 0x7fa1f806e5d0 2024-09-10T11:34:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1458418639 0 0) 0x7fa1e4003520 con 0x7fa1f806e5d0 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 >> v2:172.21.15.196:3300/0 conn(0x7fa1f80d1200 msgr2=0x7fa1f8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 --2- 172.21.15.99:0/870497074 >> v2:172.21.15.196:3300/0 conn(0x7fa1f80d1200 0x7fa1f8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 >> v1:172.21.15.196:6789/0 conn(0x7fa1f815c770 legacy=0x7fa1f8159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f8072960 con 0x7fa1f806e5d0 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa1e8001670 con 0x7fa1f806e5d0 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 >> v1:172.21.15.99:6789/0 conn(0x7fa1f806e5d0 legacy=0x7fa1f815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 0x7fa1e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1e4004800 0x7fa1e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40051c0 0x7fa1e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8072680 con 0x7fa1e40039b0 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e400c160 con 0x7fa1e40051c0 2024-09-10T11:34:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e400c360 con 0x7fa1e4004800 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 0x7fa1e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fe381640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40051c0 0x7fa1e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fd37f640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1e4004800 0x7fa1e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 -- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1e4004800 msgr2=0x7fa1e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1e4004800 0x7fa1e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 -- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40051c0 msgr2=0x7fa1e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40051c0 0x7fa1e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fe381640 1 -- 172.21.15.99:0/870497074 reap_dead start 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.407+0000 7fa1fdb80640 1 -- 172.21.15.99:0/870497074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e4003520 con 0x7fa1e40039b0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.408+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 0x7fa1e4003dd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009ac0 tx=0x7fa1e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb42421428fab6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.408+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4036070 con 0x7fa1e40039b0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.408+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f4031400 con 0x7fa1e40039b0 2024-09-10T11:34:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.408+0000 7fa1fcb7e640 1 -- 172.21.15.99:0/870497074 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4009690 con 0x7fa1e40039b0 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 -- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 msgr2=0x7fa1e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 0x7fa1e4003dd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009ac0 tx=0x7fa1e4005830 comp rx=0 tx=0).stop 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 -- 172.21.15.99:0/870497074 shutdown_connections 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/870497074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1e40039b0 0x7fa1e4003dd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 -- 172.21.15.99:0/870497074 >> 172.21.15.99:0/870497074 conn(0x7fa1f806bc10 msgr2=0x7fa1f806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 -- 172.21.15.99:0/870497074 shutdown_connections 2024-09-10T11:34:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.409+0000 7fa1ffde4640 1 -- 172.21.15.99:0/870497074 wait complete. 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.410+0000 7fa1ffde4640 1 Processor -- start 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.410+0000 7fa1ffde4640 1 -- start start 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40039b0 0x7fa1f8157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f81f6460 0x7fa1f81521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f815cd90 con 0x7fa1f80b4840 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8072f20 con 0x7fa1e40039b0 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f8073440 con 0x7fa1f81f6460 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33844/0 (socket says 172.21.15.99:33844) 2024-09-10T11:34:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 -- 172.21.15.99:0/3871457145 learned_addr learned my addr 172.21.15.99:0/3871457145 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fd37f640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f81f6460 0x7fa1f81521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40039b0 0x7fa1f8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 -- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f81f6460 msgr2=0x7fa1f81521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f81f6460 0x7fa1f81521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 -- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40039b0 msgr2=0x7fa1f8157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40039b0 0x7fa1f8157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f8158e80 con 0x7fa1f80b4840 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1fe381640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8071b80 tx=0x7fa1f000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f001a070 con 0x7fa1f80b4840 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f0006d60 con 0x7fa1f80b4840 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.411+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f81f68e0 con 0x7fa1f80b4840 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.412+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0003bb0 con 0x7fa1f80b4840 2024-09-10T11:34:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.412+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f8158ba0 con 0x7fa1f80b4840 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.413+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f806ea40 con 0x7fa1f80b4840 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.413+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1f001d070 con 0x7fa1f80b4840 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.414+0000 7fa1ee7fc640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 0x7fa1dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.414+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 0x7fa1dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.414+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f008a290 con 0x7fa1f80b4840 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.414+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7fa1dc0739c0 con 0x7fa1f80b4840 2024-09-10T11:34:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.414+0000 7fa1fdb80640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 0x7fa1dc06e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4033000 tx=0x7fa1f4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.418+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1f81f68e0 con 0x7fa1f80b4840 2024-09-10T11:34:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.598+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1f80b5230 con 0x7fa1f80b4840 2024-09-10T11:34:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.599+0000 7fa1ee7fc640 1 -- 172.21.15.99:0/3871457145 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7fa1f8158e80 con 0x7fa1f80b4840 2024-09-10T11:34:11.601 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 msgr2=0x7fa1dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 0x7fa1dc06e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4033000 tx=0x7fa1f4008030 comp rx=0 tx=0).stop 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 msgr2=0x7fa1f80b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8071b80 tx=0x7fa1f000fbe0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 shutdown_connections 2024-09-10T11:34:11.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1dc06be30 0x7fa1dc06e2f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f80b4840 0x7fa1f80b4cf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f81f6460 0x7fa1f81521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 --2- 172.21.15.99:0/3871457145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1e40039b0 0x7fa1f8157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 >> 172.21.15.99:0/3871457145 conn(0x7fa1f806bc10 msgr2=0x7fa1f806d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 shutdown_connections 2024-09-10T11:34:11.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.601+0000 7fa1ffde4640 1 -- 172.21.15.99:0/3871457145 wait complete. 2024-09-10T11:34:11.616 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:34:11.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:11.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:11.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[47006]: fuse finished with error 0 and tester_r 0 2024-09-10T11:34:11.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:34:11.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:11.641 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:34:11.697 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:11.697 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:11.697 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:11.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:34:11.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.834+0000 7f307d8ac640 1 Processor -- start 2024-09-10T11:34:11.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.834+0000 7f307d8ac640 1 -- start start 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f307d8ac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f307815eb80 0x7f30780d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f307d8ac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30780d5010 0x7f30780d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f307d8ac640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307815ff70 con 0x7f30781626c0 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f307d8ac640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f30780d1620 con 0x7f307815eb80 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f307d8ac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3078162bc0 con 0x7f30780d5010 2024-09-10T11:34:11.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f307815eb80 0x7f30780d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f307815eb80 0x7f30780d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33848/0 (socket says 172.21.15.99:33848) 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 -- 172.21.15.99:0/3244970547 learned_addr learned my addr 172.21.15.99:0/3244970547 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 -- 172.21.15.99:0/3244970547 >> v2:172.21.15.196:3300/0 conn(0x7f30780d5010 msgr2=0x7f30780d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 --2- 172.21.15.99:0/3244970547 >> v2:172.21.15.196:3300/0 conn(0x7f30780d5010 0x7f30780d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 -- 172.21.15.99:0/3244970547 >> v1:172.21.15.196:6789/0 conn(0x7f30781626c0 legacy=0x7f30780d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:11.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.835+0000 7f3076ffd640 1 -- 172.21.15.99:0/3244970547 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3078162ea0 con 0x7f307815eb80 2024-09-10T11:34:11.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.836+0000 7f3076ffd640 1 --2- 172.21.15.99:0/3244970547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780d29b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3068009870 tx=0x7f306802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa3ad5907055ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.836+0000 7f3075ffb640 1 -- 172.21.15.99:0/3244970547 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068037070 con 0x7f307815eb80 2024-09-10T11:34:11.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.836+0000 7f3075ffb640 1 -- 172.21.15.99:0/3244970547 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068032440 con 0x7f307815eb80 2024-09-10T11:34:11.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.836+0000 7f3075ffb640 1 -- 172.21.15.99:0/3244970547 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068032810 con 0x7f307815eb80 2024-09-10T11:34:11.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 -- 172.21.15.99:0/3244970547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 msgr2=0x7f30780d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 --2- 172.21.15.99:0/3244970547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780d29b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3068009870 tx=0x7f306802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 -- 172.21.15.99:0/3244970547 shutdown_connections 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 --2- 172.21.15.99:0/3244970547 >> v2:172.21.15.196:3300/0 conn(0x7f30780d5010 0x7f30780d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 --2- 172.21.15.99:0/3244970547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780d29b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 -- 172.21.15.99:0/3244970547 >> 172.21.15.99:0/3244970547 conn(0x7f307808e680 msgr2=0x7f307809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 -- 172.21.15.99:0/3244970547 shutdown_connections 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.837+0000 7f307d8ac640 1 -- 172.21.15.99:0/3244970547 wait complete. 2024-09-10T11:34:11.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.838+0000 7f307d8ac640 1 Processor -- start 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.838+0000 7f307d8ac640 1 -- start start 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30780d5010 0x7f30780c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 0x7f3078159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30780d2ef0 con 0x7f307815eb80 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3078163460 con 0x7f30780d5010 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30780d3070 con 0x7f30781626c0 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33852/0 (socket says 172.21.15.99:33852) 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 -- 172.21.15.99:0/2364065328 learned_addr learned my addr 172.21.15.99:0/2364065328 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f3076ffd640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30780d5010 0x7f30780c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30777fe640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 0x7f3078159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 -- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 msgr2=0x7f3078159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 0x7f3078159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 -- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30780d5010 msgr2=0x7f30780c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30780d5010 0x7f30780c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780c8e80 con 0x7f307815eb80 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30777fe640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 0x7f3078159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f30767fc640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b3c0 tx=0x7f306c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c011830 con 0x7f307815eb80 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c016070 con 0x7f307815eb80 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30780ca890 con 0x7f307815eb80 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.839+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c011bf0 con 0x7f307815eb80 2024-09-10T11:34:11.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.840+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307815a320 con 0x7f307815eb80 2024-09-10T11:34:11.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.841+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30780d29b0 con 0x7f307815eb80 2024-09-10T11:34:11.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.841+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f306c01b460 con 0x7f307815eb80 2024-09-10T11:34:11.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.841+0000 7f305ffff640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:11.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.842+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f306c08aa10 con 0x7f307815eb80 2024-09-10T11:34:11.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.842+0000 7f3076ffd640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:11.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.842+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f30640739c0 con 0x7f307815eb80 2024-09-10T11:34:11.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.842+0000 7f3076ffd640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 0x7f306406e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3068034000 tx=0x7f306802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:11.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:11.846+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f306c059450 con 0x7f307815eb80 2024-09-10T11:34:12.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.023+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f307809e130 con 0x7f307815eb80 2024-09-10T11:34:12.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.023+0000 7f305ffff640 1 -- 172.21.15.99:0/2364065328 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) ==== 80+0+0 (secure 0 0 0) 0x7f306c058d50 con 0x7f307815eb80 2024-09-10T11:34:12.025 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:34:12.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 msgr2=0x7f306406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 0x7f306406e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3068034000 tx=0x7f306802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 msgr2=0x7f30780cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f306c00b3c0 tx=0x7f306c00b880 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 shutdown_connections 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f306406be30 0x7f306406e2f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30781626c0 0x7f3078159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f307815eb80 0x7f30780cb370 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 --2- 172.21.15.99:0/2364065328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30780d5010 0x7f30780c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.025+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 >> 172.21.15.99:0/2364065328 conn(0x7f307808e680 msgr2=0x7f307815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.026+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 shutdown_connections 2024-09-10T11:34:12.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.026+0000 7f307d8ac640 1 -- 172.21.15.99:0/2364065328 wait complete. 2024-09-10T11:34:12.038 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:34:12.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:12.039 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:12.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36123]: fuse finished with error 0 and tester_r 0 2024-09-10T11:34:12.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:34:12.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:12.069 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:34:12.201 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:12.201 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:12.201 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:12.217 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:12.217 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:34:12.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:12.218 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:34:12.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:12.279 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:34:12.279 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:12.279 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:12.279 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:12.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:12.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:34:12.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:12.304 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:34:12.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:12.360 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:34:12.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:12.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 Processor -- start 2024-09-10T11:34:12.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 -- start start 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f23441626c0 0x7f23440d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f234415ff70 con 0x7f23440d4e40 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23440d1620 con 0x7f234415eb80 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.458+0000 7f234a95f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f23440d30c0 con 0x7f23441626c0 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f2348efc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f23440d4e40 0x7f23440d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:33942/0 (socket says 172.21.15.99:33942) 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f2348efc640 1 -- 172.21.15.99:0/992557243 learned_addr learned my addr 172.21.15.99:0/992557243 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f23437fe640 1 --2- 172.21.15.99:0/992557243 >> v2:172.21.15.196:3300/0 conn(0x7f23441626c0 0x7f23440d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f2342ffd640 1 -- 172.21.15.99:0/992557243 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817739597 0 0) 0x7f234415ff70 con 0x7f23440d4e40 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f2342ffd640 1 -- 172.21.15.99:0/992557243 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2330003520 con 0x7f23440d4e40 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f23437fe640 1 -- 172.21.15.99:0/992557243 >> v1:172.21.15.196:6789/0 conn(0x7f234415eb80 legacy=0x7f23440d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f23437fe640 1 -- 172.21.15.99:0/992557243 >> v1:172.21.15.99:6789/0 conn(0x7f23440d4e40 legacy=0x7f23440d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.459+0000 7f23437fe640 1 -- 172.21.15.99:0/992557243 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2344162bc0 con 0x7f23441626c0 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f23437fe640 1 --2- 172.21.15.99:0/992557243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f23440d4900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f232c0097d0 tx=0x7f232c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4000747dbbbf2d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f2342ffd640 1 -- 172.21.15.99:0/992557243 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c037070 con 0x7f23441626c0 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f2342ffd640 1 -- 172.21.15.99:0/992557243 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232c032440 con 0x7f23441626c0 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f2342ffd640 1 -- 172.21.15.99:0/992557243 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232c0327f0 con 0x7f23441626c0 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f234a95f640 1 -- 172.21.15.99:0/992557243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 msgr2=0x7f23440d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f234a95f640 1 --2- 172.21.15.99:0/992557243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f23440d4900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f232c0097d0 tx=0x7f232c02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f234a95f640 1 -- 172.21.15.99:0/992557243 shutdown_connections 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f234a95f640 1 --2- 172.21.15.99:0/992557243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f23440d4900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.460+0000 7f234a95f640 1 -- 172.21.15.99:0/992557243 >> 172.21.15.99:0/992557243 conn(0x7f234408e680 msgr2=0x7f234409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.461+0000 7f234a95f640 1 -- 172.21.15.99:0/992557243 shutdown_connections 2024-09-10T11:34:12.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.461+0000 7f234a95f640 1 -- 172.21.15.99:0/992557243 wait complete. 2024-09-10T11:34:12.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.462+0000 7f234a95f640 1 Processor -- start 2024-09-10T11:34:12.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.462+0000 7f234a95f640 1 -- start start 2024-09-10T11:34:12.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f234415eb80 0x7f234415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f234415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2344163460 con 0x7f23440d4e40 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2344163180 con 0x7f23441626c0 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f23440d5290 con 0x7f234415eb80 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33862/0 (socket says 172.21.15.99:33862) 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 -- 172.21.15.99:0/4292555174 learned_addr learned my addr 172.21.15.99:0/4292555174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2348efc640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f234415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f23437fe640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f234415eb80 0x7f234415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 -- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f234415eb80 msgr2=0x7f234415a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f234415eb80 0x7f234415a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 -- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 msgr2=0x7f234415b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f234415b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234415a760 con 0x7f23440d4e40 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2348efc640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f234415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2343fff640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f233400c3a0 tx=0x7f233400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334017070 con 0x7f23440d4e40 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2334005310 con 0x7f23440d4e40 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234415bff0 con 0x7f23440d4e40 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2334003850 con 0x7f23440d4e40 2024-09-10T11:34:12.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.463+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234415c5b0 con 0x7f23440d4e40 2024-09-10T11:34:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.464+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23440d4900 con 0x7f23440d4e40 2024-09-10T11:34:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.465+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f233401a070 con 0x7f23440d4e40 2024-09-10T11:34:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.465+0000 7f2340ff9640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 0x7f232406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.465+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f233408a060 con 0x7f23440d4e40 2024-09-10T11:34:12.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.465+0000 7f23437fe640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 0x7f232406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.465+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f23240739f0 con 0x7f23440d4e40 2024-09-10T11:34:12.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.466+0000 7f23437fe640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 0x7f232406e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f232c034000 tx=0x7f232c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.469+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f234415a760 con 0x7f23440d4e40 2024-09-10T11:34:12.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.669+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f234409e040 con 0x7f23440d4e40 2024-09-10T11:34:12.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.670+0000 7f2340ff9640 1 -- 172.21.15.99:0/4292555174 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7f23340583d0 con 0x7f23440d4e40 2024-09-10T11:34:12.672 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:12.672 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:34:12.672 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:34:12.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 msgr2=0x7f232406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 0x7f232406e320 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f232c034000 tx=0x7f232c02e000 comp rx=0 tx=0).stop 2024-09-10T11:34:12.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 msgr2=0x7f23440b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f233400c3a0 tx=0x7f233400c860 comp rx=0 tx=0).stop 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 shutdown_connections 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f232406be60 0x7f232406e320 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23441626c0 0x7f234415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f234415eb80 0x7f234415a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 --2- 172.21.15.99:0/4292555174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23440d4e40 0x7f23440b38b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 >> 172.21.15.99:0/4292555174 conn(0x7f234408e680 msgr2=0x7f23440b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 shutdown_connections 2024-09-10T11:34:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.672+0000 7f234a95f640 1 -- 172.21.15.99:0/4292555174 wait complete. 2024-09-10T11:34:12.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.781+0000 7fd07b8e4640 1 Processor -- start 2024-09-10T11:34:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 -- start start 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0741626c0 0x7fd0740d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0740d52d0 con 0x7fd0740d4e40 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd07415ff70 con 0x7fd0741626c0 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.782+0000 7fd07b8e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740d1620 con 0x7fd07415eb80 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0741626c0 0x7fd0740d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd0741626c0 0x7fd0740d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34976/0 (socket says 172.21.15.99:34976) 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 -- 172.21.15.99:0/4145470430 learned_addr learned my addr 172.21.15.99:0/4145470430 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd079680640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 -- 172.21.15.99:0/4145470430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 msgr2=0x7fd0740d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 -- 172.21.15.99:0/4145470430 >> v1:172.21.15.196:6789/0 conn(0x7fd0740d4e40 legacy=0x7fd0740d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:12.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 -- 172.21.15.99:0/4145470430 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074162bc0 con 0x7fd0741626c0 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.783+0000 7fd078e7f640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0741626c0 0x7fd0740d4900 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5437f8e4fc8c7e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd073fff640 1 -- 172.21.15.99:0/4145470430 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068037070 con 0x7fd0741626c0 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd073fff640 1 -- 172.21.15.99:0/4145470430 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068032440 con 0x7fd0741626c0 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd073fff640 1 -- 172.21.15.99:0/4145470430 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068032810 con 0x7fd0741626c0 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 -- 172.21.15.99:0/4145470430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0741626c0 msgr2=0x7fd0740d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0741626c0 0x7fd0740d4900 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd068009870 tx=0x7fd06802a180 comp rx=0 tx=0).stop 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 -- 172.21.15.99:0/4145470430 shutdown_connections 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0741626c0 0x7fd0740d4900 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/4145470430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 -- 172.21.15.99:0/4145470430 >> 172.21.15.99:0/4145470430 conn(0x7fd07408e680 msgr2=0x7fd07409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 -- 172.21.15.99:0/4145470430 shutdown_connections 2024-09-10T11:34:12.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.784+0000 7fd07b8e4640 1 -- 172.21.15.99:0/4145470430 wait complete. 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.785+0000 7fd07b8e4640 1 Processor -- start 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 -- start start 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 0x7fd0740cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd074163460 con 0x7fd0740d4e40 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd074163180 con 0x7fd0741626c0 2024-09-10T11:34:12.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd07b8e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd0740d2ef0 con 0x7fd07415eb80 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd079680640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd079680640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34978/0 (socket says 172.21.15.99:34978) 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd079680640 1 -- 172.21.15.99:0/1333363331 learned_addr learned my addr 172.21.15.99:0/1333363331 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.786+0000 7fd078e7f640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079e81640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 0x7fd0740cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 -- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 msgr2=0x7fd0740cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 -- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 msgr2=0x7fd0740cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 0x7fd0740cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074160ad0 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079e81640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 0x7fd0740cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd078e7f640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd079680640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ccd0 tx=0x7fd06400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064003990 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd064003b30 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd074160db0 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064004020 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.787+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd074161370 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.788+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0740d5bc0 con 0x7fd0740d4e40 2024-09-10T11:34:12.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.788+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd0640041c0 con 0x7fd0740d4e40 2024-09-10T11:34:12.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.789+0000 7fd071ffb640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:12.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.789+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7fd06408b2b0 con 0x7fd0740d4e40 2024-09-10T11:34:12.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.789+0000 7fd078e7f640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:12.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.789+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7fd060073990 con 0x7fd0740d4e40 2024-09-10T11:34:12.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.789+0000 7fd078e7f640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 0x7fd06006e320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd068034000 tx=0x7fd06802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:12.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.793+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd0740d5bc0 con 0x7fd0740d4e40 2024-09-10T11:34:12.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.993+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0740d29b0 con 0x7fd0740d4e40 2024-09-10T11:34:12.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.993+0000 7fd071ffb640 1 -- 172.21.15.99:0/1333363331 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7fd064014030 con 0x7fd0740d4e40 2024-09-10T11:34:12.995 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:12.995 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:34:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 msgr2=0x7fd06006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 0x7fd06006e320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd068034000 tx=0x7fd06802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 msgr2=0x7fd074150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ccd0 tx=0x7fd06400a5c0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 shutdown_connections 2024-09-10T11:34:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd06006be60 0x7fd06006e320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0741626c0 0x7fd0740cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd07415eb80 0x7fd0740cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 --2- 172.21.15.99:0/1333363331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0740d4e40 0x7fd074150210 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 >> 172.21.15.99:0/1333363331 conn(0x7fd07408e680 msgr2=0x7fd0740cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 shutdown_connections 2024-09-10T11:34:12.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:12.995+0000 7fd07b8e4640 1 -- 172.21.15.99:0/1333363331 wait complete. 2024-09-10T11:34:13.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:13.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.104+0000 7f63b76d9640 1 Processor -- start 2024-09-10T11:34:13.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.104+0000 7f63b76d9640 1 -- start start 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f63b01485b0 0x7f63b00b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63b0162910 0x7f63b0162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b00cebd0 0x7f63b00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f63b006dbc0 con 0x7f63b0162910 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f63b0163290 con 0x7f63b01485b0 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b76d9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63b006d8c0 con 0x7f63b00cebd0 2024-09-10T11:34:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63b0162910 0x7f63b0162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63b0162910 0x7f63b0162d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34982/0 (socket says 172.21.15.99:34982) 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 -- 172.21.15.99:0/3416978147 learned_addr learned my addr 172.21.15.99:0/3416978147 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b5475640 1 --2- 172.21.15.99:0/3416978147 >> v2:172.21.15.196:3300/0 conn(0x7f63b01485b0 0x7f63b00b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b5c76640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b00cebd0 0x7f63b00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 -- 172.21.15.99:0/3416978147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b00cebd0 msgr2=0x7f63b00cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b00cebd0 0x7f63b00cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 -- 172.21.15.99:0/3416978147 >> v2:172.21.15.196:3300/0 conn(0x7f63b01485b0 msgr2=0x7f63b00b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 --2- 172.21.15.99:0/3416978147 >> v2:172.21.15.196:3300/0 conn(0x7f63b01485b0 0x7f63b00b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.105+0000 7f63b4c74640 1 -- 172.21.15.99:0/3416978147 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b00b3870 con 0x7f63b0162910 2024-09-10T11:34:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.106+0000 7f63b4c74640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b0162910 0x7f63b0162d50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f63b006d370 tx=0x7f63a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a2acf8a78acd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.106+0000 7f63a7fff640 1 -- 172.21.15.99:0/3416978147 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0037070 con 0x7f63b0162910 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.106+0000 7f63a7fff640 1 -- 172.21.15.99:0/3416978147 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a0032440 con 0x7f63b0162910 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.106+0000 7f63a7fff640 1 -- 172.21.15.99:0/3416978147 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0032810 con 0x7f63b0162910 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 -- 172.21.15.99:0/3416978147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b0162910 msgr2=0x7f63b0162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b0162910 0x7f63b0162d50 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f63b006d370 tx=0x7f63a002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 -- 172.21.15.99:0/3416978147 shutdown_connections 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b00cebd0 0x7f63b00cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 --2- 172.21.15.99:0/3416978147 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b0162910 0x7f63b0162d50 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 --2- 172.21.15.99:0/3416978147 >> v2:172.21.15.196:3300/0 conn(0x7f63b01485b0 0x7f63b00b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 -- 172.21.15.99:0/3416978147 >> 172.21.15.99:0/3416978147 conn(0x7f63b00b2a80 msgr2=0x7f63b00b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 -- 172.21.15.99:0/3416978147 shutdown_connections 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.107+0000 7f63b76d9640 1 -- 172.21.15.99:0/3416978147 wait complete. 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.108+0000 7f63b76d9640 1 Processor -- start 2024-09-10T11:34:13.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.108+0000 7f63b76d9640 1 -- start start 2024-09-10T11:34:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b01485b0 0x7f63b0161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0162910 0x7f63b015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63b00d00e0 con 0x7f63b00cebd0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0148990 con 0x7f63b01485b0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63b00d03c0 con 0x7f63b0162910 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34992/0 (socket says 172.21.15.99:34992) 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 -- 172.21.15.99:0/924505989 learned_addr learned my addr 172.21.15.99:0/924505989 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b4c74640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b01485b0 0x7f63b0161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5c76640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0162910 0x7f63b015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 -- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0162910 msgr2=0x7f63b015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0162910 0x7f63b015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 -- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b01485b0 msgr2=0x7f63b0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b01485b0 0x7f63b0161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b009bc40 con 0x7f63b00cebd0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b5475640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c7f0 tx=0x7f63ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac015070 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63ac004240 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b009bf20 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ac007620 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.109+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b009c4e0 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.110+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b00d06a0 con 0x7f63b00cebd0 2024-09-10T11:34:13.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.111+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63ac0079d0 con 0x7f63b00cebd0 2024-09-10T11:34:13.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.111+0000 7f63a5ffb640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.111+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f63ac08b240 con 0x7f63b00cebd0 2024-09-10T11:34:13.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.111+0000 7f63b4c74640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.111+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f6394073960 con 0x7f63b00cebd0 2024-09-10T11:34:13.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.112+0000 7f63b4c74640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63a0034000 tx=0x7f63a002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.115+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f63ac059bf0 con 0x7f63b00cebd0 2024-09-10T11:34:13.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.280+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63b015f0c0 con 0x7f63b00cebd0 2024-09-10T11:34:13.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.281+0000 7f63a5ffb640 1 -- 172.21.15.99:0/924505989 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16757 (secure 0 0 0) 0x7f63ac0594f0 con 0x7f63b00cebd0 2024-09-10T11:34:13.284 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:13.284 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":89,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:01.285389+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:33:41.576087+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T11:33:42.522066+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:13.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.283+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 msgr2=0x7f639406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.283+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63a0034000 tx=0x7f63a002e000 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.283+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 msgr2=0x7f63b01f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.283+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00c7f0 tx=0x7f63ac00ccb0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 shutdown_connections 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63b0162910 0x7f63b015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63b01485b0 0x7f63b0161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 --2- 172.21.15.99:0/924505989 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b00cebd0 0x7f63b01f6350 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 >> 172.21.15.99:0/924505989 conn(0x7f63b00b2a80 msgr2=0x7f63b01477c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 shutdown_connections 2024-09-10T11:34:13.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.284+0000 7f63b76d9640 1 -- 172.21.15.99:0/924505989 wait complete. 2024-09-10T11:34:13.296 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:34:13.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:13.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.393+0000 7eff61e3c640 1 Processor -- start 2024-09-10T11:34:13.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.393+0000 7eff61e3c640 1 -- start start 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.393+0000 7eff61e3c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff5c15f1e0 0x7eff5c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.393+0000 7eff61e3c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0b40e0 con 0x7eff5c0b0090 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff61e3c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0b5ab0 con 0x7eff5c15ee00 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff61e3c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c073c50 con 0x7eff5c15f1e0 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7eff5c0b0090 0x7eff5c0b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59220/0 (socket says 172.21.15.99:59220) 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5bfff640 1 -- 172.21.15.99:0/4159452853 learned_addr learned my addr 172.21.15.99:0/4159452853 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5affd640 1 --2- 172.21.15.99:0/4159452853 >> v2:172.21.15.99:3300/0 conn(0x7eff5c15f1e0 0x7eff5c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5affd640 1 -- 172.21.15.99:0/4159452853 >> v1:172.21.15.196:6789/0 conn(0x7eff5c15ee00 legacy=0x7eff5c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5affd640 1 -- 172.21.15.99:0/4159452853 >> v1:172.21.15.99:6789/0 conn(0x7eff5c0b0090 legacy=0x7eff5c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5a7fc640 1 -- 172.21.15.99:0/4159452853 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022225949 0 0) 0x7eff5c0b40e0 con 0x7eff5c0b0090 2024-09-10T11:34:13.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.394+0000 7eff5affd640 1 -- 172.21.15.99:0/4159452853 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0b4380 con 0x7eff5c15f1e0 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff5affd640 1 --2- 172.21.15.99:0/4159452853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15f1e0 0x7eff5c0b5f90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7eff500097a0 tx=0x7eff5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2cf8590616fe99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff5a7fc640 1 -- 172.21.15.99:0/4159452853 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50037070 con 0x7eff5c15f1e0 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff5a7fc640 1 -- 172.21.15.99:0/4159452853 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff50032440 con 0x7eff5c15f1e0 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff5a7fc640 1 -- 172.21.15.99:0/4159452853 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50032810 con 0x7eff5c15f1e0 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 -- 172.21.15.99:0/4159452853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15f1e0 msgr2=0x7eff5c0b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 --2- 172.21.15.99:0/4159452853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15f1e0 0x7eff5c0b5f90 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7eff500097a0 tx=0x7eff5002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 -- 172.21.15.99:0/4159452853 shutdown_connections 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 --2- 172.21.15.99:0/4159452853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15f1e0 0x7eff5c0b5f90 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 -- 172.21.15.99:0/4159452853 >> 172.21.15.99:0/4159452853 conn(0x7eff5c0730c0 msgr2=0x7eff5c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 -- 172.21.15.99:0/4159452853 shutdown_connections 2024-09-10T11:34:13.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.395+0000 7eff61e3c640 1 -- 172.21.15.99:0/4159452853 wait complete. 2024-09-10T11:34:13.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.397+0000 7eff61e3c640 1 Processor -- start 2024-09-10T11:34:13.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.397+0000 7eff61e3c640 1 -- start start 2024-09-10T11:34:13.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.397+0000 7eff61e3c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c0b0090 0x7eff5c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5c15f1e0 0x7eff5c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0b0600 con 0x7eff5c15ee00 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0afd60 con 0x7eff5c0b0090 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c0b0780 con 0x7eff5c15f1e0 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35006/0 (socket says 172.21.15.99:35006) 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 -- 172.21.15.99:0/1791981992 learned_addr learned my addr 172.21.15.99:0/1791981992 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5bfff640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5c15f1e0 0x7eff5c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5b7fe640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c0b0090 0x7eff5c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 -- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5c15f1e0 msgr2=0x7eff5c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5c15f1e0 0x7eff5c0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 -- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c0b0090 msgr2=0x7eff5c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c0b0090 0x7eff5c156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c158740 con 0x7eff5c15ee00 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff5affd640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50037040 con 0x7eff5c15ee00 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff50004230 con 0x7eff5c15ee00 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c158a20 con 0x7eff5c15ee00 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50032e40 con 0x7eff5c15ee00 2024-09-10T11:34:13.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.398+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c158fe0 con 0x7eff5c15ee00 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.399+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff5c0b5f90 con 0x7eff5c15ee00 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.400+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff50030770 con 0x7eff5c15ee00 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.400+0000 7eff43fff640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 0x7eff4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.400+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7eff500ac0d0 con 0x7eff5c15ee00 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.400+0000 7eff5b7fe640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 0x7eff4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.400+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7eff480739c0 con 0x7eff5c15ee00 2024-09-10T11:34:13.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.401+0000 7eff5b7fe640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 0x7eff4806e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7eff4c006fd0 tx=0x7eff4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.405+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff5007ac30 con 0x7eff5c15ee00 2024-09-10T11:34:13.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.605+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff5c0b2490 con 0x7eff5c15ee00 2024-09-10T11:34:13.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.606+0000 7eff43fff640 1 -- 172.21.15.99:0/1791981992 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7eff5007a530 con 0x7eff5c15ee00 2024-09-10T11:34:13.608 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:13.608 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:34:13.608 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:34:13.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 msgr2=0x7eff4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 0x7eff4806e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7eff4c006fd0 tx=0x7eff4c008040 comp rx=0 tx=0).stop 2024-09-10T11:34:13.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 msgr2=0x7eff5c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7eff50000c00 tx=0x7eff500046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 shutdown_connections 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff4806be30 0x7eff4806e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff5c15f1e0 0x7eff5c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff5c15ee00 0x7eff5c0cda80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 --2- 172.21.15.99:0/1791981992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c0b0090 0x7eff5c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 >> 172.21.15.99:0/1791981992 conn(0x7eff5c0730c0 msgr2=0x7eff5c0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 shutdown_connections 2024-09-10T11:34:13.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.608+0000 7eff61e3c640 1 -- 172.21.15.99:0/1791981992 wait complete. 2024-09-10T11:34:13.621 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:13.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.717+0000 7f14f80e4640 1 Processor -- start 2024-09-10T11:34:13.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.717+0000 7f14f80e4640 1 -- start start 2024-09-10T11:34:13.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f80e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f80e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f14f00b0090 0x7f14f00b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f80e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f00b40e0 con 0x7f14f015ee00 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f80e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f14f00b5ab0 con 0x7f14f00b0090 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f80e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14f0073c50 con 0x7f14f015f1e0 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f14f00b0090 0x7f14f00b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f14f00b0090 0x7f14f00b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35008/0 (socket says 172.21.15.99:35008) 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 -- 172.21.15.99:0/2157144449 learned_addr learned my addr 172.21.15.99:0/2157144449 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f567f640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 -- 172.21.15.99:0/2157144449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 msgr2=0x7f14f00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f00b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.718+0000 7f14f6681640 1 -- 172.21.15.99:0/2157144449 >> v1:172.21.15.196:6789/0 conn(0x7f14f015ee00 legacy=0x7f14f00b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.719+0000 7f14f6681640 1 -- 172.21.15.99:0/2157144449 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f00b4380 con 0x7f14f00b0090 2024-09-10T11:34:13.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.719+0000 7f14f6681640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f00b0090 0x7f14f00b1c70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e50 tx=0x7f14e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5277096cc41b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.719+0000 7f14f4e7e640 1 -- 172.21.15.99:0/2157144449 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e8036070 con 0x7f14f00b0090 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.719+0000 7f14f4e7e640 1 -- 172.21.15.99:0/2157144449 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e8031600 con 0x7f14f00b0090 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.719+0000 7f14f4e7e640 1 -- 172.21.15.99:0/2157144449 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e80319d0 con 0x7f14f00b0090 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 -- 172.21.15.99:0/2157144449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f00b0090 msgr2=0x7f14f00b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f00b0090 0x7f14f00b1c70 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009e50 tx=0x7f14e802a710 comp rx=0 tx=0).stop 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 -- 172.21.15.99:0/2157144449 shutdown_connections 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f00b0090 0x7f14f00b1c70 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2157144449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f00b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 -- 172.21.15.99:0/2157144449 >> 172.21.15.99:0/2157144449 conn(0x7f14f00730c0 msgr2=0x7f14f00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 -- 172.21.15.99:0/2157144449 shutdown_connections 2024-09-10T11:34:13.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.720+0000 7f14f80e4640 1 -- 172.21.15.99:0/2157144449 wait complete. 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.721+0000 7f14f80e4640 1 Processor -- start 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.721+0000 7f14f80e4640 1 -- start start 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f00b0090 0x7f14f0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f0156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00b21b0 con 0x7f14f015ee00 2024-09-10T11:34:13.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f14f00afd60 con 0x7f14f00b0090 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f80e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f14f00b0600 con 0x7f14f015f1e0 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35014/0 (socket says 172.21.15.99:35014) 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 -- 172.21.15.99:0/2305952034 learned_addr learned my addr 172.21.15.99:0/2305952034 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f6681640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f0156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f5e80640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f00b0090 0x7f14f0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 -- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 msgr2=0x7f14f0156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f0156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 -- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f00b0090 msgr2=0x7f14f0154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f00b0090 0x7f14f0154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f0159640 con 0x7f14f015ee00 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.722+0000 7f14f567f640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d560 tx=0x7f14e000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.723+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00125f0 con 0x7f14f015ee00 2024-09-10T11:34:13.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.723+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0019070 con 0x7f14f015ee00 2024-09-10T11:34:13.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.723+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e00129f0 con 0x7f14f015ee00 2024-09-10T11:34:13.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.723+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f0159360 con 0x7f14f015ee00 2024-09-10T11:34:13.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.723+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f0155870 con 0x7f14f015ee00 2024-09-10T11:34:13.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.724+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f0073510 con 0x7f14f015ee00 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.724+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f14e0012b90 con 0x7f14f015ee00 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.725+0000 7f14e67fc640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.725+0000 7f14f5e80640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.725+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f14e008b100 con 0x7f14f015ee00 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.725+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f14d4073960 con 0x7f14f015ee00 2024-09-10T11:34:13.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.725+0000 7f14f5e80640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14ec001780 tx=0x7f14ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:13.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.729+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14f0073510 con 0x7f14f015ee00 2024-09-10T11:34:13.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.925+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14f00b5f90 con 0x7f14f015ee00 2024-09-10T11:34:13.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.925+0000 7f14e67fc640 1 -- 172.21.15.99:0/2305952034 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5231 (secure 0 0 0) 0x7f14e0059460 con 0x7f14f015ee00 2024-09-10T11:34:13.927 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:13.927 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"btime":"2024-09-10T11:33:46:235639+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:33:43.544162+0000","modified":"2024-09-10T11:33:45.565721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5922":{"gid":5922,"name":"b","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/1122563557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1122563557},{"type":"v1","addr":"172.21.15.196:6835","nonce":1122563557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5922,"qdb_cluster":[5922]},"id":5}]} 2024-09-10T11:34:13.928 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 35 2024-09-10T11:34:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 msgr2=0x7f14d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f14ec001780 tx=0x7f14ec009040 comp rx=0 tx=0).stop 2024-09-10T11:34:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 msgr2=0x7f14f01f93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f14e000d560 tx=0x7f14e000da20 comp rx=0 tx=0).stop 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 shutdown_connections 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f14d406be30 0x7f14d406e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f14f015f1e0 0x7f14f0156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14f015ee00 0x7f14f01f93f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 --2- 172.21.15.99:0/2305952034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14f00b0090 0x7f14f0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 >> 172.21.15.99:0/2305952034 conn(0x7f14f00730c0 msgr2=0x7f14f006f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 shutdown_connections 2024-09-10T11:34:13.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:13.927+0000 7f14f80e4640 1 -- 172.21.15.99:0/2305952034 wait complete. 2024-09-10T11:34:13.940 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:14.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.036+0000 7f328085e640 1 Processor -- start 2024-09-10T11:34:14.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.037+0000 7f328085e640 1 -- start start 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.037+0000 7f328085e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f327c0d0130 0x7f327c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.037+0000 7f328085e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c0ce430 con 0x7f327c0d1760 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.037+0000 7f328085e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c0ce710 con 0x7f327c160030 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.037+0000 7f328085e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f327c0ce200 con 0x7f327c0d0130 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f3279d9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f327c0d1760 0x7f327c0cd5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59226/0 (socket says 172.21.15.99:59226) 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f3279d9b640 1 -- 172.21.15.99:0/2471058055 learned_addr learned my addr 172.21.15.99:0/2471058055 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:14.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327ad9d640 1 --2- 172.21.15.99:0/2471058055 >> v2:172.21.15.196:3300/0 conn(0x7f327c0d0130 0x7f327c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 427500369 0 0) 0x7f327c0ce430 con 0x7f327c0d1760 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3268003520 con 0x7f327c0d1760 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1426813432 0 0) 0x7f3268003520 con 0x7f327c0d1760 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 >> v2:172.21.15.196:3300/0 conn(0x7f327c0d0130 msgr2=0x7f327c0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 --2- 172.21.15.99:0/2471058055 >> v2:172.21.15.196:3300/0 conn(0x7f327c0d0130 0x7f327c0cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 >> v1:172.21.15.196:6789/0 conn(0x7f327c160030 legacy=0x7f327c0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c0d1240 con 0x7f327c0d1760 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f3264001670 con 0x7f327c0d1760 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 >> v1:172.21.15.99:6789/0 conn(0x7f327c0d1760 legacy=0x7f327c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32680039b0 0x7f3268003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3268004800 0x7f3268004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f326800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f326800c030 con 0x7f32680051c0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f326800c1e0 con 0x7f3268004800 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f326800c3e0 con 0x7f32680039b0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327ad9d640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f326800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f3279d9b640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3268004800 0x7f3268004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.038+0000 7f327a59c640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32680039b0 0x7f3268003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 -- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32680039b0 msgr2=0x7f3268003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32680039b0 0x7f3268003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 -- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3268004800 msgr2=0x7f3268004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3268004800 0x7f3268004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 -- 172.21.15.99:0/2471058055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3268003520 con 0x7f32680051c0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f3279d9b640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3268004800 0x7f3268004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 -- 172.21.15.99:0/2471058055 reap_dead start 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327ad9d640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f326800b9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ac0 tx=0x7f3268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89188da7b1980116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c009690 con 0x7f32680051c0 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f326c004020 con 0x7f32680051c0 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.039+0000 7f327959a640 1 -- 172.21.15.99:0/2471058055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c004410 con 0x7f32680051c0 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 -- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 msgr2=0x7f326800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f326800b9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ac0 tx=0x7f3268005830 comp rx=0 tx=0).stop 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 -- 172.21.15.99:0/2471058055 shutdown_connections 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 --2- 172.21.15.99:0/2471058055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f326800b9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 -- 172.21.15.99:0/2471058055 >> 172.21.15.99:0/2471058055 conn(0x7f327c0b4650 msgr2=0x7f327c0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 -- 172.21.15.99:0/2471058055 shutdown_connections 2024-09-10T11:34:14.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.040+0000 7f328085e640 1 -- 172.21.15.99:0/2471058055 wait complete. 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.041+0000 7f328085e640 1 Processor -- start 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.041+0000 7f328085e640 1 -- start start 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 0x7f327c0b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f327c0b21a0 0x7f327c0c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f327c0b38f0 con 0x7f32680051c0 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f327c160620 con 0x7f327c1f3230 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f328085e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f327c0b3a70 con 0x7f327c0b21a0 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35022/0 (socket says 172.21.15.99:35022) 2024-09-10T11:34:14.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 -- 172.21.15.99:0/3219611842 learned_addr learned my addr 172.21.15.99:0/3219611842 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327ad9d640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f327c0b21a0 0x7f327c0c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f3279d9b640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 0x7f327c0b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 -- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f327c0b21a0 msgr2=0x7f327c0c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f327c0b21a0 0x7f327c0c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 -- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 msgr2=0x7f327c0b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 0x7f327c0b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.042+0000 7f327a59c640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c0b7b60 con 0x7f32680051c0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f3279d9b640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 0x7f327c0b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f327a59c640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f327400f990 tx=0x7f327400fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274003b80 con 0x7f32680051c0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274018070 con 0x7f32680051c0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c0b7e30 con 0x7f32680051c0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327400d7d0 con 0x7f32680051c0 2024-09-10T11:34:14.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.043+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1f2750 con 0x7f32680051c0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.044+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c1f3610 con 0x7f32680051c0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.044+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f327400d970 con 0x7f32680051c0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.045+0000 7f3272ffd640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 0x7f325c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.045+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f327408aff0 con 0x7f32680051c0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.045+0000 7f3279d9b640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 0x7f325c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.045+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f325c073990 con 0x7f32680051c0 2024-09-10T11:34:14.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.045+0000 7f3279d9b640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 0x7f325c06e320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f327c0b8a10 tx=0x7f3264009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.049+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f327401a310 con 0x7f32680051c0 2024-09-10T11:34:14.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.214+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f327c0b3e00 con 0x7f32680051c0 2024-09-10T11:34:14.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.215+0000 7f3272ffd640 1 -- 172.21.15.99:0/3219611842 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16757 (secure 0 0 0) 0x7f32740599a0 con 0x7f32680051c0 2024-09-10T11:34:14.217 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:14.217 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":89,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:01.285389+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:33:41.576087+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T11:33:42.522066+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:14.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 msgr2=0x7f325c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 0x7f325c06e320 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f327c0b8a10 tx=0x7f3264009040 comp rx=0 tx=0).stop 2024-09-10T11:34:14.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 msgr2=0x7f327c1f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f327400f990 tx=0x7f327400fe50 comp rx=0 tx=0).stop 2024-09-10T11:34:14.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 shutdown_connections 2024-09-10T11:34:14.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f325c06be60 0x7f325c06e320 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f327c0b21a0 0x7f327c0c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f327c1f3230 0x7f327c0b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 --2- 172.21.15.99:0/3219611842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f32680051c0 0x7f327c1f6800 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.217+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 >> 172.21.15.99:0/3219611842 conn(0x7f327c0b4650 msgr2=0x7f327c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.218+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 shutdown_connections 2024-09-10T11:34:14.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.218+0000 7f328085e640 1 -- 172.21.15.99:0/3219611842 wait complete. 2024-09-10T11:34:14.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:34:14.312 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:14.310+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.332+0000 7fc89c56d640 1 Processor -- start 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.332+0000 7fc89c56d640 1 -- start start 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89c56d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc894163620 0x7fc894160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89c56d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8940b5140 con 0x7fc89406bc10 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89c56d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc894162b40 con 0x7fc8940b4930 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89c56d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc8940b5420 con 0x7fc894163620 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89a309640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc89406bc10 0x7fc89406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59230/0 (socket says 172.21.15.99:59230) 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc899b08640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc894163620 0x7fc894160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc89a309640 1 -- 172.21.15.99:0/2800025544 learned_addr learned my addr 172.21.15.99:0/2800025544 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:14.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc899b08640 1 -- 172.21.15.99:0/2800025544 >> v1:172.21.15.196:6789/0 conn(0x7fc8940b4930 legacy=0x7fc89406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc899b08640 1 -- 172.21.15.99:0/2800025544 >> v1:172.21.15.99:6789/0 conn(0x7fc89406bc10 legacy=0x7fc89406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc899b08640 1 -- 172.21.15.99:0/2800025544 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc894162e20 con 0x7fc894163620 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.333+0000 7fc899307640 1 -- 172.21.15.99:0/2800025544 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184859488 0 0) 0x7fc8940b5140 con 0x7fc89406bc10 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.334+0000 7fc899b08640 1 --2- 172.21.15.99:0/2800025544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc894163620 0x7fc894160030 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d498fed9bf713a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.334+0000 7fc899307640 1 -- 172.21.15.99:0/2800025544 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884037070 con 0x7fc894163620 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.334+0000 7fc899307640 1 -- 172.21.15.99:0/2800025544 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88402ae50 con 0x7fc894163620 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.334+0000 7fc899307640 1 -- 172.21.15.99:0/2800025544 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884032660 con 0x7fc894163620 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 -- 172.21.15.99:0/2800025544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc894163620 msgr2=0x7fc894160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2800025544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc894163620 0x7fc894160030 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc88402a140 comp rx=0 tx=0).stop 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 -- 172.21.15.99:0/2800025544 shutdown_connections 2024-09-10T11:34:14.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2800025544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc894163620 0x7fc894160030 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 -- 172.21.15.99:0/2800025544 >> 172.21.15.99:0/2800025544 conn(0x7fc8940b3630 msgr2=0x7fc8940b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:14.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 -- 172.21.15.99:0/2800025544 shutdown_connections 2024-09-10T11:34:14.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.335+0000 7fc89c56d640 1 -- 172.21.15.99:0/2800025544 wait complete. 2024-09-10T11:34:14.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.336+0000 7fc89c56d640 1 Processor -- start 2024-09-10T11:34:14.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.336+0000 7fc89c56d640 1 -- start start 2024-09-10T11:34:14.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc89406bc10 0x7fc8941f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894163620 0x7fc89415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8940d1560 con 0x7fc8940b4930 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8940b4dc0 con 0x7fc894163620 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89c56d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8940d16e0 con 0x7fc89406bc10 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35050/0 (socket says 172.21.15.99:35050) 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 -- 172.21.15.99:0/2795261256 learned_addr learned my addr 172.21.15.99:0/2795261256 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc89ab0a640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894163620 0x7fc89415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 -- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc89406bc10 msgr2=0x7fc8941f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc89406bc10 0x7fc8941f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 -- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894163620 msgr2=0x7fc89415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894163620 0x7fc89415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc894159b10 con 0x7fc8940b4930 2024-09-10T11:34:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.337+0000 7fc899b08640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc8840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.338+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884037040 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.338+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc884004030 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.338+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc894159db0 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.338+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884006e10 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.338+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89415d100 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.339+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8940b4390 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.339+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc88403d650 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.339+0000 7fc88affd640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 0x7fc87806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.340+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fc8840ab6a0 con 0x7fc8940b4930 2024-09-10T11:34:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.340+0000 7fc89a309640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 0x7fc87806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.340+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7fc878073960 con 0x7fc8940b4930 2024-09-10T11:34:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.340+0000 7fc89a309640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 0x7fc87806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc890007fd0 tx=0x7fc890007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.344+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8940b4390 con 0x7fc8940b4930 2024-09-10T11:34:14.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.543+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8940b5140 con 0x7fc8940b4930 2024-09-10T11:34:14.545 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:14.543+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:34:14.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.545+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7fc884042020 con 0x7fc8940b4930 2024-09-10T11:34:14.546 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:34:14.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.545+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(91..91 src has 1..91) ==== 390+0+0 (secure 0 0 0) 0x7fc8840dfd50 con 0x7fc8940b4930 2024-09-10T11:34:14.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.545+0000 7fc88affd640 1 -- 172.21.15.99:0/2795261256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7fc8780743a0 con 0x7fc8940b4930 2024-09-10T11:34:14.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 msgr2=0x7fc87806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 0x7fc87806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc890007fd0 tx=0x7fc890007960 comp rx=0 tx=0).stop 2024-09-10T11:34:14.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 msgr2=0x7fc8941f77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc8840097a0 tx=0x7fc8840046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 shutdown_connections 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc87806be30 0x7fc87806e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc894163620 0x7fc89415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8940b4930 0x7fc8941f77d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 --2- 172.21.15.99:0/2795261256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc89406bc10 0x7fc8941f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 >> 172.21.15.99:0/2795261256 conn(0x7fc8940b3630 msgr2=0x7fc894072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 shutdown_connections 2024-09-10T11:34:14.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.547+0000 7fc89c56d640 1 -- 172.21.15.99:0/2795261256 wait complete. 2024-09-10T11:34:14.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:34:14.589 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:34:14.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.660+0000 7fc72a1f8640 1 Processor -- start 2024-09-10T11:34:14.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.660+0000 7fc72a1f8640 1 -- start start 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc72a1f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc724160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc72a1f8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7240b5140 con 0x7fc7240b4930 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc72a1f8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc724162b40 con 0x7fc72406bc10 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc72a1f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc7240b5420 con 0x7fc724163620 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc723fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc7240b4930 0x7fc72406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59244/0 (socket says 172.21.15.99:59244) 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc723fff640 1 -- 172.21.15.99:0/3758261043 learned_addr learned my addr 172.21.15.99:0/3758261043 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.661+0000 7fc722ffd640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc724160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 497728991 0 0) 0x7fc7240b5140 con 0x7fc7240b4930 2024-09-10T11:34:14.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc70c003520 con 0x7fc7240b4930 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 765883201 0 0) 0x7fc70c003520 con 0x7fc7240b4930 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 msgr2=0x7fc724160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc724160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 >> v1:172.21.15.196:6789/0 conn(0x7fc72406bc10 legacy=0x7fc72406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc724162e20 con 0x7fc7240b4930 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fc71c001670 con 0x7fc7240b4930 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 >> v1:172.21.15.99:6789/0 conn(0x7fc7240b4930 legacy=0x7fc72406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc70c0039e0 0x7fc70c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc70c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c004d40 0x7fc70c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7240b5420 con 0x7fc70c0039e0 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c00ba70 con 0x7fc70c004d40 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c00bc70 con 0x7fc724163620 2024-09-10T11:34:14.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc70c0039e0 0x7fc70c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc723fff640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c004d40 0x7fc70c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc722ffd640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc70c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 -- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 msgr2=0x7fc70c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc724163620 0x7fc70c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 -- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c004d40 msgr2=0x7fc70c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc70c004d40 0x7fc70c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.662+0000 7fc7237fe640 1 -- 172.21.15.99:0/3758261043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c003520 con 0x7fc70c0039e0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.663+0000 7fc7237fe640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc70c0039e0 0x7fc70c003dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc714001860 tx=0x7fc70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1be2a3db0d431e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.663+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714037070 con 0x7fc70c0039e0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.663+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71402ac30 con 0x7fc70c0039e0 2024-09-10T11:34:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.663+0000 7fc7227fc640 1 -- 172.21.15.99:0/3758261043 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714001430 con 0x7fc70c0039e0 2024-09-10T11:34:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 -- 172.21.15.99:0/3758261043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc70c0039e0 msgr2=0x7fc70c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/3758261043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc70c0039e0 0x7fc70c003dc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc714001860 tx=0x7fc70c005830 comp rx=0 tx=0).stop 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc723fff640 1 -- 172.21.15.99:0/3758261043 reap_dead start 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 -- 172.21.15.99:0/3758261043 shutdown_connections 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 -- 172.21.15.99:0/3758261043 >> 172.21.15.99:0/3758261043 conn(0x7fc7240b3630 msgr2=0x7fc7240b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 -- 172.21.15.99:0/3758261043 shutdown_connections 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.664+0000 7fc72a1f8640 1 -- 172.21.15.99:0/3758261043 wait complete. 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.665+0000 7fc72a1f8640 1 Processor -- start 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.665+0000 7fc72a1f8640 1 -- start start 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc70c0039e0 0x7fc7241f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7241433f0 0x7fc724143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc7240b4d90 con 0x7fc72413fc20 2024-09-10T11:34:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7240d1560 con 0x7fc7241433f0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc72a1f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc7241413d0 con 0x7fc70c0039e0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35066/0 (socket says 172.21.15.99:35066) 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 -- 172.21.15.99:0/47761457 learned_addr learned my addr 172.21.15.99:0/47761457 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc7237fe640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc70c0039e0 0x7fc7241f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc723fff640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7241433f0 0x7fc724143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 -- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc70c0039e0 msgr2=0x7fc7241f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc70c0039e0 0x7fc7241f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 -- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7241433f0 msgr2=0x7fc724143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7241433f0 0x7fc724143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72413eb80 con 0x7fc72413fc20 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.666+0000 7fc722ffd640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc718011e30 tx=0x7fc71800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.667+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718015630 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.667+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7180157d0 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.667+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72413ee60 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.667+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718015ba0 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.667+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72413f420 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.668+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724070ad0 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.668+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc71801d400 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.668+0000 7fc70bfff640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 0x7fc71006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.668+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fc71808bde0 con 0x7fc72413fc20 2024-09-10T11:34:14.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.668+0000 7fc7237fe640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 0x7fc71006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.669+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7fc710073a70 con 0x7fc72413fc20 2024-09-10T11:34:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.669+0000 7fc7237fe640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 0x7fc71006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc714034000 tx=0x7fc714009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:14.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.673+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc724070ad0 con 0x7fc72413fc20 2024-09-10T11:34:14.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:14.870+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc724162b40 con 0x7fc72413fc20 2024-09-10T11:34:15.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.549+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7fc718019030 con 0x7fc72413fc20 2024-09-10T11:34:15.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.550+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(92..92 src has 1..92) ==== 346+0+0 (secure 0 0 0) 0x7fc71808a3c0 con 0x7fc72413fc20 2024-09-10T11:34:15.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.550+0000 7fc70bfff640 1 -- 172.21.15.99:0/47761457 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fc710074600 con 0x7fc72413fc20 2024-09-10T11:34:15.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.554+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 msgr2=0x7fc71006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.554+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 0x7fc71006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc714034000 tx=0x7fc714009030 comp rx=0 tx=0).stop 2024-09-10T11:34:15.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.554+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 msgr2=0x7fc72413e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.554+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc718011e30 tx=0x7fc71800f580 comp rx=0 tx=0).stop 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 shutdown_connections 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc71006be30 0x7fc71006e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7241433f0 0x7fc724143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc72413fc20 0x7fc72413e4f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 --2- 172.21.15.99:0/47761457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc70c0039e0 0x7fc7241f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 >> 172.21.15.99:0/47761457 conn(0x7fc7240b3630 msgr2=0x7fc7240b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 shutdown_connections 2024-09-10T11:34:15.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.555+0000 7fc72a1f8640 1 -- 172.21.15.99:0/47761457 wait complete. 2024-09-10T11:34:15.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:34:15.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.670+0000 7fa5aef18640 1 Processor -- start 2024-09-10T11:34:15.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.670+0000 7fa5aef18640 1 -- start start 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5aef18640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa5a80af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5aef18640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a81635c0 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5aef18640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a80b7b50 con 0x7fa5a8162ef0 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5aef18640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a806d550 con 0x7fa5a815ee10 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5a7fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa5a80b6fe0 0x7fa5a81627e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59248/0 (socket says 172.21.15.99:59248) 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.671+0000 7fa5a7fff640 1 -- 172.21.15.99:0/2364980844 learned_addr learned my addr 172.21.15.99:0/2364980844 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa5a80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074294390 0 0) 0x7fa5a81635c0 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa594003520 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 606733745 0 0) 0x7fa594003520 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 msgr2=0x7fa5a80af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa5a80af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 >> v1:172.21.15.196:6789/0 conn(0x7fa5a8162ef0 legacy=0x7fa5a806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a806cf00 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa5980019f0 con 0x7fa5a80b6fe0 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 >> v1:172.21.15.99:6789/0 conn(0x7fa5a80b6fe0 legacy=0x7fa5a81627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5940039e0 0x7fa594003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa594004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a806d550 con 0x7fa5940039e0 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa59400b9c0 con 0x7fa594004d40 2024-09-10T11:34:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa59400bbc0 con 0x7fa5a815ee10 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5940039e0 0x7fa594003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5a7fff640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa594004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5ad4b5640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 -- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 msgr2=0x7fa594004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5a815ee10 0x7fa594004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 -- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa594004d40 msgr2=0x7fa5940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5accb4640 1 -- 172.21.15.99:0/2364980844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa594003520 con 0x7fa5940039e0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.672+0000 7fa5ad4b5640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa594004d40 0x7fa5940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.673+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5940039e0 0x7fa594003dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa590009ac0 tx=0x7fa594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7477d5ba7d03f71f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.673+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590036070 con 0x7fa5940039e0 2024-09-10T11:34:15.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.673+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590031400 con 0x7fa5940039e0 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.673+0000 7fa5a77fe640 1 -- 172.21.15.99:0/2364980844 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590009690 con 0x7fa5940039e0 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 -- 172.21.15.99:0/2364980844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5940039e0 msgr2=0x7fa594003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2364980844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5940039e0 0x7fa594003dc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa590009ac0 tx=0x7fa594005830 comp rx=0 tx=0).stop 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5ad4b5640 1 -- 172.21.15.99:0/2364980844 reap_dead start 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 -- 172.21.15.99:0/2364980844 shutdown_connections 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 -- 172.21.15.99:0/2364980844 >> 172.21.15.99:0/2364980844 conn(0x7fa5a8073c60 msgr2=0x7fa5a809ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 -- 172.21.15.99:0/2364980844 shutdown_connections 2024-09-10T11:34:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.674+0000 7fa5aef18640 1 -- 172.21.15.99:0/2364980844 wait complete. 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.675+0000 7fa5aef18640 1 Processor -- start 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.675+0000 7fa5aef18640 1 -- start start 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5940039e0 0x7fa5a814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8150590 0x7fa5a815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80afab0 con 0x7fa5a815c0c0 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80af930 con 0x7fa5a8150590 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5aef18640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a80afc30 con 0x7fa5940039e0 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35088/0 (socket says 172.21.15.99:35088) 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 -- 172.21.15.99:0/2371876303 learned_addr learned my addr 172.21.15.99:0/2371876303 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5940039e0 0x7fa5a814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5a7fff640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8150590 0x7fa5a815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 -- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5940039e0 msgr2=0x7fa5a814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5940039e0 0x7fa5a814fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 -- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8150590 msgr2=0x7fa5a815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8150590 0x7fa5a815db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8150260 con 0x7fa5a815c0c0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5ad4b5640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8073500 tx=0x7fa5a000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.676+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0019070 con 0x7fa5a815c0c0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.677+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a00052e0 con 0x7fa5a815c0c0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.677+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a815e500 con 0x7fa5a815c0c0 2024-09-10T11:34:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.677+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0003810 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.677+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a815b5e0 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.678+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a80b7440 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.678+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5a001c070 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.679+0000 7fa5a57fa640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 0x7fa58806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.679+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 0x7fa58806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.679+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5853+0+0 (secure 0 0 0) 0x7fa5a008a170 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.679+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=93}) -- 0x7fa588073b30 con 0x7fa5a815c0c0 2024-09-10T11:34:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.679+0000 7fa5accb4640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 0x7fa58806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.683+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5a0058440 con 0x7fa5a815c0c0 2024-09-10T11:34:15.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:15.849+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5a80d3790 con 0x7fa5a815c0c0 2024-09-10T11:34:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.551+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7fa5a005c740 con 0x7fa5a815c0c0 2024-09-10T11:34:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:34:16.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.554+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(93..93 src has 1..93) ==== 321+0+0 (secure 0 0 0) 0x7fa5a00bed50 con 0x7fa5a815c0c0 2024-09-10T11:34:16.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.554+0000 7fa5a57fa640 1 -- 172.21.15.99:0/2371876303 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=94}) -- 0x7fa588071f60 con 0x7fa5a815c0c0 2024-09-10T11:34:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 msgr2=0x7fa58806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 0x7fa58806e2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa590033000 tx=0x7fa590008030 comp rx=0 tx=0).stop 2024-09-10T11:34:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 msgr2=0x7fa5a815c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8073500 tx=0x7fa5a000e8a0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 shutdown_connections 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa58806be30 0x7fa58806e2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5a815c0c0 0x7fa5a815c570 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a8150590 0x7fa5a815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 --2- 172.21.15.99:0/2371876303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5940039e0 0x7fa5a814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 >> 172.21.15.99:0/2371876303 conn(0x7fa5a8073c60 msgr2=0x7fa5a80b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 shutdown_connections 2024-09-10T11:34:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.559+0000 7fa5aef18640 1 -- 172.21.15.99:0/2371876303 wait complete. 2024-09-10T11:34:16.574 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:34:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.678+0000 7f63270d0640 1 Processor -- start 2024-09-10T11:34:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.678+0000 7f63270d0640 1 -- start start 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f63270d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f63200b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f63270d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63200b1fa0 0x7f632015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f63270d0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63200b2510 con 0x7f63200b6510 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f63270d0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f63200b27f0 con 0x7f63200b1fa0 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f63270d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63200b2ad0 con 0x7f632015cab0 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f631ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63200b1fa0 0x7f632015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f632566d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f63200b6510 0x7f63200b35b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59256/0 (socket says 172.21.15.99:59256) 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.679+0000 7f632566d640 1 -- 172.21.15.99:0/2250375640 learned_addr learned my addr 172.21.15.99:0/2250375640 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631ffff640 1 -- 172.21.15.99:0/2250375640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 msgr2=0x7f63200b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631ffff640 1 --2- 172.21.15.99:0/2250375640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f63200b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631ffff640 1 -- 172.21.15.99:0/2250375640 >> v1:172.21.15.99:6789/0 conn(0x7f63200b6510 legacy=0x7f63200b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631f7fe640 1 -- 172.21.15.99:0/2250375640 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006703467 0 0) 0x7f63200b2510 con 0x7f63200b6510 2024-09-10T11:34:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631ffff640 1 -- 172.21.15.99:0/2250375640 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63200b2db0 con 0x7f63200b1fa0 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.680+0000 7f631ffff640 1 --2- 172.21.15.99:0/2250375640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632015ce90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6314009d90 tx=0x7f631402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed734f4ca52855c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f631f7fe640 1 -- 172.21.15.99:0/2250375640 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314007740 con 0x7f63200b1fa0 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f631f7fe640 1 -- 172.21.15.99:0/2250375640 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63140078e0 con 0x7f63200b1fa0 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f631f7fe640 1 -- 172.21.15.99:0/2250375640 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314007cb0 con 0x7f63200b1fa0 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 -- 172.21.15.99:0/2250375640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 msgr2=0x7f632015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 --2- 172.21.15.99:0/2250375640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632015ce90 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6314009d90 tx=0x7f631402a6f0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 -- 172.21.15.99:0/2250375640 shutdown_connections 2024-09-10T11:34:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 --2- 172.21.15.99:0/2250375640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632015ce90 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 --2- 172.21.15.99:0/2250375640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f63200b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 -- 172.21.15.99:0/2250375640 >> 172.21.15.99:0/2250375640 conn(0x7f6320073c60 msgr2=0x7f632009ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 -- 172.21.15.99:0/2250375640 shutdown_connections 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.681+0000 7f63270d0640 1 -- 172.21.15.99:0/2250375640 wait complete. 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.682+0000 7f63270d0640 1 Processor -- start 2024-09-10T11:34:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.682+0000 7f63270d0640 1 -- start start 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63200b6510 0x7f632009f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f632014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f632015d3d0 con 0x7f63200b1fa0 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63200b6120 con 0x7f63200b6510 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f63270d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f632015d550 con 0x7f632015cab0 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f6324e6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f6324e6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35106/0 (socket says 172.21.15.99:35106) 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.683+0000 7f6324e6c640 1 -- 172.21.15.99:0/2647454023 learned_addr learned my addr 172.21.15.99:0/2647454023 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 -- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 msgr2=0x7f632014a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f632014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 -- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63200b6510 msgr2=0x7f632009f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63200b6510 0x7f632009f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63201579d0 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f6324e6c640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f631000b410 tx=0x7f631000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310015020 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6310011850 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632009fdb0 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310011c40 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.684+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63200a0370 con 0x7f63200b1fa0 2024-09-10T11:34:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.685+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632015ce90 con 0x7f63200b1fa0 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.685+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6310018020 con 0x7f63200b1fa0 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.686+0000 7f631d7fa640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 0x7f630c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.686+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5386+0+0 (secure 0 0 0) 0x7f631008ac70 con 0x7f63200b1fa0 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.686+0000 7f631ffff640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 0x7f630c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.686+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=94}) -- 0x7f630c072cc0 con 0x7f63200b1fa0 2024-09-10T11:34:16.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.686+0000 7f631ffff640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 0x7f630c06e2f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6314034000 tx=0x7f631402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:16.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.690+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f631001f020 con 0x7f63200b1fa0 2024-09-10T11:34:16.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:16.855+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63201e4900 con 0x7f63200b1fa0 2024-09-10T11:34:17.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.557+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f63201579d0 con 0x7f63200b1fa0 2024-09-10T11:34:17.559 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:34:17.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.566+0000 7f631d7fa640 1 -- 172.21.15.99:0/2647454023 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(94..94 src has 1..94) ==== 321+0+0 (secure 0 0 0) 0x7f6310090020 con 0x7f63200b1fa0 2024-09-10T11:34:17.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.566+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 msgr2=0x7f630c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.566+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 0x7f630c06e2f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6314034000 tx=0x7f631402e000 comp rx=0 tx=0).stop 2024-09-10T11:34:17.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.566+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 msgr2=0x7f632009c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.566+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f631000b410 tx=0x7f631000b8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 shutdown_connections 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630c06be30 0x7f630c06e2f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f632015cab0 0x7f632014a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63200b6510 0x7f632009f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 --2- 172.21.15.99:0/2647454023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63200b1fa0 0x7f632009c9b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 >> 172.21.15.99:0/2647454023 conn(0x7f6320073c60 msgr2=0x7f632015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:17.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.567+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 shutdown_connections 2024-09-10T11:34:17.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.568+0000 7f63270d0640 1 -- 172.21.15.99:0/2647454023 wait complete. 2024-09-10T11:34:17.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:34:17.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.684+0000 7f53eeca4640 1 Processor -- start 2024-09-10T11:34:17.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.684+0000 7f53eeca4640 1 -- start start 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eeca4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53e8163580 0x7f53e80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eeca4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53e80b3960 0x7f53e80d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eeca4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e80b3620 con 0x7f53e806cd20 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eeca4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f53e806d960 con 0x7f53e80b3960 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eeca4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53e806dc40 con 0x7f53e8163580 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53e80b3960 0x7f53e80d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53e80b3960 0x7f53e80d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35112/0 (socket says 172.21.15.99:35112) 2024-09-10T11:34:17.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 -- 172.21.15.99:0/1368058879 learned_addr learned my addr 172.21.15.99:0/1368058879 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53ed241640 1 --1- 172.21.15.99:0/1368058879 >> v1:172.21.15.99:6789/0 conn(0x7f53e806cd20 0x7f53e806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59266/0 (socket says 172.21.15.99:59266) 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53eca40640 1 --2- 172.21.15.99:0/1368058879 >> v2:172.21.15.196:3300/0 conn(0x7f53e8163580 0x7f53e80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 -- 172.21.15.99:0/1368058879 >> v2:172.21.15.196:3300/0 conn(0x7f53e8163580 msgr2=0x7f53e80b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 --2- 172.21.15.99:0/1368058879 >> v2:172.21.15.196:3300/0 conn(0x7f53e8163580 0x7f53e80b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 -- 172.21.15.99:0/1368058879 >> v1:172.21.15.99:6789/0 conn(0x7f53e806cd20 legacy=0x7f53e806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.685+0000 7f53e7fff640 1 -- 172.21.15.99:0/1368058879 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e806df20 con 0x7f53e80b3960 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.686+0000 7f53e7fff640 1 --2- 172.21.15.99:0/1368058879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80d1be0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f53e8072600 tx=0x7f53dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47bc11e9f9bc859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.686+0000 7f53e77fe640 1 -- 172.21.15.99:0/1368058879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc037070 con 0x7f53e80b3960 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.686+0000 7f53e77fe640 1 -- 172.21.15.99:0/1368058879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc032440 con 0x7f53e80b3960 2024-09-10T11:34:17.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.686+0000 7f53e77fe640 1 -- 172.21.15.99:0/1368058879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc032810 con 0x7f53e80b3960 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 -- 172.21.15.99:0/1368058879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 msgr2=0x7f53e80d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 --2- 172.21.15.99:0/1368058879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80d1be0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f53e8072600 tx=0x7f53dc02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 -- 172.21.15.99:0/1368058879 shutdown_connections 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 --2- 172.21.15.99:0/1368058879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80d1be0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 --2- 172.21.15.99:0/1368058879 >> v2:172.21.15.196:3300/0 conn(0x7f53e8163580 0x7f53e80b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 -- 172.21.15.99:0/1368058879 >> 172.21.15.99:0/1368058879 conn(0x7f53e80b4570 msgr2=0x7f53e80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 -- 172.21.15.99:0/1368058879 shutdown_connections 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.687+0000 7f53eeca4640 1 -- 172.21.15.99:0/1368058879 wait complete. 2024-09-10T11:34:17.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.688+0000 7f53eeca4640 1 Processor -- start 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.688+0000 7f53eeca4640 1 -- start start 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e806cd20 0x7f53e8153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e8163580 0x7f53e80cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53e80b3160 con 0x7f53e80b3960 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53e80d2120 con 0x7f53e8163580 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53eeca4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53e80b32e0 con 0x7f53e806cd20 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35120/0 (socket says 172.21.15.99:35120) 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 -- 172.21.15.99:0/3823039007 learned_addr learned my addr 172.21.15.99:0/3823039007 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 -- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e806cd20 msgr2=0x7f53e8153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e806cd20 0x7f53e8153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 -- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e8163580 msgr2=0x7f53e80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e8163580 0x7f53e80cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.689+0000 7f53e7fff640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e80b8020 con 0x7f53e80b3960 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53e7fff640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02a220 tx=0x7f53dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc037040 con 0x7f53e80b3960 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53dc042070 con 0x7f53e80b3960 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e80b8300 con 0x7f53e80b3960 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc030880 con 0x7f53e80b3960 2024-09-10T11:34:17.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.690+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e80cd2a0 con 0x7f53e80b3960 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.691+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e80d1be0 con 0x7f53e80b3960 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.691+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53dc030a20 con 0x7f53e80b3960 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.691+0000 7f53e57fa640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 0x7f53d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.692+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f53dc0ab3b0 con 0x7f53e80b3960 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.692+0000 7f53eca40640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 0x7f53d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.692+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=95}) -- 0x7f53d4071620 con 0x7f53e80b3960 2024-09-10T11:34:17.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.692+0000 7f53eca40640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 0x7f53d406e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e190 tx=0x7f53d800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.696+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53dc07a0f0 con 0x7f53e80b3960 2024-09-10T11:34:17.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.861+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f53e80b89a0 con 0x7f53e80b3960 2024-09-10T11:34:17.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.864+0000 7f53e57fa640 1 -- 172.21.15.99:0/3823039007 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f53dc0799f0 con 0x7f53e80b3960 2024-09-10T11:34:17.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 msgr2=0x7f53d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 0x7f53d406e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f53d800e190 tx=0x7f53d800b040 comp rx=0 tx=0).stop 2024-09-10T11:34:17.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 msgr2=0x7f53e80ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f53dc02a220 tx=0x7f53dc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 shutdown_connections 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53d406be30 0x7f53d406e2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53e8163580 0x7f53e80cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53e80b3960 0x7f53e80ceeb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 --2- 172.21.15.99:0/3823039007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53e806cd20 0x7f53e8153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 >> 172.21.15.99:0/3823039007 conn(0x7f53e80b4570 msgr2=0x7f53e80b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 shutdown_connections 2024-09-10T11:34:17.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.866+0000 7f53eeca4640 1 -- 172.21.15.99:0/3823039007 wait complete. 2024-09-10T11:34:17.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-09-10T11:34:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.975+0000 7f44a0d9f640 1 Processor -- start 2024-09-10T11:34:17.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.975+0000 7f44a0d9f640 1 -- start start 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f449c0b0de0 0x7f449c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f449c06e0d0 0x7f449c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c072800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f449c070100 con 0x7f449c0b0de0 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f449c072d40 con 0x7f449c06e0d0 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f44a0d9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f449c06eb20 con 0x7f449c0723c0 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f449c0b0de0 0x7f449c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f449c0b0de0 0x7f449c06fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35124/0 (socket says 172.21.15.99:35124) 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 -- 172.21.15.99:0/2257716093 learned_addr learned my addr 172.21.15.99:0/2257716093 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 -- 172.21.15.99:0/2257716093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 msgr2=0x7f449c072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 --2- 172.21.15.99:0/2257716093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 -- 172.21.15.99:0/2257716093 >> v2:172.21.15.196:3300/0 conn(0x7f449c06e0d0 msgr2=0x7f449c06e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 --2- 172.21.15.99:0/2257716093 >> v2:172.21.15.196:3300/0 conn(0x7f449c06e0d0 0x7f449c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.976+0000 7f449b7fe640 1 -- 172.21.15.99:0/2257716093 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c06ee00 con 0x7f449c0b0de0 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f449b7fe640 1 --2- 172.21.15.99:0/2257716093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c06fbc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f449c0b5f90 tx=0x7f448c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d2b5b04be4f14e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f449a7fc640 1 -- 172.21.15.99:0/2257716093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c037070 con 0x7f449c0b0de0 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f449a7fc640 1 -- 172.21.15.99:0/2257716093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c032440 con 0x7f449c0b0de0 2024-09-10T11:34:17.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f449a7fc640 1 -- 172.21.15.99:0/2257716093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c032810 con 0x7f449c0b0de0 2024-09-10T11:34:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2257716093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 msgr2=0x7f449c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.977+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2257716093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c06fbc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f449c0b5f90 tx=0x7f448c02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2257716093 shutdown_connections 2024-09-10T11:34:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2257716093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2257716093 >> v2:172.21.15.196:3300/0 conn(0x7f449c06e0d0 0x7f449c06e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2257716093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c06fbc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2257716093 >> 172.21.15.99:0/2257716093 conn(0x7f449c0b3550 msgr2=0x7f449c0b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2257716093 shutdown_connections 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.978+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2257716093 wait complete. 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.979+0000 7f44a0d9f640 1 Processor -- start 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.979+0000 7f44a0d9f640 1 -- start start 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 0x7f449c146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f449c06f6a0 con 0x7f449c0b0de0 2024-09-10T11:34:17.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f449c06f3c0 con 0x7f449c06e0d0 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f449c0b11c0 con 0x7f449c0723c0 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 0x7f449c146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449b7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 0x7f449c146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:49462/0 (socket says 172.21.15.99:49462) 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35132/0 (socket says 172.21.15.99:35132) 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449b7fe640 1 -- 172.21.15.99:0/2883092724 learned_addr learned my addr 172.21.15.99:0/2883092724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449affd640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 -- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 msgr2=0x7f449c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 -- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 msgr2=0x7f449c146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:17.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 0x7f449c146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c152730 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f449bfff640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f449400b410 tx=0x7f449400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494015020 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494011830 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449c152a10 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.980+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494011c20 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.981+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449c152fd0 con 0x7f449c0b0de0 2024-09-10T11:34:17.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.981+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c0b4a10 con 0x7f449c0b0de0 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.982+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4494018020 con 0x7f449c0b0de0 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.982+0000 7f4483fff640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 0x7f448806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.982+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7f449408a8f0 con 0x7f449c0b0de0 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.982+0000 7f449b7fe640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 0x7f448806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.983+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=95}) -- 0x7f4488071620 con 0x7f449c0b0de0 2024-09-10T11:34:17.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.983+0000 7f449b7fe640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 0x7f448806e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f448c0090b0 tx=0x7f448c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:17.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:17.987+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f449401f020 con 0x7f449c0b0de0 2024-09-10T11:34:18.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.217+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7f449c157250 con 0x7f449c0b0de0 2024-09-10T11:34:18.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.559+0000 7f4483fff640 1 -- 172.21.15.99:0/2883092724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v206) ==== 133+0+0 (secure 0 0 0) 0x7f4494058f50 con 0x7f449c0b0de0 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 msgr2=0x7f448806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 0x7f448806e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f448c0090b0 tx=0x7f448c02e000 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 msgr2=0x7f449c14c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f449400b410 tx=0x7f449400b8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 shutdown_connections 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f448806be30 0x7f448806e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f449c0b0de0 0x7f449c14c2e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f449c0723c0 0x7f449c0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 --2- 172.21.15.99:0/2883092724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f449c06e0d0 0x7f449c146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 >> 172.21.15.99:0/2883092724 conn(0x7f449c0b3550 msgr2=0x7f449c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:18.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 shutdown_connections 2024-09-10T11:34:18.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.561+0000 7f44a0d9f640 1 -- 172.21.15.99:0/2883092724 wait complete. 2024-09-10T11:34:18.574 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-09-10T11:34:18.574 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-09-10T11:34:18.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:18.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.671+0000 7ff22cbb7640 1 Processor -- start 2024-09-10T11:34:18.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.671+0000 7ff22cbb7640 1 -- start start 2024-09-10T11:34:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2280d1200 0x7ff228073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff22806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff22815c060 0x7ff22815c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff22806ef50 con 0x7ff2280d1200 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff228152e60 con 0x7ff22815c060 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22cbb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff2280d15e0 con 0x7ff22806e5d0 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2280d1200 0x7ff228073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff2280d1200 0x7ff228073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35138/0 (socket says 172.21.15.99:35138) 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 -- 172.21.15.99:0/3994363061 learned_addr learned my addr 172.21.15.99:0/3994363061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff21dd9b640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff22806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 -- 172.21.15.99:0/3994363061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 msgr2=0x7ff22806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff22806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 -- 172.21.15.99:0/3994363061 >> v2:172.21.15.196:3300/0 conn(0x7ff22815c060 msgr2=0x7ff22815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 --2- 172.21.15.99:0/3994363061 >> v2:172.21.15.196:3300/0 conn(0x7ff22815c060 0x7ff22815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff22659c640 1 -- 172.21.15.99:0/3994363061 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff228072680 con 0x7ff2280d1200 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.672+0000 7ff21dd9b640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff22806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:18.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22659c640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2280d1200 0x7ff228073440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff228071b80 tx=0x7ff21802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b88b9d8ecb4e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff225d9b640 1 -- 172.21.15.99:0/3994363061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218037070 con 0x7ff2280d1200 2024-09-10T11:34:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff225d9b640 1 -- 172.21.15.99:0/3994363061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff218032440 con 0x7ff2280d1200 2024-09-10T11:34:18.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff225d9b640 1 -- 172.21.15.99:0/3994363061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff218032810 con 0x7ff2280d1200 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3994363061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2280d1200 msgr2=0x7ff228073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2280d1200 0x7ff228073440 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff228071b80 tx=0x7ff21802a140 comp rx=0 tx=0).stop 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3994363061 shutdown_connections 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3994363061 >> v2:172.21.15.196:3300/0 conn(0x7ff22815c060 0x7ff22815c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff22806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3994363061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff2280d1200 0x7ff228073440 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3994363061 >> 172.21.15.99:0/3994363061 conn(0x7ff22806bc10 msgr2=0x7ff22806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3994363061 shutdown_connections 2024-09-10T11:34:18.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.673+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3994363061 wait complete. 2024-09-10T11:34:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.675+0000 7ff22cbb7640 1 Processor -- start 2024-09-10T11:34:18.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.675+0000 7ff22cbb7640 1 -- start start 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff2281586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2280d1200 0x7ff2281f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff228072f20 con 0x7ff22815c060 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff228072c40 con 0x7ff2280d1200 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff22815cd90 con 0x7ff22806e5d0 2024-09-10T11:34:18.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35152/0 (socket says 172.21.15.99:35152) 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 -- 172.21.15.99:0/3443662474 learned_addr learned my addr 172.21.15.99:0/3443662474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22659c640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff2281586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff21dd9b640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2280d1200 0x7ff2281f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 -- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 msgr2=0x7ff2281586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff2281586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 -- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2280d1200 msgr2=0x7ff2281f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2280d1200 0x7ff2281f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2281f3ac0 con 0x7ff22815c060 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff226d9d640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b970 tx=0x7ff22000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220019070 con 0x7ff22815c060 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff22000cb60 con 0x7ff22815c060 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2281f3da0 con 0x7ff22815c060 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff220014440 con 0x7ff22815c060 2024-09-10T11:34:18.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.676+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2281f4360 con 0x7ff22815c060 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.677+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22806ea10 con 0x7ff22815c060 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.678+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff22001c070 con 0x7ff22815c060 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.678+0000 7ff21f7fe640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 0x7ff20806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.678+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 4943+0+0 (secure 0 0 0) 0x7ff22008a820 con 0x7ff22815c060 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.678+0000 7ff22659c640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 0x7ff20806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.679+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=95}) -- 0x7ff208071620 con 0x7ff22815c060 2024-09-10T11:34:18.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.679+0000 7ff22659c640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 0x7ff20806e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff218034000 tx=0x7ff21802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.683+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff220058e80 con 0x7ff22815c060 2024-09-10T11:34:18.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.847+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff22806c090 con 0x7ff22815c060 2024-09-10T11:34:18.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.848+0000 7ff21f7fe640 1 -- 172.21.15.99:0/3443662474 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13062 (secure 0 0 0) 0x7ff22005d180 con 0x7ff22815c060 2024-09-10T11:34:18.850 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:18.850 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":94,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:17.553237+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1122563557":"2024-09-11T11:34:14.544808+0000","172.21.15.196:6833/1122563557":"2024-09-11T11:34:14.544808+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 msgr2=0x7ff20806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 0x7ff20806e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff218034000 tx=0x7ff21802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 msgr2=0x7ff2281565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff22000b970 tx=0x7ff22000be30 comp rx=0 tx=0).stop 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 shutdown_connections 2024-09-10T11:34:18.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff20806be30 0x7ff20806e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff22815c060 0x7ff2281565d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2280d1200 0x7ff2281f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 --2- 172.21.15.99:0/3443662474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff22806e5d0 0x7ff2281586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 >> 172.21.15.99:0/3443662474 conn(0x7ff22806bc10 msgr2=0x7ff22815f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 shutdown_connections 2024-09-10T11:34:18.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.850+0000 7ff22cbb7640 1 -- 172.21.15.99:0/3443662474 wait complete. 2024-09-10T11:34:18.863 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:34:18.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:34:18.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.963+0000 7fdc5fd4b640 1 Processor -- start 2024-09-10T11:34:18.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.963+0000 7fdc5fd4b640 1 -- start start 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5fd4b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc58163580 0x7fdc580b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5fd4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc5806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5fd4b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc580b3620 con 0x7fdc580b3960 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5fd4b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5806d960 con 0x7fdc58163580 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5fd4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5806dc40 con 0x7fdc580b3160 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5dae7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc58163580 0x7fdc580b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5dae7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc58163580 0x7fdc580b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35156/0 (socket says 172.21.15.99:35156) 2024-09-10T11:34:18.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5d2e6640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdc580b3960 0x7fdc580d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59280/0 (socket says 172.21.15.99:59280) 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5dae7640 1 -- 172.21.15.99:0/3156535188 learned_addr learned my addr 172.21.15.99:0/3156535188 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5dae7640 1 -- 172.21.15.99:0/3156535188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 msgr2=0x7fdc5806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5e2e8640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc5806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.964+0000 7fdc5dae7640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc5806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5dae7640 1 -- 172.21.15.99:0/3156535188 >> v1:172.21.15.99:6789/0 conn(0x7fdc580b3960 legacy=0x7fdc580d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5dae7640 1 -- 172.21.15.99:0/3156535188 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5806df20 con 0x7fdc58163580 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5cae5640 1 -- 172.21.15.99:0/3156535188 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3687259787 0 0) 0x7fdc580b3620 con 0x7fdc580b3960 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5e2e8640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc5806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5dae7640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc580b5090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009e50 tx=0x7fdc5402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4909e8ceaf0048b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5cae5640 1 -- 172.21.15.99:0/3156535188 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54036070 con 0x7fdc58163580 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5cae5640 1 -- 172.21.15.99:0/3156535188 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc54031600 con 0x7fdc58163580 2024-09-10T11:34:18.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.965+0000 7fdc5cae5640 1 -- 172.21.15.99:0/3156535188 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc540319d0 con 0x7fdc58163580 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3156535188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 msgr2=0x7fdc580b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc580b5090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009e50 tx=0x7fdc5402a5f0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3156535188 shutdown_connections 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc5806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3156535188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc580b5090 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3156535188 >> 172.21.15.99:0/3156535188 conn(0x7fdc580b4570 msgr2=0x7fdc580b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3156535188 shutdown_connections 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.966+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3156535188 wait complete. 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.967+0000 7fdc5fd4b640 1 Processor -- start 2024-09-10T11:34:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.967+0000 7fdc5fd4b640 1 -- start start 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc58154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc580b3960 0x7fdc58154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5806d3e0 con 0x7fdc58163580 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5806d260 con 0x7fdc580b3960 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5fd4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5806d560 con 0x7fdc580b3160 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35160/0 (socket says 172.21.15.99:35160) 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 -- 172.21.15.99:0/3086022563 learned_addr learned my addr 172.21.15.99:0/3086022563 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5dae7640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc58154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5d2e6640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc580b3960 0x7fdc58154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 -- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 msgr2=0x7fdc58154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc58154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 -- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc580b3960 msgr2=0x7fdc58154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc580b3960 0x7fdc58154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 -- 172.21.15.99:0/3086022563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc581f5b10 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc5e2e8640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7c0 tx=0x7fdc5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.968+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50015020 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.969+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc50004230 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.969+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc580b6690 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.969+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50007620 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.969+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc580cdbe0 con 0x7fdc58163580 2024-09-10T11:34:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.969+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5806cd20 con 0x7fdc58163580 2024-09-10T11:34:18.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.970+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc50018020 con 0x7fdc58163580 2024-09-10T11:34:18.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.970+0000 7fdc4e7fc640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 0x7fdc3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:18.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.971+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7fdc5008ac40 con 0x7fdc58163580 2024-09-10T11:34:18.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.971+0000 7fdc5dae7640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 0x7fdc3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:18.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.971+0000 7fdc5dae7640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 0x7fdc3c06e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc54033000 tx=0x7fdc54009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:18.975+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc5001f020 con 0x7fdc58163580 2024-09-10T11:34:19.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.140+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc580b89a0 con 0x7fdc58163580 2024-09-10T11:34:19.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.141+0000 7fdc4e7fc640 1 -- 172.21.15.99:0/3086022563 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fdc500599b0 con 0x7fdc58163580 2024-09-10T11:34:19.143 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 msgr2=0x7fdc3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 0x7fdc3c06e2f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdc54033000 tx=0x7fdc54009000 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 msgr2=0x7fdc581f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c7c0 tx=0x7fdc5000cc80 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 shutdown_connections 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.142+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdc3c06be30 0x7fdc3c06e2f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc58163580 0x7fdc581f4bd0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc580b3960 0x7fdc58154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 --2- 172.21.15.99:0/3086022563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580b3160 0x7fdc58154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 >> 172.21.15.99:0/3086022563 conn(0x7fdc580b4570 msgr2=0x7fdc580b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 shutdown_connections 2024-09-10T11:34:19.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.143+0000 7fdc5fd4b640 1 -- 172.21.15.99:0/3086022563 wait complete. 2024-09-10T11:34:19.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-09-10T11:34:19.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.252+0000 7f6c23f9f640 1 Processor -- start 2024-09-10T11:34:19.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.252+0000 7f6c23f9f640 1 -- start start 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c23f9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c1c1605c0 0x7f6c1c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c23f9f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 0x7f6c1c160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c23f9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1c0d1bd0 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c23f9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c075620 con 0x7f6c1c1605c0 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c23f9f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6c1c0752b0 con 0x7f6c1c0ae020 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c1c1605c0 0x7f6c1c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c1c1605c0 0x7f6c1c1609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35172/0 (socket says 172.21.15.99:35172) 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 -- 172.21.15.99:0/308276154 learned_addr learned my addr 172.21.15.99:0/308276154 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c2153a640 1 --2- 172.21.15.99:0/308276154 >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 0x7f6c1c160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 -- 172.21.15.99:0/308276154 >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 msgr2=0x7f6c1c160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:19.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 --2- 172.21.15.99:0/308276154 >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 0x7f6c1c160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 -- 172.21.15.99:0/308276154 >> v1:172.21.15.196:6789/0 conn(0x7f6c1c15cfd0 legacy=0x7f6c1c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c21d3b640 1 -- 172.21.15.99:0/308276154 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c0d1eb0 con 0x7f6c1c1605c0 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.253+0000 7f6c2153a640 1 --2- 172.21.15.99:0/308276154 >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 0x7f6c1c160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.254+0000 7f6c21d3b640 1 --2- 172.21.15.99:0/308276154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c1609a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0721a0 tx=0x7f6c1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84450f0212a6afcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.254+0000 7f6c20d39640 1 -- 172.21.15.99:0/308276154 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18037070 con 0x7f6c1c1605c0 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.254+0000 7f6c20d39640 1 -- 172.21.15.99:0/308276154 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c18032440 con 0x7f6c1c1605c0 2024-09-10T11:34:19.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.254+0000 7f6c20d39640 1 -- 172.21.15.99:0/308276154 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18032810 con 0x7f6c1c1605c0 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 -- 172.21.15.99:0/308276154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1605c0 msgr2=0x7f6c1c1609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/308276154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c1609a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0721a0 tx=0x7f6c1802a230 comp rx=0 tx=0).stop 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 -- 172.21.15.99:0/308276154 shutdown_connections 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/308276154 >> v2:172.21.15.196:3300/0 conn(0x7f6c1c0ae020 0x7f6c1c160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/308276154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c1609a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 -- 172.21.15.99:0/308276154 >> 172.21.15.99:0/308276154 conn(0x7f6c1c0730c0 msgr2=0x7f6c1c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 -- 172.21.15.99:0/308276154 shutdown_connections 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.255+0000 7f6c23f9f640 1 -- 172.21.15.99:0/308276154 wait complete. 2024-09-10T11:34:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.256+0000 7f6c23f9f640 1 Processor -- start 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.256+0000 7f6c23f9f640 1 -- start start 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c1c0ae020 0x7f6c1c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c073d00 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c15a9e0 con 0x7f6c1c1605c0 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1c073e80 con 0x7f6c1c0ae020 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35186/0 (socket says 172.21.15.99:35186) 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 -- 172.21.15.99:0/1911666474 learned_addr learned my addr 172.21.15.99:0/1911666474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c21d3b640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c1c0ae020 0x7f6c1c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2253c640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 -- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c1c0ae020 msgr2=0x7f6c1c15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c1c0ae020 0x7f6c1c15e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 -- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1c1605c0 msgr2=0x7f6c1c0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c0cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 -- 172.21.15.99:0/1911666474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c1f87d0 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c2153a640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c7f0 tx=0x7f6c0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c015070 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c0c007820 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c007bd0 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c1f8ab0 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.257+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c0b1520 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.258+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c1609a0 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.259+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c0c018070 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.259+0000 7f6c127fc640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 0x7f6c0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.260+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f6c0c08ac20 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.260+0000 7f6c21d3b640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 0x7f6c0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.260+0000 7f6c21d3b640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 0x7f6c0006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6c18034000 tx=0x7f6c1802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.264+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c0c059280 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.493+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7f6c1c0d0e00 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.560+0000 7f6c127fc640 1 -- 172.21.15.99:0/1911666474 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v207) ==== 140+0+0 (secure 0 0 0) 0x7f6c0c05d580 con 0x7f6c1c15cfd0 2024-09-10T11:34:19.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 msgr2=0x7f6c0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 0x7f6c0006e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6c18034000 tx=0x7f6c1802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:19.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 msgr2=0x7f6c1c0cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00c7f0 tx=0x7f6c0c00ccb0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 shutdown_connections 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c0006be30 0x7f6c0006e2f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1c1605c0 0x7f6c1c0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c1c15cfd0 0x7f6c1c0cd3a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 --2- 172.21.15.99:0/1911666474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c1c0ae020 0x7f6c1c15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 >> 172.21.15.99:0/1911666474 conn(0x7f6c1c0730c0 msgr2=0x7f6c1c15fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 shutdown_connections 2024-09-10T11:34:19.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.562+0000 7f6c23f9f640 1 -- 172.21.15.99:0/1911666474 wait complete. 2024-09-10T11:34:19.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-09-10T11:34:19.685 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_objectstore":"memstore"} 2024-09-10T11:34:19.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-09-10T11:34:19.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:34:19.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.784+0000 7f210c9b9640 1 Processor -- start 2024-09-10T11:34:19.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.784+0000 7f210c9b9640 1 -- start start 2024-09-10T11:34:19.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210c9b9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2108146920 0x7f21080718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210c9b9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2108161270 0x7f2108160660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210c9b9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2108071e30 con 0x7f2108160ba0 2024-09-10T11:34:19.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210c9b9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f21080d0580 con 0x7f2108161270 2024-09-10T11:34:19.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210c9b9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f210806f2c0 con 0x7f2108146920 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2108161270 0x7f2108160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f2106d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2108160ba0 0x7f210806d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59290/0 (socket says 172.21.15.99:59290) 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2108161270 0x7f2108160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35190/0 (socket says 172.21.15.99:35190) 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f2106d9d640 1 -- 172.21.15.99:0/2482157452 learned_addr learned my addr 172.21.15.99:0/2482157452 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f210659c640 1 --2- 172.21.15.99:0/2482157452 >> v2:172.21.15.196:3300/0 conn(0x7f2108146920 0x7f21080718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 -- 172.21.15.99:0/2482157452 >> v2:172.21.15.196:3300/0 conn(0x7f2108146920 msgr2=0x7f21080718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 --2- 172.21.15.99:0/2482157452 >> v2:172.21.15.196:3300/0 conn(0x7f2108146920 0x7f21080718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 -- 172.21.15.99:0/2482157452 >> v1:172.21.15.99:6789/0 conn(0x7f2108160ba0 legacy=0x7f210806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f20fffff640 1 -- 172.21.15.99:0/2482157452 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210806f5a0 con 0x7f2108161270 2024-09-10T11:34:19.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.785+0000 7f2105d9b640 1 -- 172.21.15.99:0/2482157452 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344419614 0 0) 0x7f2108071e30 con 0x7f2108160ba0 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f20fffff640 1 --2- 172.21.15.99:0/2482157452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108161270 0x7f2108160660 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2108050870 tx=0x7f20f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165690f7c1352841 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f2105d9b640 1 -- 172.21.15.99:0/2482157452 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0037070 con 0x7f2108161270 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f2105d9b640 1 -- 172.21.15.99:0/2482157452 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f0032440 con 0x7f2108161270 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f2105d9b640 1 -- 172.21.15.99:0/2482157452 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f002ae60 con 0x7f2108161270 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f210c9b9640 1 -- 172.21.15.99:0/2482157452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108161270 msgr2=0x7f2108160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.786+0000 7f210c9b9640 1 --2- 172.21.15.99:0/2482157452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108161270 0x7f2108160660 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2108050870 tx=0x7f20f002a110 comp rx=0 tx=0).stop 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 -- 172.21.15.99:0/2482157452 shutdown_connections 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 --2- 172.21.15.99:0/2482157452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108161270 0x7f2108160660 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 --2- 172.21.15.99:0/2482157452 >> v2:172.21.15.196:3300/0 conn(0x7f2108146920 0x7f21080718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 -- 172.21.15.99:0/2482157452 >> 172.21.15.99:0/2482157452 conn(0x7f21080b2a80 msgr2=0x7f21080b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 -- 172.21.15.99:0/2482157452 shutdown_connections 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.787+0000 7f210c9b9640 1 -- 172.21.15.99:0/2482157452 wait complete. 2024-09-10T11:34:19.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.788+0000 7f210c9b9640 1 Processor -- start 2024-09-10T11:34:19.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.788+0000 7f210c9b9640 1 -- start start 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 0x7f210809e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 0x7f21080b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21080add60 con 0x7f2108146920 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f21080cf050 con 0x7f2108161270 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f21080adee0 con 0x7f2108160ba0 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35206/0 (socket says 172.21.15.99:35206) 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 -- 172.21.15.99:0/359870096 learned_addr learned my addr 172.21.15.99:0/359870096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f2106d9d640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 0x7f21080b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f20fffff640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 0x7f210809e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 -- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 msgr2=0x7f210809e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 0x7f210809e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 -- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 msgr2=0x7f21080b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 0x7f21080b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 -- 172.21.15.99:0/359870096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21080af330 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f20fffff640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 0x7f210809e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f2106d9d640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 0x7f21080b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210659c640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d5f0 tx=0x7f20f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80125f0 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f8019070 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21080af610 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80129d0 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.789+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21080afbd0 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.790+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2108160660 con 0x7f2108146920 2024-09-10T11:34:19.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.791+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20f8012b70 con 0x7f2108146920 2024-09-10T11:34:19.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.791+0000 7f20feffd640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 0x7f20e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:19.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.791+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f20f808af70 con 0x7f2108146920 2024-09-10T11:34:19.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.791+0000 7f20fffff640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 0x7f20e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:19.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.792+0000 7f20fffff640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 0x7f20e806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f20f0034000 tx=0x7f20f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:19.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.796+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f20f8059c20 con 0x7f2108146920 2024-09-10T11:34:19.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.960+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f21080d0710 con 0x7f2108146920 2024-09-10T11:34:19.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.964+0000 7f20feffd640 1 -- 172.21.15.99:0/359870096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f20f8059520 con 0x7f2108146920 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 msgr2=0x7f20e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 0x7f20e806e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f20f0034000 tx=0x7f20f002e000 comp rx=0 tx=0).stop 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 msgr2=0x7f210815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d5f0 tx=0x7f20f800dab0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 shutdown_connections 2024-09-10T11:34:19.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f20e806be30 0x7f20e806e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2108161270 0x7f21080b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2108160ba0 0x7f210809e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 --2- 172.21.15.99:0/359870096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2108146920 0x7f210815f5c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 >> 172.21.15.99:0/359870096 conn(0x7f21080b2a80 msgr2=0x7f21080718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 shutdown_connections 2024-09-10T11:34:19.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:19.966+0000 7f210c9b9640 1 -- 172.21.15.99:0/359870096 wait complete. 2024-09-10T11:34:19.979 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:34:19.979 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:20.027 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:20.070 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:34:20.070 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:20.070 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:20.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:20.086 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:20.086 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:20.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:20.128 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:20.128 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:20.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:20.143 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:20.144 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:20.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:20.186 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:34:20.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.282+0000 7f70cf823640 1 Processor -- start 2024-09-10T11:34:20.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.282+0000 7f70cf823640 1 -- start start 2024-09-10T11:34:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cf823640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cf823640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70c80b3160 0x7f70c806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cf823640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70c80b3620 con 0x7f70c80b3960 2024-09-10T11:34:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cf823640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f70c806d960 con 0x7f70c80b3160 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cf823640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70c806dc40 con 0x7f70c8163580 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cd5bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cddc0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70c80b3160 0x7f70c806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cd5bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:42666/0 (socket says 172.21.15.99:42666) 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cd5bf640 1 -- 172.21.15.99:0/3960859261 learned_addr learned my addr 172.21.15.99:0/3960859261 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cddc0640 1 -- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 msgr2=0x7f70c80b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cddc0640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cddc0640 1 -- 172.21.15.99:0/3960859261 >> v1:172.21.15.196:6789/0 conn(0x7f70c80b3960 legacy=0x7f70c80d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cddc0640 1 -- 172.21.15.99:0/3960859261 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c806df20 con 0x7f70c80b3160 2024-09-10T11:34:20.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.283+0000 7f70cd5bf640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.284+0000 7f70cddc0640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3160 0x7f70c806cd20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009d90 tx=0x7f70c002a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3978e4d97a42405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.284+0000 7f70bffff640 1 -- 172.21.15.99:0/3960859261 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007740 con 0x7f70c80b3160 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.284+0000 7f70bffff640 1 -- 172.21.15.99:0/3960859261 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c00078e0 con 0x7f70c80b3160 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.284+0000 7f70bffff640 1 -- 172.21.15.99:0/3960859261 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007cb0 con 0x7f70c80b3160 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 -- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3160 msgr2=0x7f70c806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3160 0x7f70c806cd20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009d90 tx=0x7f70c002a530 comp rx=0 tx=0).stop 2024-09-10T11:34:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 -- 172.21.15.99:0/3960859261 shutdown_connections 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3160 0x7f70c806cd20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 --2- 172.21.15.99:0/3960859261 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 -- 172.21.15.99:0/3960859261 >> 172.21.15.99:0/3960859261 conn(0x7f70c80b4570 msgr2=0x7f70c80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 -- 172.21.15.99:0/3960859261 shutdown_connections 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.285+0000 7f70cf823640 1 -- 172.21.15.99:0/3960859261 wait complete. 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.286+0000 7f70cf823640 1 Processor -- start 2024-09-10T11:34:20.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.286+0000 7f70cf823640 1 -- start start 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 0x7f70c80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70c806d3e0 con 0x7f70c80b3160 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f70c806d260 con 0x7f70c80b3960 2024-09-10T11:34:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cf823640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70c806d560 con 0x7f70c8163580 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35214/0 (socket says 172.21.15.99:35214) 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 -- 172.21.15.99:0/3743077691 learned_addr learned my addr 172.21.15.99:0/3743077691 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cddc0640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70ccdbe640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 0x7f70c80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 -- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 msgr2=0x7f70c80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 -- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 msgr2=0x7f70c80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 0x7f70c80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 -- 172.21.15.99:0/3743077691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c80cc9f0 con 0x7f70c80b3160 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cddc0640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70ccdbe640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 0x7f70c80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.287+0000 7f70cd5bf640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f70c400ea00 tx=0x7f70c400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.288+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4017070 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.288+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c400ca60 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.288+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c80cadd0 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.288+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c400ce10 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.288+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c80b7df0 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.289+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c806cd20 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.289+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70c401a070 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.290+0000 7f70bdffb640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 0x7f70ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.290+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f70c408aa40 con 0x7f70c80b3160 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.290+0000 7f70ccdbe640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 0x7f70ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.290+0000 7f70ccdbe640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 0x7f70ac06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70c8154da0 tx=0x7f70b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.294+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70c4059780 con 0x7f70c80b3160 2024-09-10T11:34:20.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.493+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70c80b89a0 con 0x7f70c80b3160 2024-09-10T11:34:20.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.494+0000 7f70bdffb640 1 -- 172.21.15.99:0/3743077691 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+3917 (secure 0 0 0) 0x7f70c408ae20 con 0x7f70c80b3160 2024-09-10T11:34:20.496 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:20.496 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":37,"btime":"2024-09-10T11:34:15:547285+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5330,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3334158148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3334158148},{"type":"v1","addr":"172.21.15.196:6835","nonce":3334158148}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5659,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3128302974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3128302974},{"type":"v1","addr":"172.21.15.196:6834","nonce":3128302974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5904,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3542245114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3542245114},{"type":"v1","addr":"172.21.15.99:6836","nonce":3542245114}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4292215559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4292215559},{"type":"v1","addr":"172.21.15.99:6837","nonce":4292215559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[]} 2024-09-10T11:34:20.496 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 37 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 msgr2=0x7f70ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 0x7f70ac06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70c8154da0 tx=0x7f70b8009040 comp rx=0 tx=0).stop 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 msgr2=0x7f70c8154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f70c400ea00 tx=0x7f70c400eec0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 shutdown_connections 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f70ac06be30 0x7f70ac06e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70c8163580 0x7f70c80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70c80b3960 0x7f70c80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 --2- 172.21.15.99:0/3743077691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c80b3160 0x7f70c8154860 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 >> 172.21.15.99:0/3743077691 conn(0x7f70c80b4570 msgr2=0x7f70c815f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 shutdown_connections 2024-09-10T11:34:20.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.496+0000 7f70cf823640 1 -- 172.21.15.99:0/3743077691 wait complete. 2024-09-10T11:34:20.509 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:34:20.509 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:34:20.509 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:34:20.509 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:20.510 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:34:20.510 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:34:20.510 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:34:20.510 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:20.510 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:34:20.510 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:34:20.510 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:34:20.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:20.511 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:34:20.511 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:34:20.511 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:34:20.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:20.519 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:34:20.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:34:20.520 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:34:20.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:34:20.522 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:34:20.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:34:20.523 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:34:20.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:34:20.562 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:34:20.562 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:20.563 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:34:20.563 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:20.565 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:34:20.565 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:20.566 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:34:20.566 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:20.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:34:20.637 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:34:20.638 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:34:20.646 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:34:20.646 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:34:20.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.668+0000 7fe0c60f1640 1 Processor -- start 2024-09-10T11:34:20.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.668+0000 7fe0c60f1640 1 -- start start 2024-09-10T11:34:20.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.668+0000 7fe0c60f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0c00d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0c60f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe0c00b62a0 0x7fe0c006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0c60f1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c0145740 con 0x7fe0c00ce270 2024-09-10T11:34:20.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0c60f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c006e5a0 con 0x7fe0c00b62a0 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0beffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe0c00ce270 0x7fe0c00b5b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59320/0 (socket says 172.21.15.99:59320) 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0beffd640 1 -- 172.21.15.99:0/2211029393 learned_addr learned my addr 172.21.15.99:0/2211029393 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c006e880 con 0x7fe0c0145360 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0bffff640 1 --2- 172.21.15.99:0/2211029393 >> v2:172.21.15.196:3300/0 conn(0x7fe0c00b62a0 0x7fe0c006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0c00d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705486751 0 0) 0x7fe0c0145740 con 0x7fe0c00ce270 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0a8003520 con 0x7fe0c00ce270 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2125419844 0 0) 0x7fe0a8003520 con 0x7fe0c00ce270 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 msgr2=0x7fe0c00d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0c00d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 >> v2:172.21.15.196:3300/0 conn(0x7fe0c00b62a0 msgr2=0x7fe0c006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:20.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 --2- 172.21.15.99:0/2211029393 >> v2:172.21.15.196:3300/0 conn(0x7fe0c00b62a0 0x7fe0c006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c006eb60 con 0x7fe0c00ce270 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fe0b0001670 con 0x7fe0c00ce270 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 >> v1:172.21.15.99:6789/0 conn(0x7fe0c00ce270 legacy=0x7fe0c00b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0a80039b0 0x7fe0a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c006e5a0 con 0x7fe0a8004800 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c006e880 con 0x7fe0a80039b0 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0a800dc00 con 0x7fe0c0145360 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0a80039b0 0x7fe0a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.669+0000 7fe0beffd640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bffff640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 -- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 msgr2=0x7fe0a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0c0145360 0x7fe0a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 -- 172.21.15.99:0/2211029393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0a8004800 msgr2=0x7fe0a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0a8004800 0x7fe0a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 -- 172.21.15.99:0/2211029393 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a8003520 con 0x7fe0a80039b0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.670+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0a80039b0 0x7fe0a8003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009ac0 tx=0x7fe0a8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc87a019e64b5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4036070 con 0x7fe0a80039b0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b4031400 con 0x7fe0a80039b0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0be7fc640 1 -- 172.21.15.99:0/2211029393 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4009690 con 0x7fe0a80039b0 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0a80039b0 msgr2=0x7fe0a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/2211029393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0a80039b0 0x7fe0a8003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009ac0 tx=0x7fe0a8005830 comp rx=0 tx=0).stop 2024-09-10T11:34:20.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0bffff640 1 -- 172.21.15.99:0/2211029393 reap_dead start 2024-09-10T11:34:20.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 shutdown_connections 2024-09-10T11:34:20.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 >> 172.21.15.99:0/2211029393 conn(0x7fe0c0072680 msgr2=0x7fe0c0072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:20.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 shutdown_connections 2024-09-10T11:34:20.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.671+0000 7fe0c60f1640 1 -- 172.21.15.99:0/2211029393 wait complete. 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.672+0000 7fe0c60f1640 1 Processor -- start 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 -- start start 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0a80039b0 0x7fe0c00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c00c7a60 0x7fe0c01f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c00ce830 con 0x7fe0c01f53a0 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c00ce6b0 con 0x7fe0c00c7a60 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0c60f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c00d10f0 con 0x7fe0a80039b0 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0bffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35246/0 (socket says 172.21.15.99:35246) 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0bffff640 1 -- 172.21.15.99:0/3619102217 learned_addr learned my addr 172.21.15.99:0/3619102217 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:20.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.673+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0a80039b0 0x7fe0c00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0beffd640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c00c7a60 0x7fe0c01f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 -- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0a80039b0 msgr2=0x7fe0c00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0a80039b0 0x7fe0c00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 -- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c00c7a60 msgr2=0x7fe0c01f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c00c7a60 0x7fe0c01f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 -- 172.21.15.99:0/3619102217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c00c6d20 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0bffff640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00b16c0 tx=0x7fe0b800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8015430 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b801d070 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c00c7000 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8015810 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.674+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c015db10 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.675+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c015dd10 con 0x7fe0c01f53a0 2024-09-10T11:34:20.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.676+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0b80159b0 con 0x7fe0c01f53a0 2024-09-10T11:34:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.676+0000 7fe0a7fff640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 0x7fe0ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.676+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7fe0b808b320 con 0x7fe0c01f53a0 2024-09-10T11:34:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.676+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 0x7fe0ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.677+0000 7fe0bf7fe640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 0x7fe0ac06e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4033000 tx=0x7fe0b4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:20.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.680+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe0c015dd10 con 0x7fe0c01f53a0 2024-09-10T11:34:20.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:20.857+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe0c00c71d0 con 0x7fe0c01f53a0 2024-09-10T11:34:21.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.570+0000 7fe0a7fff640 1 -- 172.21.15.99:0/3619102217 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7fe0c00c71d0 con 0x7fe0c01f53a0 2024-09-10T11:34:21.572 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 msgr2=0x7fe0ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 0x7fe0ac06e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4033000 tx=0x7fe0b4008030 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 msgr2=0x7fe0c01f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00b16c0 tx=0x7fe0b800f580 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 shutdown_connections 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe0ac06be30 0x7fe0ac06e2f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0c01f53a0 0x7fe0c01f8120 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0c00c7a60 0x7fe0c01f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 --2- 172.21.15.99:0/3619102217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0a80039b0 0x7fe0c00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 >> 172.21.15.99:0/3619102217 conn(0x7fe0c0072680 msgr2=0x7fe0c006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:21.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 shutdown_connections 2024-09-10T11:34:21.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.573+0000 7fe0c60f1640 1 -- 172.21.15.99:0/3619102217 wait complete. 2024-09-10T11:34:21.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:34:21.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.687+0000 7f670db91640 1 Processor -- start 2024-09-10T11:34:21.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.687+0000 7f670db91640 1 -- start start 2024-09-10T11:34:21.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f670db91640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f670815eb80 0x7f67080d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f670db91640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f67081626c0 0x7f67080d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f670db91640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670815ff70 con 0x7f67080d4e40 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f670db91640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f67080d1620 con 0x7f67081626c0 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f670db91640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6708162bc0 con 0x7f670815eb80 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f66fffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f67081626c0 0x7f67080d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f6707fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f67080d4e40 0x7f67080d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59326/0 (socket says 172.21.15.99:59326) 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f6707fff640 1 -- 172.21.15.99:0/2863117517 learned_addr learned my addr 172.21.15.99:0/2863117517 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f66fffff640 1 -- 172.21.15.99:0/2863117517 >> v2:172.21.15.196:3300/0 conn(0x7f670815eb80 msgr2=0x7f67080d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f66fffff640 1 --2- 172.21.15.99:0/2863117517 >> v2:172.21.15.196:3300/0 conn(0x7f670815eb80 0x7f67080d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f66fffff640 1 -- 172.21.15.99:0/2863117517 >> v1:172.21.15.99:6789/0 conn(0x7f67080d4e40 legacy=0x7f67080d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f66fffff640 1 -- 172.21.15.99:0/2863117517 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6708162ea0 con 0x7f67081626c0 2024-09-10T11:34:21.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.688+0000 7f6706ffd640 1 -- 172.21.15.99:0/2863117517 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 934363569 0 0) 0x7f670815ff70 con 0x7f67080d4e40 2024-09-10T11:34:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f66fffff640 1 --2- 172.21.15.99:0/2863117517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080d4900 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66f00097a0 tx=0x7f66f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24e8d831d22ab23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f6706ffd640 1 -- 172.21.15.99:0/2863117517 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0037070 con 0x7f67081626c0 2024-09-10T11:34:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f6706ffd640 1 -- 172.21.15.99:0/2863117517 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f002ae50 con 0x7f67081626c0 2024-09-10T11:34:21.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f6706ffd640 1 -- 172.21.15.99:0/2863117517 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0032660 con 0x7f67081626c0 2024-09-10T11:34:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 -- 172.21.15.99:0/2863117517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 msgr2=0x7f67080d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 --2- 172.21.15.99:0/2863117517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080d4900 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f66f00097a0 tx=0x7f66f002a140 comp rx=0 tx=0).stop 2024-09-10T11:34:21.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 -- 172.21.15.99:0/2863117517 shutdown_connections 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 --2- 172.21.15.99:0/2863117517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080d4900 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 --2- 172.21.15.99:0/2863117517 >> v2:172.21.15.196:3300/0 conn(0x7f670815eb80 0x7f67080d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.689+0000 7f670db91640 1 -- 172.21.15.99:0/2863117517 >> 172.21.15.99:0/2863117517 conn(0x7f670808e680 msgr2=0x7f670809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.690+0000 7f670db91640 1 -- 172.21.15.99:0/2863117517 shutdown_connections 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.690+0000 7f670db91640 1 -- 172.21.15.99:0/2863117517 wait complete. 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 Processor -- start 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 -- start start 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67080d4e40 0x7f670815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670815eb80 0x7f6708159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67080d5290 con 0x7f67081626c0 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6708163460 con 0x7f670815eb80 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.691+0000 7f670db91640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f67080d2ef0 con 0x7f67080d4e40 2024-09-10T11:34:21.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35264/0 (socket says 172.21.15.99:35264) 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 -- 172.21.15.99:0/1392025872 learned_addr learned my addr 172.21.15.99:0/1392025872 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f67077fe640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67080d4e40 0x7f670815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f66fffff640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670815eb80 0x7f6708159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 -- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67080d4e40 msgr2=0x7f670815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67080d4e40 0x7f670815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 -- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670815eb80 msgr2=0x7f6708159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670815eb80 0x7f6708159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 -- 172.21.15.99:0/1392025872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67080cbd90 con 0x7f67081626c0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6707fff640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f670000c730 tx=0x7f670000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700018070 con 0x7f67081626c0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670000ce40 con 0x7f67081626c0 2024-09-10T11:34:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67080b8830 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670000a860 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.692+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67080b8df0 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.693+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67080d29b0 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.694+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f670001b070 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.694+0000 7f6704ff9640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 0x7f66e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.694+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f670008a000 con 0x7f67081626c0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.694+0000 7f67077fe640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 0x7f66e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.695+0000 7f67077fe640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 0x7f66e806e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67080b6d30 tx=0x7f66f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.698+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6700058d90 con 0x7f67081626c0 2024-09-10T11:34:21.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.925+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f67080b5240 con 0x7f67081626c0 2024-09-10T11:34:21.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.925+0000 7f6704ff9640 1 -- 172.21.15.99:0/1392025872 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f6700058690 con 0x7f67081626c0 2024-09-10T11:34:21.927 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:21.927 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:34:21.931 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:34:21.933 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:34:21.935 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:34:21.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:34:21.937 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:34:21.938 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.939 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:21.941 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 msgr2=0x7f66e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 0x7f66e806e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f67080b6d30 tx=0x7f66f8008040 comp rx=0 tx=0).stop 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 msgr2=0x7f67080ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f670000c730 tx=0x7f670000cbf0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 shutdown_connections 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66e806be30 0x7f66e806e2f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67081626c0 0x7f67080ce300 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f670815eb80 0x7f6708159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 --2- 172.21.15.99:0/1392025872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67080d4e40 0x7f670815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 >> 172.21.15.99:0/1392025872 conn(0x7f670808e680 msgr2=0x7f67080d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 shutdown_connections 2024-09-10T11:34:21.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:21.927+0000 7f670db91640 1 -- 172.21.15.99:0/1392025872 wait complete. 2024-09-10T11:34:21.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:34:22.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 Processor -- start 2024-09-10T11:34:22.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 -- start start 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff9880d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff98815ff70 con 0x7ff98815eb80 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9880d1620 con 0x7ff9881626c0 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.041+0000 7ff98d6e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff9880d30c0 con 0x7ff9880d5010 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff986ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff98815eb80 0x7ff9880d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59328/0 (socket says 172.21.15.99:59328) 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff986ffd640 1 -- 172.21.15.99:0/339409681 learned_addr learned my addr 172.21.15.99:0/339409681 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1462398218 0 0) 0x7ff98815ff70 con 0x7ff98815eb80 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff974003520 con 0x7ff98815eb80 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff9877fe640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff9880d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1308076090 0 0) 0x7ff974003520 con 0x7ff98815eb80 2024-09-10T11:34:22.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 msgr2=0x7ff9880d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff9880d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 >> v1:172.21.15.196:6789/0 conn(0x7ff9881626c0 legacy=0x7ff9880d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff988162bc0 con 0x7ff98815eb80 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7ff978001670 con 0x7ff98815eb80 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 >> v1:172.21.15.99:6789/0 conn(0x7ff98815eb80 legacy=0x7ff9880d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740039e0 0x7ff974003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff974004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff974004d40 0x7ff9740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9880d30c0 con 0x7ff974004d40 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff97400baf0 con 0x7ff9740039e0 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff97400bcf0 con 0x7ff9880d5010 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff9877fe640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff974004d40 0x7ff9740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff986ffd640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740039e0 0x7ff974003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff9867fc640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff974004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff9877fe640 1 -- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 msgr2=0x7ff974004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.042+0000 7ff9877fe640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9880d5010 0x7ff974004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff9877fe640 1 -- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740039e0 msgr2=0x7ff974003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff9877fe640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740039e0 0x7ff974003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff9877fe640 1 -- 172.21.15.99:0/339409681 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff974003520 con 0x7ff974004d40 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff9877fe640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff974004d40 0x7ff9740051c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff980009f90 tx=0x7ff974005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34b8a12db801589 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980009c10 con 0x7ff974004d40 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980036070 con 0x7ff974004d40 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.043+0000 7ff985ffb640 1 -- 172.21.15.99:0/339409681 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980031810 con 0x7ff974004d40 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 -- 172.21.15.99:0/339409681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff974004d40 msgr2=0x7ff9740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/339409681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff974004d40 0x7ff9740051c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff980009f90 tx=0x7ff974005830 comp rx=0 tx=0).stop 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff9877fe640 1 -- 172.21.15.99:0/339409681 reap_dead start 2024-09-10T11:34:22.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 -- 172.21.15.99:0/339409681 shutdown_connections 2024-09-10T11:34:22.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 -- 172.21.15.99:0/339409681 >> 172.21.15.99:0/339409681 conn(0x7ff98808e680 msgr2=0x7ff98809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:22.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 -- 172.21.15.99:0/339409681 shutdown_connections 2024-09-10T11:34:22.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.044+0000 7ff98d6e6640 1 -- 172.21.15.99:0/339409681 wait complete. 2024-09-10T11:34:22.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.045+0000 7ff98d6e6640 1 Processor -- start 2024-09-10T11:34:22.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.045+0000 7ff98d6e6640 1 -- start start 2024-09-10T11:34:22.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff974004d40 0x7ff9880b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98814e5b0 0x7ff98814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff988163460 con 0x7ff9880ba350 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff988163180 con 0x7ff974004d40 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff98d6e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff9880d0770 con 0x7ff98814e5b0 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35272/0 (socket says 172.21.15.99:35272) 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 -- 172.21.15.99:0/169396080 learned_addr learned my addr 172.21.15.99:0/169396080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9867fc640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98814e5b0 0x7ff98814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff986ffd640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff974004d40 0x7ff9880b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 -- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98814e5b0 msgr2=0x7ff98814b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98814e5b0 0x7ff98814b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 -- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff974004d40 msgr2=0x7ff9880b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff974004d40 0x7ff9880b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.046+0000 7ff9877fe640 1 -- 172.21.15.99:0/169396080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98814ba80 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff9877fe640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff980033040 tx=0x7ff980009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980009c10 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff980045430 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98814b5e0 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff980045820 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.047+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98814bd20 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.048+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9880baba0 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.048+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff980005d10 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.049+0000 7ff97f7fe640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 0x7ff96806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.049+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7ff9800abca0 con 0x7ff9880ba350 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.049+0000 7ff986ffd640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 0x7ff96806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.049+0000 7ff986ffd640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 0x7ff96806e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff978007fd0 tx=0x7ff9780078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.053+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff98002e050 con 0x7ff9880ba350 2024-09-10T11:34:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.222+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff9880d0aa0 con 0x7ff9880ba350 2024-09-10T11:34:22.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.222+0000 7ff97f7fe640 1 -- 172.21.15.99:0/169396080 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7ff98007a350 con 0x7ff9880ba350 2024-09-10T11:34:22.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:22.224 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:34:22.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:22.225 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:34:22.225 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:34:22.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:22.225 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:34:22.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 msgr2=0x7ff96806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 0x7ff96806e2f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff978007fd0 tx=0x7ff9780078f0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 msgr2=0x7ff9880b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff980033040 tx=0x7ff980009750 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 shutdown_connections 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff96806be30 0x7ff96806e2f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9880ba350 0x7ff9880b8500 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff98814e5b0 0x7ff98814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 --2- 172.21.15.99:0/169396080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff974004d40 0x7ff9880b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 >> 172.21.15.99:0/169396080 conn(0x7ff98808e680 msgr2=0x7ff98815d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 shutdown_connections 2024-09-10T11:34:22.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.224+0000 7ff98d6e6640 1 -- 172.21.15.99:0/169396080 wait complete. 2024-09-10T11:34:22.237 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:34:22.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:34:22.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.338+0000 7faef31df640 1 Processor -- start 2024-09-10T11:34:22.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.338+0000 7faef31df640 1 -- start start 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef31df640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faeec0d1200 0x7faeec073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef31df640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 0x7faeec06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef31df640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeec152e60 con 0x7faeec06e5d0 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef31df640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faeec06ef80 con 0x7faeec0d1200 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef31df640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faeec0d15e0 con 0x7faeec15c770 2024-09-10T11:34:22.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faee3fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faeec06e5d0 0x7faeec15c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59332/0 (socket says 172.21.15.99:59332) 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faee3fff640 1 -- 172.21.15.99:0/2899091298 learned_addr learned my addr 172.21.15.99:0/2899091298 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef177c640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 0x7faeec06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef0f7b640 1 --2- 172.21.15.99:0/2899091298 >> v2:172.21.15.196:3300/0 conn(0x7faeec0d1200 0x7faeec073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3172596996 0 0) 0x7faeec152e60 con 0x7faeec06e5d0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faed4003520 con 0x7faeec06e5d0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2399461294 0 0) 0x7faed4003520 con 0x7faeec06e5d0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef0f7b640 1 -- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 msgr2=0x7faeec06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef0f7b640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 0x7faeec06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef0f7b640 1 -- 172.21.15.99:0/2899091298 >> v1:172.21.15.99:6789/0 conn(0x7faeec06e5d0 legacy=0x7faeec15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef0f7b640 1 -- 172.21.15.99:0/2899091298 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec072680 con 0x7faeec0d1200 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.339+0000 7faef177c640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 0x7faeec06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.340+0000 7faef0f7b640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec0d1200 0x7faeec073440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faee8009d90 tx=0x7faee802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3b10b8e0a5481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.340+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc001670 con 0x7faeec0d1200 2024-09-10T11:34:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.340+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee8007540 con 0x7faeec0d1200 2024-09-10T11:34:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.340+0000 7faee37fe640 1 -- 172.21.15.99:0/2899091298 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8007910 con 0x7faeec0d1200 2024-09-10T11:34:22.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 -- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec0d1200 msgr2=0x7faeec073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec0d1200 0x7faeec073440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faee8009d90 tx=0x7faee802a560 comp rx=0 tx=0).stop 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 -- 172.21.15.99:0/2899091298 shutdown_connections 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec15c770 0x7faeec06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 --2- 172.21.15.99:0/2899091298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec0d1200 0x7faeec073440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 -- 172.21.15.99:0/2899091298 >> 172.21.15.99:0/2899091298 conn(0x7faeec06bc10 msgr2=0x7faeec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 -- 172.21.15.99:0/2899091298 shutdown_connections 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.341+0000 7faef31df640 1 -- 172.21.15.99:0/2899091298 wait complete. 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.342+0000 7faef31df640 1 Processor -- start 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.342+0000 7faef31df640 1 -- start start 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec0d1200 0x7faeec13e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec15c770 0x7faeec1e8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faeec072f20 con 0x7faeec06e5d0 2024-09-10T11:34:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faeec072c40 con 0x7faeec15c770 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef31df640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faeec15cd90 con 0x7faeec0d1200 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35284/0 (socket says 172.21.15.99:35284) 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 -- 172.21.15.99:0/19047821 learned_addr learned my addr 172.21.15.99:0/19047821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 -- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec0d1200 msgr2=0x7faeec13e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec0d1200 0x7faeec13e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 -- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec15c770 msgr2=0x7faeec1e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec15c770 0x7faeec1e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.343+0000 7faef0f7b640 1 -- 172.21.15.99:0/19047821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeec13efe0 con 0x7faeec06e5d0 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faef0f7b640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faee8009d90 tx=0x7faee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee803f070 con 0x7faeec06e5d0 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee8031c40 con 0x7faeec06e5d0 2024-09-10T11:34:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeec13f250 con 0x7faeec06e5d0 2024-09-10T11:34:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8009450 con 0x7faeec06e5d0 2024-09-10T11:34:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.344+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeec13f810 con 0x7faeec06e5d0 2024-09-10T11:34:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.345+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeec073440 con 0x7faeec06e5d0 2024-09-10T11:34:22.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.345+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faee803b410 con 0x7faeec06e5d0 2024-09-10T11:34:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.346+0000 7faee17fa640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 0x7faed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.346+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7faee80ac2b0 con 0x7faeec06e5d0 2024-09-10T11:34:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.346+0000 7faee3fff640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 0x7faed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.346+0000 7faee3fff640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 0x7faed806e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faeec1e55c0 tx=0x7faedc001180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.350+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faee807afb0 con 0x7faeec06e5d0 2024-09-10T11:34:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.513+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faeec0b4640 con 0x7faeec06e5d0 2024-09-10T11:34:22.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.586+0000 7faee17fa640 1 -- 172.21.15.99:0/19047821 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7faee80ac0d0 con 0x7faeec06e5d0 2024-09-10T11:34:22.588 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 msgr2=0x7faed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 0x7faed806e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faeec1e55c0 tx=0x7faedc001180 comp rx=0 tx=0).stop 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 msgr2=0x7faeec1e5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7faee8009d90 tx=0x7faee8004a40 comp rx=0 tx=0).stop 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 shutdown_connections 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faed806be30 0x7faed806e2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faeec15c770 0x7faeec1e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faeec0d1200 0x7faeec13e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 --2- 172.21.15.99:0/19047821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faeec06e5d0 0x7faeec1e5820 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.589+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 >> 172.21.15.99:0/19047821 conn(0x7faeec06bc10 msgr2=0x7faeec15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.590+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 shutdown_connections 2024-09-10T11:34:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.590+0000 7faef31df640 1 -- 172.21.15.99:0/19047821 wait complete. 2024-09-10T11:34:22.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:34:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.708+0000 7f4d12a53640 1 Processor -- start 2024-09-10T11:34:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.708+0000 7f4d12a53640 1 -- start start 2024-09-10T11:34:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d12a53640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d0c15b2d0 0x7f4d0c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d12a53640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c0b0020 con 0x7f4d0c0afbb0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d12a53640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d0c06ee10 con 0x7f4d0c15aef0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d12a53640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4d0c0cd9a0 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d10ff0640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4d0c0afbb0 0x7f4d0c0714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54160/0 (socket says 172.21.15.99:54160) 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d10ff0640 1 -- 172.21.15.99:0/1773525983 learned_addr learned my addr 172.21.15.99:0/1773525983 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d0b7fe640 1 --2- 172.21.15.99:0/1773525983 >> v2:172.21.15.196:3300/0 conn(0x7f4d0c15b2d0 0x7f4d0c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438889317 0 0) 0x7f4d0c0b0020 con 0x7f4d0c0afbb0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.709+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cf4003440 con 0x7f4d0c0afbb0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.710+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2810608985 0 0) 0x7f4cf4003440 con 0x7f4d0c0afbb0 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.710+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/1773525983 >> v1:172.21.15.196:6789/0 conn(0x7f4d0c15aef0 legacy=0x7f4d0c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.710+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/1773525983 >> v1:172.21.15.99:6789/0 conn(0x7f4d0c0afbb0 legacy=0x7f4d0c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.710+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/1773525983 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c0b0290 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.710+0000 7f4d0b7fe640 1 --2- 172.21.15.99:0/1773525983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c15b2d0 0x7f4d0c0b4510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097d0 tx=0x7f4d0002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beb4d7842a91ea9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00037070 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0002ac90 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d0affd640 1 -- 172.21.15.99:0/1773525983 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d040019f0 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 -- 172.21.15.99:0/1773525983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c15b2d0 msgr2=0x7f4d0c0b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 --2- 172.21.15.99:0/1773525983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c15b2d0 0x7f4d0c0b4510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d000097d0 tx=0x7f4d0002a200 comp rx=0 tx=0).stop 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 -- 172.21.15.99:0/1773525983 shutdown_connections 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 --2- 172.21.15.99:0/1773525983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c15b2d0 0x7f4d0c0b4510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 -- 172.21.15.99:0/1773525983 >> 172.21.15.99:0/1773525983 conn(0x7f4d0c06bc10 msgr2=0x7f4d0c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 -- 172.21.15.99:0/1773525983 shutdown_connections 2024-09-10T11:34:22.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.711+0000 7f4d12a53640 1 -- 172.21.15.99:0/1773525983 wait complete. 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.712+0000 7f4d12a53640 1 Processor -- start 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 -- start start 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 0x7f4d0c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d0c15b2d0 0x7f4d0c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c06eb30 con 0x7f4d0c15aef0 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0c06e850 con 0x7f4d0c0afbb0 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.713+0000 7f4d12a53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0c071bb0 con 0x7f4d0c15b2d0 2024-09-10T11:34:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34370/0 (socket says 172.21.15.99:34370) 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/633008126 learned_addr learned my addr 172.21.15.99:0/633008126 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0bfff640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 0x7f4d0c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d10ff0640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d0c15b2d0 0x7f4d0c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d0c15b2d0 msgr2=0x7f4d0c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d0c15b2d0 0x7f4d0c15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 msgr2=0x7f4d0c150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 0x7f4d0c150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 -- 172.21.15.99:0/633008126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d0c1509a0 con 0x7f4d0c15aef0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0bfff640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 0x7f4d0c150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d0b7fe640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002a240 tx=0x7f4d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00041070 con 0x7f4d0c15aef0 2024-09-10T11:34:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0003c670 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d0c0ceb60 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00037040 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.714+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d0c0cf120 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.715+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d0c0b4510 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.716+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d00044070 con 0x7f4d0c15aef0 2024-09-10T11:34:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.716+0000 7f4d08ff9640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 0x7f4cf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.716+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7f4d000ab5a0 con 0x7f4d0c15aef0 2024-09-10T11:34:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.716+0000 7f4d0bfff640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 0x7f4cf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.717+0000 7f4d0bfff640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 0x7f4cf806e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c1556a0 tx=0x7f4cfc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.721+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4d0c1509a0 con 0x7f4d0c15aef0 2024-09-10T11:34:22.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:22.885+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4d0c155990 con 0x7f4d0c15aef0 2024-09-10T11:34:23.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.592+0000 7f4d08ff9640 1 -- 172.21.15.99:0/633008126 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7f4d0007a130 con 0x7f4d0c15aef0 2024-09-10T11:34:23.593 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:34:23.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.596+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 msgr2=0x7f4cf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.596+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 0x7f4cf806e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c1556a0 tx=0x7f4cfc009040 comp rx=0 tx=0).stop 2024-09-10T11:34:23.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.596+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 msgr2=0x7f4d0c0cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.596+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4d0002a240 tx=0x7f4d000046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 shutdown_connections 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4cf806be30 0x7f4cf806e2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d0c15b2d0 0x7f4d0c15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d0c15aef0 0x7f4d0c0cfa00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 --2- 172.21.15.99:0/633008126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d0c0afbb0 0x7f4d0c150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.597+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 >> 172.21.15.99:0/633008126 conn(0x7f4d0c06bc10 msgr2=0x7f4d0c159890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:23.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.601+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 shutdown_connections 2024-09-10T11:34:23.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.602+0000 7f4d12a53640 1 -- 172.21.15.99:0/633008126 wait complete. 2024-09-10T11:34:23.620 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:34:23.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.724+0000 7f912b8e4640 1 Processor -- start 2024-09-10T11:34:23.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.724+0000 7f912b8e4640 1 -- start start 2024-09-10T11:34:23.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f912b8e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f912406bc10 0x7f9124163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f912b8e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f912406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f912b8e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912406c530 con 0x7f91240b4930 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f912b8e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f91240b5420 con 0x7f912406bc10 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f912b8e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f91240b5140 con 0x7f9124160030 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f9129e81640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f91240b4930 0x7f912406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54162/0 (socket says 172.21.15.99:54162) 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f9129e81640 1 -- 172.21.15.99:0/4043038492 learned_addr learned my addr 172.21.15.99:0/4043038492 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f9129680640 1 --2- 172.21.15.99:0/4043038492 >> v2:172.21.15.196:3300/0 conn(0x7f912406bc10 0x7f9124163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f9128e7f640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f912406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817703419 0 0) 0x7f912406c530 con 0x7f91240b4930 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.725+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9110003520 con 0x7f91240b4930 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2856692722 0 0) 0x7f9110003520 con 0x7f91240b4930 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 msgr2=0x7f912406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:23.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f912406bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 >> v2:172.21.15.196:3300/0 conn(0x7f912406bc10 msgr2=0x7f9124163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 --2- 172.21.15.99:0/4043038492 >> v2:172.21.15.196:3300/0 conn(0x7f912406bc10 0x7f9124163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124162b40 con 0x7f91240b4930 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f911c0019f0 con 0x7f91240b4930 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 >> v1:172.21.15.99:6789/0 conn(0x7f91240b4930 legacy=0x7f912406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91100039b0 0x7f9110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9110004800 0x7f9110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f91100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91240b5420 con 0x7f9110004800 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f91240b5140 con 0x7f91100039b0 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f911000b9c0 con 0x7f9124160030 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9110004800 0x7f9110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9129680640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91100039b0 0x7f9110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9129e81640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f91100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 -- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 msgr2=0x7f91100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9124160030 0x7f91100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 -- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91100039b0 msgr2=0x7f9110003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91100039b0 0x7f9110003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.726+0000 7f9128e7f640 1 -- 172.21.15.99:0/4043038492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9110003520 con 0x7f9110004800 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f9128e7f640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9110004800 0x7f9110004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9114009af0 tx=0x7f9110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f58c7f0ddfecc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114036070 con 0x7f9110004800 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9114031400 con 0x7f9110004800 2024-09-10T11:34:23.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f911bfff640 1 -- 172.21.15.99:0/4043038492 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114009690 con 0x7f9110004800 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f912b8e4640 1 -- 172.21.15.99:0/4043038492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9110004800 msgr2=0x7f9110004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f912b8e4640 1 --2- 172.21.15.99:0/4043038492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9110004800 0x7f9110004c80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9114009af0 tx=0x7f9110005830 comp rx=0 tx=0).stop 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f9129e81640 1 -- 172.21.15.99:0/4043038492 reap_dead start 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f912b8e4640 1 -- 172.21.15.99:0/4043038492 shutdown_connections 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.727+0000 7f912b8e4640 1 -- 172.21.15.99:0/4043038492 >> 172.21.15.99:0/4043038492 conn(0x7f91240b3630 msgr2=0x7f91240b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.728+0000 7f912b8e4640 1 -- 172.21.15.99:0/4043038492 shutdown_connections 2024-09-10T11:34:23.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.728+0000 7f912b8e4640 1 -- 172.21.15.99:0/4043038492 wait complete. 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 Processor -- start 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 -- start start 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9110004800 0x7f912415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9124140e10 0x7f91241f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f91240b4d90 con 0x7f9124141360 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9124163100 con 0x7f9124140e10 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.729+0000 7f912b8e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9124143a40 con 0x7f9110004800 2024-09-10T11:34:23.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34384/0 (socket says 172.21.15.99:34384) 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 -- 172.21.15.99:0/2277409507 learned_addr learned my addr 172.21.15.99:0/2277409507 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9129680640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9110004800 0x7f912415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9129e81640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9124140e10 0x7f91241f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 -- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9110004800 msgr2=0x7f912415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9110004800 0x7f912415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 -- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9124140e10 msgr2=0x7f91241f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9124140e10 0x7f91241f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 -- 172.21.15.99:0/2277409507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9124142f20 con 0x7f9124141360 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9128e7f640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9114009af0 tx=0x7f9114004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114041020 con 0x7f9124141360 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911403c470 con 0x7f9124141360 2024-09-10T11:34:23.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912413e700 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114036040 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.730+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9124143480 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.731+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91240d1560 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.732+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9114044020 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.732+0000 7f9119ffb640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 0x7f910806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.732+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7f91140aac80 con 0x7f9124141360 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.732+0000 7f9129680640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 0x7f910806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.733+0000 7f9129680640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 0x7f910806e320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9120009b00 tx=0x7f9120008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:23.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.737+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9124142f20 con 0x7f9124141360 2024-09-10T11:34:23.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:23.936+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9124073d40 con 0x7f9124141360 2024-09-10T11:34:24.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.593+0000 7f9119ffb640 1 -- 172.21.15.99:0/2277409507 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v40) ==== 174+0+0 (secure 0 0 0) 0x7f9114079720 con 0x7f9124141360 2024-09-10T11:34:24.594 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.595+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 msgr2=0x7f910806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.595+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 0x7f910806e320 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9120009b00 tx=0x7f9120008040 comp rx=0 tx=0).stop 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.595+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 msgr2=0x7f912413ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.595+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9114009af0 tx=0x7f9114004730 comp rx=0 tx=0).stop 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 shutdown_connections 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f910806be60 0x7f910806e320 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9124140e10 0x7f91241f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9124141360 0x7f912413ef70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 --2- 172.21.15.99:0/2277409507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9110004800 0x7f912415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 >> 172.21.15.99:0/2277409507 conn(0x7f91240b3630 msgr2=0x7f9124072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:24.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 shutdown_connections 2024-09-10T11:34:24.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.596+0000 7f912b8e4640 1 -- 172.21.15.99:0/2277409507 wait complete. 2024-09-10T11:34:24.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:24.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.708+0000 7fe84b627640 1 Processor -- start 2024-09-10T11:34:24.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.708+0000 7fe84b627640 1 -- start start 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe84b627640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe844163620 0x7fe844160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe84b627640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8440b5140 con 0x7fe84406bc10 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe84b627640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe844162b40 con 0x7fe8440b4930 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe84b627640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe8440b5420 con 0x7fe844163620 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe8493c3640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe84406bc10 0x7fe84406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54170/0 (socket says 172.21.15.99:54170) 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe848bc2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe844163620 0x7fe844160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe8493c3640 1 -- 172.21.15.99:0/15528783 learned_addr learned my addr 172.21.15.99:0/15528783 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:24.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe848bc2640 1 -- 172.21.15.99:0/15528783 >> v1:172.21.15.196:6789/0 conn(0x7fe8440b4930 legacy=0x7fe84406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:24.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe848bc2640 1 -- 172.21.15.99:0/15528783 >> v1:172.21.15.99:6789/0 conn(0x7fe84406bc10 legacy=0x7fe84406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.709+0000 7fe848bc2640 1 -- 172.21.15.99:0/15528783 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe844162e20 con 0x7fe844163620 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.710+0000 7fe848bc2640 1 --2- 172.21.15.99:0/15528783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844160030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe8340097a0 tx=0x7fe83402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc1c89da3aff992 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.710+0000 7fe83bfff640 1 -- 172.21.15.99:0/15528783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834037070 con 0x7fe844163620 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.710+0000 7fe83bfff640 1 -- 172.21.15.99:0/15528783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83402ae50 con 0x7fe844163620 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.710+0000 7fe83bfff640 1 -- 172.21.15.99:0/15528783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834032660 con 0x7fe844163620 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 -- 172.21.15.99:0/15528783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 msgr2=0x7fe844160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 --2- 172.21.15.99:0/15528783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844160030 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe8340097a0 tx=0x7fe83402a140 comp rx=0 tx=0).stop 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 -- 172.21.15.99:0/15528783 shutdown_connections 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 --2- 172.21.15.99:0/15528783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844160030 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 -- 172.21.15.99:0/15528783 >> 172.21.15.99:0/15528783 conn(0x7fe8440b3630 msgr2=0x7fe8440b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 -- 172.21.15.99:0/15528783 shutdown_connections 2024-09-10T11:34:24.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.711+0000 7fe84b627640 1 -- 172.21.15.99:0/15528783 wait complete. 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.712+0000 7fe84b627640 1 Processor -- start 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.712+0000 7fe84b627640 1 -- start start 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 0x7fe84406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8440b4930 0x7fe844158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8440d1560 con 0x7fe844163620 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8440b4dc0 con 0x7fe84406bc10 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe84b627640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe8440d16e0 con 0x7fe8440b4930 2024-09-10T11:34:24.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34394/0 (socket says 172.21.15.99:34394) 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 -- 172.21.15.99:0/2800286775 learned_addr learned my addr 172.21.15.99:0/2800286775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe8493c3640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 0x7fe84406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe848bc2640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8440b4930 0x7fe844158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 -- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8440b4930 msgr2=0x7fe844158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8440b4930 0x7fe844158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 -- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 msgr2=0x7fe84406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 0x7fe84406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 -- 172.21.15.99:0/2800286775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84406ebb0 con 0x7fe844163620 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe8493c3640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 0x7fe84406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe849bc4640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00b990 tx=0x7fe83c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c019070 con 0x7fe844163620 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.713+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe83c00cba0 con 0x7fe844163620 2024-09-10T11:34:24.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.714+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c014440 con 0x7fe844163620 2024-09-10T11:34:24.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.714+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84406ee90 con 0x7fe844163620 2024-09-10T11:34:24.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.714+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84406f450 con 0x7fe844163620 2024-09-10T11:34:24.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.715+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe844160030 con 0x7fe844163620 2024-09-10T11:34:24.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.715+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe83c01c070 con 0x7fe844163620 2024-09-10T11:34:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.715+0000 7fe839ffb640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 0x7fe82806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.716+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe83c08ab20 con 0x7fe844163620 2024-09-10T11:34:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.716+0000 7fe8493c3640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 0x7fe82806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:24.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.716+0000 7fe8493c3640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 0x7fe82806e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe840000f60 tx=0x7fe840009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:24.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.720+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe83c059560 con 0x7fe844163620 2024-09-10T11:34:24.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.886+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe844073d40 con 0x7fe844163620 2024-09-10T11:34:24.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.887+0000 7fe839ffb640 1 -- 172.21.15.99:0/2800286775 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16756 (secure 0 0 0) 0x7fe83c058e60 con 0x7fe844163620 2024-09-10T11:34:24.888 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:24.889 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":102,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:23.937142+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:34:22.533186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T11:34:22.904618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.889+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 msgr2=0x7fe82806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.889+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 0x7fe82806e320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe840000f60 tx=0x7fe840009040 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.889+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 msgr2=0x7fe844142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.889+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00b990 tx=0x7fe83c00be50 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 shutdown_connections 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe82806be60 0x7fe82806e320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe844163620 0x7fe844142bf0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8440b4930 0x7fe844158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 --2- 172.21.15.99:0/2800286775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe84406bc10 0x7fe84406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 >> 172.21.15.99:0/2800286775 conn(0x7fe8440b3630 msgr2=0x7fe8441482b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 shutdown_connections 2024-09-10T11:34:24.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.890+0000 7fe84b627640 1 -- 172.21.15.99:0/2800286775 wait complete. 2024-09-10T11:34:24.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:24.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.998+0000 7fe17e75f640 1 Processor -- start 2024-09-10T11:34:25.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.998+0000 7fe17e75f640 1 -- start start 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe178160030 0x7fe1780b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d1760 0x7fe1780cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1780d0130 0x7fe1780cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780ce430 con 0x7fe178160030 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780ce710 con 0x7fe1780d0130 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe17e75f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780d1240 con 0x7fe1780d1760 2024-09-10T11:34:25.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe177fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe178160030 0x7fe1780b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:24.999+0000 7fe177fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe178160030 0x7fe1780b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34396/0 (socket says 172.21.15.99:34396) 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 -- 172.21.15.99:0/47959709 learned_addr learned my addr 172.21.15.99:0/47959709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 -- 172.21.15.99:0/47959709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d1760 msgr2=0x7fe1780cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 --2- 172.21.15.99:0/47959709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d1760 0x7fe1780cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 -- 172.21.15.99:0/47959709 >> v2:172.21.15.196:3300/0 conn(0x7fe1780d0130 msgr2=0x7fe1780cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe17ccfc640 1 --2- 172.21.15.99:0/47959709 >> v2:172.21.15.196:3300/0 conn(0x7fe1780d0130 0x7fe1780cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 --2- 172.21.15.99:0/47959709 >> v2:172.21.15.196:3300/0 conn(0x7fe1780d0130 0x7fe1780cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 -- 172.21.15.99:0/47959709 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780b3e30 con 0x7fe178160030 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.000+0000 7fe177fff640 1 --2- 172.21.15.99:0/47959709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe1780b38f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe1780735d0 tx=0x7fe16802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f63a24d6ef2ff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe176ffd640 1 -- 172.21.15.99:0/47959709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168037070 con 0x7fe178160030 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe176ffd640 1 -- 172.21.15.99:0/47959709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168032440 con 0x7fe178160030 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe176ffd640 1 -- 172.21.15.99:0/47959709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168032810 con 0x7fe178160030 2024-09-10T11:34:25.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 -- 172.21.15.99:0/47959709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 msgr2=0x7fe1780b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 --2- 172.21.15.99:0/47959709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe1780b38f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe1780735d0 tx=0x7fe16802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 -- 172.21.15.99:0/47959709 shutdown_connections 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 --2- 172.21.15.99:0/47959709 >> v2:172.21.15.196:3300/0 conn(0x7fe1780d0130 0x7fe1780cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 --2- 172.21.15.99:0/47959709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d1760 0x7fe1780cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 --2- 172.21.15.99:0/47959709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe1780b38f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 -- 172.21.15.99:0/47959709 >> 172.21.15.99:0/47959709 conn(0x7fe1780b4650 msgr2=0x7fe1780b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 -- 172.21.15.99:0/47959709 shutdown_connections 2024-09-10T11:34:25.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.001+0000 7fe17e75f640 1 -- 172.21.15.99:0/47959709 wait complete. 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.002+0000 7fe17e75f640 1 Processor -- start 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 -- start start 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d0130 0x7fe1780c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 0x7fe1780ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe178160590 con 0x7fe178160030 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe178160410 con 0x7fe1780d1760 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17e75f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe178160710 con 0x7fe1780d0130 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17ccfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.003+0000 7fe17ccfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34412/0 (socket says 172.21.15.99:34412) 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 -- 172.21.15.99:0/3946415895 learned_addr learned my addr 172.21.15.99:0/3946415895 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe177fff640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d0130 0x7fe1780c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe1777fe640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 0x7fe1780ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 -- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d0130 msgr2=0x7fe1780c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d0130 0x7fe1780c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 -- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 msgr2=0x7fe1780ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 0x7fe1780ca650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 -- 172.21.15.99:0/3946415895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1781543b0 con 0x7fe178160030 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe1777fe640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 0x7fe1780ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:25.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17ccfc640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cc60 tx=0x7fe17000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170003990 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170003b30 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1780c9b90 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170004020 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.004+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1780c9800 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.005+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1780b38f0 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.006+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1700041c0 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.006+0000 7fe174ff9640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 0x7fe16406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.006+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fe17008b1c0 con 0x7fe178160030 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.006+0000 7fe177fff640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 0x7fe16406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.006+0000 7fe177fff640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 0x7fe16406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.010+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe170014030 con 0x7fe178160030 2024-09-10T11:34:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.176+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe178070220 con 0x7fe178160030 2024-09-10T11:34:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.177+0000 7fe174ff9640 1 -- 172.21.15.99:0/3946415895 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16756 (secure 0 0 0) 0x7fe1700595b0 con 0x7fe178160030 2024-09-10T11:34:25.178 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:25.178 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":102,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:23.937142+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:34:22.533186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T11:34:22.904618+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 msgr2=0x7fe16406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 0x7fe16406e2f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe168034000 tx=0x7fe16802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 msgr2=0x7fe178153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe17000cc60 tx=0x7fe17000a5c0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 shutdown_connections 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16406be30 0x7fe16406e2f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe178160030 0x7fe178153e70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1780d1760 0x7fe1780ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 --2- 172.21.15.99:0/3946415895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1780d0130 0x7fe1780c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 >> 172.21.15.99:0/3946415895 conn(0x7fe1780b4650 msgr2=0x7fe17806f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 shutdown_connections 2024-09-10T11:34:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.179+0000 7fe17e75f640 1 -- 172.21.15.99:0/3946415895 wait complete. 2024-09-10T11:34:25.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:34:25.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.291+0000 7fc23299f640 1 Processor -- start 2024-09-10T11:34:25.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.291+0000 7fc23299f640 1 -- start start 2024-09-10T11:34:25.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23299f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc22c06cd20 0x7fc22c06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23299f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c163660 con 0x7fc22c0cc2a0 2024-09-10T11:34:25.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23299f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22c0d1c30 con 0x7fc22c15ed20 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23299f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc22c06d680 con 0x7fc22c06cd20 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23119c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc22c06cd20 0x7fc22c06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23219e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc22c0cc2a0 0x7fc22c162910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54180/0 (socket says 172.21.15.99:54180) 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23219e640 1 -- 172.21.15.99:0/1907077039 learned_addr learned my addr 172.21.15.99:0/1907077039 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23119c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc22c06cd20 0x7fc22c06d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34418/0 (socket says 172.21.15.99:34418) 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23119c640 1 -- 172.21.15.99:0/1907077039 >> v1:172.21.15.196:6789/0 conn(0x7fc22c15ed20 legacy=0x7fc22c0d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23119c640 1 -- 172.21.15.99:0/1907077039 >> v1:172.21.15.99:6789/0 conn(0x7fc22c0cc2a0 legacy=0x7fc22c162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.292+0000 7fc23119c640 1 -- 172.21.15.99:0/1907077039 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0d1f10 con 0x7fc22c06cd20 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.293+0000 7fc23119c640 1 --2- 172.21.15.99:0/1907077039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c06cd20 0x7fc22c06d140 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0097a0 tx=0x7fc21c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9f70f57ce4f889 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.293+0000 7fc23099b640 1 -- 172.21.15.99:0/1907077039 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c037070 con 0x7fc22c06cd20 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.293+0000 7fc23099b640 1 -- 172.21.15.99:0/1907077039 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc21c032440 con 0x7fc22c06cd20 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.293+0000 7fc23099b640 1 -- 172.21.15.99:0/1907077039 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c032810 con 0x7fc22c06cd20 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 -- 172.21.15.99:0/1907077039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c06cd20 msgr2=0x7fc22c06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 --2- 172.21.15.99:0/1907077039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c06cd20 0x7fc22c06d140 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0097a0 tx=0x7fc21c02a230 comp rx=0 tx=0).stop 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 -- 172.21.15.99:0/1907077039 shutdown_connections 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 --2- 172.21.15.99:0/1907077039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c06cd20 0x7fc22c06d140 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 -- 172.21.15.99:0/1907077039 >> 172.21.15.99:0/1907077039 conn(0x7fc22c09ba30 msgr2=0x7fc22c09be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:25.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 -- 172.21.15.99:0/1907077039 shutdown_connections 2024-09-10T11:34:25.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.294+0000 7fc23299f640 1 -- 172.21.15.99:0/1907077039 wait complete. 2024-09-10T11:34:25.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.295+0000 7fc23299f640 1 Processor -- start 2024-09-10T11:34:25.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.295+0000 7fc23299f640 1 -- start start 2024-09-10T11:34:25.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc22c06cd20 0x7fc22c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 0x7fc22c159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c0d3100 con 0x7fc22c15ed20 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c163020 con 0x7fc22c06cd20 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c0b7f30 con 0x7fc22c0cc2a0 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34428/0 (socket says 172.21.15.99:34428) 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 -- 172.21.15.99:0/2036440976 learned_addr learned my addr 172.21.15.99:0/2036440976 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23119c640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 0x7fc22c159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23199d640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc22c06cd20 0x7fc22c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 -- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 msgr2=0x7fc22c159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 0x7fc22c159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 -- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc22c06cd20 msgr2=0x7fc22c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc22c06cd20 0x7fc22c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 -- 172.21.15.99:0/2036440976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0b8740 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23119c640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 0x7fc22c159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23219e640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc22400cbc0 tx=0x7fc22400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224017070 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc22400ad40 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc224012600 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.296+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c0b6340 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.297+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c0b8460 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.297+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c06d140 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.298+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc22401a070 con 0x7fc22c15ed20 2024-09-10T11:34:25.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.298+0000 7fc2227fc640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 0x7fc21006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.298+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fc22408aa60 con 0x7fc22c15ed20 2024-09-10T11:34:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.298+0000 7fc23199d640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 0x7fc21006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.299+0000 7fc23199d640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 0x7fc21006e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc228007fd0 tx=0x7fc228009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.303+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc224058e50 con 0x7fc22c15ed20 2024-09-10T11:34:25.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.503+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc22c0cbe30 con 0x7fc22c15ed20 2024-09-10T11:34:25.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.597+0000 7fc2227fc640 1 -- 172.21.15.99:0/2036440976 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) ==== 118+0+0 (secure 0 0 0) 0x7fc22405d150 con 0x7fc22c15ed20 2024-09-10T11:34:25.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 msgr2=0x7fc21006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 0x7fc21006e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc228007fd0 tx=0x7fc228009040 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 msgr2=0x7fc22c0b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc22400cbc0 tx=0x7fc22400a430 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 shutdown_connections 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc21006be30 0x7fc21006e2f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc22c15ed20 0x7fc22c0b6b00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc22c0cc2a0 0x7fc22c159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 --2- 172.21.15.99:0/2036440976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc22c06cd20 0x7fc22c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 >> 172.21.15.99:0/2036440976 conn(0x7fc22c09ba30 msgr2=0x7fc22c0705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 shutdown_connections 2024-09-10T11:34:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.599+0000 7fc23299f640 1 -- 172.21.15.99:0/2036440976 wait complete. 2024-09-10T11:34:25.611 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:34:25.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:34:25.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.712+0000 7fb251b9f640 1 Processor -- start 2024-09-10T11:34:25.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.712+0000 7fb251b9f640 1 -- start start 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb251b9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb24c0ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb251b9f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c06fc90 con 0x7fb24c159140 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb251b9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c06c530 con 0x7fb24c06bc10 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb251b9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c159520 con 0x7fb24c06c0a0 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb250b9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb24c159140 0x7fb24c15fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54194/0 (socket says 172.21.15.99:54194) 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.713+0000 7fb250b9d640 1 -- 172.21.15.99:0/2247336522 learned_addr learned my addr 172.21.15.99:0/2247336522 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 505486889 0 0) 0x7fb24c06fc90 con 0x7fb24c159140 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb25139e640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb24c0ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb234003520 con 0x7fb24c159140 2024-09-10T11:34:25.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3971671080 0 0) 0x7fb234003520 con 0x7fb24c159140 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 msgr2=0x7fb24c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb24c0ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 >> v1:172.21.15.196:6789/0 conn(0x7fb24c06bc10 legacy=0x7fb24c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c06ff30 con 0x7fb24c159140 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fb248001670 con 0x7fb24c159140 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 >> v1:172.21.15.99:6789/0 conn(0x7fb24c159140 legacy=0x7fb24c15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb234003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb234004800 0x7fb234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb2340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c159520 con 0x7fb234003a50 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb23400ba70 con 0x7fb234004800 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb23400bc40 con 0x7fb24c06c0a0 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb234003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb243fff640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb234004800 0x7fb234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb25139e640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb2340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 -- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 msgr2=0x7fb2340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c06c0a0 0x7fb2340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 -- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb234004800 msgr2=0x7fb234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb234004800 0x7fb234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.714+0000 7fb250b9d640 1 -- 172.21.15.99:0/2247336522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb234003520 con 0x7fb234003a50 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.715+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb234003e30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c290 tx=0x7fb234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1b65dbd10bc47f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.715+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248007d50 con 0x7fb234003a50 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.715+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248005af0 con 0x7fb234003a50 2024-09-10T11:34:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.715+0000 7fb2437fe640 1 -- 172.21.15.99:0/2247336522 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248007070 con 0x7fb234003a50 2024-09-10T11:34:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 -- 172.21.15.99:0/2247336522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 msgr2=0x7fb234003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2247336522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb234003e30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb24800c290 tx=0x7fb234005830 comp rx=0 tx=0).stop 2024-09-10T11:34:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb25139e640 1 -- 172.21.15.99:0/2247336522 reap_dead start 2024-09-10T11:34:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 -- 172.21.15.99:0/2247336522 shutdown_connections 2024-09-10T11:34:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 -- 172.21.15.99:0/2247336522 >> 172.21.15.99:0/2247336522 conn(0x7fb24c0b35c0 msgr2=0x7fb24c0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 -- 172.21.15.99:0/2247336522 shutdown_connections 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.716+0000 7fb251b9f640 1 -- 172.21.15.99:0/2247336522 wait complete. 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.717+0000 7fb251b9f640 1 Processor -- start 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.717+0000 7fb251b9f640 1 -- start start 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c1e7320 0x7fb24c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c1596a0 0x7fb24c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c0732c0 con 0x7fb234003a50 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c072fe0 con 0x7fb24c1e7320 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb251b9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c0735a0 con 0x7fb24c1596a0 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34444/0 (socket says 172.21.15.99:34444) 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 -- 172.21.15.99:0/2594162115 learned_addr learned my addr 172.21.15.99:0/2594162115 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb243fff640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c1e7320 0x7fb24c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb25139e640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c1596a0 0x7fb24c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 -- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c1596a0 msgr2=0x7fb24c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c1596a0 0x7fb24c0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 -- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c1e7320 msgr2=0x7fb24c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c1e7320 0x7fb24c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 -- 172.21.15.99:0/2594162115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c145fe0 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.718+0000 7fb250b9d640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb2480017e0 tx=0x7fb248001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.719+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248038040 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.719+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb248042070 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.719+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c146560 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.719+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248007d50 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.719+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c146b20 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.720+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c0b1a60 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.720+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb248031070 con 0x7fb234003a50 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.720+0000 7fb2417fa640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 0x7fb23806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.721+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fb2480aaa20 con 0x7fb234003a50 2024-09-10T11:34:25.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.721+0000 7fb243fff640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 0x7fb23806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:25.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.721+0000 7fb243fff640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 0x7fb23806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb24c14ca20 tx=0x7fb23c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:25.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.725+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb248078e10 con 0x7fb234003a50 2024-09-10T11:34:25.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:25.921+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb24c0b1780 con 0x7fb234003a50 2024-09-10T11:34:26.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.598+0000 7fb2417fa640 1 -- 172.21.15.99:0/2594162115 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7fb24807d110 con 0x7fb234003a50 2024-09-10T11:34:26.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 msgr2=0x7fb23806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 0x7fb23806e320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb24c14ca20 tx=0x7fb23c009040 comp rx=0 tx=0).stop 2024-09-10T11:34:26.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 msgr2=0x7fb24c145aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb2480017e0 tx=0x7fb248001a40 comp rx=0 tx=0).stop 2024-09-10T11:34:26.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 shutdown_connections 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb23806be60 0x7fb23806e320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb24c1596a0 0x7fb24c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb24c1e7320 0x7fb24c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 --2- 172.21.15.99:0/2594162115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb234003a50 0x7fb24c145aa0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 >> 172.21.15.99:0/2594162115 conn(0x7fb24c0b35c0 msgr2=0x7fb24c0b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 shutdown_connections 2024-09-10T11:34:26.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.601+0000 7fb251b9f640 1 -- 172.21.15.99:0/2594162115 wait complete. 2024-09-10T11:34:26.615 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:34:26.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:26.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.714+0000 7f2eca398640 1 Processor -- start 2024-09-10T11:34:26.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 -- start start 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec415eb80 0x7f2ec40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ec41626c0 0x7f2ec40d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec40d4e40 0x7f2ec40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec40d52d0 con 0x7f2ec40d4e40 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec40d1620 con 0x7f2ec41626c0 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2eca398640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec4162bc0 con 0x7f2ec415eb80 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2ec8935640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec40d4e40 0x7f2ec40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2ec8935640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec40d4e40 0x7f2ec40d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34456/0 (socket says 172.21.15.99:34456) 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.715+0000 7f2ec8935640 1 -- 172.21.15.99:0/4140919667 learned_addr learned my addr 172.21.15.99:0/4140919667 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 -- 172.21.15.99:0/4140919667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec415eb80 msgr2=0x7f2ec40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 --2- 172.21.15.99:0/4140919667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec415eb80 0x7f2ec40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 -- 172.21.15.99:0/4140919667 >> v2:172.21.15.196:3300/0 conn(0x7f2ec41626c0 msgr2=0x7f2ec40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:26.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 --2- 172.21.15.99:0/4140919667 >> v2:172.21.15.196:3300/0 conn(0x7f2ec41626c0 0x7f2ec40d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 -- 172.21.15.99:0/4140919667 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec4162ea0 con 0x7f2ec40d4e40 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.716+0000 7f2ec8935640 1 --2- 172.21.15.99:0/4140919667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec40d4e40 0x7f2ec40d1b20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2ec408db70 tx=0x7f2ebc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c5a881585db87e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2ec2ffd640 1 -- 172.21.15.99:0/4140919667 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc036070 con 0x7f2ec40d4e40 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2ec2ffd640 1 -- 172.21.15.99:0/4140919667 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ebc031600 con 0x7f2ec40d4e40 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2ec2ffd640 1 -- 172.21.15.99:0/4140919667 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0319d0 con 0x7f2ec40d4e40 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 -- 172.21.15.99:0/4140919667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec40d4e40 msgr2=0x7f2ec40d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 --2- 172.21.15.99:0/4140919667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec40d4e40 0x7f2ec40d1b20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2ec408db70 tx=0x7f2ebc02a610 comp rx=0 tx=0).stop 2024-09-10T11:34:26.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 -- 172.21.15.99:0/4140919667 shutdown_connections 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 --2- 172.21.15.99:0/4140919667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec40d4e40 0x7f2ec40d1b20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 --2- 172.21.15.99:0/4140919667 >> v2:172.21.15.196:3300/0 conn(0x7f2ec41626c0 0x7f2ec40d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 --2- 172.21.15.99:0/4140919667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec415eb80 0x7f2ec40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 -- 172.21.15.99:0/4140919667 >> 172.21.15.99:0/4140919667 conn(0x7f2ec408e680 msgr2=0x7f2ec409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 -- 172.21.15.99:0/4140919667 shutdown_connections 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.717+0000 7f2eca398640 1 -- 172.21.15.99:0/4140919667 wait complete. 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.718+0000 7f2eca398640 1 Processor -- start 2024-09-10T11:34:26.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 -- start start 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec40d4e40 0x7f2ec41e84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec41626c0 0x7f2ec40b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec40d2ef0 con 0x7f2ec415eb80 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec4163460 con 0x7f2ec41626c0 2024-09-10T11:34:26.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2eca398640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec40d3070 con 0x7f2ec40d4e40 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2ec37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2ec37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34462/0 (socket says 172.21.15.99:34462) 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2ec37fe640 1 -- 172.21.15.99:0/4033076179 learned_addr learned my addr 172.21.15.99:0/4033076179 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2ec8935640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec41626c0 0x7f2ec40b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.719+0000 7f2ec3fff640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec40d4e40 0x7f2ec41e84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 -- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec40d4e40 msgr2=0x7f2ec41e84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec40d4e40 0x7f2ec41e84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 -- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec41626c0 msgr2=0x7f2ec40b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec41626c0 0x7f2ec40b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 -- 172.21.15.99:0/4033076179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec40cc940 con 0x7f2ec415eb80 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec37fe640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800b410 tx=0x7f2eb800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8015020 con 0x7f2ec415eb80 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb8011850 con 0x7f2ec415eb80 2024-09-10T11:34:26.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec40ccc20 con 0x7f2ec415eb80 2024-09-10T11:34:26.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8011c40 con 0x7f2ec415eb80 2024-09-10T11:34:26.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.720+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec40cd1e0 con 0x7f2ec415eb80 2024-09-10T11:34:26.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.721+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec40cd3e0 con 0x7f2ec415eb80 2024-09-10T11:34:26.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.722+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb8018020 con 0x7f2ec415eb80 2024-09-10T11:34:26.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.722+0000 7f2ec0ff9640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 0x7f2eb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:26.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.722+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f2eb808ae30 con 0x7f2ec415eb80 2024-09-10T11:34:26.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.722+0000 7f2ec3fff640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 0x7f2eb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:26.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.722+0000 7f2ec3fff640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 0x7f2eb006e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b810 tx=0x7f2eb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:26.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.726+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ec40cd3e0 con 0x7f2ec415eb80 2024-09-10T11:34:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.930+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ec40b71d0 con 0x7f2ec415eb80 2024-09-10T11:34:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.931+0000 7f2ec0ff9640 1 -- 172.21.15.99:0/4033076179 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7f2ec40b71d0 con 0x7f2ec415eb80 2024-09-10T11:34:26.932 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:26.932 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 msgr2=0x7f2eb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 0x7f2eb006e320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b810 tx=0x7f2eb4008040 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 msgr2=0x7f2ec40ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800b410 tx=0x7f2eb800b8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 shutdown_connections 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2eb006be60 0x7f2eb006e320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec41626c0 0x7f2ec40b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec415eb80 0x7f2ec40ce720 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 --2- 172.21.15.99:0/4033076179 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec40d4e40 0x7f2ec41e84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:26.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 >> 172.21.15.99:0/4033076179 conn(0x7f2ec408e680 msgr2=0x7f2ec40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:26.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 shutdown_connections 2024-09-10T11:34:26.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:26.933+0000 7f2eca398640 1 -- 172.21.15.99:0/4033076179 wait complete. 2024-09-10T11:34:26.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:27.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.043+0000 7f88ec874640 1 Processor -- start 2024-09-10T11:34:27.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.044+0000 7f88ec874640 1 -- start start 2024-09-10T11:34:27.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.044+0000 7f88ec874640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.044+0000 7f88ec874640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f88e81626c0 0x7f88e80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88ec874640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e80d52d0 con 0x7f88e80d4e40 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88ec874640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f88e815ff70 con 0x7f88e81626c0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88ec874640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f88e80d1620 con 0x7f88e815eb80 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f88e81626c0 0x7f88e80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f88e81626c0 0x7f88e80d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34474/0 (socket says 172.21.15.99:34474) 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 -- 172.21.15.99:0/1066665175 learned_addr learned my addr 172.21.15.99:0/1066665175 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e659c640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 -- 172.21.15.99:0/1066665175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 msgr2=0x7f88e80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 -- 172.21.15.99:0/1066665175 >> v1:172.21.15.196:6789/0 conn(0x7f88e80d4e40 legacy=0x7f88e80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.045+0000 7f88e5d9b640 1 -- 172.21.15.99:0/1066665175 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e8162bc0 con 0x7f88e81626c0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88e5d9b640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e80d4900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009870 tx=0x7f88d002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3249a8e83ae5da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88e559a640 1 -- 172.21.15.99:0/1066665175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0037070 con 0x7f88e81626c0 2024-09-10T11:34:27.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88e559a640 1 -- 172.21.15.99:0/1066665175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d0032440 con 0x7f88e81626c0 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88e559a640 1 -- 172.21.15.99:0/1066665175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0032810 con 0x7f88e81626c0 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88ec874640 1 -- 172.21.15.99:0/1066665175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 msgr2=0x7f88e80d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88ec874640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e80d4900 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009870 tx=0x7f88d002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.046+0000 7f88ec874640 1 -- 172.21.15.99:0/1066665175 shutdown_connections 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.047+0000 7f88ec874640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e80d4900 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.047+0000 7f88ec874640 1 --2- 172.21.15.99:0/1066665175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.047+0000 7f88ec874640 1 -- 172.21.15.99:0/1066665175 >> 172.21.15.99:0/1066665175 conn(0x7f88e808e680 msgr2=0x7f88e809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.047+0000 7f88ec874640 1 -- 172.21.15.99:0/1066665175 shutdown_connections 2024-09-10T11:34:27.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.047+0000 7f88ec874640 1 -- 172.21.15.99:0/1066665175 wait complete. 2024-09-10T11:34:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.048+0000 7f88ec874640 1 Processor -- start 2024-09-10T11:34:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.048+0000 7f88ec874640 1 -- start start 2024-09-10T11:34:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e80d4e40 0x7f88e80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8163460 con 0x7f88e81626c0 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88e8163180 con 0x7f88e80d4e40 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f88e80d2ef0 con 0x7f88e815eb80 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34486/0 (socket says 172.21.15.99:34486) 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 -- 172.21.15.99:0/3643129930 learned_addr learned my addr 172.21.15.99:0/3643129930 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e659c640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e80d4e40 0x7f88e80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e5d9b640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 -- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 msgr2=0x7f88e8159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e8159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 -- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e80d4e40 msgr2=0x7f88e80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e80d4e40 0x7f88e80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 -- 172.21.15.99:0/3643129930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e81e88a0 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88e6d9d640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c6e0 tx=0x7f88e000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0003b70 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88e0003d30 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0005310 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.049+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81eac80 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.050+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81e84e0 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.050+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e80d29b0 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.051+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88e00054b0 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.051+0000 7f88deffd640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 0x7f88c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.051+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f88e008ab60 con 0x7f88e81626c0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.051+0000 7f88e659c640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 0x7f88c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.052+0000 7f88e659c640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 0x7f88c806e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88e80cb940 tx=0x7f88d8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.055+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f88e0058ec0 con 0x7f88e81626c0 2024-09-10T11:34:27.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.222+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88e806e1f0 con 0x7f88e81626c0 2024-09-10T11:34:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.223+0000 7f88deffd640 1 -- 172.21.15.99:0/3643129930 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16743 (secure 0 0 0) 0x7f88e005d1c0 con 0x7f88e81626c0 2024-09-10T11:34:27.224 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:27.224 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":103,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:25.573515+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:34:22.533186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T11:34:22.904618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 msgr2=0x7f88c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 0x7f88c806e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88e80cb940 tx=0x7f88d8009190 comp rx=0 tx=0).stop 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 msgr2=0x7f88e806c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c6e0 tx=0x7f88e000cba0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 shutdown_connections 2024-09-10T11:34:27.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f88c806be30 0x7f88c806e2f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e81626c0 0x7f88e806c3c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88e815eb80 0x7f88e8159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 --2- 172.21.15.99:0/3643129930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88e80d4e40 0x7f88e80b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 >> 172.21.15.99:0/3643129930 conn(0x7f88e808e680 msgr2=0x7f88e80b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 shutdown_connections 2024-09-10T11:34:27.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.225+0000 7f88ec874640 1 -- 172.21.15.99:0/3643129930 wait complete. 2024-09-10T11:34:27.237 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:34:27.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:34:27.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.334+0000 7f86eb25c640 1 Processor -- start 2024-09-10T11:34:27.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.334+0000 7f86eb25c640 1 -- start start 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e415efc0 0x7f86e40d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86e40d3a30 0x7f86e40d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f86e40d15a0 0x7f86e40d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f86e40d2990 con 0x7f86e40d3a30 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f86e40d2c70 con 0x7f86e40d15a0 2024-09-10T11:34:27.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86eb25c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86e40d2f50 con 0x7f86e415efc0 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86e3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86e40d3a30 0x7f86e40d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86e3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86e40d3a30 0x7f86e40d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34488/0 (socket says 172.21.15.99:34488) 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86e3fff640 1 -- 172.21.15.99:0/4135383277 learned_addr learned my addr 172.21.15.99:0/4135383277 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86e97f9640 1 --2- 172.21.15.99:0/4135383277 >> v2:172.21.15.196:3300/0 conn(0x7f86e40d15a0 0x7f86e40d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.335+0000 7f86e8ff8640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e415efc0 0x7f86e40d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 -- 172.21.15.99:0/4135383277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e415efc0 msgr2=0x7f86e40d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e415efc0 0x7f86e40d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 -- 172.21.15.99:0/4135383277 >> v2:172.21.15.196:3300/0 conn(0x7f86e40d15a0 msgr2=0x7f86e40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 --2- 172.21.15.99:0/4135383277 >> v2:172.21.15.196:3300/0 conn(0x7f86e40d15a0 0x7f86e40d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 -- 172.21.15.99:0/4135383277 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e40d3230 con 0x7f86e40d3a30 2024-09-10T11:34:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e3fff640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e40d3a30 0x7f86e40d0930 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d634d941f676ff4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e37fe640 1 -- 172.21.15.99:0/4135383277 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8037070 con 0x7f86e40d3a30 2024-09-10T11:34:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e37fe640 1 -- 172.21.15.99:0/4135383277 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d8032440 con 0x7f86e40d3a30 2024-09-10T11:34:27.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.336+0000 7f86e37fe640 1 -- 172.21.15.99:0/4135383277 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8032810 con 0x7f86e40d3a30 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 -- 172.21.15.99:0/4135383277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e40d3a30 msgr2=0x7f86e40d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e40d3a30 0x7f86e40d0930 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a210 comp rx=0 tx=0).stop 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 -- 172.21.15.99:0/4135383277 shutdown_connections 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 --2- 172.21.15.99:0/4135383277 >> v2:172.21.15.196:3300/0 conn(0x7f86e40d15a0 0x7f86e40d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e40d3a30 0x7f86e40d0930 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 --2- 172.21.15.99:0/4135383277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e415efc0 0x7f86e40d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 -- 172.21.15.99:0/4135383277 >> 172.21.15.99:0/4135383277 conn(0x7f86e40d4c20 msgr2=0x7f86e40d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 -- 172.21.15.99:0/4135383277 shutdown_connections 2024-09-10T11:34:27.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.337+0000 7f86eb25c640 1 -- 172.21.15.99:0/4135383277 wait complete. 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.338+0000 7f86eb25c640 1 Processor -- start 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.338+0000 7f86eb25c640 1 -- start start 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e40d15a0 0x7f86e4154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e40d3a30 0x7f86e40b9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4162d90 con 0x7f86e415efc0 2024-09-10T11:34:27.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4162c10 con 0x7f86e40d3a30 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86eb25c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86e4162f10 con 0x7f86e40d15a0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86e97f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86e8ff8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e40d15a0 0x7f86e4154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86e97f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34504/0 (socket says 172.21.15.99:34504) 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e40d3a30 0x7f86e40b9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.339+0000 7f86e97f9640 1 -- 172.21.15.99:0/292626423 learned_addr learned my addr 172.21.15.99:0/292626423 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 -- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e40d15a0 msgr2=0x7f86e4154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e40d15a0 0x7f86e4154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 -- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e40d3a30 msgr2=0x7f86e40b9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e40d3a30 0x7f86e40b9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 -- 172.21.15.99:0/292626423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e4160a30 con 0x7f86e415efc0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e97f9640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007e60 tx=0x7f86dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc014620 con 0x7f86e415efc0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86dc019070 con 0x7f86e415efc0 2024-09-10T11:34:27.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e4160d10 con 0x7f86e415efc0 2024-09-10T11:34:27.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc014a20 con 0x7f86e415efc0 2024-09-10T11:34:27.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.340+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e41612d0 con 0x7f86e415efc0 2024-09-10T11:34:27.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.341+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e40d1a50 con 0x7f86e415efc0 2024-09-10T11:34:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.341+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86dc014bc0 con 0x7f86e415efc0 2024-09-10T11:34:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.342+0000 7f86e17fa640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 0x7f86d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.342+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f86dc08b4c0 con 0x7f86e415efc0 2024-09-10T11:34:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.342+0000 7f86e8ff8640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 0x7f86d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.342+0000 7f86e8ff8640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 0x7f86d006e320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f86d40079a0 tx=0x7f86d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.346+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86dc0598b0 con 0x7f86e415efc0 2024-09-10T11:34:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.512+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86e41546b0 con 0x7f86d006be60 2024-09-10T11:34:27.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.517+0000 7f86e17fa640 1 -- 172.21.15.99:0/292626423 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247992 (secure 0 0 0) 0x7f86e41546b0 con 0x7f86d006be60 2024-09-10T11:34:27.521 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:27.522 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 msgr2=0x7f86d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 0x7f86d006e320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f86d40079a0 tx=0x7f86d4008040 comp rx=0 tx=0).stop 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 msgr2=0x7f86e41f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f86dc007e60 tx=0x7f86dc00e4b0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 shutdown_connections 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86d006be60 0x7f86d006e320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86e415efc0 0x7f86e41f4a70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86e40d3a30 0x7f86e40b9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 --2- 172.21.15.99:0/292626423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86e40d15a0 0x7f86e4154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 >> 172.21.15.99:0/292626423 conn(0x7f86e40d4c20 msgr2=0x7f86e40d0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 shutdown_connections 2024-09-10T11:34:27.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.520+0000 7f86eb25c640 1 -- 172.21.15.99:0/292626423 wait complete. 2024-09-10T11:34:27.532 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T11:34:26.959540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556987,"statfs":{"total":1600000000,"available":1594355588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.370577"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576788+0000","last_change":"2024-09-10T11:34:24.587358+0000","last_active":"2024-09-10T11:34:25.576788+0000","last_peered":"2024-09-10T11:34:25.576788+0000","last_clean":"2024-09-10T11:34:25.576788+0000","last_became_active":"2024-09-10T11:34:24.587094+0000","last_became_peered":"2024-09-10T11:34:24.587094+0000","last_unstale":"2024-09-10T11:34:25.576788+0000","last_undegraded":"2024-09-10T11:34:25.576788+0000","last_fullsized":"2024-09-10T11:34:25.576788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:27.164698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492230+0000","last_change":"2024-09-10T11:34:24.589433+0000","last_active":"2024-09-10T11:34:26.492230+0000","last_peered":"2024-09-10T11:34:26.492230+0000","last_clean":"2024-09-10T11:34:26.492230+0000","last_became_active":"2024-09-10T11:34:24.588944+0000","last_became_peered":"2024-09-10T11:34:24.588944+0000","last_unstale":"2024-09-10T11:34:26.492230+0000","last_undegraded":"2024-09-10T11:34:26.492230+0000","last_fullsized":"2024-09-10T11:34:26.492230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:50.778882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492025+0000","last_change":"2024-09-10T11:34:24.585835+0000","last_active":"2024-09-10T11:34:26.492025+0000","last_peered":"2024-09-10T11:34:26.492025+0000","last_clean":"2024-09-10T11:34:26.492025+0000","last_became_active":"2024-09-10T11:34:24.585623+0000","last_became_peered":"2024-09-10T11:34:24.585623+0000","last_unstale":"2024-09-10T11:34:26.492025+0000","last_undegraded":"2024-09-10T11:34:26.492025+0000","last_fullsized":"2024-09-10T11:34:26.492025+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:02.607420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212802+0000","last_change":"2024-09-10T11:34:24.587968+0000","last_active":"2024-09-10T11:34:26.212802+0000","last_peered":"2024-09-10T11:34:26.212802+0000","last_clean":"2024-09-10T11:34:26.212802+0000","last_became_active":"2024-09-10T11:34:24.587711+0000","last_became_peered":"2024-09-10T11:34:24.587711+0000","last_unstale":"2024-09-10T11:34:26.212802+0000","last_undegraded":"2024-09-10T11:34:26.212802+0000","last_fullsized":"2024-09-10T11:34:26.212802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:07.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492720+0000","last_change":"2024-09-10T11:34:24.585664+0000","last_active":"2024-09-10T11:34:26.492720+0000","last_peered":"2024-09-10T11:34:26.492720+0000","last_clean":"2024-09-10T11:34:26.492720+0000","last_became_active":"2024-09-10T11:34:24.585426+0000","last_became_peered":"2024-09-10T11:34:24.585426+0000","last_unstale":"2024-09-10T11:34:26.492720+0000","last_undegraded":"2024-09-10T11:34:26.492720+0000","last_fullsized":"2024-09-10T11:34:26.492720+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:53.501159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492357+0000","last_change":"2024-09-10T11:34:24.589286+0000","last_active":"2024-09-10T11:34:26.492357+0000","last_peered":"2024-09-10T11:34:26.492357+0000","last_clean":"2024-09-10T11:34:26.492357+0000","last_became_active":"2024-09-10T11:34:24.589116+0000","last_became_peered":"2024-09-10T11:34:24.589116+0000","last_unstale":"2024-09-10T11:34:26.492357+0000","last_undegraded":"2024-09-10T11:34:26.492357+0000","last_fullsized":"2024-09-10T11:34:26.492357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:20.162620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492449+0000","last_change":"2024-09-10T11:34:24.589240+0000","last_active":"2024-09-10T11:34:26.492449+0000","last_peered":"2024-09-10T11:34:26.492449+0000","last_clean":"2024-09-10T11:34:26.492449+0000","last_became_active":"2024-09-10T11:34:24.587491+0000","last_became_peered":"2024-09-10T11:34:24.587491+0000","last_unstale":"2024-09-10T11:34:26.492449+0000","last_undegraded":"2024-09-10T11:34:26.492449+0000","last_fullsized":"2024-09-10T11:34:26.492449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:14.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576776+0000","last_change":"2024-09-10T11:34:24.577483+0000","last_active":"2024-09-10T11:34:25.576776+0000","last_peered":"2024-09-10T11:34:25.576776+0000","last_clean":"2024-09-10T11:34:25.576776+0000","last_became_active":"2024-09-10T11:34:24.577186+0000","last_became_peered":"2024-09-10T11:34:24.577186+0000","last_unstale":"2024-09-10T11:34:25.576776+0000","last_undegraded":"2024-09-10T11:34:25.576776+0000","last_fullsized":"2024-09-10T11:34:25.576776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:57.540166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576458+0000","last_change":"2024-09-10T11:34:24.587049+0000","last_active":"2024-09-10T11:34:25.576458+0000","last_peered":"2024-09-10T11:34:25.576458+0000","last_clean":"2024-09-10T11:34:25.576458+0000","last_became_active":"2024-09-10T11:34:24.586800+0000","last_became_peered":"2024-09-10T11:34:24.586800+0000","last_unstale":"2024-09-10T11:34:25.576458+0000","last_undegraded":"2024-09-10T11:34:25.576458+0000","last_fullsized":"2024-09-10T11:34:25.576458+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:18.282012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492188+0000","last_change":"2024-09-10T11:34:24.588067+0000","last_active":"2024-09-10T11:34:26.492188+0000","last_peered":"2024-09-10T11:34:26.492188+0000","last_clean":"2024-09-10T11:34:26.492188+0000","last_became_active":"2024-09-10T11:34:24.587714+0000","last_became_peered":"2024-09-10T11:34:24.587714+0000","last_unstale":"2024-09-10T11:34:26.492188+0000","last_undegraded":"2024-09-10T11:34:26.492188+0000","last_fullsized":"2024-09-10T11:34:26.492188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:45.186673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576942+0000","last_change":"2024-09-10T11:34:24.585763+0000","last_active":"2024-09-10T11:34:25.576942+0000","last_peered":"2024-09-10T11:34:25.576942+0000","last_clean":"2024-09-10T11:34:25.576942+0000","last_became_active":"2024-09-10T11:34:24.585122+0000","last_became_peered":"2024-09-10T11:34:24.585122+0000","last_unstale":"2024-09-10T11:34:25.576942+0000","last_undegraded":"2024-09-10T11:34:25.576942+0000","last_fullsized":"2024-09-10T11:34:25.576942+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:43.133411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577118+0000","last_change":"2024-09-10T11:34:24.584319+0000","last_active":"2024-09-10T11:34:25.577118+0000","last_peered":"2024-09-10T11:34:25.577118+0000","last_clean":"2024-09-10T11:34:25.577118+0000","last_became_active":"2024-09-10T11:34:24.584111+0000","last_became_peered":"2024-09-10T11:34:24.584111+0000","last_unstale":"2024-09-10T11:34:25.577118+0000","last_undegraded":"2024-09-10T11:34:25.577118+0000","last_fullsized":"2024-09-10T11:34:25.577118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:34.575572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492606+0000","last_change":"2024-09-10T11:34:24.585155+0000","last_active":"2024-09-10T11:34:26.492606+0000","last_peered":"2024-09-10T11:34:26.492606+0000","last_clean":"2024-09-10T11:34:26.492606+0000","last_became_active":"2024-09-10T11:34:24.584908+0000","last_became_peered":"2024-09-10T11:34:24.584908+0000","last_unstale":"2024-09-10T11:34:26.492606+0000","last_undegraded":"2024-09-10T11:34:26.492606+0000","last_fullsized":"2024-09-10T11:34:26.492606+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:48.369692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213103+0000","last_change":"2024-09-10T11:34:24.590357+0000","last_active":"2024-09-10T11:34:26.213103+0000","last_peered":"2024-09-10T11:34:26.213103+0000","last_clean":"2024-09-10T11:34:26.213103+0000","last_became_active":"2024-09-10T11:34:24.589626+0000","last_became_peered":"2024-09-10T11:34:24.589626+0000","last_unstale":"2024-09-10T11:34:26.213103+0000","last_undegraded":"2024-09-10T11:34:26.213103+0000","last_fullsized":"2024-09-10T11:34:26.213103+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:24.731255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492407+0000","last_change":"2024-09-10T11:34:24.592234+0000","last_active":"2024-09-10T11:34:26.492407+0000","last_peered":"2024-09-10T11:34:26.492407+0000","last_clean":"2024-09-10T11:34:26.492407+0000","last_became_active":"2024-09-10T11:34:24.591974+0000","last_became_peered":"2024-09-10T11:34:24.591974+0000","last_unstale":"2024-09-10T11:34:26.492407+0000","last_undegraded":"2024-09-10T11:34:26.492407+0000","last_fullsized":"2024-09-10T11:34:26.492407+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:46.022677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492638+0000","last_change":"2024-09-10T11:34:24.587956+0000","last_active":"2024-09-10T11:34:26.492638+0000","last_peered":"2024-09-10T11:34:26.492638+0000","last_clean":"2024-09-10T11:34:26.492638+0000","last_became_active":"2024-09-10T11:34:24.587360+0000","last_became_peered":"2024-09-10T11:34:24.587360+0000","last_unstale":"2024-09-10T11:34:26.492638+0000","last_undegraded":"2024-09-10T11:34:26.492638+0000","last_fullsized":"2024-09-10T11:34:26.492638+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:40.069884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577255+0000","last_change":"2024-09-10T11:34:24.583393+0000","last_active":"2024-09-10T11:34:25.577255+0000","last_peered":"2024-09-10T11:34:25.577255+0000","last_clean":"2024-09-10T11:34:25.577255+0000","last_became_active":"2024-09-10T11:34:24.583013+0000","last_became_peered":"2024-09-10T11:34:24.583013+0000","last_unstale":"2024-09-10T11:34:25.577255+0000","last_undegraded":"2024-09-10T11:34:25.577255+0000","last_fullsized":"2024-09-10T11:34:25.577255+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:34.954454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576702+0000","last_change":"2024-09-10T11:34:24.579255+0000","last_active":"2024-09-10T11:34:25.576702+0000","last_peered":"2024-09-10T11:34:25.576702+0000","last_clean":"2024-09-10T11:34:25.576702+0000","last_became_active":"2024-09-10T11:34:24.578863+0000","last_became_peered":"2024-09-10T11:34:24.578863+0000","last_unstale":"2024-09-10T11:34:25.576702+0000","last_undegraded":"2024-09-10T11:34:25.576702+0000","last_fullsized":"2024-09-10T11:34:25.576702+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:26.776397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213278+0000","last_change":"2024-09-10T11:34:24.590638+0000","last_active":"2024-09-10T11:34:26.213278+0000","last_peered":"2024-09-10T11:34:26.213278+0000","last_clean":"2024-09-10T11:34:26.213278+0000","last_became_active":"2024-09-10T11:34:24.590049+0000","last_became_peered":"2024-09-10T11:34:24.590049+0000","last_unstale":"2024-09-10T11:34:26.213278+0000","last_undegraded":"2024-09-10T11:34:26.213278+0000","last_fullsized":"2024-09-10T11:34:26.213278+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:27.422736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213389+0000","last_change":"2024-09-10T11:34:24.592072+0000","last_active":"2024-09-10T11:34:26.213389+0000","last_peered":"2024-09-10T11:34:26.213389+0000","last_clean":"2024-09-10T11:34:26.213389+0000","last_became_active":"2024-09-10T11:34:24.591750+0000","last_became_peered":"2024-09-10T11:34:24.591750+0000","last_unstale":"2024-09-10T11:34:26.213389+0000","last_undegraded":"2024-09-10T11:34:26.213389+0000","last_fullsized":"2024-09-10T11:34:26.213389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:19.702469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212784+0000","last_change":"2024-09-10T11:34:24.588653+0000","last_active":"2024-09-10T11:34:26.212784+0000","last_peered":"2024-09-10T11:34:26.212784+0000","last_clean":"2024-09-10T11:34:26.212784+0000","last_became_active":"2024-09-10T11:34:24.588406+0000","last_became_peered":"2024-09-10T11:34:24.588406+0000","last_unstale":"2024-09-10T11:34:26.212784+0000","last_undegraded":"2024-09-10T11:34:26.212784+0000","last_fullsized":"2024-09-10T11:34:26.212784+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:27.113558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213602+0000","last_change":"2024-09-10T11:34:24.586975+0000","last_active":"2024-09-10T11:34:26.213602+0000","last_peered":"2024-09-10T11:34:26.213602+0000","last_clean":"2024-09-10T11:34:26.213602+0000","last_became_active":"2024-09-10T11:34:24.586721+0000","last_became_peered":"2024-09-10T11:34:24.586721+0000","last_unstale":"2024-09-10T11:34:26.213602+0000","last_undegraded":"2024-09-10T11:34:26.213602+0000","last_fullsized":"2024-09-10T11:34:26.213602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:17.936666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576897+0000","last_change":"2024-09-10T11:34:24.582533+0000","last_active":"2024-09-10T11:34:25.576897+0000","last_peered":"2024-09-10T11:34:25.576897+0000","last_clean":"2024-09-10T11:34:25.576897+0000","last_became_active":"2024-09-10T11:34:24.582309+0000","last_became_peered":"2024-09-10T11:34:24.582309+0000","last_unstale":"2024-09-10T11:34:25.576897+0000","last_undegraded":"2024-09-10T11:34:25.576897+0000","last_fullsized":"2024-09-10T11:34:25.576897+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:15.632768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576543+0000","last_change":"2024-09-10T11:34:24.590536+0000","last_active":"2024-09-10T11:34:25.576543+0000","last_peered":"2024-09-10T11:34:25.576543+0000","last_clean":"2024-09-10T11:34:25.576543+0000","last_became_active":"2024-09-10T11:34:24.590259+0000","last_became_peered":"2024-09-10T11:34:24.590259+0000","last_unstale":"2024-09-10T11:34:25.576543+0000","last_undegraded":"2024-09-10T11:34:25.576543+0000","last_fullsized":"2024-09-10T11:34:25.576543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:19.287456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576585+0000","last_change":"2024-09-10T11:34:24.583161+0000","last_active":"2024-09-10T11:34:25.576585+0000","last_peered":"2024-09-10T11:34:25.576585+0000","last_clean":"2024-09-10T11:34:25.576585+0000","last_became_active":"2024-09-10T11:34:24.582445+0000","last_became_peered":"2024-09-10T11:34:24.582445+0000","last_unstale":"2024-09-10T11:34:25.576585+0000","last_undegraded":"2024-09-10T11:34:25.576585+0000","last_fullsized":"2024-09-10T11:34:25.576585+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:14.951048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492571+0000","last_change":"2024-09-10T11:34:24.588425+0000","last_active":"2024-09-10T11:34:26.492571+0000","last_peered":"2024-09-10T11:34:26.492571+0000","last_clean":"2024-09-10T11:34:26.492571+0000","last_became_active":"2024-09-10T11:34:24.588197+0000","last_became_peered":"2024-09-10T11:34:24.588197+0000","last_unstale":"2024-09-10T11:34:26.492571+0000","last_undegraded":"2024-09-10T11:34:26.492571+0000","last_fullsized":"2024-09-10T11:34:26.492571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:38.979273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492558+0000","last_change":"2024-09-10T11:34:24.585431+0000","last_active":"2024-09-10T11:34:26.492558+0000","last_peered":"2024-09-10T11:34:26.492558+0000","last_clean":"2024-09-10T11:34:26.492558+0000","last_became_active":"2024-09-10T11:34:24.584866+0000","last_became_peered":"2024-09-10T11:34:24.584866+0000","last_unstale":"2024-09-10T11:34:26.492558+0000","last_undegraded":"2024-09-10T11:34:26.492558+0000","last_fullsized":"2024-09-10T11:34:26.492558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.799864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576572+0000","last_change":"2024-09-10T11:34:24.584025+0000","last_active":"2024-09-10T11:34:25.576572+0000","last_peered":"2024-09-10T11:34:25.576572+0000","last_clean":"2024-09-10T11:34:25.576572+0000","last_became_active":"2024-09-10T11:34:24.583545+0000","last_became_peered":"2024-09-10T11:34:24.583545+0000","last_unstale":"2024-09-10T11:34:25.576572+0000","last_undegraded":"2024-09-10T11:34:25.576572+0000","last_fullsized":"2024-09-10T11:34:25.576572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:20.303092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212970+0000","last_change":"2024-09-10T11:34:24.589627+0000","last_active":"2024-09-10T11:34:26.212970+0000","last_peered":"2024-09-10T11:34:26.212970+0000","last_clean":"2024-09-10T11:34:26.212970+0000","last_became_active":"2024-09-10T11:34:24.589379+0000","last_became_peered":"2024-09-10T11:34:24.589379+0000","last_unstale":"2024-09-10T11:34:26.212970+0000","last_undegraded":"2024-09-10T11:34:26.212970+0000","last_fullsized":"2024-09-10T11:34:26.212970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:44.241164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213276+0000","last_change":"2024-09-10T11:34:24.588104+0000","last_active":"2024-09-10T11:34:26.213276+0000","last_peered":"2024-09-10T11:34:26.213276+0000","last_clean":"2024-09-10T11:34:26.213276+0000","last_became_active":"2024-09-10T11:34:24.587851+0000","last_became_peered":"2024-09-10T11:34:24.587851+0000","last_unstale":"2024-09-10T11:34:26.213276+0000","last_undegraded":"2024-09-10T11:34:26.213276+0000","last_fullsized":"2024-09-10T11:34:26.213276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:05.283370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492508+0000","last_change":"2024-09-10T11:34:24.587794+0000","last_active":"2024-09-10T11:34:26.492508+0000","last_peered":"2024-09-10T11:34:26.492508+0000","last_clean":"2024-09-10T11:34:26.492508+0000","last_became_active":"2024-09-10T11:34:24.586812+0000","last_became_peered":"2024-09-10T11:34:24.586812+0000","last_unstale":"2024-09-10T11:34:26.492508+0000","last_undegraded":"2024-09-10T11:34:26.492508+0000","last_fullsized":"2024-09-10T11:34:26.492508+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:46:33.760645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492191+0000","last_change":"2024-09-10T11:34:24.586382+0000","last_active":"2024-09-10T11:34:26.492191+0000","last_peered":"2024-09-10T11:34:26.492191+0000","last_clean":"2024-09-10T11:34:26.492191+0000","last_became_active":"2024-09-10T11:34:24.585798+0000","last_became_peered":"2024-09-10T11:34:24.585798+0000","last_unstale":"2024-09-10T11:34:26.492191+0000","last_undegraded":"2024-09-10T11:34:26.492191+0000","last_fullsized":"2024-09-10T11:34:26.492191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:03.796650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576673+0000","last_change":"2024-09-10T11:34:24.581388+0000","last_active":"2024-09-10T11:34:25.576673+0000","last_peered":"2024-09-10T11:34:25.576673+0000","last_clean":"2024-09-10T11:34:25.576673+0000","last_became_active":"2024-09-10T11:34:24.581135+0000","last_became_peered":"2024-09-10T11:34:24.581135+0000","last_unstale":"2024-09-10T11:34:25.576673+0000","last_undegraded":"2024-09-10T11:34:25.576673+0000","last_fullsized":"2024-09-10T11:34:25.576673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:37.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.493105+0000","last_change":"2024-09-10T11:34:23.607410+0000","last_active":"2024-09-10T11:34:26.493105+0000","last_peered":"2024-09-10T11:34:26.493105+0000","last_clean":"2024-09-10T11:34:26.493105+0000","last_became_active":"2024-09-10T11:34:23.606877+0000","last_became_peered":"2024-09-10T11:34:23.606877+0000","last_unstale":"2024-09-10T11:34:26.493105+0000","last_undegraded":"2024-09-10T11:34:26.493105+0000","last_fullsized":"2024-09-10T11:34:26.493105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:29.403856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576714+0000","last_change":"2024-09-10T11:34:24.583831+0000","last_active":"2024-09-10T11:34:25.576714+0000","last_peered":"2024-09-10T11:34:25.576714+0000","last_clean":"2024-09-10T11:34:25.576714+0000","last_became_active":"2024-09-10T11:34:24.583224+0000","last_became_peered":"2024-09-10T11:34:24.583224+0000","last_unstale":"2024-09-10T11:34:25.576714+0000","last_undegraded":"2024-09-10T11:34:25.576714+0000","last_fullsized":"2024-09-10T11:34:25.576714+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.194039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492280+0000","last_change":"2024-09-10T11:34:23.600685+0000","last_active":"2024-09-10T11:34:26.492280+0000","last_peered":"2024-09-10T11:34:26.492280+0000","last_clean":"2024-09-10T11:34:26.492280+0000","last_became_active":"2024-09-10T11:34:23.600433+0000","last_became_peered":"2024-09-10T11:34:23.600433+0000","last_unstale":"2024-09-10T11:34:26.492280+0000","last_undegraded":"2024-09-10T11:34:26.492280+0000","last_fullsized":"2024-09-10T11:34:26.492280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:26.301084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577294+0000","last_change":"2024-09-10T11:34:24.586442+0000","last_active":"2024-09-10T11:34:25.577294+0000","last_peered":"2024-09-10T11:34:25.577294+0000","last_clean":"2024-09-10T11:34:25.577294+0000","last_became_active":"2024-09-10T11:34:24.586077+0000","last_became_peered":"2024-09-10T11:34:24.586077+0000","last_unstale":"2024-09-10T11:34:25.577294+0000","last_undegraded":"2024-09-10T11:34:25.577294+0000","last_fullsized":"2024-09-10T11:34:25.577294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:13.618383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576763+0000","last_change":"2024-09-10T11:34:23.601612+0000","last_active":"2024-09-10T11:34:25.576763+0000","last_peered":"2024-09-10T11:34:25.576763+0000","last_clean":"2024-09-10T11:34:25.576763+0000","last_became_active":"2024-09-10T11:34:23.601356+0000","last_became_peered":"2024-09-10T11:34:23.601356+0000","last_unstale":"2024-09-10T11:34:25.576763+0000","last_undegraded":"2024-09-10T11:34:25.576763+0000","last_fullsized":"2024-09-10T11:34:25.576763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:46.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576824+0000","last_change":"2024-09-10T11:34:24.580441+0000","last_active":"2024-09-10T11:34:25.576824+0000","last_peered":"2024-09-10T11:34:25.576824+0000","last_clean":"2024-09-10T11:34:25.576824+0000","last_became_active":"2024-09-10T11:34:24.579943+0000","last_became_peered":"2024-09-10T11:34:24.579943+0000","last_unstale":"2024-09-10T11:34:25.576824+0000","last_undegraded":"2024-09-10T11:34:25.576824+0000","last_fullsized":"2024-09-10T11:34:25.576824+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:19.471823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213297+0000","last_change":"2024-09-10T11:34:23.603868+0000","last_active":"2024-09-10T11:34:26.213297+0000","last_peered":"2024-09-10T11:34:26.213297+0000","last_clean":"2024-09-10T11:34:26.213297+0000","last_became_active":"2024-09-10T11:34:23.603620+0000","last_became_peered":"2024-09-10T11:34:23.603620+0000","last_unstale":"2024-09-10T11:34:26.213297+0000","last_undegraded":"2024-09-10T11:34:26.213297+0000","last_fullsized":"2024-09-10T11:34:26.213297+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:24.624623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212723+0000","last_change":"2024-09-10T11:34:24.588426+0000","last_active":"2024-09-10T11:34:26.212723+0000","last_peered":"2024-09-10T11:34:26.212723+0000","last_clean":"2024-09-10T11:34:26.212723+0000","last_became_active":"2024-09-10T11:34:24.588163+0000","last_became_peered":"2024-09-10T11:34:24.588163+0000","last_unstale":"2024-09-10T11:34:26.212723+0000","last_undegraded":"2024-09-10T11:34:26.212723+0000","last_fullsized":"2024-09-10T11:34:26.212723+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:33.251352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577411+0000","last_change":"2024-09-10T11:34:23.601930+0000","last_active":"2024-09-10T11:34:25.577411+0000","last_peered":"2024-09-10T11:34:25.577411+0000","last_clean":"2024-09-10T11:34:25.577411+0000","last_became_active":"2024-09-10T11:34:23.601703+0000","last_became_peered":"2024-09-10T11:34:23.601703+0000","last_unstale":"2024-09-10T11:34:25.577411+0000","last_undegraded":"2024-09-10T11:34:25.577411+0000","last_fullsized":"2024-09-10T11:34:25.577411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:35.739429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492028+0000","last_change":"2024-09-10T11:34:24.586969+0000","last_active":"2024-09-10T11:34:26.492028+0000","last_peered":"2024-09-10T11:34:26.492028+0000","last_clean":"2024-09-10T11:34:26.492028+0000","last_became_active":"2024-09-10T11:34:24.586681+0000","last_became_peered":"2024-09-10T11:34:24.586681+0000","last_unstale":"2024-09-10T11:34:26.492028+0000","last_undegraded":"2024-09-10T11:34:26.492028+0000","last_fullsized":"2024-09-10T11:34:26.492028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:05.019855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492154+0000","last_change":"2024-09-10T11:34:23.604097+0000","last_active":"2024-09-10T11:34:26.492154+0000","last_peered":"2024-09-10T11:34:26.492154+0000","last_clean":"2024-09-10T11:34:26.492154+0000","last_became_active":"2024-09-10T11:34:23.603848+0000","last_became_peered":"2024-09-10T11:34:23.603848+0000","last_unstale":"2024-09-10T11:34:26.492154+0000","last_undegraded":"2024-09-10T11:34:26.492154+0000","last_fullsized":"2024-09-10T11:34:26.492154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:24.476679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576937+0000","last_change":"2024-09-10T11:34:24.580624+0000","last_active":"2024-09-10T11:34:25.576937+0000","last_peered":"2024-09-10T11:34:25.576937+0000","last_clean":"2024-09-10T11:34:25.576937+0000","last_became_active":"2024-09-10T11:34:24.580231+0000","last_became_peered":"2024-09-10T11:34:24.580231+0000","last_unstale":"2024-09-10T11:34:25.576937+0000","last_undegraded":"2024-09-10T11:34:25.576937+0000","last_fullsized":"2024-09-10T11:34:25.576937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:30.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213660+0000","last_change":"2024-09-10T11:34:23.606119+0000","last_active":"2024-09-10T11:34:26.213660+0000","last_peered":"2024-09-10T11:34:26.213660+0000","last_clean":"2024-09-10T11:34:26.213660+0000","last_became_active":"2024-09-10T11:34:23.605898+0000","last_became_peered":"2024-09-10T11:34:23.605898+0000","last_unstale":"2024-09-10T11:34:26.213660+0000","last_undegraded":"2024-09-10T11:34:26.213660+0000","last_fullsized":"2024-09-10T11:34:26.213660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:21.621299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213449+0000","last_change":"2024-09-10T11:34:24.588905+0000","last_active":"2024-09-10T11:34:26.213449+0000","last_peered":"2024-09-10T11:34:26.213449+0000","last_clean":"2024-09-10T11:34:26.213449+0000","last_became_active":"2024-09-10T11:34:24.585583+0000","last_became_peered":"2024-09-10T11:34:24.585583+0000","last_unstale":"2024-09-10T11:34:26.213449+0000","last_undegraded":"2024-09-10T11:34:26.213449+0000","last_fullsized":"2024-09-10T11:34:26.213449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:51.832758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213452+0000","last_change":"2024-09-10T11:34:23.603835+0000","last_active":"2024-09-10T11:34:26.213452+0000","last_peered":"2024-09-10T11:34:26.213452+0000","last_clean":"2024-09-10T11:34:26.213452+0000","last_became_active":"2024-09-10T11:34:23.603420+0000","last_became_peered":"2024-09-10T11:34:23.603420+0000","last_unstale":"2024-09-10T11:34:26.213452+0000","last_undegraded":"2024-09-10T11:34:26.213452+0000","last_fullsized":"2024-09-10T11:34:26.213452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:02.707677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213479+0000","last_change":"2024-09-10T11:34:24.585980+0000","last_active":"2024-09-10T11:34:26.213479+0000","last_peered":"2024-09-10T11:34:26.213479+0000","last_clean":"2024-09-10T11:34:26.213479+0000","last_became_active":"2024-09-10T11:34:24.584344+0000","last_became_peered":"2024-09-10T11:34:24.584344+0000","last_unstale":"2024-09-10T11:34:26.213479+0000","last_undegraded":"2024-09-10T11:34:26.213479+0000","last_fullsized":"2024-09-10T11:34:26.213479+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:03.535465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576880+0000","last_change":"2024-09-10T11:34:23.601301+0000","last_active":"2024-09-10T11:34:25.576880+0000","last_peered":"2024-09-10T11:34:25.576880+0000","last_clean":"2024-09-10T11:34:25.576880+0000","last_became_active":"2024-09-10T11:34:23.601068+0000","last_became_peered":"2024-09-10T11:34:23.601068+0000","last_unstale":"2024-09-10T11:34:25.576880+0000","last_undegraded":"2024-09-10T11:34:25.576880+0000","last_fullsized":"2024-09-10T11:34:25.576880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:23.727005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576655+0000","last_change":"2024-09-10T11:34:24.578058+0000","last_active":"2024-09-10T11:34:25.576655+0000","last_peered":"2024-09-10T11:34:25.576655+0000","last_clean":"2024-09-10T11:34:25.576655+0000","last_became_active":"2024-09-10T11:34:24.577794+0000","last_became_peered":"2024-09-10T11:34:24.577794+0000","last_unstale":"2024-09-10T11:34:25.576655+0000","last_undegraded":"2024-09-10T11:34:25.576655+0000","last_fullsized":"2024-09-10T11:34:25.576655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492355+0000","last_change":"2024-09-10T11:34:23.608036+0000","last_active":"2024-09-10T11:34:26.492355+0000","last_peered":"2024-09-10T11:34:26.492355+0000","last_clean":"2024-09-10T11:34:26.492355+0000","last_became_active":"2024-09-10T11:34:23.607371+0000","last_became_peered":"2024-09-10T11:34:23.607371+0000","last_unstale":"2024-09-10T11:34:26.492355+0000","last_undegraded":"2024-09-10T11:34:26.492355+0000","last_fullsized":"2024-09-10T11:34:26.492355+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.960115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576746+0000","last_change":"2024-09-10T11:34:24.583788+0000","last_active":"2024-09-10T11:34:25.576746+0000","last_peered":"2024-09-10T11:34:25.576746+0000","last_clean":"2024-09-10T11:34:25.576746+0000","last_became_active":"2024-09-10T11:34:24.583504+0000","last_became_peered":"2024-09-10T11:34:24.583504+0000","last_unstale":"2024-09-10T11:34:25.576746+0000","last_undegraded":"2024-09-10T11:34:25.576746+0000","last_fullsized":"2024-09-10T11:34:25.576746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:40.364999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576952+0000","last_change":"2024-09-10T11:34:23.607971+0000","last_active":"2024-09-10T11:34:25.576952+0000","last_peered":"2024-09-10T11:34:25.576952+0000","last_clean":"2024-09-10T11:34:25.576952+0000","last_became_active":"2024-09-10T11:34:23.607431+0000","last_became_peered":"2024-09-10T11:34:23.607431+0000","last_unstale":"2024-09-10T11:34:25.576952+0000","last_undegraded":"2024-09-10T11:34:25.576952+0000","last_fullsized":"2024-09-10T11:34:25.576952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:29.999547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492756+0000","last_change":"2024-09-10T11:34:24.588470+0000","last_active":"2024-09-10T11:34:26.492756+0000","last_peered":"2024-09-10T11:34:26.492756+0000","last_clean":"2024-09-10T11:34:26.492756+0000","last_became_active":"2024-09-10T11:34:24.588201+0000","last_became_peered":"2024-09-10T11:34:24.588201+0000","last_unstale":"2024-09-10T11:34:26.492756+0000","last_undegraded":"2024-09-10T11:34:26.492756+0000","last_fullsized":"2024-09-10T11:34:26.492756+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:44.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576867+0000","last_change":"2024-09-10T11:34:23.607510+0000","last_active":"2024-09-10T11:34:25.576867+0000","last_peered":"2024-09-10T11:34:25.576867+0000","last_clean":"2024-09-10T11:34:25.576867+0000","last_became_active":"2024-09-10T11:34:23.607277+0000","last_became_peered":"2024-09-10T11:34:23.607277+0000","last_unstale":"2024-09-10T11:34:25.576867+0000","last_undegraded":"2024-09-10T11:34:25.576867+0000","last_fullsized":"2024-09-10T11:34:25.576867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:55.433412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576491+0000","last_change":"2024-09-10T11:34:24.578411+0000","last_active":"2024-09-10T11:34:25.576491+0000","last_peered":"2024-09-10T11:34:25.576491+0000","last_clean":"2024-09-10T11:34:25.576491+0000","last_became_active":"2024-09-10T11:34:24.578123+0000","last_became_peered":"2024-09-10T11:34:24.578123+0000","last_unstale":"2024-09-10T11:34:25.576491+0000","last_undegraded":"2024-09-10T11:34:25.576491+0000","last_fullsized":"2024-09-10T11:34:25.576491+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:19.225665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212970+0000","last_change":"2024-09-10T11:34:23.606581+0000","last_active":"2024-09-10T11:34:26.212970+0000","last_peered":"2024-09-10T11:34:26.212970+0000","last_clean":"2024-09-10T11:34:26.212970+0000","last_became_active":"2024-09-10T11:34:23.606302+0000","last_became_peered":"2024-09-10T11:34:23.606302+0000","last_unstale":"2024-09-10T11:34:26.212970+0000","last_undegraded":"2024-09-10T11:34:26.212970+0000","last_fullsized":"2024-09-10T11:34:26.212970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:49.084821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577127+0000","last_change":"2024-09-10T11:34:24.581276+0000","last_active":"2024-09-10T11:34:25.577127+0000","last_peered":"2024-09-10T11:34:25.577127+0000","last_clean":"2024-09-10T11:34:25.577127+0000","last_became_active":"2024-09-10T11:34:24.581111+0000","last_became_peered":"2024-09-10T11:34:24.581111+0000","last_unstale":"2024-09-10T11:34:25.577127+0000","last_undegraded":"2024-09-10T11:34:25.577127+0000","last_fullsized":"2024-09-10T11:34:25.577127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:16.731228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492268+0000","last_change":"2024-09-10T11:34:23.600717+0000","last_active":"2024-09-10T11:34:26.492268+0000","last_peered":"2024-09-10T11:34:26.492268+0000","last_clean":"2024-09-10T11:34:26.492268+0000","last_became_active":"2024-09-10T11:34:23.600406+0000","last_became_peered":"2024-09-10T11:34:23.600406+0000","last_unstale":"2024-09-10T11:34:26.492268+0000","last_undegraded":"2024-09-10T11:34:26.492268+0000","last_fullsized":"2024-09-10T11:34:26.492268+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:07.359175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213055+0000","last_change":"2024-09-10T11:34:24.588591+0000","last_active":"2024-09-10T11:34:26.213055+0000","last_peered":"2024-09-10T11:34:26.213055+0000","last_clean":"2024-09-10T11:34:26.213055+0000","last_became_active":"2024-09-10T11:34:24.587124+0000","last_became_peered":"2024-09-10T11:34:24.587124+0000","last_unstale":"2024-09-10T11:34:26.213055+0000","last_undegraded":"2024-09-10T11:34:26.213055+0000","last_fullsized":"2024-09-10T11:34:26.213055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:30.062420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492445+0000","last_change":"2024-09-10T11:34:23.607644+0000","last_active":"2024-09-10T11:34:26.492445+0000","last_peered":"2024-09-10T11:34:26.492445+0000","last_clean":"2024-09-10T11:34:26.492445+0000","last_became_active":"2024-09-10T11:34:23.605127+0000","last_became_peered":"2024-09-10T11:34:23.605127+0000","last_unstale":"2024-09-10T11:34:26.492445+0000","last_undegraded":"2024-09-10T11:34:26.492445+0000","last_fullsized":"2024-09-10T11:34:26.492445+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:46.582048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576406+0000","last_change":"2024-09-10T11:34:24.586861+0000","last_active":"2024-09-10T11:34:25.576406+0000","last_peered":"2024-09-10T11:34:25.576406+0000","last_clean":"2024-09-10T11:34:25.576406+0000","last_became_active":"2024-09-10T11:34:24.586611+0000","last_became_peered":"2024-09-10T11:34:24.586611+0000","last_unstale":"2024-09-10T11:34:25.576406+0000","last_undegraded":"2024-09-10T11:34:25.576406+0000","last_fullsized":"2024-09-10T11:34:25.576406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:50.126003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576820+0000","last_change":"2024-09-10T11:34:23.607623+0000","last_active":"2024-09-10T11:34:25.576820+0000","last_peered":"2024-09-10T11:34:25.576820+0000","last_clean":"2024-09-10T11:34:25.576820+0000","last_became_active":"2024-09-10T11:34:23.607376+0000","last_became_peered":"2024-09-10T11:34:23.607376+0000","last_unstale":"2024-09-10T11:34:25.576820+0000","last_undegraded":"2024-09-10T11:34:25.576820+0000","last_fullsized":"2024-09-10T11:34:25.576820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:29.507835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577165+0000","last_change":"2024-09-10T11:34:24.578393+0000","last_active":"2024-09-10T11:34:25.577165+0000","last_peered":"2024-09-10T11:34:25.577165+0000","last_clean":"2024-09-10T11:34:25.577165+0000","last_became_active":"2024-09-10T11:34:24.578191+0000","last_became_peered":"2024-09-10T11:34:24.578191+0000","last_unstale":"2024-09-10T11:34:25.577165+0000","last_undegraded":"2024-09-10T11:34:25.577165+0000","last_fullsized":"2024-09-10T11:34:25.577165+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:52.236893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492402+0000","last_change":"2024-09-10T11:34:23.605060+0000","last_active":"2024-09-10T11:34:26.492402+0000","last_peered":"2024-09-10T11:34:26.492402+0000","last_clean":"2024-09-10T11:34:26.492402+0000","last_became_active":"2024-09-10T11:34:23.604788+0000","last_became_peered":"2024-09-10T11:34:23.604788+0000","last_unstale":"2024-09-10T11:34:26.492402+0000","last_undegraded":"2024-09-10T11:34:26.492402+0000","last_fullsized":"2024-09-10T11:34:26.492402+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:18.339996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492015+0000","last_change":"2024-09-10T11:34:24.587007+0000","last_active":"2024-09-10T11:34:26.492015+0000","last_peered":"2024-09-10T11:34:26.492015+0000","last_clean":"2024-09-10T11:34:26.492015+0000","last_became_active":"2024-09-10T11:34:24.586441+0000","last_became_peered":"2024-09-10T11:34:24.586441+0000","last_unstale":"2024-09-10T11:34:26.492015+0000","last_undegraded":"2024-09-10T11:34:26.492015+0000","last_fullsized":"2024-09-10T11:34:26.492015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:11.725799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213587+0000","last_change":"2024-09-10T11:34:23.602635+0000","last_active":"2024-09-10T11:34:26.213587+0000","last_peered":"2024-09-10T11:34:26.213587+0000","last_clean":"2024-09-10T11:34:26.213587+0000","last_became_active":"2024-09-10T11:34:23.602353+0000","last_became_peered":"2024-09-10T11:34:23.602353+0000","last_unstale":"2024-09-10T11:34:26.213587+0000","last_undegraded":"2024-09-10T11:34:26.213587+0000","last_fullsized":"2024-09-10T11:34:26.213587+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:08.548995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577188+0000","last_change":"2024-09-10T11:34:24.578586+0000","last_active":"2024-09-10T11:34:25.577188+0000","last_peered":"2024-09-10T11:34:25.577188+0000","last_clean":"2024-09-10T11:34:25.577188+0000","last_became_active":"2024-09-10T11:34:24.578265+0000","last_became_peered":"2024-09-10T11:34:24.578265+0000","last_unstale":"2024-09-10T11:34:25.577188+0000","last_undegraded":"2024-09-10T11:34:25.577188+0000","last_fullsized":"2024-09-10T11:34:25.577188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:06.590310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213665+0000","last_change":"2024-09-10T11:34:23.601885+0000","last_active":"2024-09-10T11:34:26.213665+0000","last_peered":"2024-09-10T11:34:26.213665+0000","last_clean":"2024-09-10T11:34:26.213665+0000","last_became_active":"2024-09-10T11:34:23.601657+0000","last_became_peered":"2024-09-10T11:34:23.601657+0000","last_unstale":"2024-09-10T11:34:26.213665+0000","last_undegraded":"2024-09-10T11:34:26.213665+0000","last_fullsized":"2024-09-10T11:34:26.213665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:42.356871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492252+0000","last_change":"2024-09-10T11:34:24.588620+0000","last_active":"2024-09-10T11:34:26.492252+0000","last_peered":"2024-09-10T11:34:26.492252+0000","last_clean":"2024-09-10T11:34:26.492252+0000","last_became_active":"2024-09-10T11:34:24.587205+0000","last_became_peered":"2024-09-10T11:34:24.587205+0000","last_unstale":"2024-09-10T11:34:26.492252+0000","last_undegraded":"2024-09-10T11:34:26.492252+0000","last_fullsized":"2024-09-10T11:34:26.492252+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:19.666210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213168+0000","last_change":"2024-09-10T11:34:23.603320+0000","last_active":"2024-09-10T11:34:26.213168+0000","last_peered":"2024-09-10T11:34:26.213168+0000","last_clean":"2024-09-10T11:34:26.213168+0000","last_became_active":"2024-09-10T11:34:23.603068+0000","last_became_peered":"2024-09-10T11:34:23.603068+0000","last_unstale":"2024-09-10T11:34:26.213168+0000","last_undegraded":"2024-09-10T11:34:26.213168+0000","last_fullsized":"2024-09-10T11:34:26.213168+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:09.307907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.493010+0000","last_change":"2024-09-10T11:34:24.587370+0000","last_active":"2024-09-10T11:34:26.493010+0000","last_peered":"2024-09-10T11:34:26.493010+0000","last_clean":"2024-09-10T11:34:26.493010+0000","last_became_active":"2024-09-10T11:34:24.587157+0000","last_became_peered":"2024-09-10T11:34:24.587157+0000","last_unstale":"2024-09-10T11:34:26.493010+0000","last_undegraded":"2024-09-10T11:34:26.493010+0000","last_fullsized":"2024-09-10T11:34:26.493010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:56.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492443+0000","last_change":"2024-09-10T11:34:23.602978+0000","last_active":"2024-09-10T11:34:26.492443+0000","last_peered":"2024-09-10T11:34:26.492443+0000","last_clean":"2024-09-10T11:34:26.492443+0000","last_became_active":"2024-09-10T11:34:23.602700+0000","last_became_peered":"2024-09-10T11:34:23.602700+0000","last_unstale":"2024-09-10T11:34:26.492443+0000","last_undegraded":"2024-09-10T11:34:26.492443+0000","last_fullsized":"2024-09-10T11:34:26.492443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:05.451285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213045+0000","last_change":"2024-09-10T11:34:24.583236+0000","last_active":"2024-09-10T11:34:26.213045+0000","last_peered":"2024-09-10T11:34:26.213045+0000","last_clean":"2024-09-10T11:34:26.213045+0000","last_became_active":"2024-09-10T11:34:24.582998+0000","last_became_peered":"2024-09-10T11:34:24.582998+0000","last_unstale":"2024-09-10T11:34:26.213045+0000","last_undegraded":"2024-09-10T11:34:26.213045+0000","last_fullsized":"2024-09-10T11:34:26.213045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:39.938728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576840+0000","last_change":"2024-09-10T11:34:23.603968+0000","last_active":"2024-09-10T11:34:25.576840+0000","last_peered":"2024-09-10T11:34:25.576840+0000","last_clean":"2024-09-10T11:34:25.576840+0000","last_became_active":"2024-09-10T11:34:23.603479+0000","last_became_peered":"2024-09-10T11:34:23.603479+0000","last_unstale":"2024-09-10T11:34:25.576840+0000","last_undegraded":"2024-09-10T11:34:25.576840+0000","last_fullsized":"2024-09-10T11:34:25.576840+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:01.063801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577042+0000","last_change":"2024-09-10T11:34:24.585738+0000","last_active":"2024-09-10T11:34:25.577042+0000","last_peered":"2024-09-10T11:34:25.577042+0000","last_clean":"2024-09-10T11:34:25.577042+0000","last_became_active":"2024-09-10T11:34:24.585517+0000","last_became_peered":"2024-09-10T11:34:24.585517+0000","last_unstale":"2024-09-10T11:34:25.577042+0000","last_undegraded":"2024-09-10T11:34:25.577042+0000","last_fullsized":"2024-09-10T11:34:25.577042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:18.002678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213046+0000","last_change":"2024-09-10T11:34:23.604116+0000","last_active":"2024-09-10T11:34:26.213046+0000","last_peered":"2024-09-10T11:34:26.213046+0000","last_clean":"2024-09-10T11:34:26.213046+0000","last_became_active":"2024-09-10T11:34:23.603849+0000","last_became_peered":"2024-09-10T11:34:23.603849+0000","last_unstale":"2024-09-10T11:34:26.213046+0000","last_undegraded":"2024-09-10T11:34:26.213046+0000","last_fullsized":"2024-09-10T11:34:26.213046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:23.415771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492888+0000","last_change":"2024-09-10T11:34:24.584486+0000","last_active":"2024-09-10T11:34:26.492888+0000","last_peered":"2024-09-10T11:34:26.492888+0000","last_clean":"2024-09-10T11:34:26.492888+0000","last_became_active":"2024-09-10T11:34:24.584260+0000","last_became_peered":"2024-09-10T11:34:24.584260+0000","last_unstale":"2024-09-10T11:34:26.492888+0000","last_undegraded":"2024-09-10T11:34:26.492888+0000","last_fullsized":"2024-09-10T11:34:26.492888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:22.464319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213721+0000","last_change":"2024-09-10T11:34:23.605000+0000","last_active":"2024-09-10T11:34:26.213721+0000","last_peered":"2024-09-10T11:34:26.213721+0000","last_clean":"2024-09-10T11:34:26.213721+0000","last_became_active":"2024-09-10T11:34:23.604652+0000","last_became_peered":"2024-09-10T11:34:23.604652+0000","last_unstale":"2024-09-10T11:34:26.213721+0000","last_undegraded":"2024-09-10T11:34:26.213721+0000","last_fullsized":"2024-09-10T11:34:26.213721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:05.588865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492703+0000","last_change":"2024-09-10T11:34:24.582510+0000","last_active":"2024-09-10T11:34:26.492703+0000","last_peered":"2024-09-10T11:34:26.492703+0000","last_clean":"2024-09-10T11:34:26.492703+0000","last_became_active":"2024-09-10T11:34:24.582231+0000","last_became_peered":"2024-09-10T11:34:24.582231+0000","last_unstale":"2024-09-10T11:34:26.492703+0000","last_undegraded":"2024-09-10T11:34:26.492703+0000","last_fullsized":"2024-09-10T11:34:26.492703+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:30.945082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.598556+0000","last_change":"2024-09-10T11:34:23.606790+0000","last_active":"2024-09-10T11:34:26.598556+0000","last_peered":"2024-09-10T11:34:26.598556+0000","last_clean":"2024-09-10T11:34:26.598556+0000","last_became_active":"2024-09-10T11:34:23.606309+0000","last_became_peered":"2024-09-10T11:34:23.606309+0000","last_unstale":"2024-09-10T11:34:26.598556+0000","last_undegraded":"2024-09-10T11:34:26.598556+0000","last_fullsized":"2024-09-10T11:34:26.598556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:44.009394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492835+0000","last_change":"2024-09-10T11:34:24.588090+0000","last_active":"2024-09-10T11:34:26.492835+0000","last_peered":"2024-09-10T11:34:26.492835+0000","last_clean":"2024-09-10T11:34:26.492835+0000","last_became_active":"2024-09-10T11:34:24.587851+0000","last_became_peered":"2024-09-10T11:34:24.587851+0000","last_unstale":"2024-09-10T11:34:26.492835+0000","last_undegraded":"2024-09-10T11:34:26.492835+0000","last_fullsized":"2024-09-10T11:34:26.492835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:46.293799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576948+0000","last_change":"2024-09-10T11:34:23.606318+0000","last_active":"2024-09-10T11:34:25.576948+0000","last_peered":"2024-09-10T11:34:25.576948+0000","last_clean":"2024-09-10T11:34:25.576948+0000","last_became_active":"2024-09-10T11:34:23.606086+0000","last_became_peered":"2024-09-10T11:34:23.606086+0000","last_unstale":"2024-09-10T11:34:25.576948+0000","last_undegraded":"2024-09-10T11:34:25.576948+0000","last_fullsized":"2024-09-10T11:34:25.576948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.883635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492180+0000","last_change":"2024-09-10T11:34:24.583152+0000","last_active":"2024-09-10T11:34:26.492180+0000","last_peered":"2024-09-10T11:34:26.492180+0000","last_clean":"2024-09-10T11:34:26.492180+0000","last_became_active":"2024-09-10T11:34:24.582840+0000","last_became_peered":"2024-09-10T11:34:24.582840+0000","last_unstale":"2024-09-10T11:34:26.492180+0000","last_undegraded":"2024-09-10T11:34:26.492180+0000","last_fullsized":"2024-09-10T11:34:26.492180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:07.033562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492350+0000","last_change":"2024-09-10T11:34:23.602999+0000","last_active":"2024-09-10T11:34:26.492350+0000","last_peered":"2024-09-10T11:34:26.492350+0000","last_clean":"2024-09-10T11:34:26.492350+0000","last_became_active":"2024-09-10T11:34:23.602551+0000","last_became_peered":"2024-09-10T11:34:23.602551+0000","last_unstale":"2024-09-10T11:34:26.492350+0000","last_undegraded":"2024-09-10T11:34:26.492350+0000","last_fullsized":"2024-09-10T11:34:26.492350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:03.869297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576633+0000","last_change":"2024-09-10T11:34:24.587198+0000","last_active":"2024-09-10T11:34:25.576633+0000","last_peered":"2024-09-10T11:34:25.576633+0000","last_clean":"2024-09-10T11:34:25.576633+0000","last_became_active":"2024-09-10T11:34:24.586882+0000","last_became_peered":"2024-09-10T11:34:24.586882+0000","last_unstale":"2024-09-10T11:34:25.576633+0000","last_undegraded":"2024-09-10T11:34:25.576633+0000","last_fullsized":"2024-09-10T11:34:25.576633+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:27.103552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492354+0000","last_change":"2024-09-10T11:34:23.605133+0000","last_active":"2024-09-10T11:34:26.492354+0000","last_peered":"2024-09-10T11:34:26.492354+0000","last_clean":"2024-09-10T11:34:26.492354+0000","last_became_active":"2024-09-10T11:34:23.604890+0000","last_became_peered":"2024-09-10T11:34:23.604890+0000","last_unstale":"2024-09-10T11:34:26.492354+0000","last_undegraded":"2024-09-10T11:34:26.492354+0000","last_fullsized":"2024-09-10T11:34:26.492354+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:19.279117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576708+0000","last_change":"2024-09-10T11:34:24.585969+0000","last_active":"2024-09-10T11:34:25.576708+0000","last_peered":"2024-09-10T11:34:25.576708+0000","last_clean":"2024-09-10T11:34:25.576708+0000","last_became_active":"2024-09-10T11:34:24.585465+0000","last_became_peered":"2024-09-10T11:34:24.585465+0000","last_unstale":"2024-09-10T11:34:25.576708+0000","last_undegraded":"2024-09-10T11:34:25.576708+0000","last_fullsized":"2024-09-10T11:34:25.576708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:54.574404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492545+0000","last_change":"2024-09-10T11:34:23.607846+0000","last_active":"2024-09-10T11:34:26.492545+0000","last_peered":"2024-09-10T11:34:26.492545+0000","last_clean":"2024-09-10T11:34:26.492545+0000","last_became_active":"2024-09-10T11:34:23.606912+0000","last_became_peered":"2024-09-10T11:34:23.606912+0000","last_unstale":"2024-09-10T11:34:26.492545+0000","last_undegraded":"2024-09-10T11:34:26.492545+0000","last_fullsized":"2024-09-10T11:34:26.492545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:08.282111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577151+0000","last_change":"2024-09-10T11:34:24.583576+0000","last_active":"2024-09-10T11:34:25.577151+0000","last_peered":"2024-09-10T11:34:25.577151+0000","last_clean":"2024-09-10T11:34:25.577151+0000","last_became_active":"2024-09-10T11:34:24.583331+0000","last_became_peered":"2024-09-10T11:34:24.583331+0000","last_unstale":"2024-09-10T11:34:25.577151+0000","last_undegraded":"2024-09-10T11:34:25.577151+0000","last_fullsized":"2024-09-10T11:34:25.577151+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:57.369960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577039+0000","last_change":"2024-09-10T11:34:23.607717+0000","last_active":"2024-09-10T11:34:25.577039+0000","last_peered":"2024-09-10T11:34:25.577039+0000","last_clean":"2024-09-10T11:34:25.577039+0000","last_became_active":"2024-09-10T11:34:23.607095+0000","last_became_peered":"2024-09-10T11:34:23.607095+0000","last_unstale":"2024-09-10T11:34:25.577039+0000","last_undegraded":"2024-09-10T11:34:25.577039+0000","last_fullsized":"2024-09-10T11:34:25.577039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:06.822644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576853+0000","last_change":"2024-09-10T11:34:24.587129+0000","last_active":"2024-09-10T11:34:25.576853+0000","last_peered":"2024-09-10T11:34:25.576853+0000","last_clean":"2024-09-10T11:34:25.576853+0000","last_became_active":"2024-09-10T11:34:24.586905+0000","last_became_peered":"2024-09-10T11:34:24.586905+0000","last_unstale":"2024-09-10T11:34:25.576853+0000","last_undegraded":"2024-09-10T11:34:25.576853+0000","last_fullsized":"2024-09-10T11:34:25.576853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:07.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577304+0000","last_change":"2024-09-10T11:34:23.604364+0000","last_active":"2024-09-10T11:34:25.577304+0000","last_peered":"2024-09-10T11:34:25.577304+0000","last_clean":"2024-09-10T11:34:25.577304+0000","last_became_active":"2024-09-10T11:34:23.603996+0000","last_became_peered":"2024-09-10T11:34:23.603996+0000","last_unstale":"2024-09-10T11:34:25.577304+0000","last_undegraded":"2024-09-10T11:34:25.577304+0000","last_fullsized":"2024-09-10T11:34:25.577304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:50.711505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576587+0000","last_change":"2024-09-10T11:34:24.582791+0000","last_active":"2024-09-10T11:34:25.576587+0000","last_peered":"2024-09-10T11:34:25.576587+0000","last_clean":"2024-09-10T11:34:25.576587+0000","last_became_active":"2024-09-10T11:34:24.582532+0000","last_became_peered":"2024-09-10T11:34:24.582532+0000","last_unstale":"2024-09-10T11:34:25.576587+0000","last_undegraded":"2024-09-10T11:34:25.576587+0000","last_fullsized":"2024-09-10T11:34:25.576587+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:24.562608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492662+0000","last_change":"2024-09-10T11:34:23.599066+0000","last_active":"2024-09-10T11:34:26.492662+0000","last_peered":"2024-09-10T11:34:26.492662+0000","last_clean":"2024-09-10T11:34:26.492662+0000","last_became_active":"2024-09-10T11:34:23.597806+0000","last_became_peered":"2024-09-10T11:34:23.597806+0000","last_unstale":"2024-09-10T11:34:26.492662+0000","last_undegraded":"2024-09-10T11:34:26.492662+0000","last_fullsized":"2024-09-10T11:34:26.492662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:35.775712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":339,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577198+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:34:25.577198+0000","last_peered":"2024-09-10T11:34:25.577198+0000","last_clean":"2024-09-10T11:34:25.577198+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:34:25.577198+0000","last_undegraded":"2024-09-10T11:34:25.577198+0000","last_fullsized":"2024-09-10T11:34:25.577198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:19.979634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442573,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441971,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1122,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198850629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442609,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441969,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442643,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1120,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194191,"statfs":{"total":200000000,"available":198852185,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199441009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:34:27.536 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:34:27.576 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-09-10T11:34:27.575+0000 7fcbd6d61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:34:27.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.670+0000 7f7842752640 1 Processor -- start 2024-09-10T11:34:27.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.670+0000 7f7842752640 1 -- start start 2024-09-10T11:34:27.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7842752640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f783c15f1e0 0x7f783c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7842752640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 0x7f783c0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7842752640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c0b40e0 con 0x7f783c15ee00 2024-09-10T11:34:27.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7842752640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f783c0b5ab0 con 0x7f783c15f1e0 2024-09-10T11:34:27.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7842752640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f783c073c50 con 0x7f783c0b0090 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783b7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f783c15f1e0 0x7f783c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f7840cef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 0x7f783c0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783bfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f783c15ee00 0x7f783c0b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:44026/0 (socket says 172.21.15.99:44026) 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783bfff640 1 -- 172.21.15.99:0/62822046 learned_addr learned my addr 172.21.15.99:0/62822046 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783affd640 1 -- 172.21.15.99:0/62822046 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923762581 0 0) 0x7f783c0b40e0 con 0x7f783c15ee00 2024-09-10T11:34:27.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783affd640 1 -- 172.21.15.99:0/62822046 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7824003520 con 0x7f783c15ee00 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783b7fe640 1 -- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 msgr2=0x7f783c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.671+0000 7f783b7fe640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 0x7f783c0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783b7fe640 1 -- 172.21.15.99:0/62822046 >> v1:172.21.15.196:6789/0 conn(0x7f783c15ee00 legacy=0x7f783c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783b7fe640 1 -- 172.21.15.99:0/62822046 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0b4380 con 0x7f783c15f1e0 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f7840cef640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 0x7f783c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783b7fe640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c0b5f90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f78300097a0 tx=0x7f783002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e9e90fb5e08f909 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783affd640 1 -- 172.21.15.99:0/62822046 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830037070 con 0x7f783c15f1e0 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783affd640 1 -- 172.21.15.99:0/62822046 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7830032440 con 0x7f783c15f1e0 2024-09-10T11:34:27.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.672+0000 7f783affd640 1 -- 172.21.15.99:0/62822046 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78300327f0 con 0x7f783c15f1e0 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 -- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 msgr2=0x7f783c0b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c0b5f90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f78300097a0 tx=0x7f783002a180 comp rx=0 tx=0).stop 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 -- 172.21.15.99:0/62822046 shutdown_connections 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c0b0090 0x7f783c0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 --2- 172.21.15.99:0/62822046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c0b5f90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 -- 172.21.15.99:0/62822046 >> 172.21.15.99:0/62822046 conn(0x7f783c0730c0 msgr2=0x7f783c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 -- 172.21.15.99:0/62822046 shutdown_connections 2024-09-10T11:34:27.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.673+0000 7f7842752640 1 -- 172.21.15.99:0/62822046 wait complete. 2024-09-10T11:34:27.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.674+0000 7f7842752640 1 Processor -- start 2024-09-10T11:34:27.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.674+0000 7f7842752640 1 -- start start 2024-09-10T11:34:27.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c15ee00 0x7f783c154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f783c0b21b0 con 0x7f783c0b0090 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f783c0afd60 con 0x7f783c15f1e0 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7842752640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f783c0b0600 con 0x7f783c15ee00 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34508/0 (socket says 172.21.15.99:34508) 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 -- 172.21.15.99:0/3287270447 learned_addr learned my addr 172.21.15.99:0/3287270447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f7840cef640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783b7fe640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c15ee00 0x7f783c154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 -- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c15ee00 msgr2=0x7f783c154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c15ee00 0x7f783c154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 -- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 msgr2=0x7f783c154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.675+0000 7f783bfff640 1 -- 172.21.15.99:0/3287270447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c15b920 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f783bfff640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c460 tx=0x7f782c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c017070 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f782c005ce0 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c155620 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c007290 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.676+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c155be0 con 0x7f783c0b0090 2024-09-10T11:34:27.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.677+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c0b5f90 con 0x7f783c0b0090 2024-09-10T11:34:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.677+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f782c01a070 con 0x7f783c0b0090 2024-09-10T11:34:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.678+0000 7f7838ff9640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 0x7f782806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.678+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f782c08a120 con 0x7f783c0b0090 2024-09-10T11:34:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.678+0000 7f783b7fe640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 0x7f782806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:27.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.678+0000 7f783b7fe640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 0x7f782806e320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7830034000 tx=0x7f783002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:27.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.682+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f782c058b40 con 0x7f783c0b0090 2024-09-10T11:34:27.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.843+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f783c0d2000 con 0x7f782806be60 2024-09-10T11:34:27.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.849+0000 7f7838ff9640 1 -- 172.21.15.99:0/3287270447 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247992 (secure 0 0 0) 0x7f7830032660 con 0x7f782806be60 2024-09-10T11:34:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:27.853 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:34:27.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 msgr2=0x7f782806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 0x7f782806e320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7830034000 tx=0x7f783002e000 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 msgr2=0x7f783c1566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c460 tx=0x7f782c00c920 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 shutdown_connections 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f782806be60 0x7f782806e320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f783c15f1e0 0x7f783c154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f783c15ee00 0x7f783c154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 --2- 172.21.15.99:0/3287270447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f783c0b0090 0x7f783c1566c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 >> 172.21.15.99:0/3287270447 conn(0x7f783c0730c0 msgr2=0x7f783c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 shutdown_connections 2024-09-10T11:34:27.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:27.852+0000 7f7842752640 1 -- 172.21.15.99:0/3287270447 wait complete. 2024-09-10T11:34:27.864 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-10T11:34:26.959540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5512,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556987,"statfs":{"total":1600000000,"available":1594355588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.370577"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576788+0000","last_change":"2024-09-10T11:34:24.587358+0000","last_active":"2024-09-10T11:34:25.576788+0000","last_peered":"2024-09-10T11:34:25.576788+0000","last_clean":"2024-09-10T11:34:25.576788+0000","last_became_active":"2024-09-10T11:34:24.587094+0000","last_became_peered":"2024-09-10T11:34:24.587094+0000","last_unstale":"2024-09-10T11:34:25.576788+0000","last_undegraded":"2024-09-10T11:34:25.576788+0000","last_fullsized":"2024-09-10T11:34:25.576788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:21:27.164698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492230+0000","last_change":"2024-09-10T11:34:24.589433+0000","last_active":"2024-09-10T11:34:26.492230+0000","last_peered":"2024-09-10T11:34:26.492230+0000","last_clean":"2024-09-10T11:34:26.492230+0000","last_became_active":"2024-09-10T11:34:24.588944+0000","last_became_peered":"2024-09-10T11:34:24.588944+0000","last_unstale":"2024-09-10T11:34:26.492230+0000","last_undegraded":"2024-09-10T11:34:26.492230+0000","last_fullsized":"2024-09-10T11:34:26.492230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:50.778882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492025+0000","last_change":"2024-09-10T11:34:24.585835+0000","last_active":"2024-09-10T11:34:26.492025+0000","last_peered":"2024-09-10T11:34:26.492025+0000","last_clean":"2024-09-10T11:34:26.492025+0000","last_became_active":"2024-09-10T11:34:24.585623+0000","last_became_peered":"2024-09-10T11:34:24.585623+0000","last_unstale":"2024-09-10T11:34:26.492025+0000","last_undegraded":"2024-09-10T11:34:26.492025+0000","last_fullsized":"2024-09-10T11:34:26.492025+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:02.607420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212802+0000","last_change":"2024-09-10T11:34:24.587968+0000","last_active":"2024-09-10T11:34:26.212802+0000","last_peered":"2024-09-10T11:34:26.212802+0000","last_clean":"2024-09-10T11:34:26.212802+0000","last_became_active":"2024-09-10T11:34:24.587711+0000","last_became_peered":"2024-09-10T11:34:24.587711+0000","last_unstale":"2024-09-10T11:34:26.212802+0000","last_undegraded":"2024-09-10T11:34:26.212802+0000","last_fullsized":"2024-09-10T11:34:26.212802+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:07.649525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492720+0000","last_change":"2024-09-10T11:34:24.585664+0000","last_active":"2024-09-10T11:34:26.492720+0000","last_peered":"2024-09-10T11:34:26.492720+0000","last_clean":"2024-09-10T11:34:26.492720+0000","last_became_active":"2024-09-10T11:34:24.585426+0000","last_became_peered":"2024-09-10T11:34:24.585426+0000","last_unstale":"2024-09-10T11:34:26.492720+0000","last_undegraded":"2024-09-10T11:34:26.492720+0000","last_fullsized":"2024-09-10T11:34:26.492720+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:53.501159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492357+0000","last_change":"2024-09-10T11:34:24.589286+0000","last_active":"2024-09-10T11:34:26.492357+0000","last_peered":"2024-09-10T11:34:26.492357+0000","last_clean":"2024-09-10T11:34:26.492357+0000","last_became_active":"2024-09-10T11:34:24.589116+0000","last_became_peered":"2024-09-10T11:34:24.589116+0000","last_unstale":"2024-09-10T11:34:26.492357+0000","last_undegraded":"2024-09-10T11:34:26.492357+0000","last_fullsized":"2024-09-10T11:34:26.492357+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:20.162620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492449+0000","last_change":"2024-09-10T11:34:24.589240+0000","last_active":"2024-09-10T11:34:26.492449+0000","last_peered":"2024-09-10T11:34:26.492449+0000","last_clean":"2024-09-10T11:34:26.492449+0000","last_became_active":"2024-09-10T11:34:24.587491+0000","last_became_peered":"2024-09-10T11:34:24.587491+0000","last_unstale":"2024-09-10T11:34:26.492449+0000","last_undegraded":"2024-09-10T11:34:26.492449+0000","last_fullsized":"2024-09-10T11:34:26.492449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:14.319805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576776+0000","last_change":"2024-09-10T11:34:24.577483+0000","last_active":"2024-09-10T11:34:25.576776+0000","last_peered":"2024-09-10T11:34:25.576776+0000","last_clean":"2024-09-10T11:34:25.576776+0000","last_became_active":"2024-09-10T11:34:24.577186+0000","last_became_peered":"2024-09-10T11:34:24.577186+0000","last_unstale":"2024-09-10T11:34:25.576776+0000","last_undegraded":"2024-09-10T11:34:25.576776+0000","last_fullsized":"2024-09-10T11:34:25.576776+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:57.540166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576458+0000","last_change":"2024-09-10T11:34:24.587049+0000","last_active":"2024-09-10T11:34:25.576458+0000","last_peered":"2024-09-10T11:34:25.576458+0000","last_clean":"2024-09-10T11:34:25.576458+0000","last_became_active":"2024-09-10T11:34:24.586800+0000","last_became_peered":"2024-09-10T11:34:24.586800+0000","last_unstale":"2024-09-10T11:34:25.576458+0000","last_undegraded":"2024-09-10T11:34:25.576458+0000","last_fullsized":"2024-09-10T11:34:25.576458+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:18.282012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492188+0000","last_change":"2024-09-10T11:34:24.588067+0000","last_active":"2024-09-10T11:34:26.492188+0000","last_peered":"2024-09-10T11:34:26.492188+0000","last_clean":"2024-09-10T11:34:26.492188+0000","last_became_active":"2024-09-10T11:34:24.587714+0000","last_became_peered":"2024-09-10T11:34:24.587714+0000","last_unstale":"2024-09-10T11:34:26.492188+0000","last_undegraded":"2024-09-10T11:34:26.492188+0000","last_fullsized":"2024-09-10T11:34:26.492188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:45.186673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576942+0000","last_change":"2024-09-10T11:34:24.585763+0000","last_active":"2024-09-10T11:34:25.576942+0000","last_peered":"2024-09-10T11:34:25.576942+0000","last_clean":"2024-09-10T11:34:25.576942+0000","last_became_active":"2024-09-10T11:34:24.585122+0000","last_became_peered":"2024-09-10T11:34:24.585122+0000","last_unstale":"2024-09-10T11:34:25.576942+0000","last_undegraded":"2024-09-10T11:34:25.576942+0000","last_fullsized":"2024-09-10T11:34:25.576942+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:43.133411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577118+0000","last_change":"2024-09-10T11:34:24.584319+0000","last_active":"2024-09-10T11:34:25.577118+0000","last_peered":"2024-09-10T11:34:25.577118+0000","last_clean":"2024-09-10T11:34:25.577118+0000","last_became_active":"2024-09-10T11:34:24.584111+0000","last_became_peered":"2024-09-10T11:34:24.584111+0000","last_unstale":"2024-09-10T11:34:25.577118+0000","last_undegraded":"2024-09-10T11:34:25.577118+0000","last_fullsized":"2024-09-10T11:34:25.577118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:34.575572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492606+0000","last_change":"2024-09-10T11:34:24.585155+0000","last_active":"2024-09-10T11:34:26.492606+0000","last_peered":"2024-09-10T11:34:26.492606+0000","last_clean":"2024-09-10T11:34:26.492606+0000","last_became_active":"2024-09-10T11:34:24.584908+0000","last_became_peered":"2024-09-10T11:34:24.584908+0000","last_unstale":"2024-09-10T11:34:26.492606+0000","last_undegraded":"2024-09-10T11:34:26.492606+0000","last_fullsized":"2024-09-10T11:34:26.492606+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:48.369692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213103+0000","last_change":"2024-09-10T11:34:24.590357+0000","last_active":"2024-09-10T11:34:26.213103+0000","last_peered":"2024-09-10T11:34:26.213103+0000","last_clean":"2024-09-10T11:34:26.213103+0000","last_became_active":"2024-09-10T11:34:24.589626+0000","last_became_peered":"2024-09-10T11:34:24.589626+0000","last_unstale":"2024-09-10T11:34:26.213103+0000","last_undegraded":"2024-09-10T11:34:26.213103+0000","last_fullsized":"2024-09-10T11:34:26.213103+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:24.731255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492407+0000","last_change":"2024-09-10T11:34:24.592234+0000","last_active":"2024-09-10T11:34:26.492407+0000","last_peered":"2024-09-10T11:34:26.492407+0000","last_clean":"2024-09-10T11:34:26.492407+0000","last_became_active":"2024-09-10T11:34:24.591974+0000","last_became_peered":"2024-09-10T11:34:24.591974+0000","last_unstale":"2024-09-10T11:34:26.492407+0000","last_undegraded":"2024-09-10T11:34:26.492407+0000","last_fullsized":"2024-09-10T11:34:26.492407+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:46.022677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492638+0000","last_change":"2024-09-10T11:34:24.587956+0000","last_active":"2024-09-10T11:34:26.492638+0000","last_peered":"2024-09-10T11:34:26.492638+0000","last_clean":"2024-09-10T11:34:26.492638+0000","last_became_active":"2024-09-10T11:34:24.587360+0000","last_became_peered":"2024-09-10T11:34:24.587360+0000","last_unstale":"2024-09-10T11:34:26.492638+0000","last_undegraded":"2024-09-10T11:34:26.492638+0000","last_fullsized":"2024-09-10T11:34:26.492638+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:40.069884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577255+0000","last_change":"2024-09-10T11:34:24.583393+0000","last_active":"2024-09-10T11:34:25.577255+0000","last_peered":"2024-09-10T11:34:25.577255+0000","last_clean":"2024-09-10T11:34:25.577255+0000","last_became_active":"2024-09-10T11:34:24.583013+0000","last_became_peered":"2024-09-10T11:34:24.583013+0000","last_unstale":"2024-09-10T11:34:25.577255+0000","last_undegraded":"2024-09-10T11:34:25.577255+0000","last_fullsized":"2024-09-10T11:34:25.577255+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:34.954454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576702+0000","last_change":"2024-09-10T11:34:24.579255+0000","last_active":"2024-09-10T11:34:25.576702+0000","last_peered":"2024-09-10T11:34:25.576702+0000","last_clean":"2024-09-10T11:34:25.576702+0000","last_became_active":"2024-09-10T11:34:24.578863+0000","last_became_peered":"2024-09-10T11:34:24.578863+0000","last_unstale":"2024-09-10T11:34:25.576702+0000","last_undegraded":"2024-09-10T11:34:25.576702+0000","last_fullsized":"2024-09-10T11:34:25.576702+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:58:26.776397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213278+0000","last_change":"2024-09-10T11:34:24.590638+0000","last_active":"2024-09-10T11:34:26.213278+0000","last_peered":"2024-09-10T11:34:26.213278+0000","last_clean":"2024-09-10T11:34:26.213278+0000","last_became_active":"2024-09-10T11:34:24.590049+0000","last_became_peered":"2024-09-10T11:34:24.590049+0000","last_unstale":"2024-09-10T11:34:26.213278+0000","last_undegraded":"2024-09-10T11:34:26.213278+0000","last_fullsized":"2024-09-10T11:34:26.213278+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:27.422736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213389+0000","last_change":"2024-09-10T11:34:24.592072+0000","last_active":"2024-09-10T11:34:26.213389+0000","last_peered":"2024-09-10T11:34:26.213389+0000","last_clean":"2024-09-10T11:34:26.213389+0000","last_became_active":"2024-09-10T11:34:24.591750+0000","last_became_peered":"2024-09-10T11:34:24.591750+0000","last_unstale":"2024-09-10T11:34:26.213389+0000","last_undegraded":"2024-09-10T11:34:26.213389+0000","last_fullsized":"2024-09-10T11:34:26.213389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:19.702469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212784+0000","last_change":"2024-09-10T11:34:24.588653+0000","last_active":"2024-09-10T11:34:26.212784+0000","last_peered":"2024-09-10T11:34:26.212784+0000","last_clean":"2024-09-10T11:34:26.212784+0000","last_became_active":"2024-09-10T11:34:24.588406+0000","last_became_peered":"2024-09-10T11:34:24.588406+0000","last_unstale":"2024-09-10T11:34:26.212784+0000","last_undegraded":"2024-09-10T11:34:26.212784+0000","last_fullsized":"2024-09-10T11:34:26.212784+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:22:27.113558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213602+0000","last_change":"2024-09-10T11:34:24.586975+0000","last_active":"2024-09-10T11:34:26.213602+0000","last_peered":"2024-09-10T11:34:26.213602+0000","last_clean":"2024-09-10T11:34:26.213602+0000","last_became_active":"2024-09-10T11:34:24.586721+0000","last_became_peered":"2024-09-10T11:34:24.586721+0000","last_unstale":"2024-09-10T11:34:26.213602+0000","last_undegraded":"2024-09-10T11:34:26.213602+0000","last_fullsized":"2024-09-10T11:34:26.213602+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:17.936666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576897+0000","last_change":"2024-09-10T11:34:24.582533+0000","last_active":"2024-09-10T11:34:25.576897+0000","last_peered":"2024-09-10T11:34:25.576897+0000","last_clean":"2024-09-10T11:34:25.576897+0000","last_became_active":"2024-09-10T11:34:24.582309+0000","last_became_peered":"2024-09-10T11:34:24.582309+0000","last_unstale":"2024-09-10T11:34:25.576897+0000","last_undegraded":"2024-09-10T11:34:25.576897+0000","last_fullsized":"2024-09-10T11:34:25.576897+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:15.632768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576543+0000","last_change":"2024-09-10T11:34:24.590536+0000","last_active":"2024-09-10T11:34:25.576543+0000","last_peered":"2024-09-10T11:34:25.576543+0000","last_clean":"2024-09-10T11:34:25.576543+0000","last_became_active":"2024-09-10T11:34:24.590259+0000","last_became_peered":"2024-09-10T11:34:24.590259+0000","last_unstale":"2024-09-10T11:34:25.576543+0000","last_undegraded":"2024-09-10T11:34:25.576543+0000","last_fullsized":"2024-09-10T11:34:25.576543+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:19.287456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576585+0000","last_change":"2024-09-10T11:34:24.583161+0000","last_active":"2024-09-10T11:34:25.576585+0000","last_peered":"2024-09-10T11:34:25.576585+0000","last_clean":"2024-09-10T11:34:25.576585+0000","last_became_active":"2024-09-10T11:34:24.582445+0000","last_became_peered":"2024-09-10T11:34:24.582445+0000","last_unstale":"2024-09-10T11:34:25.576585+0000","last_undegraded":"2024-09-10T11:34:25.576585+0000","last_fullsized":"2024-09-10T11:34:25.576585+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:14.951048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492571+0000","last_change":"2024-09-10T11:34:24.588425+0000","last_active":"2024-09-10T11:34:26.492571+0000","last_peered":"2024-09-10T11:34:26.492571+0000","last_clean":"2024-09-10T11:34:26.492571+0000","last_became_active":"2024-09-10T11:34:24.588197+0000","last_became_peered":"2024-09-10T11:34:24.588197+0000","last_unstale":"2024-09-10T11:34:26.492571+0000","last_undegraded":"2024-09-10T11:34:26.492571+0000","last_fullsized":"2024-09-10T11:34:26.492571+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:38.979273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492558+0000","last_change":"2024-09-10T11:34:24.585431+0000","last_active":"2024-09-10T11:34:26.492558+0000","last_peered":"2024-09-10T11:34:26.492558+0000","last_clean":"2024-09-10T11:34:26.492558+0000","last_became_active":"2024-09-10T11:34:24.584866+0000","last_became_peered":"2024-09-10T11:34:24.584866+0000","last_unstale":"2024-09-10T11:34:26.492558+0000","last_undegraded":"2024-09-10T11:34:26.492558+0000","last_fullsized":"2024-09-10T11:34:26.492558+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:19:51.799864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576572+0000","last_change":"2024-09-10T11:34:24.584025+0000","last_active":"2024-09-10T11:34:25.576572+0000","last_peered":"2024-09-10T11:34:25.576572+0000","last_clean":"2024-09-10T11:34:25.576572+0000","last_became_active":"2024-09-10T11:34:24.583545+0000","last_became_peered":"2024-09-10T11:34:24.583545+0000","last_unstale":"2024-09-10T11:34:25.576572+0000","last_undegraded":"2024-09-10T11:34:25.576572+0000","last_fullsized":"2024-09-10T11:34:25.576572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:20.303092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212970+0000","last_change":"2024-09-10T11:34:24.589627+0000","last_active":"2024-09-10T11:34:26.212970+0000","last_peered":"2024-09-10T11:34:26.212970+0000","last_clean":"2024-09-10T11:34:26.212970+0000","last_became_active":"2024-09-10T11:34:24.589379+0000","last_became_peered":"2024-09-10T11:34:24.589379+0000","last_unstale":"2024-09-10T11:34:26.212970+0000","last_undegraded":"2024-09-10T11:34:26.212970+0000","last_fullsized":"2024-09-10T11:34:26.212970+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:44.241164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213276+0000","last_change":"2024-09-10T11:34:24.588104+0000","last_active":"2024-09-10T11:34:26.213276+0000","last_peered":"2024-09-10T11:34:26.213276+0000","last_clean":"2024-09-10T11:34:26.213276+0000","last_became_active":"2024-09-10T11:34:24.587851+0000","last_became_peered":"2024-09-10T11:34:24.587851+0000","last_unstale":"2024-09-10T11:34:26.213276+0000","last_undegraded":"2024-09-10T11:34:26.213276+0000","last_fullsized":"2024-09-10T11:34:26.213276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:05.283370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492508+0000","last_change":"2024-09-10T11:34:24.587794+0000","last_active":"2024-09-10T11:34:26.492508+0000","last_peered":"2024-09-10T11:34:26.492508+0000","last_clean":"2024-09-10T11:34:26.492508+0000","last_became_active":"2024-09-10T11:34:24.586812+0000","last_became_peered":"2024-09-10T11:34:24.586812+0000","last_unstale":"2024-09-10T11:34:26.492508+0000","last_undegraded":"2024-09-10T11:34:26.492508+0000","last_fullsized":"2024-09-10T11:34:26.492508+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:46:33.760645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492191+0000","last_change":"2024-09-10T11:34:24.586382+0000","last_active":"2024-09-10T11:34:26.492191+0000","last_peered":"2024-09-10T11:34:26.492191+0000","last_clean":"2024-09-10T11:34:26.492191+0000","last_became_active":"2024-09-10T11:34:24.585798+0000","last_became_peered":"2024-09-10T11:34:24.585798+0000","last_unstale":"2024-09-10T11:34:26.492191+0000","last_undegraded":"2024-09-10T11:34:26.492191+0000","last_fullsized":"2024-09-10T11:34:26.492191+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:03.796650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576673+0000","last_change":"2024-09-10T11:34:24.581388+0000","last_active":"2024-09-10T11:34:25.576673+0000","last_peered":"2024-09-10T11:34:25.576673+0000","last_clean":"2024-09-10T11:34:25.576673+0000","last_became_active":"2024-09-10T11:34:24.581135+0000","last_became_peered":"2024-09-10T11:34:24.581135+0000","last_unstale":"2024-09-10T11:34:25.576673+0000","last_undegraded":"2024-09-10T11:34:25.576673+0000","last_fullsized":"2024-09-10T11:34:25.576673+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:37.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.493105+0000","last_change":"2024-09-10T11:34:23.607410+0000","last_active":"2024-09-10T11:34:26.493105+0000","last_peered":"2024-09-10T11:34:26.493105+0000","last_clean":"2024-09-10T11:34:26.493105+0000","last_became_active":"2024-09-10T11:34:23.606877+0000","last_became_peered":"2024-09-10T11:34:23.606877+0000","last_unstale":"2024-09-10T11:34:26.493105+0000","last_undegraded":"2024-09-10T11:34:26.493105+0000","last_fullsized":"2024-09-10T11:34:26.493105+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:29.403856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576714+0000","last_change":"2024-09-10T11:34:24.583831+0000","last_active":"2024-09-10T11:34:25.576714+0000","last_peered":"2024-09-10T11:34:25.576714+0000","last_clean":"2024-09-10T11:34:25.576714+0000","last_became_active":"2024-09-10T11:34:24.583224+0000","last_became_peered":"2024-09-10T11:34:24.583224+0000","last_unstale":"2024-09-10T11:34:25.576714+0000","last_undegraded":"2024-09-10T11:34:25.576714+0000","last_fullsized":"2024-09-10T11:34:25.576714+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:45.194039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492280+0000","last_change":"2024-09-10T11:34:23.600685+0000","last_active":"2024-09-10T11:34:26.492280+0000","last_peered":"2024-09-10T11:34:26.492280+0000","last_clean":"2024-09-10T11:34:26.492280+0000","last_became_active":"2024-09-10T11:34:23.600433+0000","last_became_peered":"2024-09-10T11:34:23.600433+0000","last_unstale":"2024-09-10T11:34:26.492280+0000","last_undegraded":"2024-09-10T11:34:26.492280+0000","last_fullsized":"2024-09-10T11:34:26.492280+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:26.301084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577294+0000","last_change":"2024-09-10T11:34:24.586442+0000","last_active":"2024-09-10T11:34:25.577294+0000","last_peered":"2024-09-10T11:34:25.577294+0000","last_clean":"2024-09-10T11:34:25.577294+0000","last_became_active":"2024-09-10T11:34:24.586077+0000","last_became_peered":"2024-09-10T11:34:24.586077+0000","last_unstale":"2024-09-10T11:34:25.577294+0000","last_undegraded":"2024-09-10T11:34:25.577294+0000","last_fullsized":"2024-09-10T11:34:25.577294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:13.618383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576763+0000","last_change":"2024-09-10T11:34:23.601612+0000","last_active":"2024-09-10T11:34:25.576763+0000","last_peered":"2024-09-10T11:34:25.576763+0000","last_clean":"2024-09-10T11:34:25.576763+0000","last_became_active":"2024-09-10T11:34:23.601356+0000","last_became_peered":"2024-09-10T11:34:23.601356+0000","last_unstale":"2024-09-10T11:34:25.576763+0000","last_undegraded":"2024-09-10T11:34:25.576763+0000","last_fullsized":"2024-09-10T11:34:25.576763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:08:46.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576824+0000","last_change":"2024-09-10T11:34:24.580441+0000","last_active":"2024-09-10T11:34:25.576824+0000","last_peered":"2024-09-10T11:34:25.576824+0000","last_clean":"2024-09-10T11:34:25.576824+0000","last_became_active":"2024-09-10T11:34:24.579943+0000","last_became_peered":"2024-09-10T11:34:24.579943+0000","last_unstale":"2024-09-10T11:34:25.576824+0000","last_undegraded":"2024-09-10T11:34:25.576824+0000","last_fullsized":"2024-09-10T11:34:25.576824+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:19.471823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213297+0000","last_change":"2024-09-10T11:34:23.603868+0000","last_active":"2024-09-10T11:34:26.213297+0000","last_peered":"2024-09-10T11:34:26.213297+0000","last_clean":"2024-09-10T11:34:26.213297+0000","last_became_active":"2024-09-10T11:34:23.603620+0000","last_became_peered":"2024-09-10T11:34:23.603620+0000","last_unstale":"2024-09-10T11:34:26.213297+0000","last_undegraded":"2024-09-10T11:34:26.213297+0000","last_fullsized":"2024-09-10T11:34:26.213297+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:47:24.624623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212723+0000","last_change":"2024-09-10T11:34:24.588426+0000","last_active":"2024-09-10T11:34:26.212723+0000","last_peered":"2024-09-10T11:34:26.212723+0000","last_clean":"2024-09-10T11:34:26.212723+0000","last_became_active":"2024-09-10T11:34:24.588163+0000","last_became_peered":"2024-09-10T11:34:24.588163+0000","last_unstale":"2024-09-10T11:34:26.212723+0000","last_undegraded":"2024-09-10T11:34:26.212723+0000","last_fullsized":"2024-09-10T11:34:26.212723+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:33.251352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577411+0000","last_change":"2024-09-10T11:34:23.601930+0000","last_active":"2024-09-10T11:34:25.577411+0000","last_peered":"2024-09-10T11:34:25.577411+0000","last_clean":"2024-09-10T11:34:25.577411+0000","last_became_active":"2024-09-10T11:34:23.601703+0000","last_became_peered":"2024-09-10T11:34:23.601703+0000","last_unstale":"2024-09-10T11:34:25.577411+0000","last_undegraded":"2024-09-10T11:34:25.577411+0000","last_fullsized":"2024-09-10T11:34:25.577411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:35.739429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492028+0000","last_change":"2024-09-10T11:34:24.586969+0000","last_active":"2024-09-10T11:34:26.492028+0000","last_peered":"2024-09-10T11:34:26.492028+0000","last_clean":"2024-09-10T11:34:26.492028+0000","last_became_active":"2024-09-10T11:34:24.586681+0000","last_became_peered":"2024-09-10T11:34:24.586681+0000","last_unstale":"2024-09-10T11:34:26.492028+0000","last_undegraded":"2024-09-10T11:34:26.492028+0000","last_fullsized":"2024-09-10T11:34:26.492028+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:05.019855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492154+0000","last_change":"2024-09-10T11:34:23.604097+0000","last_active":"2024-09-10T11:34:26.492154+0000","last_peered":"2024-09-10T11:34:26.492154+0000","last_clean":"2024-09-10T11:34:26.492154+0000","last_became_active":"2024-09-10T11:34:23.603848+0000","last_became_peered":"2024-09-10T11:34:23.603848+0000","last_unstale":"2024-09-10T11:34:26.492154+0000","last_undegraded":"2024-09-10T11:34:26.492154+0000","last_fullsized":"2024-09-10T11:34:26.492154+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:24.476679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576937+0000","last_change":"2024-09-10T11:34:24.580624+0000","last_active":"2024-09-10T11:34:25.576937+0000","last_peered":"2024-09-10T11:34:25.576937+0000","last_clean":"2024-09-10T11:34:25.576937+0000","last_became_active":"2024-09-10T11:34:24.580231+0000","last_became_peered":"2024-09-10T11:34:24.580231+0000","last_unstale":"2024-09-10T11:34:25.576937+0000","last_undegraded":"2024-09-10T11:34:25.576937+0000","last_fullsized":"2024-09-10T11:34:25.576937+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:30.207889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213660+0000","last_change":"2024-09-10T11:34:23.606119+0000","last_active":"2024-09-10T11:34:26.213660+0000","last_peered":"2024-09-10T11:34:26.213660+0000","last_clean":"2024-09-10T11:34:26.213660+0000","last_became_active":"2024-09-10T11:34:23.605898+0000","last_became_peered":"2024-09-10T11:34:23.605898+0000","last_unstale":"2024-09-10T11:34:26.213660+0000","last_undegraded":"2024-09-10T11:34:26.213660+0000","last_fullsized":"2024-09-10T11:34:26.213660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:21.621299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213449+0000","last_change":"2024-09-10T11:34:24.588905+0000","last_active":"2024-09-10T11:34:26.213449+0000","last_peered":"2024-09-10T11:34:26.213449+0000","last_clean":"2024-09-10T11:34:26.213449+0000","last_became_active":"2024-09-10T11:34:24.585583+0000","last_became_peered":"2024-09-10T11:34:24.585583+0000","last_unstale":"2024-09-10T11:34:26.213449+0000","last_undegraded":"2024-09-10T11:34:26.213449+0000","last_fullsized":"2024-09-10T11:34:26.213449+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:36:51.832758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213452+0000","last_change":"2024-09-10T11:34:23.603835+0000","last_active":"2024-09-10T11:34:26.213452+0000","last_peered":"2024-09-10T11:34:26.213452+0000","last_clean":"2024-09-10T11:34:26.213452+0000","last_became_active":"2024-09-10T11:34:23.603420+0000","last_became_peered":"2024-09-10T11:34:23.603420+0000","last_unstale":"2024-09-10T11:34:26.213452+0000","last_undegraded":"2024-09-10T11:34:26.213452+0000","last_fullsized":"2024-09-10T11:34:26.213452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:02.707677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213479+0000","last_change":"2024-09-10T11:34:24.585980+0000","last_active":"2024-09-10T11:34:26.213479+0000","last_peered":"2024-09-10T11:34:26.213479+0000","last_clean":"2024-09-10T11:34:26.213479+0000","last_became_active":"2024-09-10T11:34:24.584344+0000","last_became_peered":"2024-09-10T11:34:24.584344+0000","last_unstale":"2024-09-10T11:34:26.213479+0000","last_undegraded":"2024-09-10T11:34:26.213479+0000","last_fullsized":"2024-09-10T11:34:26.213479+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:25:03.535465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576880+0000","last_change":"2024-09-10T11:34:23.601301+0000","last_active":"2024-09-10T11:34:25.576880+0000","last_peered":"2024-09-10T11:34:25.576880+0000","last_clean":"2024-09-10T11:34:25.576880+0000","last_became_active":"2024-09-10T11:34:23.601068+0000","last_became_peered":"2024-09-10T11:34:23.601068+0000","last_unstale":"2024-09-10T11:34:25.576880+0000","last_undegraded":"2024-09-10T11:34:25.576880+0000","last_fullsized":"2024-09-10T11:34:25.576880+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:23.727005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576655+0000","last_change":"2024-09-10T11:34:24.578058+0000","last_active":"2024-09-10T11:34:25.576655+0000","last_peered":"2024-09-10T11:34:25.576655+0000","last_clean":"2024-09-10T11:34:25.576655+0000","last_became_active":"2024-09-10T11:34:24.577794+0000","last_became_peered":"2024-09-10T11:34:24.577794+0000","last_unstale":"2024-09-10T11:34:25.576655+0000","last_undegraded":"2024-09-10T11:34:25.576655+0000","last_fullsized":"2024-09-10T11:34:25.576655+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.890958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492355+0000","last_change":"2024-09-10T11:34:23.608036+0000","last_active":"2024-09-10T11:34:26.492355+0000","last_peered":"2024-09-10T11:34:26.492355+0000","last_clean":"2024-09-10T11:34:26.492355+0000","last_became_active":"2024-09-10T11:34:23.607371+0000","last_became_peered":"2024-09-10T11:34:23.607371+0000","last_unstale":"2024-09-10T11:34:26.492355+0000","last_undegraded":"2024-09-10T11:34:26.492355+0000","last_fullsized":"2024-09-10T11:34:26.492355+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:18.960115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576746+0000","last_change":"2024-09-10T11:34:24.583788+0000","last_active":"2024-09-10T11:34:25.576746+0000","last_peered":"2024-09-10T11:34:25.576746+0000","last_clean":"2024-09-10T11:34:25.576746+0000","last_became_active":"2024-09-10T11:34:24.583504+0000","last_became_peered":"2024-09-10T11:34:24.583504+0000","last_unstale":"2024-09-10T11:34:25.576746+0000","last_undegraded":"2024-09-10T11:34:25.576746+0000","last_fullsized":"2024-09-10T11:34:25.576746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:40.364999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576952+0000","last_change":"2024-09-10T11:34:23.607971+0000","last_active":"2024-09-10T11:34:25.576952+0000","last_peered":"2024-09-10T11:34:25.576952+0000","last_clean":"2024-09-10T11:34:25.576952+0000","last_became_active":"2024-09-10T11:34:23.607431+0000","last_became_peered":"2024-09-10T11:34:23.607431+0000","last_unstale":"2024-09-10T11:34:25.576952+0000","last_undegraded":"2024-09-10T11:34:25.576952+0000","last_fullsized":"2024-09-10T11:34:25.576952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:29.999547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492756+0000","last_change":"2024-09-10T11:34:24.588470+0000","last_active":"2024-09-10T11:34:26.492756+0000","last_peered":"2024-09-10T11:34:26.492756+0000","last_clean":"2024-09-10T11:34:26.492756+0000","last_became_active":"2024-09-10T11:34:24.588201+0000","last_became_peered":"2024-09-10T11:34:24.588201+0000","last_unstale":"2024-09-10T11:34:26.492756+0000","last_undegraded":"2024-09-10T11:34:26.492756+0000","last_fullsized":"2024-09-10T11:34:26.492756+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:44.887495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576867+0000","last_change":"2024-09-10T11:34:23.607510+0000","last_active":"2024-09-10T11:34:25.576867+0000","last_peered":"2024-09-10T11:34:25.576867+0000","last_clean":"2024-09-10T11:34:25.576867+0000","last_became_active":"2024-09-10T11:34:23.607277+0000","last_became_peered":"2024-09-10T11:34:23.607277+0000","last_unstale":"2024-09-10T11:34:25.576867+0000","last_undegraded":"2024-09-10T11:34:25.576867+0000","last_fullsized":"2024-09-10T11:34:25.576867+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:55.433412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576491+0000","last_change":"2024-09-10T11:34:24.578411+0000","last_active":"2024-09-10T11:34:25.576491+0000","last_peered":"2024-09-10T11:34:25.576491+0000","last_clean":"2024-09-10T11:34:25.576491+0000","last_became_active":"2024-09-10T11:34:24.578123+0000","last_became_peered":"2024-09-10T11:34:24.578123+0000","last_unstale":"2024-09-10T11:34:25.576491+0000","last_undegraded":"2024-09-10T11:34:25.576491+0000","last_fullsized":"2024-09-10T11:34:25.576491+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:19.225665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.212970+0000","last_change":"2024-09-10T11:34:23.606581+0000","last_active":"2024-09-10T11:34:26.212970+0000","last_peered":"2024-09-10T11:34:26.212970+0000","last_clean":"2024-09-10T11:34:26.212970+0000","last_became_active":"2024-09-10T11:34:23.606302+0000","last_became_peered":"2024-09-10T11:34:23.606302+0000","last_unstale":"2024-09-10T11:34:26.212970+0000","last_undegraded":"2024-09-10T11:34:26.212970+0000","last_fullsized":"2024-09-10T11:34:26.212970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:31:49.084821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577127+0000","last_change":"2024-09-10T11:34:24.581276+0000","last_active":"2024-09-10T11:34:25.577127+0000","last_peered":"2024-09-10T11:34:25.577127+0000","last_clean":"2024-09-10T11:34:25.577127+0000","last_became_active":"2024-09-10T11:34:24.581111+0000","last_became_peered":"2024-09-10T11:34:24.581111+0000","last_unstale":"2024-09-10T11:34:25.577127+0000","last_undegraded":"2024-09-10T11:34:25.577127+0000","last_fullsized":"2024-09-10T11:34:25.577127+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:16.731228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492268+0000","last_change":"2024-09-10T11:34:23.600717+0000","last_active":"2024-09-10T11:34:26.492268+0000","last_peered":"2024-09-10T11:34:26.492268+0000","last_clean":"2024-09-10T11:34:26.492268+0000","last_became_active":"2024-09-10T11:34:23.600406+0000","last_became_peered":"2024-09-10T11:34:23.600406+0000","last_unstale":"2024-09-10T11:34:26.492268+0000","last_undegraded":"2024-09-10T11:34:26.492268+0000","last_fullsized":"2024-09-10T11:34:26.492268+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:07.359175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213055+0000","last_change":"2024-09-10T11:34:24.588591+0000","last_active":"2024-09-10T11:34:26.213055+0000","last_peered":"2024-09-10T11:34:26.213055+0000","last_clean":"2024-09-10T11:34:26.213055+0000","last_became_active":"2024-09-10T11:34:24.587124+0000","last_became_peered":"2024-09-10T11:34:24.587124+0000","last_unstale":"2024-09-10T11:34:26.213055+0000","last_undegraded":"2024-09-10T11:34:26.213055+0000","last_fullsized":"2024-09-10T11:34:26.213055+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:30.062420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492445+0000","last_change":"2024-09-10T11:34:23.607644+0000","last_active":"2024-09-10T11:34:26.492445+0000","last_peered":"2024-09-10T11:34:26.492445+0000","last_clean":"2024-09-10T11:34:26.492445+0000","last_became_active":"2024-09-10T11:34:23.605127+0000","last_became_peered":"2024-09-10T11:34:23.605127+0000","last_unstale":"2024-09-10T11:34:26.492445+0000","last_undegraded":"2024-09-10T11:34:26.492445+0000","last_fullsized":"2024-09-10T11:34:26.492445+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:46.582048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576406+0000","last_change":"2024-09-10T11:34:24.586861+0000","last_active":"2024-09-10T11:34:25.576406+0000","last_peered":"2024-09-10T11:34:25.576406+0000","last_clean":"2024-09-10T11:34:25.576406+0000","last_became_active":"2024-09-10T11:34:24.586611+0000","last_became_peered":"2024-09-10T11:34:24.586611+0000","last_unstale":"2024-09-10T11:34:25.576406+0000","last_undegraded":"2024-09-10T11:34:25.576406+0000","last_fullsized":"2024-09-10T11:34:25.576406+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:50.126003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576820+0000","last_change":"2024-09-10T11:34:23.607623+0000","last_active":"2024-09-10T11:34:25.576820+0000","last_peered":"2024-09-10T11:34:25.576820+0000","last_clean":"2024-09-10T11:34:25.576820+0000","last_became_active":"2024-09-10T11:34:23.607376+0000","last_became_peered":"2024-09-10T11:34:23.607376+0000","last_unstale":"2024-09-10T11:34:25.576820+0000","last_undegraded":"2024-09-10T11:34:25.576820+0000","last_fullsized":"2024-09-10T11:34:25.576820+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:29.507835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577165+0000","last_change":"2024-09-10T11:34:24.578393+0000","last_active":"2024-09-10T11:34:25.577165+0000","last_peered":"2024-09-10T11:34:25.577165+0000","last_clean":"2024-09-10T11:34:25.577165+0000","last_became_active":"2024-09-10T11:34:24.578191+0000","last_became_peered":"2024-09-10T11:34:24.578191+0000","last_unstale":"2024-09-10T11:34:25.577165+0000","last_undegraded":"2024-09-10T11:34:25.577165+0000","last_fullsized":"2024-09-10T11:34:25.577165+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:52.236893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492402+0000","last_change":"2024-09-10T11:34:23.605060+0000","last_active":"2024-09-10T11:34:26.492402+0000","last_peered":"2024-09-10T11:34:26.492402+0000","last_clean":"2024-09-10T11:34:26.492402+0000","last_became_active":"2024-09-10T11:34:23.604788+0000","last_became_peered":"2024-09-10T11:34:23.604788+0000","last_unstale":"2024-09-10T11:34:26.492402+0000","last_undegraded":"2024-09-10T11:34:26.492402+0000","last_fullsized":"2024-09-10T11:34:26.492402+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:18.339996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492015+0000","last_change":"2024-09-10T11:34:24.587007+0000","last_active":"2024-09-10T11:34:26.492015+0000","last_peered":"2024-09-10T11:34:26.492015+0000","last_clean":"2024-09-10T11:34:26.492015+0000","last_became_active":"2024-09-10T11:34:24.586441+0000","last_became_peered":"2024-09-10T11:34:24.586441+0000","last_unstale":"2024-09-10T11:34:26.492015+0000","last_undegraded":"2024-09-10T11:34:26.492015+0000","last_fullsized":"2024-09-10T11:34:26.492015+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:11.725799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213587+0000","last_change":"2024-09-10T11:34:23.602635+0000","last_active":"2024-09-10T11:34:26.213587+0000","last_peered":"2024-09-10T11:34:26.213587+0000","last_clean":"2024-09-10T11:34:26.213587+0000","last_became_active":"2024-09-10T11:34:23.602353+0000","last_became_peered":"2024-09-10T11:34:23.602353+0000","last_unstale":"2024-09-10T11:34:26.213587+0000","last_undegraded":"2024-09-10T11:34:26.213587+0000","last_fullsized":"2024-09-10T11:34:26.213587+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:08.548995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577188+0000","last_change":"2024-09-10T11:34:24.578586+0000","last_active":"2024-09-10T11:34:25.577188+0000","last_peered":"2024-09-10T11:34:25.577188+0000","last_clean":"2024-09-10T11:34:25.577188+0000","last_became_active":"2024-09-10T11:34:24.578265+0000","last_became_peered":"2024-09-10T11:34:24.578265+0000","last_unstale":"2024-09-10T11:34:25.577188+0000","last_undegraded":"2024-09-10T11:34:25.577188+0000","last_fullsized":"2024-09-10T11:34:25.577188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:06.590310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213665+0000","last_change":"2024-09-10T11:34:23.601885+0000","last_active":"2024-09-10T11:34:26.213665+0000","last_peered":"2024-09-10T11:34:26.213665+0000","last_clean":"2024-09-10T11:34:26.213665+0000","last_became_active":"2024-09-10T11:34:23.601657+0000","last_became_peered":"2024-09-10T11:34:23.601657+0000","last_unstale":"2024-09-10T11:34:26.213665+0000","last_undegraded":"2024-09-10T11:34:26.213665+0000","last_fullsized":"2024-09-10T11:34:26.213665+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:42.356871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492252+0000","last_change":"2024-09-10T11:34:24.588620+0000","last_active":"2024-09-10T11:34:26.492252+0000","last_peered":"2024-09-10T11:34:26.492252+0000","last_clean":"2024-09-10T11:34:26.492252+0000","last_became_active":"2024-09-10T11:34:24.587205+0000","last_became_peered":"2024-09-10T11:34:24.587205+0000","last_unstale":"2024-09-10T11:34:26.492252+0000","last_undegraded":"2024-09-10T11:34:26.492252+0000","last_fullsized":"2024-09-10T11:34:26.492252+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:19.666210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213168+0000","last_change":"2024-09-10T11:34:23.603320+0000","last_active":"2024-09-10T11:34:26.213168+0000","last_peered":"2024-09-10T11:34:26.213168+0000","last_clean":"2024-09-10T11:34:26.213168+0000","last_became_active":"2024-09-10T11:34:23.603068+0000","last_became_peered":"2024-09-10T11:34:23.603068+0000","last_unstale":"2024-09-10T11:34:26.213168+0000","last_undegraded":"2024-09-10T11:34:26.213168+0000","last_fullsized":"2024-09-10T11:34:26.213168+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:09.307907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.493010+0000","last_change":"2024-09-10T11:34:24.587370+0000","last_active":"2024-09-10T11:34:26.493010+0000","last_peered":"2024-09-10T11:34:26.493010+0000","last_clean":"2024-09-10T11:34:26.493010+0000","last_became_active":"2024-09-10T11:34:24.587157+0000","last_became_peered":"2024-09-10T11:34:24.587157+0000","last_unstale":"2024-09-10T11:34:26.493010+0000","last_undegraded":"2024-09-10T11:34:26.493010+0000","last_fullsized":"2024-09-10T11:34:26.493010+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:29:56.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492443+0000","last_change":"2024-09-10T11:34:23.602978+0000","last_active":"2024-09-10T11:34:26.492443+0000","last_peered":"2024-09-10T11:34:26.492443+0000","last_clean":"2024-09-10T11:34:26.492443+0000","last_became_active":"2024-09-10T11:34:23.602700+0000","last_became_peered":"2024-09-10T11:34:23.602700+0000","last_unstale":"2024-09-10T11:34:26.492443+0000","last_undegraded":"2024-09-10T11:34:26.492443+0000","last_fullsized":"2024-09-10T11:34:26.492443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:05.451285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213045+0000","last_change":"2024-09-10T11:34:24.583236+0000","last_active":"2024-09-10T11:34:26.213045+0000","last_peered":"2024-09-10T11:34:26.213045+0000","last_clean":"2024-09-10T11:34:26.213045+0000","last_became_active":"2024-09-10T11:34:24.582998+0000","last_became_peered":"2024-09-10T11:34:24.582998+0000","last_unstale":"2024-09-10T11:34:26.213045+0000","last_undegraded":"2024-09-10T11:34:26.213045+0000","last_fullsized":"2024-09-10T11:34:26.213045+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:39.938728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576840+0000","last_change":"2024-09-10T11:34:23.603968+0000","last_active":"2024-09-10T11:34:25.576840+0000","last_peered":"2024-09-10T11:34:25.576840+0000","last_clean":"2024-09-10T11:34:25.576840+0000","last_became_active":"2024-09-10T11:34:23.603479+0000","last_became_peered":"2024-09-10T11:34:23.603479+0000","last_unstale":"2024-09-10T11:34:25.576840+0000","last_undegraded":"2024-09-10T11:34:25.576840+0000","last_fullsized":"2024-09-10T11:34:25.576840+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:01.063801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577042+0000","last_change":"2024-09-10T11:34:24.585738+0000","last_active":"2024-09-10T11:34:25.577042+0000","last_peered":"2024-09-10T11:34:25.577042+0000","last_clean":"2024-09-10T11:34:25.577042+0000","last_became_active":"2024-09-10T11:34:24.585517+0000","last_became_peered":"2024-09-10T11:34:24.585517+0000","last_unstale":"2024-09-10T11:34:25.577042+0000","last_undegraded":"2024-09-10T11:34:25.577042+0000","last_fullsized":"2024-09-10T11:34:25.577042+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:10:18.002678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213046+0000","last_change":"2024-09-10T11:34:23.604116+0000","last_active":"2024-09-10T11:34:26.213046+0000","last_peered":"2024-09-10T11:34:26.213046+0000","last_clean":"2024-09-10T11:34:26.213046+0000","last_became_active":"2024-09-10T11:34:23.603849+0000","last_became_peered":"2024-09-10T11:34:23.603849+0000","last_unstale":"2024-09-10T11:34:26.213046+0000","last_undegraded":"2024-09-10T11:34:26.213046+0000","last_fullsized":"2024-09-10T11:34:26.213046+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:23.415771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492888+0000","last_change":"2024-09-10T11:34:24.584486+0000","last_active":"2024-09-10T11:34:26.492888+0000","last_peered":"2024-09-10T11:34:26.492888+0000","last_clean":"2024-09-10T11:34:26.492888+0000","last_became_active":"2024-09-10T11:34:24.584260+0000","last_became_peered":"2024-09-10T11:34:24.584260+0000","last_unstale":"2024-09-10T11:34:26.492888+0000","last_undegraded":"2024-09-10T11:34:26.492888+0000","last_fullsized":"2024-09-10T11:34:26.492888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:19:22.464319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.213721+0000","last_change":"2024-09-10T11:34:23.605000+0000","last_active":"2024-09-10T11:34:26.213721+0000","last_peered":"2024-09-10T11:34:26.213721+0000","last_clean":"2024-09-10T11:34:26.213721+0000","last_became_active":"2024-09-10T11:34:23.604652+0000","last_became_peered":"2024-09-10T11:34:23.604652+0000","last_unstale":"2024-09-10T11:34:26.213721+0000","last_undegraded":"2024-09-10T11:34:26.213721+0000","last_fullsized":"2024-09-10T11:34:26.213721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:05.588865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492703+0000","last_change":"2024-09-10T11:34:24.582510+0000","last_active":"2024-09-10T11:34:26.492703+0000","last_peered":"2024-09-10T11:34:26.492703+0000","last_clean":"2024-09-10T11:34:26.492703+0000","last_became_active":"2024-09-10T11:34:24.582231+0000","last_became_peered":"2024-09-10T11:34:24.582231+0000","last_unstale":"2024-09-10T11:34:26.492703+0000","last_undegraded":"2024-09-10T11:34:26.492703+0000","last_fullsized":"2024-09-10T11:34:26.492703+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:30.945082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.598556+0000","last_change":"2024-09-10T11:34:23.606790+0000","last_active":"2024-09-10T11:34:26.598556+0000","last_peered":"2024-09-10T11:34:26.598556+0000","last_clean":"2024-09-10T11:34:26.598556+0000","last_became_active":"2024-09-10T11:34:23.606309+0000","last_became_peered":"2024-09-10T11:34:23.606309+0000","last_unstale":"2024-09-10T11:34:26.598556+0000","last_undegraded":"2024-09-10T11:34:26.598556+0000","last_fullsized":"2024-09-10T11:34:26.598556+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:44.009394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492835+0000","last_change":"2024-09-10T11:34:24.588090+0000","last_active":"2024-09-10T11:34:26.492835+0000","last_peered":"2024-09-10T11:34:26.492835+0000","last_clean":"2024-09-10T11:34:26.492835+0000","last_became_active":"2024-09-10T11:34:24.587851+0000","last_became_peered":"2024-09-10T11:34:24.587851+0000","last_unstale":"2024-09-10T11:34:26.492835+0000","last_undegraded":"2024-09-10T11:34:26.492835+0000","last_fullsized":"2024-09-10T11:34:26.492835+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:46.293799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576948+0000","last_change":"2024-09-10T11:34:23.606318+0000","last_active":"2024-09-10T11:34:25.576948+0000","last_peered":"2024-09-10T11:34:25.576948+0000","last_clean":"2024-09-10T11:34:25.576948+0000","last_became_active":"2024-09-10T11:34:23.606086+0000","last_became_peered":"2024-09-10T11:34:23.606086+0000","last_unstale":"2024-09-10T11:34:25.576948+0000","last_undegraded":"2024-09-10T11:34:25.576948+0000","last_fullsized":"2024-09-10T11:34:25.576948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:24.883635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492180+0000","last_change":"2024-09-10T11:34:24.583152+0000","last_active":"2024-09-10T11:34:26.492180+0000","last_peered":"2024-09-10T11:34:26.492180+0000","last_clean":"2024-09-10T11:34:26.492180+0000","last_became_active":"2024-09-10T11:34:24.582840+0000","last_became_peered":"2024-09-10T11:34:24.582840+0000","last_unstale":"2024-09-10T11:34:26.492180+0000","last_undegraded":"2024-09-10T11:34:26.492180+0000","last_fullsized":"2024-09-10T11:34:26.492180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:07.033562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492350+0000","last_change":"2024-09-10T11:34:23.602999+0000","last_active":"2024-09-10T11:34:26.492350+0000","last_peered":"2024-09-10T11:34:26.492350+0000","last_clean":"2024-09-10T11:34:26.492350+0000","last_became_active":"2024-09-10T11:34:23.602551+0000","last_became_peered":"2024-09-10T11:34:23.602551+0000","last_unstale":"2024-09-10T11:34:26.492350+0000","last_undegraded":"2024-09-10T11:34:26.492350+0000","last_fullsized":"2024-09-10T11:34:26.492350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:03.869297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576633+0000","last_change":"2024-09-10T11:34:24.587198+0000","last_active":"2024-09-10T11:34:25.576633+0000","last_peered":"2024-09-10T11:34:25.576633+0000","last_clean":"2024-09-10T11:34:25.576633+0000","last_became_active":"2024-09-10T11:34:24.586882+0000","last_became_peered":"2024-09-10T11:34:24.586882+0000","last_unstale":"2024-09-10T11:34:25.576633+0000","last_undegraded":"2024-09-10T11:34:25.576633+0000","last_fullsized":"2024-09-10T11:34:25.576633+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:27.103552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492354+0000","last_change":"2024-09-10T11:34:23.605133+0000","last_active":"2024-09-10T11:34:26.492354+0000","last_peered":"2024-09-10T11:34:26.492354+0000","last_clean":"2024-09-10T11:34:26.492354+0000","last_became_active":"2024-09-10T11:34:23.604890+0000","last_became_peered":"2024-09-10T11:34:23.604890+0000","last_unstale":"2024-09-10T11:34:26.492354+0000","last_undegraded":"2024-09-10T11:34:26.492354+0000","last_fullsized":"2024-09-10T11:34:26.492354+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:19.279117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576708+0000","last_change":"2024-09-10T11:34:24.585969+0000","last_active":"2024-09-10T11:34:25.576708+0000","last_peered":"2024-09-10T11:34:25.576708+0000","last_clean":"2024-09-10T11:34:25.576708+0000","last_became_active":"2024-09-10T11:34:24.585465+0000","last_became_peered":"2024-09-10T11:34:24.585465+0000","last_unstale":"2024-09-10T11:34:25.576708+0000","last_undegraded":"2024-09-10T11:34:25.576708+0000","last_fullsized":"2024-09-10T11:34:25.576708+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:54.574404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492545+0000","last_change":"2024-09-10T11:34:23.607846+0000","last_active":"2024-09-10T11:34:26.492545+0000","last_peered":"2024-09-10T11:34:26.492545+0000","last_clean":"2024-09-10T11:34:26.492545+0000","last_became_active":"2024-09-10T11:34:23.606912+0000","last_became_peered":"2024-09-10T11:34:23.606912+0000","last_unstale":"2024-09-10T11:34:26.492545+0000","last_undegraded":"2024-09-10T11:34:26.492545+0000","last_fullsized":"2024-09-10T11:34:26.492545+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:08.282111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577151+0000","last_change":"2024-09-10T11:34:24.583576+0000","last_active":"2024-09-10T11:34:25.577151+0000","last_peered":"2024-09-10T11:34:25.577151+0000","last_clean":"2024-09-10T11:34:25.577151+0000","last_became_active":"2024-09-10T11:34:24.583331+0000","last_became_peered":"2024-09-10T11:34:24.583331+0000","last_unstale":"2024-09-10T11:34:25.577151+0000","last_undegraded":"2024-09-10T11:34:25.577151+0000","last_fullsized":"2024-09-10T11:34:25.577151+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:57.369960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577039+0000","last_change":"2024-09-10T11:34:23.607717+0000","last_active":"2024-09-10T11:34:25.577039+0000","last_peered":"2024-09-10T11:34:25.577039+0000","last_clean":"2024-09-10T11:34:25.577039+0000","last_became_active":"2024-09-10T11:34:23.607095+0000","last_became_peered":"2024-09-10T11:34:23.607095+0000","last_unstale":"2024-09-10T11:34:25.577039+0000","last_undegraded":"2024-09-10T11:34:25.577039+0000","last_fullsized":"2024-09-10T11:34:25.577039+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:06.822644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576853+0000","last_change":"2024-09-10T11:34:24.587129+0000","last_active":"2024-09-10T11:34:25.576853+0000","last_peered":"2024-09-10T11:34:25.576853+0000","last_clean":"2024-09-10T11:34:25.576853+0000","last_became_active":"2024-09-10T11:34:24.586905+0000","last_became_peered":"2024-09-10T11:34:24.586905+0000","last_unstale":"2024-09-10T11:34:25.576853+0000","last_undegraded":"2024-09-10T11:34:25.576853+0000","last_fullsized":"2024-09-10T11:34:25.576853+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:07.968594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577304+0000","last_change":"2024-09-10T11:34:23.604364+0000","last_active":"2024-09-10T11:34:25.577304+0000","last_peered":"2024-09-10T11:34:25.577304+0000","last_clean":"2024-09-10T11:34:25.577304+0000","last_became_active":"2024-09-10T11:34:23.603996+0000","last_became_peered":"2024-09-10T11:34:23.603996+0000","last_unstale":"2024-09-10T11:34:25.577304+0000","last_undegraded":"2024-09-10T11:34:25.577304+0000","last_fullsized":"2024-09-10T11:34:25.577304+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:50.711505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.576587+0000","last_change":"2024-09-10T11:34:24.582791+0000","last_active":"2024-09-10T11:34:25.576587+0000","last_peered":"2024-09-10T11:34:25.576587+0000","last_clean":"2024-09-10T11:34:25.576587+0000","last_became_active":"2024-09-10T11:34:24.582532+0000","last_became_peered":"2024-09-10T11:34:24.582532+0000","last_unstale":"2024-09-10T11:34:25.576587+0000","last_undegraded":"2024-09-10T11:34:25.576587+0000","last_fullsized":"2024-09-10T11:34:25.576587+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:23.588559+0000","last_clean_scrub_stamp":"2024-09-10T11:34:23.588559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:24.562608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:26.492662+0000","last_change":"2024-09-10T11:34:23.599066+0000","last_active":"2024-09-10T11:34:26.492662+0000","last_peered":"2024-09-10T11:34:26.492662+0000","last_clean":"2024-09-10T11:34:26.492662+0000","last_became_active":"2024-09-10T11:34:23.597806+0000","last_became_peered":"2024-09-10T11:34:23.597806+0000","last_unstale":"2024-09-10T11:34:26.492662+0000","last_undegraded":"2024-09-10T11:34:26.492662+0000","last_fullsized":"2024-09-10T11:34:26.492662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:34:22.586535+0000","last_clean_scrub_stamp":"2024-09-10T11:34:22.586535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:35.775712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":339,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-10T11:34:25.577198+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:34:25.577198+0000","last_peered":"2024-09-10T11:34:25.577198+0000","last_clean":"2024-09-10T11:34:25.577198+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:34:25.577198+0000","last_undegraded":"2024-09-10T11:34:25.577198+0000","last_fullsized":"2024-09-10T11:34:25.577198+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:19.979634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442573,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441971,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1122,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194190,"statfs":{"total":200000000,"available":198850629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574900,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442609,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194767,"statfs":{"total":200000000,"available":199441969,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":544,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194768,"statfs":{"total":200000000,"available":199442643,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1120,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194191,"statfs":{"total":200000000,"available":198852185,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194766,"statfs":{"total":200000000,"available":199441009,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:34:27.867 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:34:27.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:34:28.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.008+0000 7fc20802c640 1 Processor -- start 2024-09-10T11:34:28.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.008+0000 7fc20802c640 1 -- start start 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc20802c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20006ece0 0x7fc200073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc20802c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc20015c060 0x7fc20015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc20802c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc200152e60 con 0x7fc2000d1200 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc20802c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc2000d15e0 con 0x7fc20015c060 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc20802c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc200072680 con 0x7fc20006ece0 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc20015c060 0x7fc20015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc20015c060 0x7fc20015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34516/0 (socket says 172.21.15.99:34516) 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 -- 172.21.15.99:0/2980186673 learned_addr learned my addr 172.21.15.99:0/2980186673 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2055c7640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20006ece0 0x7fc200073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 -- 172.21.15.99:0/2980186673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20006ece0 msgr2=0x7fc200073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20006ece0 0x7fc200073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 -- 172.21.15.99:0/2980186673 >> v1:172.21.15.196:6789/0 conn(0x7fc2000d1200 legacy=0x7fc20006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.009+0000 7fc2065c9640 1 -- 172.21.15.99:0/2980186673 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc200072960 con 0x7fc20015c060 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.010+0000 7fc2065c9640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20015c060 0x7fc20015c4a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009870 tx=0x7fc1f802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d368c93dbb6cb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.010+0000 7fc204dc6640 1 -- 172.21.15.99:0/2980186673 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8037070 con 0x7fc20015c060 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.010+0000 7fc204dc6640 1 -- 172.21.15.99:0/2980186673 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f8032440 con 0x7fc20015c060 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.010+0000 7fc204dc6640 1 -- 172.21.15.99:0/2980186673 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8032810 con 0x7fc20015c060 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 -- 172.21.15.99:0/2980186673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20015c060 msgr2=0x7fc20015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20015c060 0x7fc20015c4a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009870 tx=0x7fc1f802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 -- 172.21.15.99:0/2980186673 shutdown_connections 2024-09-10T11:34:28.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20015c060 0x7fc20015c4a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 --2- 172.21.15.99:0/2980186673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20006ece0 0x7fc200073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 -- 172.21.15.99:0/2980186673 >> 172.21.15.99:0/2980186673 conn(0x7fc20006bc10 msgr2=0x7fc20006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:28.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 -- 172.21.15.99:0/2980186673 shutdown_connections 2024-09-10T11:34:28.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.011+0000 7fc20802c640 1 -- 172.21.15.99:0/2980186673 wait complete. 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.012+0000 7fc20802c640 1 Processor -- start 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.012+0000 7fc20802c640 1 -- start start 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2000d1200 0x7fc2001522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20015c060 0x7fc2000c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc20015cd90 con 0x7fc20006ece0 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc200072f20 con 0x7fc2000d1200 2024-09-10T11:34:28.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc20015c9e0 con 0x7fc20015c060 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34524/0 (socket says 172.21.15.99:34524) 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 -- 172.21.15.99:0/4218241738 learned_addr learned my addr 172.21.15.99:0/4218241738 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc2055c7640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2000d1200 0x7fc2001522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc2065c9640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20015c060 0x7fc2000c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 -- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20015c060 msgr2=0x7fc2000c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20015c060 0x7fc2000c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 -- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2000d1200 msgr2=0x7fc2001522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2000d1200 0x7fc2001522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 -- 172.21.15.99:0/4218241738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc200152830 con 0x7fc20006ece0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc205dc8640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c730 tx=0x7fc1fc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc016020 con 0x7fc20006ece0 2024-09-10T11:34:28.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1fc006d50 con 0x7fc20006ece0 2024-09-10T11:34:28.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc003bb0 con 0x7fc20006ece0 2024-09-10T11:34:28.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2000c5cf0 con 0x7fc20006ece0 2024-09-10T11:34:28.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.013+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2000afb40 con 0x7fc20006ece0 2024-09-10T11:34:28.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.014+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc200073440 con 0x7fc20006ece0 2024-09-10T11:34:28.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.015+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc1fc00a850 con 0x7fc20006ece0 2024-09-10T11:34:28.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.015+0000 7fc1f67fc640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 0x7fc1e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.016+0000 7fc2055c7640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 0x7fc1e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.016+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fc1fc08a990 con 0x7fc20006ece0 2024-09-10T11:34:28.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.016+0000 7fc2055c7640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 0x7fc1e406e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2000cfc10 tx=0x7fc1f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.020+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc1fc020020 con 0x7fc20006ece0 2024-09-10T11:34:28.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.246+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc20015c5c0 con 0x7fc20006ece0 2024-09-10T11:34:28.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.248+0000 7fc1f67fc640 1 -- 172.21.15.99:0/4218241738 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fc200152830 con 0x7fc20006ece0 2024-09-10T11:34:28.248 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:34:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:34:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:34:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:34:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:34:28.249 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 msgr2=0x7fc1e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 0x7fc1e406e2f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc2000cfc10 tx=0x7fc1f0008040 comp rx=0 tx=0).stop 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 msgr2=0x7fc2000b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c730 tx=0x7fc1fc00cbf0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 shutdown_connections 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc1e406be30 0x7fc1e406e2f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20015c060 0x7fc2000c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc2000d1200 0x7fc2001522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 --2- 172.21.15.99:0/4218241738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc20006ece0 0x7fc2000b0900 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 >> 172.21.15.99:0/4218241738 conn(0x7fc20006bc10 msgr2=0x7fc20015cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:28.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 shutdown_connections 2024-09-10T11:34:28.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.250+0000 7fc20802c640 1 -- 172.21.15.99:0/4218241738 wait complete. 2024-09-10T11:34:28.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.359+0000 7ff6a03fd640 1 Processor -- start 2024-09-10T11:34:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.359+0000 7ff6a03fd640 1 -- start start 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff6a03fd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff69815f1e0 0x7ff6980b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff6a03fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6980b0090 0x7ff6980b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff6a03fd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6980b40e0 con 0x7ff69815ee00 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff6a03fd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff6980b5ab0 con 0x7ff69815f1e0 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff6a03fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff698073c50 con 0x7ff6980b0090 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff69815f1e0 0x7ff6980b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff69815f1e0 0x7ff6980b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34532/0 (socket says 172.21.15.99:34532) 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 -- 172.21.15.99:0/3658842539 learned_addr learned my addr 172.21.15.99:0/3658842539 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69e199640 1 --1- 172.21.15.99:0/3658842539 >> v1:172.21.15.99:6789/0 conn(0x7ff69815ee00 0x7ff6980b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54200/0 (socket says 172.21.15.99:54200) 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 -- 172.21.15.99:0/3658842539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6980b0090 msgr2=0x7ff6980b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 --2- 172.21.15.99:0/3658842539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6980b0090 0x7ff6980b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 -- 172.21.15.99:0/3658842539 >> v1:172.21.15.99:6789/0 conn(0x7ff69815ee00 legacy=0x7ff6980b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d197640 1 -- 172.21.15.99:0/3658842539 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390212250 0 0) 0x7ff6980b40e0 con 0x7ff69815ee00 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.360+0000 7ff69d998640 1 -- 172.21.15.99:0/3658842539 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6980b4380 con 0x7ff69815f1e0 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff69d998640 1 --2- 172.21.15.99:0/3658842539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815f1e0 0x7ff6980b5f90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff68802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f0a9569c38f435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff69d197640 1 -- 172.21.15.99:0/3658842539 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688037070 con 0x7ff69815f1e0 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff69d197640 1 -- 172.21.15.99:0/3658842539 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff688032440 con 0x7ff69815f1e0 2024-09-10T11:34:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff69d197640 1 -- 172.21.15.99:0/3658842539 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688032810 con 0x7ff69815f1e0 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3658842539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815f1e0 msgr2=0x7ff6980b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.361+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3658842539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815f1e0 0x7ff6980b5f90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff68802a270 comp rx=0 tx=0).stop 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3658842539 shutdown_connections 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3658842539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6980b0090 0x7ff6980b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3658842539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815f1e0 0x7ff6980b5f90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3658842539 >> 172.21.15.99:0/3658842539 conn(0x7ff6980730c0 msgr2=0x7ff6980734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3658842539 shutdown_connections 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.362+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3658842539 wait complete. 2024-09-10T11:34:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.363+0000 7ff6a03fd640 1 Processor -- start 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.363+0000 7ff6a03fd640 1 -- start start 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.363+0000 7ff6a03fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6980b0090 0x7ff69814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.363+0000 7ff6a03fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff69815f1e0 0x7ff6981586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980b21b0 con 0x7ff69815ee00 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980afd60 con 0x7ff6980b0090 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6980b0600 con 0x7ff69815f1e0 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34542/0 (socket says 172.21.15.99:34542) 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 -- 172.21.15.99:0/3798291727 learned_addr learned my addr 172.21.15.99:0/3798291727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 -- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff69815f1e0 msgr2=0x7ff6981586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff69815f1e0 0x7ff6981586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 -- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6980b0090 msgr2=0x7ff69814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6980b0090 0x7ff69814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 -- 172.21.15.99:0/3798291727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6980ca560 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff69d998640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff688037040 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff688004250 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6980ca840 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68803e670 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.364+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6980cae00 con 0x7ff69815ee00 2024-09-10T11:34:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.365+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6980b5f90 con 0x7ff69815ee00 2024-09-10T11:34:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.366+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff68803a040 con 0x7ff69815ee00 2024-09-10T11:34:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.366+0000 7ff68effd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 0x7ff67c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.366+0000 7ff69e199640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 0x7ff67c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.366+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff6880ab710 con 0x7ff69815ee00 2024-09-10T11:34:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.367+0000 7ff69e199640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 0x7ff67c06e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff694001560 tx=0x7ff694007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:28.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.371+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff688042020 con 0x7ff69815ee00 2024-09-10T11:34:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.568+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff69815de20 con 0x7ff69815ee00 2024-09-10T11:34:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.569+0000 7ff68effd640 1 -- 172.21.15.99:0/3798291727 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7ff68807a1e0 con 0x7ff69815ee00 2024-09-10T11:34:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:28.570 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 msgr2=0x7ff67c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 0x7ff67c06e2f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff694001560 tx=0x7ff694007ca0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 msgr2=0x7ff6981581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff6880046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 shutdown_connections 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff67c06be30 0x7ff67c06e2f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff69815f1e0 0x7ff6981586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff69815ee00 0x7ff6981581b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 --2- 172.21.15.99:0/3798291727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6980b0090 0x7ff69814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 >> 172.21.15.99:0/3798291727 conn(0x7ff6980730c0 msgr2=0x7ff6980b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 shutdown_connections 2024-09-10T11:34:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:28.571+0000 7ff6a03fd640 1 -- 172.21.15.99:0/3798291727 wait complete. 2024-09-10T11:34:28.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:34:28.631 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:34:28.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:34:23.937446+0000', 'modified': '2024-09-10T11:34:25.922393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6243}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6243': {'gid': 6243, 'name': 'b', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.196:6835/1425455454', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 1425455454}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1425455454}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6243, 'qdb_cluster': [6243]} 2024-09-10T11:34:28.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:34:28.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T11:34:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.086+0000 7fd7a01b9640 1 Processor -- start 2024-09-10T11:34:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 -- start start 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7980d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd7980d4e40 0x7fd7980d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7980d52f0 con 0x7fd79815eb80 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd7980d30c0 con 0x7fd7980d4e40 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.087+0000 7fd7a01b9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd79815ff70 con 0x7fd7981626c0 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79df55640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd79815eb80 0x7fd7980d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54210/0 (socket says 172.21.15.99:54210) 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79df55640 1 -- 172.21.15.99:0/2069041318 learned_addr learned my addr 172.21.15.99:0/2069041318 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097807351 0 0) 0x7fd794001110 con 0x7fd79815eb80 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd784003520 con 0x7fd79815eb80 2024-09-10T11:34:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79e756640 1 --2- 172.21.15.99:0/2069041318 >> v2:172.21.15.196:3300/0 conn(0x7fd7980d4e40 0x7fd7980d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79d754640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7980d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 742891310 0 0) 0x7fd784003520 con 0x7fd79815eb80 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 msgr2=0x7fd7980d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7980d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 >> v2:172.21.15.196:3300/0 conn(0x7fd7980d4e40 msgr2=0x7fd7980d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 --2- 172.21.15.99:0/2069041318 >> v2:172.21.15.196:3300/0 conn(0x7fd7980d4e40 0x7fd7980d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7980d1620 con 0x7fd79815eb80 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fd794001840 con 0x7fd79815eb80 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 >> v1:172.21.15.99:6789/0 conn(0x7fd79815eb80 legacy=0x7fd7980d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd784004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd784004d40 0x7fd7840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980d30c0 con 0x7fd784003fe0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd79815ff70 con 0x7fd784004d40 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd78400bb80 con 0x7fd7981626c0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79d754640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd784004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79df55640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.088+0000 7fd79e756640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd784004d40 0x7fd7840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 -- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 msgr2=0x7fd7840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7981626c0 0x7fd7840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 -- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd784004d40 msgr2=0x7fd7840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd784004d40 0x7fd7840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 -- 172.21.15.99:0/2069041318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd784003520 con 0x7fd784003fe0 2024-09-10T11:34:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79d754640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd784004800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00ba10 tx=0x7fd784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f29713d56ede7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c030070 con 0x7fd784003fe0 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c02adb0 con 0x7fd784003fe0 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.089+0000 7fd79cf53640 1 -- 172.21.15.99:0/2069041318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c00b610 con 0x7fd784003fe0 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 -- 172.21.15.99:0/2069041318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 msgr2=0x7fd784004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/2069041318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd784004800 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00ba10 tx=0x7fd784005830 comp rx=0 tx=0).stop 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd79e756640 1 -- 172.21.15.99:0/2069041318 reap_dead start 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 -- 172.21.15.99:0/2069041318 shutdown_connections 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 -- 172.21.15.99:0/2069041318 >> 172.21.15.99:0/2069041318 conn(0x7fd79808e680 msgr2=0x7fd79809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 -- 172.21.15.99:0/2069041318 shutdown_connections 2024-09-10T11:34:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.090+0000 7fd7a01b9640 1 -- 172.21.15.99:0/2069041318 wait complete. 2024-09-10T11:34:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.091+0000 7fd7a01b9640 1 Processor -- start 2024-09-10T11:34:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.091+0000 7fd7a01b9640 1 -- start start 2024-09-10T11:34:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 0x7fd79815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7980cd4c0 0x7fd7980cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd798163180 con 0x7fd784003fe0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd798162ea0 con 0x7fd7980cd4c0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd7a01b9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd798163460 con 0x7fd79815bce0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34556/0 (socket says 172.21.15.99:34556) 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 -- 172.21.15.99:0/3125469682 learned_addr learned my addr 172.21.15.99:0/3125469682 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79d754640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 0x7fd79815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79e756640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7980cd4c0 0x7fd7980cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 -- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 msgr2=0x7fd79815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 0x7fd79815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 -- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7980cd4c0 msgr2=0x7fd7980cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7980cd4c0 0x7fd7980cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 -- 172.21.15.99:0/3125469682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79815bab0 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79d754640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 0x7fd79815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.092+0000 7fd79df55640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd794007340 tx=0x7fd794007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.093+0000 7fd78a7fc640 1 -- 172.21.15.99:0/3125469682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79400f940 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.093+0000 7fd78a7fc640 1 -- 172.21.15.99:0/3125469682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794018070 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.093+0000 7fd78a7fc640 1 -- 172.21.15.99:0/3125469682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79400fd20 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.093+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7980ce3b0 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.093+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7980ce970 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.094+0000 7fd78a7fc640 1 -- 172.21.15.99:0/3125469682 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd79401d460 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.094+0000 7fd78a7fc640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 0x7fd77c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.095+0000 7fd78a7fc640 1 -- 172.21.15.99:0/3125469682 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd79408ae40 con 0x7fd784003fe0 2024-09-10T11:34:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.095+0000 7fd79d754640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 0x7fd77c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.095+0000 7fd79d754640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 0x7fd77c06e320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd798162af0 tx=0x7fd78c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 Processor -- start 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 -- start start 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefee7a80 0x556cefeea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cefe4c520 con 0x556cefeea2c0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556cefdf43e0 con 0x556cefeeba50 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556cefe484e0 con 0x556cefee7a80 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.096+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34570/0 (socket says 172.21.15.99:34570) 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 -- 172.21.15.99:0/4230819861 learned_addr learned my addr 172.21.15.99:0/4230819861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79d754640 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefee7a80 0x556cefeea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79e756640 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 -- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefee7a80 msgr2=0x556cefeea6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefee7a80 0x556cefeea6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 -- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 msgr2=0x556cefee48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 -- 172.21.15.99:0/4230819861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cefe0f2f0 con 0x556cefeea2c0 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd79df55640 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd794089390 tx=0x7fd794058a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176e8f24593567bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd77b7fe640 1 -- 172.21.15.99:0/4230819861 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79405d4a0 con 0x556cefeea2c0 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd77b7fe640 1 -- 172.21.15.99:0/4230819861 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794061cd0 con 0x556cefeea2c0 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.097+0000 7fd77b7fe640 1 -- 172.21.15.99:0/4230819861 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79405d170 con 0x556cefeea2c0 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- 172.21.15.99:0/4230819861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 msgr2=0x556ceff2fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd794089390 tx=0x7fd794058a50 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- 172.21.15.99:0/4230819861 shutdown_connections 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefee7a80 0x556cefeea6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4230819861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff2fa60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- 172.21.15.99:0/4230819861 >> 172.21.15.99:0/4230819861 conn(0x556cefee6a80 msgr2=0x556ceff2ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- 172.21.15.99:0/4230819861 shutdown_connections 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- 172.21.15.99:0/4230819861 wait complete. 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 -- start start 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.098+0000 7fd7a3027740 1 Processor -- start 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 0x556cefee8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cefee5f20 con 0x556cefeea2c0 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556cefdd5450 con 0x556cefeeba50 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556ceff30290 con 0x556ceff0aa30 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd7a3027740 10 client.0 _ll_register_callbacks cb 0x556cefb5eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.099+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34586/0 (socket says 172.21.15.99:34586) 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 -- 172.21.15.99:0/4166764535 learned_addr learned my addr 172.21.15.99:0/4166764535 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79d754640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79e756640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 0x556cefee8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 -- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 msgr2=0x556cefee8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 0x556cefee8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 -- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 msgr2=0x556cefee89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 -- 172.21.15.99:0/4166764535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cefb5f2d0 con 0x556cefeea2c0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79e756640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 0x556cefee8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd79df55640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd794089340 tx=0x7fd794015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbb8981e593f6c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79405d4a0 con 0x556cefeea2c0 2024-09-10T11:34:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd794057190 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd794061cd0 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd794058540 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 10 client.6360 fetch_fsmap learned FSMap version 42 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x556cefc296b0 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794061cd0 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 3502+0+0 (secure 0 0 0) 0x7fd7940586e0 con 0x556cefeea2c0 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 10 client.6360 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 10 client.6360 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.100+0000 7fd7a3027740 20 client.6360 populate_metadata read hostname 'smithi099' 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556cefee4350 0x556ceff4cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd7a3027740 4 client.6360 mds_command: new command op to 6243 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 --> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cefe59c50 con 0x556cefee4350 2024-09-10T11:34:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd79d754640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556cefee4350 0x556ceff4cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd778ff9640 10 client.6360 ms_handle_connect on v2:172.21.15.196:6833/1425455454 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.101+0000 7fd79d754640 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556cefee4350 0x556ceff4cb80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1000f01f1b5c373 server_cookie=6330016b3c9bd7ea in_seq=0 out_seq=0 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd778ff9640 1 -- 172.21.15.99:0/4166764535 <== mds.0 v2:172.21.15.196:6833/1425455454 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cefe59c50 con 0x556cefee4350 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd778ff9640 10 client.6360 handle_command_reply: tid=0 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 client.6360 shutdown 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 msgr2=0x556ceff08f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd794089340 tx=0x7fd794015040 comp rx=0 tx=0).stop 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 shutdown_connections 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556cefee4350 0x556ceff4cb80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556ceff0aa30 0x556cefee8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd778ff9640 0 client.6360 ms_handle_reset on v2:172.21.15.196:6833/1425455454 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 --2- 172.21.15.99:0/4166764535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefeea2c0 0x556ceff08f00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 >> 172.21.15.99:0/4166764535 conn(0x556cefee7a80 msgr2=0x556ceff2e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 shutdown_connections 2024-09-10T11:34:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 1 -- 172.21.15.99:0/4166764535 wait complete. 2024-09-10T11:34:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.102+0000 7fd7a3027740 20 client.6360 trim_cache size 0 max 16384 2024-09-10T11:34:29.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.120+0000 7fd7a3027740 1 Processor -- start 2024-09-10T11:34:29.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.120+0000 7fd7a3027740 1 -- start start 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeea2c0 0x556ceff0ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeeba50 0x556ceff0f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd798200830 con 0x556cefee7a80 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980b2e50 con 0x556cefeea2c0 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd7982001e0 con 0x556cefeeba50 2024-09-10T11:34:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34592/0 (socket says 172.21.15.99:34592) 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 -- 172.21.15.99:0/1959039682 learned_addr learned my addr 172.21.15.99:0/1959039682 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79d754640 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeea2c0 0x556ceff0ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79e756640 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeeba50 0x556ceff0f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 -- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeeba50 msgr2=0x556ceff0f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeeba50 0x556ceff0f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 -- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeea2c0 msgr2=0x556ceff0ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeea2c0 0x556ceff0ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 -- 172.21.15.99:0/1959039682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cefee58d0 con 0x556cefee7a80 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd79df55640 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd794089450 tx=0x7fd79406f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c75165f244bb4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd77affd640 1 -- 172.21.15.99:0/1959039682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79405d4a0 con 0x556cefee7a80 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd77affd640 1 -- 172.21.15.99:0/1959039682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794061cd0 con 0x556cefee7a80 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.121+0000 7fd77affd640 1 -- 172.21.15.99:0/1959039682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940586e0 con 0x556cefee7a80 2024-09-10T11:34:29.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- 172.21.15.99:0/1959039682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 msgr2=0x556ceff2d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd794089450 tx=0x7fd79406f040 comp rx=0 tx=0).stop 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- 172.21.15.99:0/1959039682 shutdown_connections 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeeba50 0x556ceff0f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeea2c0 0x556ceff0ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1959039682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff2d1a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- 172.21.15.99:0/1959039682 >> 172.21.15.99:0/1959039682 conn(0x556ceff12130 msgr2=0x556ceff2ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- 172.21.15.99:0/1959039682 shutdown_connections 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- 172.21.15.99:0/1959039682 wait complete. 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 -- start start 2024-09-10T11:34:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.122+0000 7fd7a3027740 1 Processor -- start 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 0x556ceffa8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cefee5f20 con 0x556cefee7a80 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556ceff30290 con 0x556cefeeba50 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x556cefdd5450 con 0x556cefeea2c0 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd7a3027740 10 client.0 _ll_register_callbacks cb 0x7fd794070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:34:29.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd79df55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34596/0 (socket says 172.21.15.99:34596) 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.123+0000 7fd79df55640 1 -- 172.21.15.99:0/1516920711 learned_addr learned my addr 172.21.15.99:0/1516920711 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79d754640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 0x556ceffa8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79e756640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 -- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 msgr2=0x556ceffa8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 0x556ceffa8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 -- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 msgr2=0x556cefee4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 -- 172.21.15.99:0/1516920711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ceff414c0 con 0x556cefee7a80 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79d754640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 0x556ceffa8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd79df55640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd794055550 tx=0x7fd79400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12729f3bfcdc608 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:34:29.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79405d4a0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fd7940550a0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd794061cd0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd794057fb0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 10 client.6366 fetch_fsmap learned FSMap version 42 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x556ceff414c0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd794061cd0 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 42) ==== 3502+0+0 (secure 0 0 0) 0x7fd79407f030 con 0x556cefee7a80 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 10 client.6366 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 10 client.6366 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 20 client.6366 populate_metadata read hostname 'smithi099' 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556ceff0aa30 0x556cefee86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 4 client.6366 mds_command: new command op to 6243 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.124+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 --> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] -- command(tid 0: {"prefix": "status"}) -- 0x556cefe53fa0 con 0x556ceff0aa30 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd79d754640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556ceff0aa30 0x556cefee86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd79d754640 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556ceff0aa30 0x556cefee86c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af12648004c14dc5 server_cookie=74df9d7336375e04 in_seq=0 out_seq=0 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd778ff9640 10 client.6366 ms_handle_connect on v2:172.21.15.196:6833/1425455454 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd778ff9640 1 -- 172.21.15.99:0/1516920711 <== mds.0 v2:172.21.15.196:6833/1425455454 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x556cefe53fa0 con 0x556ceff0aa30 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd778ff9640 10 client.6366 handle_command_reply: tid=0 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 client.6366 shutdown 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 msgr2=0x556ceff11390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd794055550 tx=0x7fd79400ff40 comp rx=0 tx=0).stop 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 shutdown_connections 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:6833/1425455454,v1:172.21.15.196:6835/1425455454] conn(0x556ceff0aa30 0x556cefee86c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556cefeeba50 0x556cefee4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556cefeea2c0 0x556ceffa8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd778ff9640 0 client.6366 ms_handle_reset on v2:172.21.15.196:6833/1425455454 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 --2- 172.21.15.99:0/1516920711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cefee7a80 0x556ceff11390 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 >> 172.21.15.99:0/1516920711 conn(0x556ceff12130 msgr2=0x556ceff3ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 shutdown_connections 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 1 -- 172.21.15.99:0/1516920711 wait complete. 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.125+0000 7fd7a3027740 20 client.6366 trim_cache size 0 max 16384 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6243, 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:34:29.133 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5312114939999999, 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 42, 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 103, 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 103, 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 8.4780939170000007 2024-09-10T11:34:29.134 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 msgr2=0x7fd77c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 0x7fd77c06e320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd798162af0 tx=0x7fd78c030000 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 msgr2=0x7fd79815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd794007340 tx=0x7fd794007800 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 shutdown_connections 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd77c06be60 0x7fd77c06e320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7980cd4c0 0x7fd7980cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd79815bce0 0x7fd79815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 --2- 172.21.15.99:0/3125469682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd784003fe0 0x7fd79815b570 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 >> 172.21.15.99:0/3125469682 conn(0x7fd79808e680 msgr2=0x7fd79815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 shutdown_connections 2024-09-10T11:34:29.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.126+0000 7fd7a01b9640 1 -- 172.21.15.99:0/3125469682 wait complete. 2024-09-10T11:34:29.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:34:29.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.236+0000 7f5485bd6640 1 Processor -- start 2024-09-10T11:34:29.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.236+0000 7f5485bd6640 1 -- start start 2024-09-10T11:34:29.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f5485bd6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54800d1200 0x7f5480073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f5485bd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f548006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f5485bd6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5480152e60 con 0x7f548015c060 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f5485bd6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f548006ef50 con 0x7f54800d1200 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f5485bd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54800d15e0 con 0x7f548006e5d0 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54800d1200 0x7f5480073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54800d1200 0x7f5480073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34612/0 (socket says 172.21.15.99:34612) 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 -- 172.21.15.99:0/2003584716 learned_addr learned my addr 172.21.15.99:0/2003584716 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f548015c060 0x7f548015c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54226/0 (socket says 172.21.15.99:54226) 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 -- 172.21.15.99:0/2003584716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 msgr2=0x7f548006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 --2- 172.21.15.99:0/2003584716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f548006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 -- 172.21.15.99:0/2003584716 >> v1:172.21.15.99:6789/0 conn(0x7f548015c060 legacy=0x7f548015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547f7fe640 1 -- 172.21.15.99:0/2003584716 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5480072680 con 0x7f54800d1200 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.237+0000 7f547e7fc640 1 -- 172.21.15.99:0/2003584716 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187040620 0 0) 0x7f5480152e60 con 0x7f548015c060 2024-09-10T11:34:29.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f547f7fe640 1 --2- 172.21.15.99:0/2003584716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54800d1200 0x7f5480073440 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5470009870 tx=0x7f547002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82517e8d620835 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f547e7fc640 1 -- 172.21.15.99:0/2003584716 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470037070 con 0x7f54800d1200 2024-09-10T11:34:29.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f547e7fc640 1 -- 172.21.15.99:0/2003584716 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5470032440 con 0x7f54800d1200 2024-09-10T11:34:29.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f547e7fc640 1 -- 172.21.15.99:0/2003584716 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470032810 con 0x7f54800d1200 2024-09-10T11:34:29.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 -- 172.21.15.99:0/2003584716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54800d1200 msgr2=0x7f5480073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 --2- 172.21.15.99:0/2003584716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54800d1200 0x7f5480073440 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f5470009870 tx=0x7f547002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 -- 172.21.15.99:0/2003584716 shutdown_connections 2024-09-10T11:34:29.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 --2- 172.21.15.99:0/2003584716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f548006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 --2- 172.21.15.99:0/2003584716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54800d1200 0x7f5480073440 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 -- 172.21.15.99:0/2003584716 >> 172.21.15.99:0/2003584716 conn(0x7f548006bc10 msgr2=0x7f548006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 -- 172.21.15.99:0/2003584716 shutdown_connections 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.238+0000 7f5485bd6640 1 -- 172.21.15.99:0/2003584716 wait complete. 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.239+0000 7f5485bd6640 1 Processor -- start 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 -- start start 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f5480152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 0x7f54801582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5480072f20 con 0x7f548015c060 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5480072c40 con 0x7f54800d1200 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.240+0000 7f5485bd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f548015cd90 con 0x7f548006e5d0 2024-09-10T11:34:29.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34622/0 (socket says 172.21.15.99:34622) 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 -- 172.21.15.99:0/643707444 learned_addr learned my addr 172.21.15.99:0/643707444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547f7fe640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f5480152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547effd640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 0x7f54801582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 -- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 msgr2=0x7f5480152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f5480152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 -- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 msgr2=0x7f54801582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 0x7f54801582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 -- 172.21.15.99:0/643707444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5480158d10 con 0x7f548015c060 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547effd640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 0x7f54801582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f547ffff640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f547800c7d0 tx=0x7f547800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478018070 con 0x7f548015c060 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547800a640 con 0x7f548015c060 2024-09-10T11:34:29.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5480158ff0 con 0x7f548015c060 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547800aa70 con 0x7f548015c060 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.241+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54801595b0 con 0x7f548015c060 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.242+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548006ea10 con 0x7f548015c060 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.243+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f547801b070 con 0x7f548015c060 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.243+0000 7f5467fff640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:29.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.243+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f547808a400 con 0x7f548015c060 2024-09-10T11:34:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.243+0000 7f547f7fe640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:29.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.243+0000 7f547f7fe640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 0x7f546c06e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5470034000 tx=0x7f547002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:29.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.247+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5478058e40 con 0x7f548015c060 2024-09-10T11:34:29.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.445+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f54800761b0 con 0x7f548015c060 2024-09-10T11:34:29.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.445+0000 7f5467fff640 1 -- 172.21.15.99:0/643707444 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) ==== 53+0+73 (secure 0 0 0) 0x7f5478058740 con 0x7f548015c060 2024-09-10T11:34:29.447 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:34:29.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 msgr2=0x7f546c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 0x7f546c06e2f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5470034000 tx=0x7f547002e000 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 msgr2=0x7f54801ef5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f547800c7d0 tx=0x7f547800cc90 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 shutdown_connections 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f546c06be30 0x7f546c06e2f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f548015c060 0x7f54801ef5a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54800d1200 0x7f54801582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 --2- 172.21.15.99:0/643707444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f548006e5d0 0x7f5480152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.447+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 >> 172.21.15.99:0/643707444 conn(0x7f548006bc10 msgr2=0x7f548015e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.448+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 shutdown_connections 2024-09-10T11:34:29.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29.448+0000 7f5485bd6640 1 -- 172.21.15.99:0/643707444 wait complete. 2024-09-10T11:34:29.459 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:34:29.459 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:34:29.459 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:34:29.459 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:34:29.460 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:29.460 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:34:29.460 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:34:29.460 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:34:29.460 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:34:29.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:29.460 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:29.476 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42579sec preferred_lft 42579sec 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:34:29.478 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:34:29.479 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:29.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:29.479 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:34:29.540 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:34:29.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:29.541 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:34:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:34:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:34:29.597 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:34:29.598 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:34:29.598 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:34:29.598 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:34:29.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:34:29.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:34:29.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:29.723 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:34:29.738 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:34:29.738 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:34:29.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:34:29.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:29.739 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:34:29.803 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:34:29.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:34:29.803 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:29.859 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:34:29.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:34:29.859 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:29.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:34:29.980 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:34:30.035 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:34:30.036 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:34:30.037 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:34:30.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.038 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:34:30.090 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:34:30.091 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:34:30.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:34:30.157 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:34:30.213 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:34:30.214 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:34:30.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.215 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:34:30.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:34:30.213+0000 7f5bd691b480 -1 init, newargv = 0x56367e55efd0 newargc=15 2024-09-10T11:34:30.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[49398]: starting ceph client 2024-09-10T11:34:30.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[49398]: starting fuse 2024-09-10T11:34:30.231 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:34:30.231 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:34:30.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:34:30.232 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:34:30.233 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:34:30.348 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.49398.asok 2024-09-10T11:34:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:34:30.359 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49398.asok 2024-09-10T11:34:30.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.359 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49398.asok status 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:34:30.492 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "49398", 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6384, 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6384 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:34:30.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2939076856 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2939076856 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6384 192.168.0.1:0/2939076856", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2939076856", 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 42, 2024-09-10T11:34:30.494 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 103, 2024-09-10T11:34:30.495 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:34:30.495 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:34:30.495 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:34:30.495 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:30.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.501 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:34:30.501 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:30.516 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:34:30.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:30.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:30.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:34:30.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.720+0000 7f702cf86640 1 Processor -- start 2024-09-10T11:34:30.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.721+0000 7f702cf86640 1 -- start start 2024-09-10T11:34:30.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.721+0000 7f702cf86640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70280cfb90 0x7f70280b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.721+0000 7f702cf86640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 0x7f70280cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.721+0000 7f702cf86640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f702806df80 con 0x7f70280b3660 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702cf86640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f70280d18a0 con 0x7f70280cfb90 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702cf86640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70280d2120 con 0x7f70280d1410 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70280cfb90 0x7f70280b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f7026d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 0x7f70280cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70280cfb90 0x7f70280b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:40878/0 (socket says 172.21.15.99:40878) 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 -- 172.21.15.99:0/1820888931 learned_addr learned my addr 172.21.15.99:0/1820888931 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:30.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f7025d9b640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f70280b3660 0x7f70280d0d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:44042/0 (socket says 172.21.15.99:44042) 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 -- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 msgr2=0x7f70280cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 0x7f70280cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 -- 172.21.15.99:0/1820888931 >> v1:172.21.15.196:6789/0 conn(0x7f70280b3660 legacy=0x7f70280d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702659c640 1 -- 172.21.15.99:0/1820888931 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70280d1ad0 con 0x7f70280cfb90 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f7026d9d640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 0x7f70280cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.722+0000 7f702559a640 1 -- 172.21.15.99:0/1820888931 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3569835071 0 0) 0x7f702806df80 con 0x7f70280b3660 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.723+0000 7f702659c640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cfb90 0x7f70280b5170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7010009870 tx=0x7f701002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb31dfdf22059a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.723+0000 7f702559a640 1 -- 172.21.15.99:0/1820888931 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010037070 con 0x7f70280cfb90 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.723+0000 7f702559a640 1 -- 172.21.15.99:0/1820888931 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7010032440 con 0x7f70280cfb90 2024-09-10T11:34:30.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.723+0000 7f702559a640 1 -- 172.21.15.99:0/1820888931 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010032810 con 0x7f70280cfb90 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 -- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cfb90 msgr2=0x7f70280b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cfb90 0x7f70280b5170 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7010009870 tx=0x7f701002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 -- 172.21.15.99:0/1820888931 shutdown_connections 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280d1410 0x7f70280cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 --2- 172.21.15.99:0/1820888931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280cfb90 0x7f70280b5170 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 -- 172.21.15.99:0/1820888931 >> 172.21.15.99:0/1820888931 conn(0x7f70280b4650 msgr2=0x7f70280b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 -- 172.21.15.99:0/1820888931 shutdown_connections 2024-09-10T11:34:30.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.724+0000 7f702cf86640 1 -- 172.21.15.99:0/1820888931 wait complete. 2024-09-10T11:34:30.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.725+0000 7f702cf86640 1 Processor -- start 2024-09-10T11:34:30.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.725+0000 7f702cf86640 1 -- start start 2024-09-10T11:34:30.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280cfb90 0x7f70280cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280d1410 0x7f70281f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f702806dd30 con 0x7f70280b3660 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f702806e260 con 0x7f70280d1410 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702cf86640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f70280cff70 con 0x7f70280cfb90 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34630/0 (socket says 172.21.15.99:34630) 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 -- 172.21.15.99:0/119281545 learned_addr learned my addr 172.21.15.99:0/119281545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f7026d9d640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280d1410 0x7f70281f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f7025d9b640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280cfb90 0x7f70280cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 -- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280cfb90 msgr2=0x7f70280cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280cfb90 0x7f70280cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 -- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280d1410 msgr2=0x7f70281f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280d1410 0x7f70281f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.726+0000 7f702659c640 1 -- 172.21.15.99:0/119281545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70281f88c0 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f702659c640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7010035040 tx=0x7f70100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010037040 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7010004230 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010003610 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70281f8ba0 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.727+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281f47e0 con 0x7f70280b3660 2024-09-10T11:34:30.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.728+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702806df80 con 0x7f70280b3660 2024-09-10T11:34:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.728+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f701002ad10 con 0x7f70280b3660 2024-09-10T11:34:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.729+0000 7f701effd640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.729+0000 7f7025d9b640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.729+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f70100ab7e0 con 0x7f70280b3660 2024-09-10T11:34:30.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.729+0000 7f7025d9b640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 0x7f700806e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f70280c9820 tx=0x7f7018007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:30.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.733+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f701007a220 con 0x7f70280b3660 2024-09-10T11:34:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.929+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70280b8980 con 0x7f70280b3660 2024-09-10T11:34:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.930+0000 7f701effd640 1 -- 172.21.15.99:0/119281545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) ==== 53+0+73 (secure 0 0 0) 0x7f7010079b20 con 0x7f70280b3660 2024-09-10T11:34:30.931 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 msgr2=0x7f700806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 0x7f700806e2f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f70280c9820 tx=0x7f7018007c00 comp rx=0 tx=0).stop 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 msgr2=0x7f70281f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7010035040 tx=0x7f70100046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 shutdown_connections 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f700806be30 0x7f700806e2f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70280d1410 0x7f70281f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70280cfb90 0x7f70280cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 --2- 172.21.15.99:0/119281545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70280b3660 0x7f70281f5340 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 >> 172.21.15.99:0/119281545 conn(0x7f70280b4650 msgr2=0x7f70280cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 shutdown_connections 2024-09-10T11:34:30.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:30.932+0000 7f702cf86640 1 -- 172.21.15.99:0/119281545 wait complete. 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:34:30.943 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:34:30.944 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:34:30.944 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:34:30.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.944 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:30.964 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42593sec preferred_lft 42593sec 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:34:30.965 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:34:30.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:30.967 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:34:31.031 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:34:31.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.033 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:34:31.092 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:34:31.092 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:34:31.093 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:34:31.093 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:34:31.093 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:34:31.093 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:34:31.184 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:34:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:34:31.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:34:31.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.224 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:34:31.240 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:34:31.240 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:34:31.241 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:34:31.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.241 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:34:31.305 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:34:31.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:34:31.306 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:31.361 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:34:31.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:34:31.361 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:31.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:34:31.480 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:34:31.537 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:34:31.537 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:34:31.537 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:34:31.537 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:34:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:34:31.539 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:34:31.540 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:34:31.540 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:34:31.540 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:34:31.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.540 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:34:31.593 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:34:31.593 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:34:31.636 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:34:31.661 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:34:31.717 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:34:31.718 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:34:31.719 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:34:31.720 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:34:31.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:34:31.716+0000 7f5d2b1dd480 -1 init, newargv = 0x5580a10144c0 newargc=15 2024-09-10T11:34:31.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36823]: starting ceph client 2024-09-10T11:34:31.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.721 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:34:31.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36823]: starting fuse 2024-09-10T11:34:31.777 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:34:31.777 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:34:31.777 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:34:31.777 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:34:31.778 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:34:31.779 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:34:31.895 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.36823.asok 2024-09-10T11:34:31.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:34:31.905 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36823.asok 2024-09-10T11:34:31.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:31.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36823.asok status 2024-09-10T11:34:32.039 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:34:32.039 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:34:32.039 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "36823", 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5992, 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:34:32.040 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 5992 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1345568691 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1345568691 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.5992 192.168.0.1:0/1345568691", 2024-09-10T11:34:32.041 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/1345568691", 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 42, 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 103, 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:34:32.042 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:34:32.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:32.047 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:34:32.047 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:32.063 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:34:32.063 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:32.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:32.063 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:32.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:32.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.240+0000 7f60a5db0640 1 Processor -- start 2024-09-10T11:34:32.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.240+0000 7f60a5db0640 1 -- start start 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.240+0000 7f60a5db0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60a006ece0 0x7f60a0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f60a5db0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a015c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f60a5db0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a0152e60 con 0x7f60a00d1200 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f60a5db0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f60a00d15e0 con 0x7f60a006ece0 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f60a5db0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60a0072680 con 0x7f60a015c060 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60a006ece0 0x7f60a0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60a006ece0 0x7f60a0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34636/0 (socket says 172.21.15.99:34636) 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 -- 172.21.15.99:0/224137731 learned_addr learned my addr 172.21.15.99:0/224137731 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:32.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 -- 172.21.15.99:0/224137731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 msgr2=0x7f60a015c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:32.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609ffff640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a015c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a015c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 -- 172.21.15.99:0/224137731 >> v1:172.21.15.196:6789/0 conn(0x7f60a00d1200 legacy=0x7f60a006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:32.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.241+0000 7f609effd640 1 -- 172.21.15.99:0/224137731 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a0072960 con 0x7f60a006ece0 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f609effd640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a0073440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6088009870 tx=0x7f608802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a543af7f686b2d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f609e7fc640 1 -- 172.21.15.99:0/224137731 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088037070 con 0x7f60a006ece0 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f609e7fc640 1 -- 172.21.15.99:0/224137731 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6088032440 con 0x7f60a006ece0 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f609e7fc640 1 -- 172.21.15.99:0/224137731 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088032810 con 0x7f60a006ece0 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f60a5db0640 1 -- 172.21.15.99:0/224137731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 msgr2=0x7f60a0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.242+0000 7f60a5db0640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a0073440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6088009870 tx=0x7f608802a180 comp rx=0 tx=0).stop 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 -- 172.21.15.99:0/224137731 shutdown_connections 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a015c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 --2- 172.21.15.99:0/224137731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a0073440 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 -- 172.21.15.99:0/224137731 >> 172.21.15.99:0/224137731 conn(0x7f60a006bc10 msgr2=0x7f60a006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:32.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 -- 172.21.15.99:0/224137731 shutdown_connections 2024-09-10T11:34:32.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.243+0000 7f60a5db0640 1 -- 172.21.15.99:0/224137731 wait complete. 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.244+0000 7f60a5db0640 1 Processor -- start 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.244+0000 7f60a5db0640 1 -- start start 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a00d1200 0x7f60a015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a013e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60a015cd90 con 0x7f60a006ece0 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60a0072f20 con 0x7f60a00d1200 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f60a5db0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60a015ca00 con 0x7f60a015c060 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34650/0 (socket says 172.21.15.99:34650) 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 -- 172.21.15.99:0/4159907852 learned_addr learned my addr 172.21.15.99:0/4159907852 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609effd640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a00d1200 0x7f60a015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609ffff640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a013e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 -- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 msgr2=0x7f60a013e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a013e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 -- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a00d1200 msgr2=0x7f60a015a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a00d1200 0x7f60a015a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 -- 172.21.15.99:0/4159907852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a0157b30 con 0x7f60a006ece0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f609f7fe640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f609000ba50 tx=0x7f609000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090019070 con 0x7f60a006ece0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f609000cc20 con 0x7f60a006ece0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.245+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6090014680 con 0x7f60a006ece0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.246+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a013f280 con 0x7f60a006ece0 2024-09-10T11:34:32.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.246+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a013f800 con 0x7f60a006ece0 2024-09-10T11:34:32.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.246+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a0073440 con 0x7f60a006ece0 2024-09-10T11:34:32.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.247+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f609001c070 con 0x7f60a006ece0 2024-09-10T11:34:32.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.247+0000 7f6097fff640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 0x7f608006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:32.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.247+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f609008ab10 con 0x7f60a006ece0 2024-09-10T11:34:32.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.247+0000 7f609effd640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 0x7f608006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:32.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.248+0000 7f609effd640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 0x7f608006e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6088034000 tx=0x7f608802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:32.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.252+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6090058e70 con 0x7f60a006ece0 2024-09-10T11:34:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.451+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60a00d05c0 con 0x7f60a006ece0 2024-09-10T11:34:32.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.452+0000 7f6097fff640 1 -- 172.21.15.99:0/4159907852 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7f609005d170 con 0x7f60a006ece0 2024-09-10T11:34:32.453 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:32.453 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:32.453 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 msgr2=0x7f608006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 0x7f608006e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6088034000 tx=0x7f608802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 msgr2=0x7f60a01575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f609000ba50 tx=0x7f609000bf10 comp rx=0 tx=0).stop 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 shutdown_connections 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f608006be30 0x7f608006e2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60a015c060 0x7f60a013e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a00d1200 0x7f60a015a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 --2- 172.21.15.99:0/4159907852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60a006ece0 0x7f60a01575f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 >> 172.21.15.99:0/4159907852 conn(0x7f60a006bc10 msgr2=0x7f60a006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 shutdown_connections 2024-09-10T11:34:32.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:32.454+0000 7f60a5db0640 1 -- 172.21.15.99:0/4159907852 wait complete. 2024-09-10T11:34:32.466 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:34:32.584 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6243,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9807117239999998,"mdsmap_epoch":42,"osdmap_epoch":103,"osdmap_epoch_barrier":103,"uptime":11.927598033000001} 2024-09-10T11:34:32.585 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6243, "mdsmap_epoch": 42, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "rank_uptime": 7.980711724, "state": "up:active", "uptime": 11.927598033, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:34:32.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:34:32.696 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:34:32.697 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:34:32.697 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-09-10T11:34:32.697 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:32.697 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:32.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:32.713 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-09-10T11:34:33.801 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records in 2024-09-10T11:34:33.802 INFO:teuthology.orchestra.run.smithi099.stderr:265+0 records out 2024-09-10T11:34:33.802 INFO:teuthology.orchestra.run.smithi099.stderr:277872640 bytes (278 MB, 265 MiB) copied, 1.02889 s, 270 MB/s 2024-09-10T11:34:33.802 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:33.802 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:33.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:33.818 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-09-10T11:34:34.955 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records in 2024-09-10T11:34:34.955 INFO:teuthology.orchestra.run.smithi099.stderr:292+0 records out 2024-09-10T11:34:34.955 INFO:teuthology.orchestra.run.smithi099.stderr:306184192 bytes (306 MB, 292 MiB) copied, 1.07838 s, 284 MB/s 2024-09-10T11:34:34.955 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-09-10T11:34:34.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:35.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.052+0000 7fcc5ff7f640 1 Processor -- start 2024-09-10T11:34:35.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.052+0000 7fcc5ff7f640 1 -- start start 2024-09-10T11:34:35.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5ff7f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc580b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5ff7f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc580b40e0 con 0x7fcc5815ee00 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5ff7f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc580b5ab0 con 0x7fcc580b0090 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5ff7f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcc58073c50 con 0x7fcc5815f1e0 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5dd1b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcc5815ee00 0x7fcc580b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46692/0 (socket says 172.21.15.99:46692) 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5dd1b640 1 -- 172.21.15.99:0/3598607383 learned_addr learned my addr 172.21.15.99:0/3598607383 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc580b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 112106381 0 0) 0x7fcc580b40e0 con 0x7fcc5815ee00 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.053+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc44003520 con 0x7fcc5815ee00 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539974062 0 0) 0x7fcc580b5ab0 con 0x7fcc580b0090 2024-09-10T11:34:35.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc580b40e0 con 0x7fcc580b0090 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3690464602 0 0) 0x7fcc44003520 con 0x7fcc5815ee00 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 msgr2=0x7fcc580b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc580b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 >> v1:172.21.15.196:6789/0 conn(0x7fcc580b0090 legacy=0x7fcc580b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc580b4380 con 0x7fcc5815ee00 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fcc540015f0 con 0x7fcc5815ee00 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 >> v1:172.21.15.99:6789/0 conn(0x7fcc5815ee00 legacy=0x7fcc580b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc44003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc44004120 0x7fcc44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc44004d40 0x7fcc440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58073c50 con 0x7fcc44004120 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4400bb00 con 0x7fcc44004d40 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4400bd00 con 0x7fcc5815f1e0 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc44004120 0x7fcc44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5dd1b640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc44003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5e51c640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc44004d40 0x7fcc440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 -- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 msgr2=0x7fcc44003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc5815f1e0 0x7fcc44003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 -- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc44004d40 msgr2=0x7fcc440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:35.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc44004d40 0x7fcc440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.054+0000 7fcc5d51a640 1 -- 172.21.15.99:0/3598607383 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc44003520 con 0x7fcc44004120 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc44004120 0x7fcc44004800 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc5806da30 tx=0x7fcc44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f452ae0cef9566 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48036070 con 0x7fcc44004120 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc48031400 con 0x7fcc44004120 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5cd19640 1 -- 172.21.15.99:0/3598607383 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48009690 con 0x7fcc44004120 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/3598607383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc44004120 msgr2=0x7fcc44004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/3598607383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc44004120 0x7fcc44004800 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc5806da30 tx=0x7fcc44005830 comp rx=0 tx=0).stop 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.055+0000 7fcc5e51c640 1 -- 172.21.15.99:0/3598607383 reap_dead start 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.056+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/3598607383 shutdown_connections 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.056+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/3598607383 >> 172.21.15.99:0/3598607383 conn(0x7fcc580730c0 msgr2=0x7fcc580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.056+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/3598607383 shutdown_connections 2024-09-10T11:34:35.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.056+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/3598607383 wait complete. 2024-09-10T11:34:35.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.057+0000 7fcc5ff7f640 1 Processor -- start 2024-09-10T11:34:35.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.057+0000 7fcc5ff7f640 1 -- start start 2024-09-10T11:34:35.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc44004120 0x7fcc58157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc580ae8d0 0x7fcc580aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc580b0600 con 0x7fcc58156350 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc580afd60 con 0x7fcc580ae8d0 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcc580b0780 con 0x7fcc44004120 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35158/0 (socket says 172.21.15.99:35158) 2024-09-10T11:34:35.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 -- 172.21.15.99:0/2185873083 learned_addr learned my addr 172.21.15.99:0/2185873083 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5e51c640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc580ae8d0 0x7fcc580aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5dd1b640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc44004120 0x7fcc58157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 -- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc44004120 msgr2=0x7fcc58157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc44004120 0x7fcc58157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 -- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc580ae8d0 msgr2=0x7fcc580aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc580ae8d0 0x7fcc580aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 -- 172.21.15.99:0/2185873083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc580ca730 con 0x7fcc58156350 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5d51a640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48040020 con 0x7fcc58156350 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc48003a40 con 0x7fcc58156350 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc48036040 con 0x7fcc58156350 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc580caa10 con 0x7fcc58156350 2024-09-10T11:34:35.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.058+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc580cafd0 con 0x7fcc58156350 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.059+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc580b5f90 con 0x7fcc58156350 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.060+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc48043020 con 0x7fcc58156350 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.060+0000 7fcc4e7fc640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 0x7fcc3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.060+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fcc480ab090 con 0x7fcc58156350 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.060+0000 7fcc5dd1b640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 0x7fcc3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:35.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.061+0000 7fcc5dd1b640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 0x7fcc3c06e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000fd0 tx=0x7fcc54001aa0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:35.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.064+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc4804a030 con 0x7fcc58156350 2024-09-10T11:34:35.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.231+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fcc5815c900 con 0x7fcc58156350 2024-09-10T11:34:35.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.232+0000 7fcc4e7fc640 1 -- 172.21.15.99:0/2185873083 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v103) ==== 81+0+34274 (secure 0 0 0) 0x7fcc48079480 con 0x7fcc58156350 2024-09-10T11:34:35.233 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 103, 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:25.573515+0000", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:35.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:35.235 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:35.236 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:35.237 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "95", 2024-09-10T11:34:35.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:35.239 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:35.240 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.241 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:35.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:35.243 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:35.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "102", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:35.245 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:35.246 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:35.247 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:35.248 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:35.249 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:35.250 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "103", 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:35.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:35.252 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:35.253 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:35.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.255 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.256 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.259 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:35.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.261 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.262 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.263 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.264 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.265 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:35.266 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.267 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.271 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:35.272 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.273 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.277 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.279 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.280 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.286 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.288 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.289 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.290 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:35.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.294 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:35.296 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:35.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:35.298 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:35.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.300 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.303 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.304 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.305 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:35.307 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:35.308 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:35.309 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:35.310 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:35.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.234+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 msgr2=0x7fcc3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.234+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 0x7fcc3c06e320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc54000fd0 tx=0x7fcc54001aa0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.234+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 msgr2=0x7fcc5809e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:35.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.234+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc48004730 comp rx=0 tx=0).stop 2024-09-10T11:34:35.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 shutdown_connections 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcc3c06be60 0x7fcc3c06e320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc580ae8d0 0x7fcc580aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc58156350 0x7fcc5809e900 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 --2- 172.21.15.99:0/2185873083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcc44004120 0x7fcc58157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 >> 172.21.15.99:0/2185873083 conn(0x7fcc580730c0 msgr2=0x7fcc580b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 shutdown_connections 2024-09-10T11:34:35.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:35.235+0000 7fcc5ff7f640 1 -- 172.21.15.99:0/2185873083 wait complete. 2024-09-10T11:34:35.314 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:34:37.632 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:37.628+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-09-10T11:34:40.315 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:40.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.425+0000 7feed100e640 1 Processor -- start 2024-09-10T11:34:40.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.425+0000 7feed100e640 1 -- start start 2024-09-10T11:34:40.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feed100e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3960 0x7feecc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feed100e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feecc0b3160 0x7feecc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feed100e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0b3620 con 0x7feecc163580 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feed100e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7feecc06d960 con 0x7feecc0b3160 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feed100e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feecc06dc40 con 0x7feecc0b3960 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feecb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feecc0b3160 0x7feecc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feecb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feecc0b3160 0x7feecc06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35162/0 (socket says 172.21.15.99:35162) 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feecb59e640 1 -- 172.21.15.99:0/1207956332 learned_addr learned my addr 172.21.15.99:0/1207956332 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.426+0000 7feeca59c640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3960 0x7feecc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feecb59e640 1 -- 172.21.15.99:0/1207956332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3960 msgr2=0x7feecc0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feecb59e640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3960 0x7feecc0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feecb59e640 1 -- 172.21.15.99:0/1207956332 >> v1:172.21.15.196:6789/0 conn(0x7feecc163580 legacy=0x7feecc0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feecb59e640 1 -- 172.21.15.99:0/1207956332 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc06df20 con 0x7feecc0b3160 2024-09-10T11:34:40.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feecb59e640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3160 0x7feecc06cd20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a33739d0a6756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feec9d9b640 1 -- 172.21.15.99:0/1207956332 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4037070 con 0x7feecc0b3160 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feec9d9b640 1 -- 172.21.15.99:0/1207956332 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec4032440 con 0x7feecc0b3160 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.427+0000 7feec9d9b640 1 -- 172.21.15.99:0/1207956332 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4032810 con 0x7feecc0b3160 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 -- 172.21.15.99:0/1207956332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3160 msgr2=0x7feecc06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3160 0x7feecc06cd20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec402a130 comp rx=0 tx=0).stop 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 -- 172.21.15.99:0/1207956332 shutdown_connections 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3160 0x7feecc06cd20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 --2- 172.21.15.99:0/1207956332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3960 0x7feecc0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 -- 172.21.15.99:0/1207956332 >> 172.21.15.99:0/1207956332 conn(0x7feecc0b4570 msgr2=0x7feecc0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 -- 172.21.15.99:0/1207956332 shutdown_connections 2024-09-10T11:34:40.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.428+0000 7feed100e640 1 -- 172.21.15.99:0/1207956332 wait complete. 2024-09-10T11:34:40.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 Processor -- start 2024-09-10T11:34:40.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 -- start start 2024-09-10T11:34:40.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3160 0x7feecc159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc163580 0x7feecc0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feecc06d3e0 con 0x7feecc0b3960 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feecc06d260 con 0x7feecc163580 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feed100e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feecc06d560 con 0x7feecc0b3160 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feeca59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.430+0000 7feeca59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35178/0 (socket says 172.21.15.99:35178) 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 -- 172.21.15.99:0/1321822016 learned_addr learned my addr 172.21.15.99:0/1321822016 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feecb59e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc163580 0x7feecc0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feecad9d640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3160 0x7feecc159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 -- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3160 msgr2=0x7feecc159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3160 0x7feecc159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 -- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc163580 msgr2=0x7feecc0b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc163580 0x7feecc0b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc15a250 con 0x7feecc0b3960 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeca59c640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feec000d5f0 tx=0x7feec000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00125f0 con 0x7feecc0b3960 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec0019070 con 0x7feecc0b3960 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc15a530 con 0x7feecc0b3960 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00129d0 con 0x7feecc0b3960 2024-09-10T11:34:40.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.431+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc15add0 con 0x7feecc0b3960 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.432+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc0b5470 con 0x7feecc0b3960 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.433+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feec0012b70 con 0x7feecc0b3960 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.433+0000 7feeb37fe640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 0x7feeb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.433+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7feec008b0d0 con 0x7feecc0b3960 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.433+0000 7feecad9d640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 0x7feeb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.433+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=108}) -- 0x7feeb80739c0 con 0x7feecc0b3960 2024-09-10T11:34:40.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.434+0000 7feecad9d640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 0x7feeb806e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feecc1f86d0 tx=0x7feebc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.438+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feec00594c0 con 0x7feecc0b3960 2024-09-10T11:34:40.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.606+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7feecc06cd20 con 0x7feecc0b3960 2024-09-10T11:34:40.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.607+0000 7feeb37fe640 1 -- 172.21.15.99:0/1321822016 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) ==== 81+0+34426 (secure 0 0 0) 0x7feec005d7c0 con 0x7feecc0b3960 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 107, 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:39.636236+0000", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:40.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:40.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:40.677 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.678 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.679 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.680 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:40.682 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:40.683 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:40.684 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:40.685 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:40.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:40.686 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:40.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 msgr2=0x7feeb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 0x7feeb806e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feecc1f86d0 tx=0x7feebc009040 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 msgr2=0x7feecc15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7feec000d5f0 tx=0x7feec000dab0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 shutdown_connections 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb806be30 0x7feeb806e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc163580 0x7feecc0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc0b3960 0x7feecc15b3f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 --2- 172.21.15.99:0/1321822016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc0b3160 0x7feecc159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.609+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 >> 172.21.15.99:0/1321822016 conn(0x7feecc0b4570 msgr2=0x7feecc0b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.610+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 shutdown_connections 2024-09-10T11:34:40.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.610+0000 7feed100e640 1 -- 172.21.15.99:0/1321822016 wait complete. 2024-09-10T11:34:40.689 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:34:40.689 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:40.689 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:40.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:40.706 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-09-10T11:34:40.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:40.822 INFO:teuthology.orchestra.run.smithi099.stderr:dd: error writing 'large_file_b': No space left on device 2024-09-10T11:34:40.822 INFO:teuthology.orchestra.run.smithi099.stderr:dd: closing output file 'large_file_b': No space left on device 2024-09-10T11:34:40.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.919+0000 7f828070a640 1 Processor -- start 2024-09-10T11:34:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.919+0000 7f828070a640 1 -- start start 2024-09-10T11:34:40.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f828070a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f827815f1e0 0x7f82780b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f828070a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f82780b0090 0x7f82780b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f828070a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82780b40e0 con 0x7f827815ee00 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f828070a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f82780b5ab0 con 0x7f82780b0090 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f828070a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8278073c50 con 0x7f827815f1e0 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f82780b0090 0x7f82780b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f82780b0090 0x7f82780b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35180/0 (socket says 172.21.15.99:35180) 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 -- 172.21.15.99:0/342850416 learned_addr learned my addr 172.21.15.99:0/342850416 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 -- 172.21.15.99:0/342850416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f827815f1e0 msgr2=0x7f82780b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 --2- 172.21.15.99:0/342850416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f827815f1e0 0x7f82780b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 -- 172.21.15.99:0/342850416 >> v1:172.21.15.196:6789/0 conn(0x7f827815ee00 legacy=0x7f82780b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.920+0000 7f827eca7640 1 -- 172.21.15.99:0/342850416 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780b4380 con 0x7f82780b0090 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.921+0000 7f827eca7640 1 --2- 172.21.15.99:0/342850416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82780b0090 0x7f82780b1c70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db89b4eded67887a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.921+0000 7f827d4a4640 1 -- 172.21.15.99:0/342850416 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270037070 con 0x7f82780b0090 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.921+0000 7f827d4a4640 1 -- 172.21.15.99:0/342850416 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8270032440 con 0x7f82780b0090 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.921+0000 7f827d4a4640 1 -- 172.21.15.99:0/342850416 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270032810 con 0x7f82780b0090 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 -- 172.21.15.99:0/342850416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82780b0090 msgr2=0x7f82780b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 --2- 172.21.15.99:0/342850416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82780b0090 0x7f82780b1c70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8270009870 tx=0x7f827002a230 comp rx=0 tx=0).stop 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 -- 172.21.15.99:0/342850416 shutdown_connections 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 --2- 172.21.15.99:0/342850416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f82780b0090 0x7f82780b1c70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 --2- 172.21.15.99:0/342850416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f827815f1e0 0x7f82780b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 -- 172.21.15.99:0/342850416 >> 172.21.15.99:0/342850416 conn(0x7f82780730c0 msgr2=0x7f82780734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 -- 172.21.15.99:0/342850416 shutdown_connections 2024-09-10T11:34:40.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.922+0000 7f828070a640 1 -- 172.21.15.99:0/342850416 wait complete. 2024-09-10T11:34:40.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.923+0000 7f828070a640 1 Processor -- start 2024-09-10T11:34:40.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.923+0000 7f828070a640 1 -- start start 2024-09-10T11:34:40.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 0x7f82781613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827815ee00 0x7f827809d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f82780b21b0 con 0x7f827815f1e0 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82780afd60 con 0x7f827815ee00 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f82780b0600 con 0x7f82780b0090 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35196/0 (socket says 172.21.15.99:35196) 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 -- 172.21.15.99:0/1004821920 learned_addr learned my addr 172.21.15.99:0/1004821920 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827dca5640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827815ee00 0x7f827809d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827e4a6640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 0x7f82781613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 -- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 msgr2=0x7f82781613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 0x7f82781613f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 -- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827815ee00 msgr2=0x7f827809d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827815ee00 0x7f827809d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827809de00 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827e4a6640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 0x7f82781613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f827eca7640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f827002a270 tx=0x7f82700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270041070 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827003c950 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270037040 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827809e0e0 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.924+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827809e6a0 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.925+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82780b5f90 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.926+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8270044070 con 0x7f827815f1e0 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.926+0000 7f826effd640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.926+0000 7f827e4a6640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.926+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f82700aba50 con 0x7f827815f1e0 2024-09-10T11:34:40.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.927+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=108}) -- 0x7f825c0739c0 con 0x7f827815f1e0 2024-09-10T11:34:40.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.927+0000 7f827e4a6640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 0x7f825c06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8274001e50 tx=0x7f8274009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:40.931+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8270079f60 con 0x7f827815f1e0 2024-09-10T11:34:41.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.098+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f82780b3850 con 0x7f827815f1e0 2024-09-10T11:34:41.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.099+0000 7f826effd640 1 -- 172.21.15.99:0/1004821920 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v107) ==== 81+0+34426 (secure 0 0 0) 0x7f827007e260 con 0x7f827815f1e0 2024-09-10T11:34:41.100 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 107, 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:39.636236+0000", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:41.101 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:41.102 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:41.103 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:41.104 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:41.105 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:41.106 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.107 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:41.108 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:41.109 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:41.110 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:41.111 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:41.112 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:41.113 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:41.114 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:41.116 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.118 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:41.119 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:41.120 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:41.121 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:41.122 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:41.122 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:41.122 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:41.122 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:41.123 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:41.124 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.125 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:41.126 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:41.127 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:41.128 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:41.129 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.130 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.131 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.133 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.134 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.135 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:41.136 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.137 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.140 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:41.142 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.143 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.146 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.147 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:41.148 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.150 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.154 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.155 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.156 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:41.159 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.160 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:41.161 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.162 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.163 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.164 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.176 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.177 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:41.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:41.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:41.181 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.182 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.183 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:41.184 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:41.186 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:41.187 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:41.188 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:41.189 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 msgr2=0x7f825c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 0x7f825c06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8274001e50 tx=0x7f8274009040 comp rx=0 tx=0).stop 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 msgr2=0x7f8278149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f827002a270 tx=0x7f82700046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 shutdown_connections 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f827815f1e0 0x7f8278149560 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827815ee00 0x7f827809d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 --2- 172.21.15.99:0/1004821920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f82780b0090 0x7f82781613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 >> 172.21.15.99:0/1004821920 conn(0x7f82780730c0 msgr2=0x7f82780b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 shutdown_connections 2024-09-10T11:34:41.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.101+0000 7f828070a640 1 -- 172.21.15.99:0/1004821920 wait complete. 2024-09-10T11:34:41.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:41.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.290+0000 7f3fdc911640 1 Processor -- start 2024-09-10T11:34:41.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.290+0000 7f3fdc911640 1 -- start start 2024-09-10T11:34:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fdc911640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3fd80d1760 0x7f3fd80cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fdc911640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3fd80d0130 0x7f3fd80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fdc911640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fd80ce430 con 0x7f3fd8160030 2024-09-10T11:34:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fdc911640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd80ce710 con 0x7f3fd80d1760 2024-09-10T11:34:41.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fdc911640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3fd80d1240 con 0x7f3fd80d0130 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fd5d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3fd80d1760 0x7f3fd80cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fd5d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3fd80d1760 0x7f3fd80cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35210/0 (socket says 172.21.15.99:35210) 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.291+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/3266939799 learned_addr learned my addr 172.21.15.99:0/3266939799 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd6d9d640 1 --2- 172.21.15.99:0/3266939799 >> v2:172.21.15.196:3300/0 conn(0x7f3fd80d0130 0x7f3fd80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/3266939799 >> v2:172.21.15.196:3300/0 conn(0x7f3fd80d0130 msgr2=0x7f3fd80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd5d9b640 1 --2- 172.21.15.99:0/3266939799 >> v2:172.21.15.196:3300/0 conn(0x7f3fd80d0130 0x7f3fd80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/3266939799 >> v1:172.21.15.196:6789/0 conn(0x7f3fd8160030 legacy=0x7f3fd80b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/3266939799 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd80ce030 con 0x7f3fd80d1760 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.292+0000 7f3fd5d9b640 1 --2- 172.21.15.99:0/3266939799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd80cd5b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80761c0 tx=0x7f3fcc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5649a8ae229d2b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fd559a640 1 -- 172.21.15.99:0/3266939799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc037070 con 0x7f3fd80d1760 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fd559a640 1 -- 172.21.15.99:0/3266939799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc032440 con 0x7f3fd80d1760 2024-09-10T11:34:41.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fd559a640 1 -- 172.21.15.99:0/3266939799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc032810 con 0x7f3fd80d1760 2024-09-10T11:34:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 -- 172.21.15.99:0/3266939799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 msgr2=0x7f3fd80cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 --2- 172.21.15.99:0/3266939799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd80cd5b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80761c0 tx=0x7f3fcc02a270 comp rx=0 tx=0).stop 2024-09-10T11:34:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 -- 172.21.15.99:0/3266939799 shutdown_connections 2024-09-10T11:34:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 --2- 172.21.15.99:0/3266939799 >> v2:172.21.15.196:3300/0 conn(0x7f3fd80d0130 0x7f3fd80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 --2- 172.21.15.99:0/3266939799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd80cd5b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 -- 172.21.15.99:0/3266939799 >> 172.21.15.99:0/3266939799 conn(0x7f3fd80b4650 msgr2=0x7f3fd80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 -- 172.21.15.99:0/3266939799 shutdown_connections 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.293+0000 7f3fdc911640 1 -- 172.21.15.99:0/3266939799 wait complete. 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 Processor -- start 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 -- start start 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fd80d0130 0x7f3fd80c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 0x7f3fd81f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd81606a0 con 0x7f3fd80d1760 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd80b5950 con 0x7f3fd8160030 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fdc911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd80c7450 con 0x7f3fd80d0130 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fd5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fd5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35224/0 (socket says 172.21.15.99:35224) 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.295+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/222293660 learned_addr learned my addr 172.21.15.99:0/222293660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd659c640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fd80d0130 0x7f3fd80c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd6d9d640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 0x7f3fd81f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fd80d0130 msgr2=0x7f3fd80c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fd80d0130 0x7f3fd80c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 msgr2=0x7f3fd81f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 0x7f3fd81f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd80cc950 con 0x7f3fd80d1760 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd6d9d640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 0x7f3fd81f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fd5d9b640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc02a2b0 tx=0x7f3fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc041020 con 0x7f3fd80d1760 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fcc0306c0 con 0x7f3fd80d1760 2024-09-10T11:34:41.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd80c6e90 con 0x7f3fd80d1760 2024-09-10T11:34:41.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc037040 con 0x7f3fd80d1760 2024-09-10T11:34:41.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.296+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd80cae50 con 0x7f3fd80d1760 2024-09-10T11:34:41.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.297+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd80b38f0 con 0x7f3fd80d1760 2024-09-10T11:34:41.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.298+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fcc044020 con 0x7f3fd80d1760 2024-09-10T11:34:41.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.298+0000 7f3fc6ffd640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 0x7f3fb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.298+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5709+0+0 (secure 0 0 0) 0x7f3fcc0ab620 con 0x7f3fd80d1760 2024-09-10T11:34:41.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.298+0000 7f3fd659c640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 0x7f3fb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.298+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=108}) -- 0x7f3fb80739c0 con 0x7f3fd80d1760 2024-09-10T11:34:41.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.299+0000 7f3fd659c640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 0x7f3fb806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80ca690 tx=0x7f3fc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.303+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3fcc04b030 con 0x7f3fd80d1760 2024-09-10T11:34:41.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.502+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fd80b6e10 con 0x7f3fd80d1760 2024-09-10T11:34:41.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.502+0000 7f3fc6ffd640 1 -- 172.21.15.99:0/222293660 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7f3fcc079fd0 con 0x7f3fd80d1760 2024-09-10T11:34:41.503 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:41.504 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:41.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 msgr2=0x7f3fb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 0x7f3fb806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80ca690 tx=0x7f3fc8009040 comp rx=0 tx=0).stop 2024-09-10T11:34:41.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 msgr2=0x7f3fd81f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc02a2b0 tx=0x7f3fcc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 shutdown_connections 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3fb806be30 0x7f3fb806e2f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fd8160030 0x7f3fd81f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3fd80d1760 0x7f3fd81f4cb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 --2- 172.21.15.99:0/222293660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fd80d0130 0x7f3fd80c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 >> 172.21.15.99:0/222293660 conn(0x7f3fd80b4650 msgr2=0x7f3fd80b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 shutdown_connections 2024-09-10T11:34:41.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.504+0000 7f3fdc911640 1 -- 172.21.15.99:0/222293660 wait complete. 2024-09-10T11:34:41.516 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:34:41.646 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6243,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":17.042514654000001,"mdsmap_epoch":42,"osdmap_epoch":107,"osdmap_epoch_barrier":103,"uptime":20.989397371999999} 2024-09-10T11:34:41.647 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6243, "mdsmap_epoch": 42, "osdmap_epoch": 107, "osdmap_epoch_barrier": 103, "rank_uptime": 17.042514654, "state": "up:active", "uptime": 20.989397372, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:34:41.647 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:34:41.648 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:41.648 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:41.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:41.666 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-09-10T11:34:41.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:34:41.721+0000 7f5bbb7fe640 -1 client.6384 handle_client_reply no pending request on tid 14 safe is:1 2024-09-10T11:34:41.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:41.723 INFO:teuthology.orchestra.run.smithi099.stderr:dd: failed to open 'small_file_1': Permission denied 2024-09-10T11:34:41.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:41.723 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-09-10T11:34:41.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:41.782 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-09-10T11:34:41.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:41.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.996+0000 7f315d598640 1 Processor -- start 2024-09-10T11:34:41.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.996+0000 7f315d598640 1 -- start start 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f315d598640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f315d598640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3158160030 0x7f315806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f315d598640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f315806c530 con 0x7f31580b4930 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f315d598640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f31580b5420 con 0x7f3158160030 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f315d598640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f31580b5140 con 0x7f315806bc10 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3158160030 0x7f315806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3158160030 0x7f315806bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35236/0 (socket says 172.21.15.99:35236) 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 -- 172.21.15.99:0/2562106517 learned_addr learned my addr 172.21.15.99:0/2562106517 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f3156ffd640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 -- 172.21.15.99:0/2562106517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 msgr2=0x7f3158163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 -- 172.21.15.99:0/2562106517 >> v1:172.21.15.196:6789/0 conn(0x7f31580b4930 legacy=0x7f315806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.997+0000 7f314e7fc640 1 -- 172.21.15.99:0/2562106517 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3158162b40 con 0x7f3158160030 2024-09-10T11:34:42.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.998+0000 7f314e7fc640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3158160030 0x7f315806bff0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3140009870 tx=0x7f314002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d5513db03d3e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.998+0000 7f31567fc640 1 -- 172.21.15.99:0/2562106517 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140037070 con 0x7f3158160030 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.998+0000 7f31567fc640 1 -- 172.21.15.99:0/2562106517 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3140032440 con 0x7f3158160030 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.998+0000 7f31567fc640 1 -- 172.21.15.99:0/2562106517 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140032810 con 0x7f3158160030 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 -- 172.21.15.99:0/2562106517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3158160030 msgr2=0x7f315806bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3158160030 0x7f315806bff0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3140009870 tx=0x7f314002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 -- 172.21.15.99:0/2562106517 shutdown_connections 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3158160030 0x7f315806bff0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 --2- 172.21.15.99:0/2562106517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 -- 172.21.15.99:0/2562106517 >> 172.21.15.99:0/2562106517 conn(0x7f31580b3630 msgr2=0x7f31580b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 -- 172.21.15.99:0/2562106517 shutdown_connections 2024-09-10T11:34:42.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:41.999+0000 7f315d598640 1 -- 172.21.15.99:0/2562106517 wait complete. 2024-09-10T11:34:42.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.000+0000 7f315d598640 1 Processor -- start 2024-09-10T11:34:42.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.000+0000 7f315d598640 1 -- start start 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3158160030 0x7f315806f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31580b4db0 con 0x7f31580b4930 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3158163100 con 0x7f3158160030 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f315d598640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f31580d1560 con 0x7f315806bc10 2024-09-10T11:34:42.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35252/0 (socket says 172.21.15.99:35252) 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 -- 172.21.15.99:0/1641898860 learned_addr learned my addr 172.21.15.99:0/1641898860 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f3156ffd640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f31577fe640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3158160030 0x7f315806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 -- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 msgr2=0x7f3158143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 -- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3158160030 msgr2=0x7f315806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3158160030 0x7f315806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.001+0000 7f314e7fc640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31581430d0 con 0x7f31580b4930 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f314e7fc640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3140000c00 tx=0x7f31400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140041020 con 0x7f31580b4930 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31400306c0 con 0x7f31580b4930 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f315806e670 con 0x7f31580b4930 2024-09-10T11:34:42.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140037040 con 0x7f31580b4930 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.002+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f315806ec30 con 0x7f31580b4930 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.003+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31580b2fa0 con 0x7f31580b4930 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.003+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3140044020 con 0x7f31580b4930 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.004+0000 7f314ffff640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 0x7f313806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.004+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f31400ab5c0 con 0x7f31580b4930 2024-09-10T11:34:42.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.004+0000 7f3156ffd640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 0x7f313806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.004+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7f3138073960 con 0x7f31580b4930 2024-09-10T11:34:42.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.004+0000 7f3156ffd640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 0x7f313806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3148006fd0 tx=0x7f3148008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.008+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f31580b2fa0 con 0x7f31580b4930 2024-09-10T11:34:42.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.173+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f3158163620 con 0x7f31580b4930 2024-09-10T11:34:42.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.175+0000 7f314ffff640 1 -- 172.21.15.99:0/1641898860 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) ==== 81+0+34450 (secure 0 0 0) 0x7f314004b030 con 0x7f31580b4930 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 108, 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:41.643846+0000", 2024-09-10T11:34:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:42.182 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:42.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:42.184 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:42.185 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:42.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:42.187 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:42.188 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:42.189 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.190 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:42.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 3, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full", 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:42.192 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "105", 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:42.193 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:42.194 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:42.195 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:42.196 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:42.197 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.198 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:42.199 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.204 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.209 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.211 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.212 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:42.214 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull", 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.215 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.218 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:42.220 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.221 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:42.223 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.224 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.226 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.230 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:42.231 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.232 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:42.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:42.241 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:42.242 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "full", 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:42.244 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.245 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:42.246 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.247 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:42.248 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:42.249 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:42.250 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:42.251 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:42.251 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:42.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:42.251 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:42.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 msgr2=0x7f313806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 0x7f313806e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3148006fd0 tx=0x7f3148008040 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 msgr2=0x7f315815f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3140000c00 tx=0x7f31400046c0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 shutdown_connections 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f313806be30 0x7f313806e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3158160030 0x7f315806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31580b4930 0x7f315815f400 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 --2- 172.21.15.99:0/1641898860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f315806bc10 0x7f3158143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 >> 172.21.15.99:0/1641898860 conn(0x7f31580b3630 msgr2=0x7f3158160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 shutdown_connections 2024-09-10T11:34:42.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:42.177+0000 7f315d598640 1 -- 172.21.15.99:0/1641898860 wait complete. 2024-09-10T11:34:42.254 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:34:45.670 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:45.668+0000 7f266687b640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-09-10T11:34:47.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:47.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.365+0000 7f4370f9f640 1 Processor -- start 2024-09-10T11:34:47.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.365+0000 7f4370f9f640 1 -- start start 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4370f9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 0x7f436c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4370f9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f436c15c060 0x7f436c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4370f9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436c152e60 con 0x7f436c0d1200 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4370f9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f436c0d15e0 con 0x7f436c15c060 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4370f9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f436c072680 con 0x7f436c06ece0 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f436c15c060 0x7f436c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f436c15c060 0x7f436c15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50348/0 (socket says 172.21.15.99:50348) 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 -- 172.21.15.99:0/3423299077 learned_addr learned my addr 172.21.15.99:0/3423299077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4361d9b640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 0x7f436c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 -- 172.21.15.99:0/3423299077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 msgr2=0x7f436c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 0x7f436c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 -- 172.21.15.99:0/3423299077 >> v1:172.21.15.196:6789/0 conn(0x7f436c0d1200 legacy=0x7f436c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f436ad9d640 1 -- 172.21.15.99:0/3423299077 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c072960 con 0x7f436c15c060 2024-09-10T11:34:47.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.366+0000 7f4361d9b640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 0x7f436c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.367+0000 7f436ad9d640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c15c060 0x7f436c15c4a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f436400d150 tx=0x7f436402c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97479c17df95cda8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.367+0000 7f4369d9b640 1 -- 172.21.15.99:0/3423299077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364039070 con 0x7f436c15c060 2024-09-10T11:34:47.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.367+0000 7f4369d9b640 1 -- 172.21.15.99:0/3423299077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364009600 con 0x7f436c15c060 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.367+0000 7f4369d9b640 1 -- 172.21.15.99:0/3423299077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43640099d0 con 0x7f436c15c060 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 -- 172.21.15.99:0/3423299077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c15c060 msgr2=0x7f436c15c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c15c060 0x7f436c15c4a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f436400d150 tx=0x7f436402c840 comp rx=0 tx=0).stop 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 -- 172.21.15.99:0/3423299077 shutdown_connections 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c15c060 0x7f436c15c4a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 --2- 172.21.15.99:0/3423299077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c06ece0 0x7f436c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 -- 172.21.15.99:0/3423299077 >> 172.21.15.99:0/3423299077 conn(0x7f436c06bc10 msgr2=0x7f436c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 -- 172.21.15.99:0/3423299077 shutdown_connections 2024-09-10T11:34:47.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.368+0000 7f4370f9f640 1 -- 172.21.15.99:0/3423299077 wait complete. 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.369+0000 7f4370f9f640 1 Processor -- start 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.369+0000 7f4370f9f640 1 -- start start 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 0x7f436c1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 0x7f436c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f436c15cd90 con 0x7f436c06ece0 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f436c072f20 con 0x7f436c15c060 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4370f9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f436c15c9e0 con 0x7f436c0d1200 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50364/0 (socket says 172.21.15.99:50364) 2024-09-10T11:34:47.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 -- 172.21.15.99:0/1053592093 learned_addr learned my addr 172.21.15.99:0/1053592093 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4361d9b640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 0x7f436c1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436ad9d640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 0x7f436c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 -- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 msgr2=0x7f436c1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 0x7f436c1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 -- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 msgr2=0x7f436c158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 0x7f436c158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436a59c640 1 -- 172.21.15.99:0/1053592093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c158960 con 0x7f436c06ece0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f436ad9d640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 0x7f436c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.370+0000 7f4361d9b640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 0x7f436c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f436a59c640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b420 tx=0x7f435c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c011830 con 0x7f436c06ece0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c016070 con 0x7f436c06ece0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c1f0940 con 0x7f436c06ece0 2024-09-10T11:34:47.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c011c10 con 0x7f436c06ece0 2024-09-10T11:34:47.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.371+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c1f0f00 con 0x7f436c06ece0 2024-09-10T11:34:47.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.372+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c1524f0 con 0x7f436c06ece0 2024-09-10T11:34:47.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.372+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f435c01b460 con 0x7f436c06ece0 2024-09-10T11:34:47.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.373+0000 7f43637fe640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 0x7f434c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.373+0000 7f4361d9b640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 0x7f434c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.373+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f435c08a8e0 con 0x7f436c06ece0 2024-09-10T11:34:47.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.373+0000 7f4361d9b640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 0x7f434c06e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f436c1556b0 tx=0x7f4354008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.377+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f436c1524f0 con 0x7f436c06ece0 2024-09-10T11:34:47.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.542+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f436c1f1620 con 0x7f436c06ece0 2024-09-10T11:34:47.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.544+0000 7f43637fe640 1 -- 172.21.15.99:0/1053592093 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) ==== 81+0+34275 (secure 0 0 0) 0x7f435c020020 con 0x7f436c06ece0 2024-09-10T11:34:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:47.546 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 111, 2024-09-10T11:34:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:47.546 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:46.712521+0000", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:47.547 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:47.548 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.549 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.550 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.551 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.552 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.553 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:47.554 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.555 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:47.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.557 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.559 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.560 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.561 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.562 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.563 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.564 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.565 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.566 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.567 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:47.568 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.574 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.575 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.580 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:47.585 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.586 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.592 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.593 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.594 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.595 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.596 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.597 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:47.598 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:47.599 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:47.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.603 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.622 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 msgr2=0x7f434c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 0x7f434c06e2f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f436c1556b0 tx=0x7f4354008040 comp rx=0 tx=0).stop 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 msgr2=0x7f436c158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f435c00b420 tx=0x7f435c00b8e0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 shutdown_connections 2024-09-10T11:34:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f434c06be30 0x7f434c06e2f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f436c15c060 0x7f436c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f436c0d1200 0x7f436c1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 --2- 172.21.15.99:0/1053592093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f436c06ece0 0x7f436c158420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 >> 172.21.15.99:0/1053592093 conn(0x7f436c06bc10 msgr2=0x7f436c0b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 shutdown_connections 2024-09-10T11:34:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.546+0000 7f4370f9f640 1 -- 172.21.15.99:0/1053592093 wait complete. 2024-09-10T11:34:47.625 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:34:47.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:34:47.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.721+0000 7fb1ee1c4640 1 Processor -- start 2024-09-10T11:34:47.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.721+0000 7fb1ee1c4640 1 -- start start 2024-09-10T11:34:47.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1ee1c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1e81627f0 0x7fb1e8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1ee1c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1e806cd00 0x7fb1e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1ee1c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e80b6d40 con 0x7fb1e8162bd0 2024-09-10T11:34:47.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1ee1c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e80b7000 con 0x7fb1e81627f0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1ee1c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e80b72e0 con 0x7fb1e806cd00 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1e77fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1e81627f0 0x7fb1e8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1e6ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb1e8162bd0 0x7fb1e80b6470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36022/0 (socket says 172.21.15.99:36022) 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.722+0000 7fb1e6ffd640 1 -- 172.21.15.99:0/2318315722 learned_addr learned my addr 172.21.15.99:0/2318315722 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e7fff640 1 --2- 172.21.15.99:0/2318315722 >> v2:172.21.15.196:3300/0 conn(0x7fb1e806cd00 0x7fb1e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e77fe640 1 -- 172.21.15.99:0/2318315722 >> v2:172.21.15.196:3300/0 conn(0x7fb1e806cd00 msgr2=0x7fb1e806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e77fe640 1 --2- 172.21.15.99:0/2318315722 >> v2:172.21.15.196:3300/0 conn(0x7fb1e806cd00 0x7fb1e806d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e77fe640 1 -- 172.21.15.99:0/2318315722 >> v1:172.21.15.99:6789/0 conn(0x7fb1e8162bd0 legacy=0x7fb1e80b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e77fe640 1 -- 172.21.15.99:0/2318315722 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80b75c0 con 0x7fb1e81627f0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.723+0000 7fb1e77fe640 1 --2- 172.21.15.99:0/2318315722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e81627f0 0x7fb1e8070c80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8072aa0 tx=0x7fb1d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe00a98842f5b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1e67fc640 1 -- 172.21.15.99:0/2318315722 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8037070 con 0x7fb1e81627f0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1e67fc640 1 -- 172.21.15.99:0/2318315722 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8032440 con 0x7fb1e81627f0 2024-09-10T11:34:47.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1e67fc640 1 -- 172.21.15.99:0/2318315722 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8032810 con 0x7fb1e81627f0 2024-09-10T11:34:47.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/2318315722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e81627f0 msgr2=0x7fb1e8070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/2318315722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e81627f0 0x7fb1e8070c80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8072aa0 tx=0x7fb1d802a1a0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/2318315722 shutdown_connections 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/2318315722 >> v2:172.21.15.196:3300/0 conn(0x7fb1e806cd00 0x7fb1e806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/2318315722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e81627f0 0x7fb1e8070c80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/2318315722 >> 172.21.15.99:0/2318315722 conn(0x7fb1e809ba30 msgr2=0x7fb1e809be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/2318315722 shutdown_connections 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.724+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/2318315722 wait complete. 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.725+0000 7fb1ee1c4640 1 Processor -- start 2024-09-10T11:34:47.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 -- start start 2024-09-10T11:34:47.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1e806cd00 0x7fb1e81f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 0x7fb1e8152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b2430 con 0x7fb1e8162bd0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b22b0 con 0x7fb1e81627f0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1ee1c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e8162fb0 con 0x7fb1e806cd00 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50374/0 (socket says 172.21.15.99:50374) 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.726+0000 7fb1e7fff640 1 -- 172.21.15.99:0/1583857772 learned_addr learned my addr 172.21.15.99:0/1583857772 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e6ffd640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 0x7fb1e8152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e77fe640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1e806cd00 0x7fb1e81f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 -- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1e806cd00 msgr2=0x7fb1e81f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1e806cd00 0x7fb1e81f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 -- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 msgr2=0x7fb1e8152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 0x7fb1e8152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 -- 172.21.15.99:0/1583857772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e8152aa0 con 0x7fb1e8162bd0 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e6ffd640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 0x7fb1e8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:47.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1e7fff640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007e40 tx=0x7fb1e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0014620 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0019070 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e80cbe80 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0014a20 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.727+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e80cc400 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.728+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e8070c80 con 0x7fb1e8162bd0 2024-09-10T11:34:47.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.729+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1e0014bc0 con 0x7fb1e8162bd0 2024-09-10T11:34:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.729+0000 7fb1cffff640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 0x7fb1d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.729+0000 7fb1e77fe640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 0x7fb1d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.730+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e008e070 con 0x7fb1e8162bd0 2024-09-10T11:34:47.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.730+0000 7fb1e77fe640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 0x7fb1d406e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8034000 tx=0x7fb1d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:47.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.734+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1e8152aa0 con 0x7fb1e8162bd0 2024-09-10T11:34:47.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.899+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fb1e815e5b0 con 0x7fb1e8162bd0 2024-09-10T11:34:47.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.900+0000 7fb1cffff640 1 -- 172.21.15.99:0/1583857772 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) ==== 81+0+34275 (secure 0 0 0) 0x7fb1e005a120 con 0x7fb1e8162bd0 2024-09-10T11:34:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:47.905 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:47.905 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 111, 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:34:46.712521+0000", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:34:47.906 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 13, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.907 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.908 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.909 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.910 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.912 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:34:47.913 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 12, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.533186+0000", 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:34:47.915 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.917 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.918 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:34:47.919 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.75, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.75, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.75, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.75, 2024-09-10T11:34:47.920 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 13, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:34:22.904618+0000", 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:34:47.921 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:34:47.922 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:34:47.923 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:34:47.924 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:34:47.925 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:34:47.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:34:47.927 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.928 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:34:47.932 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.933 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.934 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:34:47.935 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:34:47.936 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.940 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.941 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.942 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.943 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.946 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.947 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.948 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:34:47.951 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.952 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:34:47.956 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.957 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.958 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:34:47.960 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.961 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.962 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.964 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:34:47.969 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 101, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:34:47.973 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.974 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.975 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.976 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.978 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.979 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:34:47.981 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:47.982 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:47.983 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 msgr2=0x7fb1d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 0x7fb1d406e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8034000 tx=0x7fb1d802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 msgr2=0x7fb1e80a0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0007e40 tx=0x7fb1e000e4b0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 shutdown_connections 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb1d406be30 0x7fb1d406e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e8162bd0 0x7fb1e80a0360 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e81627f0 0x7fb1e8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 --2- 172.21.15.99:0/1583857772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1e806cd00 0x7fb1e81f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:47.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 >> 172.21.15.99:0/1583857772 conn(0x7fb1e809ba30 msgr2=0x7fb1e8070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:47.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 shutdown_connections 2024-09-10T11:34:47.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:47.902+0000 7fb1ee1c4640 1 -- 172.21.15.99:0/1583857772 wait complete. 2024-09-10T11:34:47.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:48.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.084+0000 7f3dca1a1640 1 Processor -- start 2024-09-10T11:34:48.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.084+0000 7f3dca1a1640 1 -- start start 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.084+0000 7f3dca1a1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 0x7f3dc406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.084+0000 7f3dca1a1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3dc406ee30 0x7f3dc406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.084+0000 7f3dca1a1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dc40ceb10 con 0x7f3dc415c7b0 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dca1a1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc406f280 con 0x7f3dc406ee30 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dca1a1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc40b3840 con 0x7f3dc40d1180 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3dc406ee30 0x7f3dc406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3dc406ee30 0x7f3dc406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50388/0 (socket says 172.21.15.99:50388) 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc37fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3dc415c7b0 0x7f3dc40d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36026/0 (socket says 172.21.15.99:36026) 2024-09-10T11:34:48.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 -- 172.21.15.99:0/366316884 learned_addr learned my addr 172.21.15.99:0/366316884 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dbaffd640 1 --2- 172.21.15.99:0/366316884 >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 0x7f3dc406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 -- 172.21.15.99:0/366316884 >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 msgr2=0x7f3dc406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 --2- 172.21.15.99:0/366316884 >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 0x7f3dc406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 -- 172.21.15.99:0/366316884 >> v1:172.21.15.99:6789/0 conn(0x7f3dc415c7b0 legacy=0x7f3dc40d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc3fff640 1 -- 172.21.15.99:0/366316884 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc40b3a90 con 0x7f3dc406ee30 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dc2ffd640 1 -- 172.21.15.99:0/366316884 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2709102751 0 0) 0x7f3db4001110 con 0x7f3dc415c7b0 2024-09-10T11:34:48.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.085+0000 7f3dbaffd640 1 --2- 172.21.15.99:0/366316884 >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 0x7f3dc406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:48.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dc3fff640 1 --2- 172.21.15.99:0/366316884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc406d4c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40d0280 tx=0x7f3dbc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17cf7cba105425 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dc2ffd640 1 -- 172.21.15.99:0/366316884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc036070 con 0x7f3dc406ee30 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dc2ffd640 1 -- 172.21.15.99:0/366316884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dbc031600 con 0x7f3dc406ee30 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dc2ffd640 1 -- 172.21.15.99:0/366316884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0319d0 con 0x7f3dc406ee30 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 -- 172.21.15.99:0/366316884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 msgr2=0x7f3dc406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/366316884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc406d4c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40d0280 tx=0x7f3dbc02a730 comp rx=0 tx=0).stop 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 -- 172.21.15.99:0/366316884 shutdown_connections 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/366316884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc406d4c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/366316884 >> v2:172.21.15.196:3300/0 conn(0x7f3dc40d1180 0x7f3dc406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 -- 172.21.15.99:0/366316884 >> 172.21.15.99:0/366316884 conn(0x7f3dc406bc10 msgr2=0x7f3dc406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 -- 172.21.15.99:0/366316884 shutdown_connections 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.086+0000 7f3dca1a1640 1 -- 172.21.15.99:0/366316884 wait complete. 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.088+0000 7f3dca1a1640 1 Processor -- start 2024-09-10T11:34:48.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.088+0000 7f3dca1a1640 1 -- start start 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.088+0000 7f3dca1a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3dc40d1180 0x7f3dc41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc415c7b0 0x7f3dc4072960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc40b4330 con 0x7f3dc406ee30 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc40b4050 con 0x7f3dc415c7b0 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc40b4610 con 0x7f3dc40d1180 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50394/0 (socket says 172.21.15.99:50394) 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 -- 172.21.15.99:0/2968410820 learned_addr learned my addr 172.21.15.99:0/2968410820 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 -- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3dc40d1180 msgr2=0x7f3dc41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3dc40d1180 0x7f3dc41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 -- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc415c7b0 msgr2=0x7f3dc4072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc415c7b0 0x7f3dc4072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 -- 172.21.15.99:0/2968410820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc40718c0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc37fe640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c430 tx=0x7f3db400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db40074e0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db40047e0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc4071ba0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4004bd0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.089+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc4072160 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.090+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc4069640 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.091+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3db4011070 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.091+0000 7f3dc0ff9640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 0x7f3da406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.091+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f3db408a9d0 con 0x7f3dc406ee30 2024-09-10T11:34:48.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.091+0000 7f3dbaffd640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 0x7f3da406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.091+0000 7f3dbaffd640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 0x7f3da406e320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006e80 tx=0x7f3dac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.095+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3dc4069640 con 0x7f3dc406ee30 2024-09-10T11:34:48.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.294+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc406bff0 con 0x7f3dc406ee30 2024-09-10T11:34:48.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.294+0000 7f3dc0ff9640 1 -- 172.21.15.99:0/2968410820 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7f3dc406bff0 con 0x7f3dc406ee30 2024-09-10T11:34:48.296 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:48.297 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:48.297 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:48.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 msgr2=0x7f3da406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 0x7f3da406e320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3dac006e80 tx=0x7f3dac008040 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 msgr2=0x7f3dc41f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3db400c430 tx=0x7f3db400c8f0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 shutdown_connections 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3da406be60 0x7f3da406e320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3dc415c7b0 0x7f3dc4072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3dc40d1180 0x7f3dc41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 --2- 172.21.15.99:0/2968410820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3dc406ee30 0x7f3dc41f9030 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 >> 172.21.15.99:0/2968410820 conn(0x7f3dc406bc10 msgr2=0x7f3dc40b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:48.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 shutdown_connections 2024-09-10T11:34:48.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.296+0000 7f3dca1a1640 1 -- 172.21.15.99:0/2968410820 wait complete. 2024-09-10T11:34:48.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:34:48.439 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6243,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.835611948,"mdsmap_epoch":42,"osdmap_epoch":111,"osdmap_epoch_barrier":107,"uptime":27.782496809000001} 2024-09-10T11:34:48.440 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6243, "mdsmap_epoch": 42, "osdmap_epoch": 111, "osdmap_epoch_barrier": 107, "rank_uptime": 23.835611948, "state": "up:active", "uptime": 27.782496809, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:34:48.440 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:34:48.440 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:48.440 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:48.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:48.457 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-09-10T11:34:48.721 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2024-09-10T11:34:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2024-09-10T11:34:48.722 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.20642 s, 254 MB/s 2024-09-10T11:34:48.722 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:34:48.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:48.761 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:48.807 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:34:48.807 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:48.808 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:48.808 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:48.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:34:48.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.960+0000 7f72cf22e640 1 Processor -- start 2024-09-10T11:34:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 -- start start 2024-09-10T11:34:48.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 0x7f72c80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f72c80d5010 0x7f72c80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c815ff70 con 0x7f72c81626c0 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f72c80d1620 con 0x7f72c80d5010 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.961+0000 7f72cf22e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f72c8162bc0 con 0x7f72c815eb80 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72cd7cb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f72c80d5010 0x7f72c80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72ccfca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 0x7f72c80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72c7fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f72c81626c0 0x7f72c80d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:46172/0 (socket says 172.21.15.99:46172) 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72c7fff640 1 -- 172.21.15.99:0/2093365228 learned_addr learned my addr 172.21.15.99:0/2093365228 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72c77fe640 1 -- 172.21.15.99:0/2093365228 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394293692 0 0) 0x7f72c815ff70 con 0x7f72c81626c0 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72c77fe640 1 -- 172.21.15.99:0/2093365228 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72b4003520 con 0x7f72c81626c0 2024-09-10T11:34:48.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72cd7cb640 1 -- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 msgr2=0x7f72c80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72cd7cb640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 0x7f72c80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72cd7cb640 1 -- 172.21.15.99:0/2093365228 >> v1:172.21.15.196:6789/0 conn(0x7f72c81626c0 legacy=0x7f72c80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72cd7cb640 1 -- 172.21.15.99:0/2093365228 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c8162ea0 con 0x7f72c80d5010 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.962+0000 7f72ccfca640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 0x7f72c80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72cd7cb640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c80d1b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72c00097a0 tx=0x7f72c002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70daca9e14ef27ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72c77fe640 1 -- 172.21.15.99:0/2093365228 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0037070 con 0x7f72c80d5010 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72c77fe640 1 -- 172.21.15.99:0/2093365228 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72c002ae50 con 0x7f72c80d5010 2024-09-10T11:34:48.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72c77fe640 1 -- 172.21.15.99:0/2093365228 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0032640 con 0x7f72c80d5010 2024-09-10T11:34:48.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72cf22e640 1 -- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 msgr2=0x7f72c80d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.963+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c80d1b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f72c00097a0 tx=0x7f72c002a100 comp rx=0 tx=0).stop 2024-09-10T11:34:48.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 -- 172.21.15.99:0/2093365228 shutdown_connections 2024-09-10T11:34:48.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c80d1b20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2093365228 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c815eb80 0x7f72c80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 -- 172.21.15.99:0/2093365228 >> 172.21.15.99:0/2093365228 conn(0x7f72c808e680 msgr2=0x7f72c809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 -- 172.21.15.99:0/2093365228 shutdown_connections 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.964+0000 7f72cf22e640 1 -- 172.21.15.99:0/2093365228 wait complete. 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.965+0000 7f72cf22e640 1 Processor -- start 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.965+0000 7f72cf22e640 1 -- start start 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c81626c0 0x7f72c80b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f72c80d2ef0 con 0x7f72c815eb80 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8163460 con 0x7f72c80d5010 2024-09-10T11:34:48.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cf22e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f72c80d3070 con 0x7f72c81626c0 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50404/0 (socket says 172.21.15.99:50404) 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72ccfca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 -- 172.21.15.99:0/2900651858 learned_addr learned my addr 172.21.15.99:0/2900651858 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72cd7cb640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c81626c0 0x7f72c80b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 -- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c81626c0 msgr2=0x7f72c80b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c81626c0 0x7f72c80b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 -- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 msgr2=0x7f72c815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 -- 172.21.15.99:0/2900651858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c81e9ca0 con 0x7f72c815eb80 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72ccfca640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:48.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.966+0000 7f72c7fff640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c410 tx=0x7f72b000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.967+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0017070 con 0x7f72c815eb80 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.967+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b0005ce0 con 0x7f72c815eb80 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.967+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c81e9f80 con 0x7f72c815eb80 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.967+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00072f0 con 0x7f72c815eb80 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.967+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c80b44d0 con 0x7f72c815eb80 2024-09-10T11:34:48.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.968+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c80d29b0 con 0x7f72c815eb80 2024-09-10T11:34:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.968+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72b001a070 con 0x7f72c815eb80 2024-09-10T11:34:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.968+0000 7f72c57fa640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 0x7f72a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.968+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f72b008a1c0 con 0x7f72c815eb80 2024-09-10T11:34:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.968+0000 7f72ccfca640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 0x7f72a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:48.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.969+0000 7f72ccfca640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 0x7f72a806e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009fd0 tx=0x7f72b800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:48.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:48.973+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72b00585e0 con 0x7f72c815eb80 2024-09-10T11:34:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.148+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f72c809e040 con 0x7f72c815eb80 2024-09-10T11:34:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.148+0000 7f72c57fa640 1 -- 172.21.15.99:0/2900651858 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7f72b005c8e0 con 0x7f72c815eb80 2024-09-10T11:34:49.150 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:34:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 msgr2=0x7f72a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 0x7f72a806e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009fd0 tx=0x7f72b800b040 comp rx=0 tx=0).stop 2024-09-10T11:34:49.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 msgr2=0x7f72c8159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c410 tx=0x7f72b000c8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 shutdown_connections 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f72a806be30 0x7f72a806e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72c81626c0 0x7f72c80b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72c815eb80 0x7f72c8159430 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 --2- 172.21.15.99:0/2900651858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72c80d5010 0x7f72c815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 >> 172.21.15.99:0/2900651858 conn(0x7f72c808e680 msgr2=0x7f72c806dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 shutdown_connections 2024-09-10T11:34:49.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.150+0000 7f72cf22e640 1 -- 172.21.15.99:0/2900651858 wait complete. 2024-09-10T11:34:49.163 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:34:49.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:34:49.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[49398]: fuse finished with error 0 and tester_r 0 2024-09-10T11:34:49.194 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:34:49.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.194 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:34:49.360 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:49.360 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:49.361 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:49.378 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:34:49.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.497+0000 7ff1bc6ac640 1 Processor -- start 2024-09-10T11:34:49.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.497+0000 7ff1bc6ac640 1 -- start start 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1bc6ac640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1b415eb80 0x7ff1b40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1bc6ac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 0x7ff1b40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1bc6ac640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b415ff70 con 0x7ff1b41626c0 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1bc6ac640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b40d1620 con 0x7ff1b415eb80 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1bc6ac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b4162bc0 con 0x7ff1b40d5010 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.498+0000 7ff1ba448640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1b415eb80 0x7ff1b40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1b9c47640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff1b41626c0 0x7ff1b40d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36036/0 (socket says 172.21.15.99:36036) 2024-09-10T11:34:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1b9c47640 1 -- 172.21.15.99:0/1195977427 learned_addr learned my addr 172.21.15.99:0/1195977427 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1bac49640 1 --2- 172.21.15.99:0/1195977427 >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 0x7ff1b40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1ba448640 1 -- 172.21.15.99:0/1195977427 >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 msgr2=0x7ff1b40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1195977427 >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 0x7ff1b40d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1ba448640 1 -- 172.21.15.99:0/1195977427 >> v1:172.21.15.99:6789/0 conn(0x7ff1b41626c0 legacy=0x7ff1b40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1ba448640 1 -- 172.21.15.99:0/1195977427 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b4162ea0 con 0x7ff1b415eb80 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1b9446640 1 -- 172.21.15.99:0/1195977427 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2280138244 0 0) 0x7ff1b415ff70 con 0x7ff1b41626c0 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.499+0000 7ff1bac49640 1 --2- 172.21.15.99:0/1195977427 >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 0x7ff1b40d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1195977427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b40d29b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009d90 tx=0x7ff1a402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297d42981749b5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1b9446640 1 -- 172.21.15.99:0/1195977427 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4031600 con 0x7ff1b415eb80 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1b9446640 1 -- 172.21.15.99:0/1195977427 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a40317a0 con 0x7ff1b415eb80 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1b9446640 1 -- 172.21.15.99:0/1195977427 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4031b70 con 0x7ff1b415eb80 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1195977427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 msgr2=0x7ff1b40d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1195977427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b40d29b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009d90 tx=0x7ff1a402a610 comp rx=0 tx=0).stop 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1195977427 shutdown_connections 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1195977427 >> v2:172.21.15.196:3300/0 conn(0x7ff1b40d5010 0x7ff1b40d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1195977427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b40d29b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1195977427 >> 172.21.15.99:0/1195977427 conn(0x7ff1b408e680 msgr2=0x7ff1b409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1195977427 shutdown_connections 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.500+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1195977427 wait complete. 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.501+0000 7ff1bc6ac640 1 Processor -- start 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 -- start start 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 0x7ff1b41f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1b41626c0 0x7ff1b40d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b40d2ef0 con 0x7ff1b415eb80 2024-09-10T11:34:49.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b4163460 con 0x7ff1b41626c0 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bc6ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b40d3070 con 0x7ff1b40d5010 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1b9c47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1b9c47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50426/0 (socket says 172.21.15.99:50426) 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1b9c47640 1 -- 172.21.15.99:0/1095793504 learned_addr learned my addr 172.21.15.99:0/1095793504 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 0x7ff1b41f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.502+0000 7ff1bac49640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1b41626c0 0x7ff1b40d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 -- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 msgr2=0x7ff1b41f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 0x7ff1b41f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 -- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1b41626c0 msgr2=0x7ff1b40d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1b41626c0 0x7ff1b40d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 -- 172.21.15.99:0/1095793504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b41f4ed0 con 0x7ff1b415eb80 2024-09-10T11:34:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 0x7ff1b41f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b9c47640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c410 tx=0x7ff1a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80074e0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80047e0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b41f51b0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8004bd0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.503+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b40b25d0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.504+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b40d29b0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.504+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1a8011070 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.505+0000 7ff1b2ffd640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 0x7ff1a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.505+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 0x7ff1a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.505+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7ff1a808a9a0 con 0x7ff1b415eb80 2024-09-10T11:34:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.505+0000 7ff1ba448640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 0x7ff1a006e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4034000 tx=0x7ff1a4033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:49.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.509+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1a8059350 con 0x7ff1b415eb80 2024-09-10T11:34:49.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.686+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff1b409e320 con 0x7ff1b415eb80 2024-09-10T11:34:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.686+0000 7ff1b2ffd640 1 -- 172.21.15.99:0/1095793504 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7ff1a8058c50 con 0x7ff1b415eb80 2024-09-10T11:34:49.688 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 msgr2=0x7ff1a006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 0x7ff1a006e320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4034000 tx=0x7ff1a4033040 comp rx=0 tx=0).stop 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 msgr2=0x7ff1b41f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800c410 tx=0x7ff1a800c8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 shutdown_connections 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1a006be60 0x7ff1a006e320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1b41626c0 0x7ff1b40d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1b415eb80 0x7ff1b41f4610 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 --2- 172.21.15.99:0/1095793504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1b40d5010 0x7ff1b41f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 >> 172.21.15.99:0/1095793504 conn(0x7ff1b408e680 msgr2=0x7ff1b40d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 shutdown_connections 2024-09-10T11:34:49.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:49.688+0000 7ff1bc6ac640 1 -- 172.21.15.99:0/1095793504 wait complete. 2024-09-10T11:34:49.701 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:34:49.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.701 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:34:49.727 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[36823]: fuse finished with error 0 and tester_r 0 2024-09-10T11:34:49.729 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:34:49.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.730 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:34:49.896 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:49.897 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:49.897 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:49.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:49.914 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:34:49.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.915 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:34:49.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:49.970 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:34:49.970 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:34:49.970 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:49.971 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:49.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:49.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:34:49.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:34:49.988 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:34:50.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:34:50.044 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:34:50.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:50.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.141+0000 7f2ad414b640 1 Processor -- start 2024-09-10T11:34:50.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.141+0000 7f2ad414b640 1 -- start start 2024-09-10T11:34:50.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 0x7f2acc0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2acc06ee30 0x7f2acc1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc0d1420 con 0x7f2acc06ee30 2024-09-10T11:34:50.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc06f2a0 con 0x7f2acc15c7b0 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad414b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc0b3840 con 0x7f2acc0d0fb0 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad26e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2acc06ee30 0x7f2acc1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad26e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2acc06ee30 0x7f2acc1604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50438/0 (socket says 172.21.15.99:50438) 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.142+0000 7f2ad26e8640 1 -- 172.21.15.99:0/3557481739 learned_addr learned my addr 172.21.15.99:0/3557481739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad1ee7640 1 --2- 172.21.15.99:0/3557481739 >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 0x7f2acc0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad16e6640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 -- 172.21.15.99:0/3557481739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 msgr2=0x7f2acc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 -- 172.21.15.99:0/3557481739 >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 msgr2=0x7f2acc0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 --2- 172.21.15.99:0/3557481739 >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 0x7f2acc0d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 -- 172.21.15.99:0/3557481739 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc0ceb10 con 0x7f2acc06ee30 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad1ee7640 1 --2- 172.21.15.99:0/3557481739 >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 0x7f2acc0d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:50.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.143+0000 7f2ad26e8640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc1604b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0cd7c0 tx=0x7f2ac402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44352c836d27336d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad0ee5640 1 -- 172.21.15.99:0/3557481739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4036070 con 0x7f2acc06ee30 2024-09-10T11:34:50.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad0ee5640 1 -- 172.21.15.99:0/3557481739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac4031600 con 0x7f2acc06ee30 2024-09-10T11:34:50.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad0ee5640 1 -- 172.21.15.99:0/3557481739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40319d0 con 0x7f2acc06ee30 2024-09-10T11:34:50.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 -- 172.21.15.99:0/3557481739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 msgr2=0x7f2acc1604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc1604b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0cd7c0 tx=0x7f2ac402a6a0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 -- 172.21.15.99:0/3557481739 shutdown_connections 2024-09-10T11:34:50.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc1604b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 --2- 172.21.15.99:0/3557481739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 --2- 172.21.15.99:0/3557481739 >> v2:172.21.15.196:3300/0 conn(0x7f2acc15c7b0 0x7f2acc0d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 -- 172.21.15.99:0/3557481739 >> 172.21.15.99:0/3557481739 conn(0x7f2acc06bc10 msgr2=0x7f2acc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 -- 172.21.15.99:0/3557481739 shutdown_connections 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.144+0000 7f2ad414b640 1 -- 172.21.15.99:0/3557481739 wait complete. 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.145+0000 7f2ad414b640 1 Processor -- start 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.145+0000 7f2ad414b640 1 -- start start 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 0x7f2acc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0b4050 con 0x7f2acc06ee30 2024-09-10T11:34:50.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0b3d70 con 0x7f2acc15c7b0 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad414b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc0b4330 con 0x7f2acc0d0fb0 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50442/0 (socket says 172.21.15.99:50442) 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 -- 172.21.15.99:0/46316052 learned_addr learned my addr 172.21.15.99:0/46316052 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad16e6640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad26e8640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 0x7f2acc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 -- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 msgr2=0x7f2acc0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 -- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 msgr2=0x7f2acc0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 0x7f2acc0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad1ee7640 1 -- 172.21.15.99:0/46316052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc1511c0 con 0x7f2acc06ee30 2024-09-10T11:34:50.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad26e8640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 0x7f2acc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.146+0000 7f2ad16e6640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2ad1ee7640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d7d0 tx=0x7f2abc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc004030 con 0x7f2acc06ee30 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0041d0 con 0x7f2acc06ee30 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc1514a0 con 0x7f2acc06ee30 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00b6c0 con 0x7f2acc06ee30 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.147+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc151a60 con 0x7f2acc06ee30 2024-09-10T11:34:50.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.148+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc15cb90 con 0x7f2acc06ee30 2024-09-10T11:34:50.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.148+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2abc00b860 con 0x7f2acc06ee30 2024-09-10T11:34:50.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.148+0000 7f2aca7fc640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 0x7f2ab806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.149+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f2abc08b0e0 con 0x7f2acc06ee30 2024-09-10T11:34:50.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.149+0000 7f2ad16e6640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 0x7f2ab806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.149+0000 7f2ad16e6640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 0x7f2ab806e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2acc150400 tx=0x7f2ac0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.153+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2abc059a90 con 0x7f2acc06ee30 2024-09-10T11:34:50.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.353+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2acc0b26c0 con 0x7f2acc06ee30 2024-09-10T11:34:50.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.353+0000 7f2aca7fc640 1 -- 172.21.15.99:0/46316052 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7f2abc059390 con 0x7f2acc06ee30 2024-09-10T11:34:50.355 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:50.355 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:50.356 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 msgr2=0x7f2ab806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 0x7f2ab806e2f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2acc150400 tx=0x7f2ac0008040 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 msgr2=0x7f2acc140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00d7d0 tx=0x7f2abc00dc90 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 shutdown_connections 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab806be30 0x7f2ab806e2f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2acc15c7b0 0x7f2acc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2acc0d0fb0 0x7f2acc0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 --2- 172.21.15.99:0/46316052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2acc06ee30 0x7f2acc140a00 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 >> 172.21.15.99:0/46316052 conn(0x7f2acc06bc10 msgr2=0x7f2acc06e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 shutdown_connections 2024-09-10T11:34:50.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.355+0000 7f2ad414b640 1 -- 172.21.15.99:0/46316052 wait complete. 2024-09-10T11:34:50.368 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:50.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.465+0000 7fb45188d640 1 Processor -- start 2024-09-10T11:34:50.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.465+0000 7fb45188d640 1 -- start start 2024-09-10T11:34:50.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb45188d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 0x7fb44c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb45188d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb44c0d5010 0x7fb44c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb45188d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb44c15ff70 con 0x7fb44c1626c0 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb45188d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb44c0d1620 con 0x7fb44c0d5010 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb45188d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb44c162bc0 con 0x7fb44c15eb80 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb44c0d5010 0x7fb44c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb44c0d5010 0x7fb44c0d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50452/0 (socket says 172.21.15.99:50452) 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 -- 172.21.15.99:0/2121404775 learned_addr learned my addr 172.21.15.99:0/2121404775 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44affd640 1 --2- 172.21.15.99:0/2121404775 >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 0x7fb44c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 -- 172.21.15.99:0/2121404775 >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 msgr2=0x7fb44c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 --2- 172.21.15.99:0/2121404775 >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 0x7fb44c0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 -- 172.21.15.99:0/2121404775 >> v1:172.21.15.196:6789/0 conn(0x7fb44c1626c0 legacy=0x7fb44c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:50.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44b7fe640 1 -- 172.21.15.99:0/2121404775 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c162ea0 con 0x7fb44c0d5010 2024-09-10T11:34:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.466+0000 7fb44affd640 1 --2- 172.21.15.99:0/2121404775 >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 0x7fb44c0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb44b7fe640 1 --2- 172.21.15.99:0/2121404775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0d5010 0x7fb44c0d1b20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb444009e50 tx=0x7fb44402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0cf43866b9de0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb44a7fc640 1 -- 172.21.15.99:0/2121404775 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444036070 con 0x7fb44c0d5010 2024-09-10T11:34:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb44a7fc640 1 -- 172.21.15.99:0/2121404775 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb444031600 con 0x7fb44c0d5010 2024-09-10T11:34:50.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb44a7fc640 1 -- 172.21.15.99:0/2121404775 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44402ae60 con 0x7fb44c0d5010 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 -- 172.21.15.99:0/2121404775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0d5010 msgr2=0x7fb44c0d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 --2- 172.21.15.99:0/2121404775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0d5010 0x7fb44c0d1b20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb444009e50 tx=0x7fb44402a680 comp rx=0 tx=0).stop 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 -- 172.21.15.99:0/2121404775 shutdown_connections 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 --2- 172.21.15.99:0/2121404775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c0d5010 0x7fb44c0d1b20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 --2- 172.21.15.99:0/2121404775 >> v2:172.21.15.196:3300/0 conn(0x7fb44c15eb80 0x7fb44c0d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.467+0000 7fb45188d640 1 -- 172.21.15.99:0/2121404775 >> 172.21.15.99:0/2121404775 conn(0x7fb44c08e680 msgr2=0x7fb44c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.468+0000 7fb45188d640 1 -- 172.21.15.99:0/2121404775 shutdown_connections 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.468+0000 7fb45188d640 1 -- 172.21.15.99:0/2121404775 wait complete. 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.469+0000 7fb45188d640 1 Processor -- start 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.469+0000 7fb45188d640 1 -- start start 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c0d5010 0x7fb44c14abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb44c1626c0 0x7fb44c14d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0d2ef0 con 0x7fb44c15eb80 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c163460 con 0x7fb44c0d5010 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c0d3070 con 0x7fb44c1626c0 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50458/0 (socket says 172.21.15.99:50458) 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 -- 172.21.15.99:0/2069967538 learned_addr learned my addr 172.21.15.99:0/2069967538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb44affd640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c0d5010 0x7fb44c14abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb44b7fe640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb44c1626c0 0x7fb44c14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 -- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb44c1626c0 msgr2=0x7fb44c14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb44c1626c0 0x7fb44c14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 -- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c0d5010 msgr2=0x7fb44c14abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c0d5010 0x7fb44c14abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 -- 172.21.15.99:0/2069967538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c0b37b0 con 0x7fb44c15eb80 2024-09-10T11:34:50.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb4427fc640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb43400b3f0 tx=0x7fb43400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434015020 con 0x7fb44c15eb80 2024-09-10T11:34:50.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb434011830 con 0x7fb44c15eb80 2024-09-10T11:34:50.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c0b3a90 con 0x7fb44c15eb80 2024-09-10T11:34:50.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb434011c20 con 0x7fb44c15eb80 2024-09-10T11:34:50.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.470+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c14b730 con 0x7fb44c15eb80 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.471+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c0d29b0 con 0x7fb44c15eb80 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.472+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb434018020 con 0x7fb44c15eb80 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.472+0000 7fb448ff9640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 0x7fb42c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.472+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fb43408a940 con 0x7fb44c15eb80 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.472+0000 7fb44affd640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 0x7fb42c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.473+0000 7fb44affd640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 0x7fb42c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00aac0 tx=0x7fb43c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.477+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb43401f030 con 0x7fb44c15eb80 2024-09-10T11:34:50.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.677+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb44c0b75c0 con 0x7fb44c15eb80 2024-09-10T11:34:50.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.677+0000 7fb448ff9640 1 -- 172.21.15.99:0/2069967538 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7fb434058ca0 con 0x7fb44c15eb80 2024-09-10T11:34:50.679 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:50.679 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:50.680 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 msgr2=0x7fb42c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 0x7fb42c06e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00aac0 tx=0x7fb43c008040 comp rx=0 tx=0).stop 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 msgr2=0x7fb44c0b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb43400b3f0 tx=0x7fb43400b8b0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 shutdown_connections 2024-09-10T11:34:50.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb42c06be30 0x7fb42c06e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb44c1626c0 0x7fb44c14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb44c15eb80 0x7fb44c0b7af0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 --2- 172.21.15.99:0/2069967538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb44c0d5010 0x7fb44c14abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 >> 172.21.15.99:0/2069967538 conn(0x7fb44c08e680 msgr2=0x7fb44c0b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 shutdown_connections 2024-09-10T11:34:50.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.679+0000 7fb45188d640 1 -- 172.21.15.99:0/2069967538 wait complete. 2024-09-10T11:34:50.692 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:50.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.789+0000 7f258dd11640 1 Processor -- start 2024-09-10T11:34:50.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.789+0000 7f258dd11640 1 -- start start 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f258dd11640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f258815aef0 0x7f25880b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f258dd11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f258dd11640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25880b0020 con 0x7f258815b2d0 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f258dd11640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f258806ee10 con 0x7f258815aef0 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f258dd11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f25880cd9a0 con 0x7f25880afbb0 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f25877fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f258815aef0 0x7f25880b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f25877fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f258815aef0 0x7f25880b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50468/0 (socket says 172.21.15.99:50468) 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f25877fe640 1 -- 172.21.15.99:0/3410941336 learned_addr learned my addr 172.21.15.99:0/3410941336 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f2587fff640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f25877fe640 1 -- 172.21.15.99:0/3410941336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 msgr2=0x7f25880714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.790+0000 7f25877fe640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25877fe640 1 -- 172.21.15.99:0/3410941336 >> v1:172.21.15.99:6789/0 conn(0x7f258815b2d0 legacy=0x7f25880cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25867fc640 1 -- 172.21.15.99:0/3410941336 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 329689140 0 0) 0x7f25880b0020 con 0x7f258815b2d0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25877fe640 1 -- 172.21.15.99:0/3410941336 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880b0290 con 0x7f258815aef0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25877fe640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815aef0 0x7f25880b4510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f257800bb10 tx=0x7f257802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ef2a5ce5d3a23c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25867fc640 1 -- 172.21.15.99:0/3410941336 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578036020 con 0x7f258815aef0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25867fc640 1 -- 172.21.15.99:0/3410941336 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2578032820 con 0x7f258815aef0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.791+0000 7f25867fc640 1 -- 172.21.15.99:0/3410941336 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578032bf0 con 0x7f258815aef0 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 -- 172.21.15.99:0/3410941336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815aef0 msgr2=0x7f25880b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815aef0 0x7f25880b4510 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f257800bb10 tx=0x7f257802c920 comp rx=0 tx=0).stop 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 -- 172.21.15.99:0/3410941336 shutdown_connections 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 --2- 172.21.15.99:0/3410941336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815aef0 0x7f25880b4510 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 -- 172.21.15.99:0/3410941336 >> 172.21.15.99:0/3410941336 conn(0x7f258806bc10 msgr2=0x7f258806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 -- 172.21.15.99:0/3410941336 shutdown_connections 2024-09-10T11:34:50.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.792+0000 7f258dd11640 1 -- 172.21.15.99:0/3410941336 wait complete. 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.793+0000 7f258dd11640 1 Processor -- start 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.793+0000 7f258dd11640 1 -- start start 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 0x7f258815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f258806eb30 con 0x7f258815b2d0 2024-09-10T11:34:50.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f258806e850 con 0x7f258815aef0 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f258dd11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f25880719e0 con 0x7f25880afbb0 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f2587fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f2587fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50472/0 (socket says 172.21.15.99:50472) 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f2587fff640 1 -- 172.21.15.99:0/1493046719 learned_addr learned my addr 172.21.15.99:0/1493046719 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f25877fe640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.794+0000 7f2586ffd640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 0x7f258815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 -- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 msgr2=0x7f25880ace90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880ace90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 -- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 msgr2=0x7f258815c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 0x7f258815c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 -- 172.21.15.99:0/1493046719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258815d640 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2586ffd640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 0x7f258815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f2587fff640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2580007e70 tx=0x7f258000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580014620 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2580019070 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258815d080 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580014a20 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.795+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25880aa5a0 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.796+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25880aaa80 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.796+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2580014bc0 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.797+0000 7f256ffff640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 0x7f257406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.797+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7f258008b700 con 0x7f258815b2d0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.797+0000 7f25877fe640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 0x7f257406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:50.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.797+0000 7f25877fe640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 0x7f257406e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2578037000 tx=0x7f257803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:50.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.801+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25880aaa80 con 0x7f258815b2d0 2024-09-10T11:34:50.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.968+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25880b0700 con 0x7f258815b2d0 2024-09-10T11:34:50.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.969+0000 7f256ffff640 1 -- 172.21.15.99:0/1493046719 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16744 (secure 0 0 0) 0x7f258005a0b0 con 0x7f258815b2d0 2024-09-10T11:34:50.971 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:50.972 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":111,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:46.712521+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:34:22.533186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T11:34:22.904618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:50.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 msgr2=0x7f257406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 0x7f257406e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2578037000 tx=0x7f257803d040 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 msgr2=0x7f258814cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2580007e70 tx=0x7f258000e4b0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 shutdown_connections 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f257406be30 0x7f257406e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f258815b2d0 0x7f258814cd90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f258815aef0 0x7f258815c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 --2- 172.21.15.99:0/1493046719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25880afbb0 0x7f25880ace90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.971+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 >> 172.21.15.99:0/1493046719 conn(0x7f258806bc10 msgr2=0x7f25880b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.972+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 shutdown_connections 2024-09-10T11:34:50.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:50.972+0000 7f258dd11640 1 -- 172.21.15.99:0/1493046719 wait complete. 2024-09-10T11:34:50.985 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:34:50.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:51.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.082+0000 7fbf0a81e640 1 Processor -- start 2024-09-10T11:34:51.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.082+0000 7fbf0a81e640 1 -- start start 2024-09-10T11:34:51.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf0a81e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbf0415c7b0 0x7fbf040d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf0a81e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf040ceb10 con 0x7fbf040d0fb0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf0a81e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf040b3a90 con 0x7fbf0406f000 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf0a81e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbf040d1420 con 0x7fbf0415c7b0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf037fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbf040d0fb0 0x7fbf0406e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36066/0 (socket says 172.21.15.99:36066) 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf037fe640 1 -- 172.21.15.99:0/1090110042 learned_addr learned my addr 172.21.15.99:0/1090110042 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf03fff640 1 --2- 172.21.15.99:0/1090110042 >> v2:172.21.15.196:3300/0 conn(0x7fbf0415c7b0 0x7fbf040d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386176818 0 0) 0x7fbef8001110 con 0x7fbf040d0fb0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeec003520 con 0x7fbf040d0fb0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 538910913 0 0) 0x7fbeec003520 con 0x7fbf040d0fb0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 >> v2:172.21.15.196:3300/0 conn(0x7fbf0415c7b0 msgr2=0x7fbf040d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 --2- 172.21.15.99:0/1090110042 >> v2:172.21.15.196:3300/0 conn(0x7fbf0415c7b0 0x7fbf040d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.083+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 >> v1:172.21.15.196:6789/0 conn(0x7fbf0406f000 legacy=0x7fbf0406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf040b3d70 con 0x7fbf040d0fb0 2024-09-10T11:34:51.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fbef8001840 con 0x7fbf040d0fb0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 >> v1:172.21.15.99:6789/0 conn(0x7fbf040d0fb0 legacy=0x7fbf0406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 0x7fbeec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040d1420 con 0x7fbeec0051c0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec00c060 con 0x7fbeec004800 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec00c260 con 0x7fbeec0039b0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 0x7fbeec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf03fff640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf037fe640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 -- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0039b0 msgr2=0x7fbeec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbeec0039b0 0x7fbeec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 -- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec004800 msgr2=0x7fbeec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec004800 0x7fbeec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.084+0000 7fbf08dbb640 1 -- 172.21.15.99:0/1090110042 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec003520 con 0x7fbeec0051c0 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf08dbb640 1 -- 172.21.15.99:0/1090110042 reap_dead start 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 0x7fbeec00b9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbefc001e50 tx=0x7fbeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637c9d358679805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc001a70 con 0x7fbeec0051c0 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbefc003970 con 0x7fbeec0051c0 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf02ffd640 1 -- 172.21.15.99:0/1090110042 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc003d60 con 0x7fbeec0051c0 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 -- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 msgr2=0x7fbeec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 0x7fbeec00b9c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbefc001e50 tx=0x7fbeec005830 comp rx=0 tx=0).stop 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 -- 172.21.15.99:0/1090110042 shutdown_connections 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/1090110042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeec0051c0 0x7fbeec00b9c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 -- 172.21.15.99:0/1090110042 >> 172.21.15.99:0/1090110042 conn(0x7fbf0406bc10 msgr2=0x7fbf0406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 -- 172.21.15.99:0/1090110042 shutdown_connections 2024-09-10T11:34:51.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.085+0000 7fbf0a81e640 1 -- 172.21.15.99:0/1090110042 wait complete. 2024-09-10T11:34:51.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 Processor -- start 2024-09-10T11:34:51.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 -- start start 2024-09-10T11:34:51.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0051c0 0x7fbf04141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 0x7fbf04160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040b4330 con 0x7fbf041558d0 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040b3840 con 0x7fbeec0051c0 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.087+0000 7fbf0a81e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040b4610 con 0x7fbf04144f30 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50478/0 (socket says 172.21.15.99:50478) 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 -- 172.21.15.99:0/385197086 learned_addr learned my addr 172.21.15.99:0/385197086 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf037fe640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 0x7fbf04160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf03fff640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0051c0 0x7fbf04141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 -- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 msgr2=0x7fbf04160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 0x7fbf04160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 -- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0051c0 msgr2=0x7fbf04141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0051c0 0x7fbf04141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 -- 172.21.15.99:0/385197086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041f28c0 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf037fe640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 0x7fbf04160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf08dbb640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbefc001f70 tx=0x7fbefc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc007db0 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbefc001250 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf04160580 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc006b50 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.088+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf04160800 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.089+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0415ccb0 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.090+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbefc03b030 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.090+0000 7fbf00ff9640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.090+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fbefc0a9e40 con 0x7fbf041558d0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.090+0000 7fbf03fff640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.091+0000 7fbf03fff640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009b40 tx=0x7fbef4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.095+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbf04160580 con 0x7fbf041558d0 2024-09-10T11:34:51.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.295+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0415cf50 con 0x7fbf041558d0 2024-09-10T11:34:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.296+0000 7fbf00ff9640 1 -- 172.21.15.99:0/385197086 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7fbf041f28c0 con 0x7fbf041558d0 2024-09-10T11:34:51.298 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:51.298 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:51.298 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 msgr2=0x7fbef006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 0x7fbef006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009b40 tx=0x7fbef4009190 comp rx=0 tx=0).stop 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 msgr2=0x7fbf04155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbefc001f70 tx=0x7fbefc0014a0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 shutdown_connections 2024-09-10T11:34:51.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbef006be30 0x7fbef006e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf041558d0 0x7fbf04155d50 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf04144f30 0x7fbf04160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 --2- 172.21.15.99:0/385197086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbeec0051c0 0x7fbf04141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 >> 172.21.15.99:0/385197086 conn(0x7fbf0406bc10 msgr2=0x7fbf040b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 shutdown_connections 2024-09-10T11:34:51.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.298+0000 7fbf0a81e640 1 -- 172.21.15.99:0/385197086 wait complete. 2024-09-10T11:34:51.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:51.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 Processor -- start 2024-09-10T11:34:51.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 -- start start 2024-09-10T11:34:51.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcd3415eb80 0x7fcd340d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd340d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd340d52f0 con 0x7fcd340d4e40 2024-09-10T11:34:51.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcd3415ff70 con 0x7fcd3415eb80 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.407+0000 7fcd3b22e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340d1620 con 0x7fcd341626c0 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd38fca640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcd3415eb80 0x7fcd340d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd340d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd397cb640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcd340d4e40 0x7fcd340d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:46204/0 (socket says 172.21.15.99:46204) 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd397cb640 1 -- 172.21.15.99:0/2069622867 learned_addr learned my addr 172.21.15.99:0/2069622867 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd38fca640 1 -- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 msgr2=0x7fcd340d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd38fca640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd340d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd38fca640 1 -- 172.21.15.99:0/2069622867 >> v1:172.21.15.196:6789/0 conn(0x7fcd340d4e40 legacy=0x7fcd340d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd38fca640 1 -- 172.21.15.99:0/2069622867 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd34162bc0 con 0x7fcd3415eb80 2024-09-10T11:34:51.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.408+0000 7fcd33fff640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd340d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.409+0000 7fcd38fca640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3415eb80 0x7fcd340d29b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cd8b9679da10c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.409+0000 7fcd337fe640 1 -- 172.21.15.99:0/2069622867 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28037070 con 0x7fcd3415eb80 2024-09-10T11:34:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.409+0000 7fcd337fe640 1 -- 172.21.15.99:0/2069622867 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2802ae50 con 0x7fcd3415eb80 2024-09-10T11:34:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.409+0000 7fcd337fe640 1 -- 172.21.15.99:0/2069622867 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28032660 con 0x7fcd3415eb80 2024-09-10T11:34:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3415eb80 msgr2=0x7fcd340d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3415eb80 0x7fcd340d29b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802a100 comp rx=0 tx=0).stop 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2069622867 shutdown_connections 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd340d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2069622867 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd3415eb80 0x7fcd340d29b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2069622867 >> 172.21.15.99:0/2069622867 conn(0x7fcd3408e680 msgr2=0x7fcd3409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2069622867 shutdown_connections 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.410+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2069622867 wait complete. 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.411+0000 7fcd3b22e640 1 Processor -- start 2024-09-10T11:34:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.411+0000 7fcd3b22e640 1 -- start start 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd340d4e40 0x7fcd3415b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd34159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34163460 con 0x7fcd3415eb80 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34163180 con 0x7fcd340d4e40 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd3b22e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd340d2ef0 con 0x7fcd341626c0 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50492/0 (socket says 172.21.15.99:50492) 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 -- 172.21.15.99:0/2485555879 learned_addr learned my addr 172.21.15.99:0/2485555879 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd38fca640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd340d4e40 0x7fcd3415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd397cb640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd34159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 -- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 msgr2=0x7fcd34159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd34159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 -- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd340d4e40 msgr2=0x7fcd3415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd340d4e40 0x7fcd3415b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.412+0000 7fcd33fff640 1 -- 172.21.15.99:0/2485555879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3415bb40 con 0x7fcd3415eb80 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd33fff640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000ebb0 tx=0x7fcd2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20017070 con 0x7fcd3415eb80 2024-09-10T11:34:51.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2000cc30 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd20013470 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3415be20 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.413+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3415c3e0 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.414+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd340d29b0 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.414+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd2001a070 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.415+0000 7fcd317fa640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 0x7fcd1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.415+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fcd2008b1c0 con 0x7fcd3415eb80 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.415+0000 7fcd38fca640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 0x7fcd1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.415+0000 7fcd38fca640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 0x7fcd1806e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.419+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd20059ae0 con 0x7fcd3415eb80 2024-09-10T11:34:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.618+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd3409e3b0 con 0x7fcd3415eb80 2024-09-10T11:34:51.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.618+0000 7fcd317fa640 1 -- 172.21.15.99:0/2485555879 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5231 (secure 0 0 0) 0x7fcd200593e0 con 0x7fcd3415eb80 2024-09-10T11:34:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:51.621 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"btime":"2024-09-10T11:34:26:597855+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:34:23.937446+0000","modified":"2024-09-10T11:34:25.922393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6243},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6243":{"gid":6243,"name":"b","rank":0,"incarnation":40,"state":"up:active","state_seq":2,"addr":"172.21.15.196:6835/1425455454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1425455454},{"type":"v1","addr":"172.21.15.196:6835","nonce":1425455454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6243,"qdb_cluster":[6243]},"id":6}]} 2024-09-10T11:34:51.621 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 42 2024-09-10T11:34:51.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 msgr2=0x7fcd1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 0x7fcd1806e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 msgr2=0x7fcd340b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000ebb0 tx=0x7fcd2000c520 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 shutdown_connections 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd1806be30 0x7fcd1806e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd341626c0 0x7fcd34159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd3415eb80 0x7fcd340b8ac0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 --2- 172.21.15.99:0/2485555879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd340d4e40 0x7fcd3415b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 >> 172.21.15.99:0/2485555879 conn(0x7fcd3408e680 msgr2=0x7fcd3415d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.620+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 shutdown_connections 2024-09-10T11:34:51.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.621+0000 7fcd3b22e640 1 -- 172.21.15.99:0/2485555879 wait complete. 2024-09-10T11:34:51.633 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:51.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.730+0000 7fdfb2a77640 1 Processor -- start 2024-09-10T11:34:51.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.730+0000 7fdfb2a77640 1 -- start start 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb2a77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdfac073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb2a77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfac152e60 con 0x7fdfac0d1200 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb2a77640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfac0d15e0 con 0x7fdfac15c060 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb2a77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac072680 con 0x7fdfac06ece0 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb0813640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdfac0d1200 0x7fdfac06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36074/0 (socket says 172.21.15.99:36074) 2024-09-10T11:34:51.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfb0813640 1 -- 172.21.15.99:0/1829848793 learned_addr learned my addr 172.21.15.99:0/1829848793 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33663790 0 0) 0x7fdfac152e60 con 0x7fdfac0d1200 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf9c003520 con 0x7fdfac0d1200 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfabfff640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdfac073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2186074300 0 0) 0x7fdf9c003520 con 0x7fdfac0d1200 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 msgr2=0x7fdfac073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdfac073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 >> v1:172.21.15.196:6789/0 conn(0x7fdfac15c060 legacy=0x7fdfac15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac072960 con 0x7fdfac0d1200 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fdfa0001670 con 0x7fdfac0d1200 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 >> v1:172.21.15.99:6789/0 conn(0x7fdfac0d1200 legacy=0x7fdfac06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0039e0 0x7fdf9c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdf9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c004d40 0x7fdf9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac072680 con 0x7fdf9c0039e0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c00bbd0 con 0x7fdf9c004d40 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.731+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9c00bdd0 con 0x7fdfac06ece0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0039e0 0x7fdf9c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfabfff640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdf9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb1014640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c004d40 0x7fdf9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 -- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 msgr2=0x7fdf9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfac06ece0 0x7fdf9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 -- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c004d40 msgr2=0x7fdf9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf9c004d40 0x7fdf9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 -- 172.21.15.99:0/1829848793 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9c003520 con 0x7fdf9c0039e0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.732+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0039e0 0x7fdf9c003e00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c210 tx=0x7fdf9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2a3d614da2235e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0007d30 con 0x7fdf9c0039e0 2024-09-10T11:34:51.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0005ad0 con 0x7fdf9c0039e0 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfab7fe640 1 -- 172.21.15.99:0/1829848793 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0007070 con 0x7fdf9c0039e0 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1829848793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0039e0 msgr2=0x7fdf9c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1829848793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdf9c0039e0 0x7fdf9c003e00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c210 tx=0x7fdf9c005830 comp rx=0 tx=0).stop 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb1014640 1 -- 172.21.15.99:0/1829848793 reap_dead start 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1829848793 shutdown_connections 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1829848793 >> 172.21.15.99:0/1829848793 conn(0x7fdfac06bc10 msgr2=0x7fdfac06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1829848793 shutdown_connections 2024-09-10T11:34:51.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.733+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1829848793 wait complete. 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.734+0000 7fdfb2a77640 1 Processor -- start 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.734+0000 7fdfb2a77640 1 -- start start 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf9c0039e0 0x7fdfac0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 0x7fdfac14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac15cd90 con 0x7fdfac150230 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac072f20 con 0x7fdfac1421f0 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb2a77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfac073440 con 0x7fdf9c0039e0 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb1014640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb1014640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50504/0 (socket says 172.21.15.99:50504) 2024-09-10T11:34:51.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb1014640 1 -- 172.21.15.99:0/1156636935 learned_addr learned my addr 172.21.15.99:0/1156636935 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf9c0039e0 0x7fdfac0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.735+0000 7fdfabfff640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 0x7fdfac14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 -- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf9c0039e0 msgr2=0x7fdfac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf9c0039e0 0x7fdfac0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 -- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 msgr2=0x7fdfac14fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 0x7fdfac14fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 -- 172.21.15.99:0/1156636935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac0ce720 con 0x7fdfac150230 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfabfff640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 0x7fdfac14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.736+0000 7fdfb1014640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0cd5a0 tx=0x7fdfa400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.738+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4003970 con 0x7fdfac150230 2024-09-10T11:34:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.738+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4018070 con 0x7fdfac150230 2024-09-10T11:34:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.738+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac0ca3c0 con 0x7fdfac150230 2024-09-10T11:34:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.738+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4003d70 con 0x7fdfac150230 2024-09-10T11:34:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.738+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac0ca980 con 0x7fdfac150230 2024-09-10T11:34:51.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.739+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac06e5d0 con 0x7fdfac150230 2024-09-10T11:34:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.740+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfa40050d0 con 0x7fdfac150230 2024-09-10T11:34:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.740+0000 7fdfa97fa640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 0x7fdf9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.740+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fdfa408a9a0 con 0x7fdfac150230 2024-09-10T11:34:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.740+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 0x7fdf9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:51.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.741+0000 7fdfb0813640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 0x7fdf9006e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0033000 tx=0x7fdfa002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:51.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.745+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfac0d15e0 con 0x7fdfac150230 2024-09-10T11:34:51.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.912+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfac0d0c70 con 0x7fdfac150230 2024-09-10T11:34:51.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.913+0000 7fdfa97fa640 1 -- 172.21.15.99:0/1156636935 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16744 (secure 0 0 0) 0x7fdfa4059350 con 0x7fdfac150230 2024-09-10T11:34:51.915 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:51.915 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":111,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:46.712521+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:34:22.533186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T11:34:22.904618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 msgr2=0x7fdf9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 0x7fdf9006e2f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0033000 tx=0x7fdfa002e000 comp rx=0 tx=0).stop 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 msgr2=0x7fdfac0cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0cd5a0 tx=0x7fdfa400ec10 comp rx=0 tx=0).stop 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 shutdown_connections 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9006be30 0x7fdf9006e2f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfac150230 0x7fdfac0cda30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfac1421f0 0x7fdfac14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 --2- 172.21.15.99:0/1156636935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf9c0039e0 0x7fdfac0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:51.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 >> 172.21.15.99:0/1156636935 conn(0x7fdfac06bc10 msgr2=0x7fdfac0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:51.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 shutdown_connections 2024-09-10T11:34:51.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:51.915+0000 7fdfb2a77640 1 -- 172.21.15.99:0/1156636935 wait complete. 2024-09-10T11:34:51.928 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:34:52.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.026+0000 7feed375f640 1 Processor -- start 2024-09-10T11:34:52.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.026+0000 7feed375f640 1 -- start start 2024-09-10T11:34:52.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed375f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feecc148620 0x7feecc0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed375f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0b38a0 con 0x7feecc162910 2024-09-10T11:34:52.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed375f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0d00c0 con 0x7feecc06f0d0 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed375f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7feecc06dc30 con 0x7feecc148620 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed14fb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7feecc148620 0x7feecc0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed0cfa640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7feecc162910 0x7feecc0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36076/0 (socket says 172.21.15.99:36076) 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed0cfa640 1 -- 172.21.15.99:0/3291245857 learned_addr learned my addr 172.21.15.99:0/3291245857 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed14fb640 1 -- 172.21.15.99:0/3291245857 >> v1:172.21.15.196:6789/0 conn(0x7feecc06f0d0 legacy=0x7feecc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed14fb640 1 -- 172.21.15.99:0/3291245857 >> v1:172.21.15.99:6789/0 conn(0x7feecc162910 legacy=0x7feecc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feed14fb640 1 -- 172.21.15.99:0/3291245857 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc0d03a0 con 0x7feecc148620 2024-09-10T11:34:52.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.027+0000 7feec3fff640 1 -- 172.21.15.99:0/3291245857 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792805695 0 0) 0x7feecc0b38a0 con 0x7feecc162910 2024-09-10T11:34:52.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.028+0000 7feed14fb640 1 --2- 172.21.15.99:0/3291245857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc148620 0x7feecc0b5ac0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7feec80097a0 tx=0x7feec802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a5334c9903bbff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.028+0000 7feec3fff640 1 -- 172.21.15.99:0/3291245857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8037070 con 0x7feecc148620 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.028+0000 7feec3fff640 1 -- 172.21.15.99:0/3291245857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8032440 con 0x7feecc148620 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.028+0000 7feec3fff640 1 -- 172.21.15.99:0/3291245857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8032810 con 0x7feecc148620 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 -- 172.21.15.99:0/3291245857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc148620 msgr2=0x7feecc0b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 --2- 172.21.15.99:0/3291245857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc148620 0x7feecc0b5ac0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7feec80097a0 tx=0x7feec802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 -- 172.21.15.99:0/3291245857 shutdown_connections 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 --2- 172.21.15.99:0/3291245857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc148620 0x7feecc0b5ac0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 -- 172.21.15.99:0/3291245857 >> 172.21.15.99:0/3291245857 conn(0x7feecc0b2a80 msgr2=0x7feecc0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 -- 172.21.15.99:0/3291245857 shutdown_connections 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.029+0000 7feed375f640 1 -- 172.21.15.99:0/3291245857 wait complete. 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.030+0000 7feed375f640 1 Processor -- start 2024-09-10T11:34:52.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.030+0000 7feed375f640 1 -- start start 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 0x7feecc09c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 0x7feecc1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feecc0b6000 con 0x7feecc162910 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feecc06d8c0 con 0x7feecc06f0d0 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed375f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7feecc148a00 con 0x7feecc148620 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50524/0 (socket says 172.21.15.99:50524) 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 -- 172.21.15.99:0/2532166629 learned_addr learned my addr 172.21.15.99:0/2532166629 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed0cfa640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 0x7feecc1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed14fb640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 0x7feecc09c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 -- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 msgr2=0x7feecc1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 0x7feecc1616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 -- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 msgr2=0x7feecc09c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 0x7feecc09c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 -- 172.21.15.99:0/2532166629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc09ba30 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed0cfa640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 0x7feecc1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed14fb640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 0x7feecc09c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feed1cfc640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7feec400c730 tx=0x7feec400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.031+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4016020 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.032+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec400aa50 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.032+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc09dba0 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.032+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec400ce40 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.032+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc09bcf0 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.033+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc0b5ac0 con 0x7feecc162910 2024-09-10T11:34:52.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.033+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feec4019020 con 0x7feecc162910 2024-09-10T11:34:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.033+0000 7feec1ffb640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 0x7feeb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.034+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7feec408adb0 con 0x7feecc162910 2024-09-10T11:34:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.034+0000 7feed14fb640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 0x7feeb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.034+0000 7feed14fb640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 0x7feeb006e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.038+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feec4020030 con 0x7feecc162910 2024-09-10T11:34:52.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.236+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7feecc0d0950 con 0x7feecc162910 2024-09-10T11:34:52.238 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:34:52.236+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:34:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.237+0000 7feec1ffb640 1 -- 172.21.15.99:0/2532166629 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7feec40212b0 con 0x7feecc162910 2024-09-10T11:34:52.239 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:34:52.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 msgr2=0x7feeb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 0x7feeb006e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7feec8034000 tx=0x7feec802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 msgr2=0x7feecc09d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7feec400c730 tx=0x7feec400cbf0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 shutdown_connections 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7feeb006be30 0x7feeb006e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feecc162910 0x7feecc09d230 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feecc148620 0x7feecc1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 --2- 172.21.15.99:0/2532166629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc06f0d0 0x7feecc09c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.241+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 >> 172.21.15.99:0/2532166629 conn(0x7feecc0b2a80 msgr2=0x7feecc0718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.242+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 shutdown_connections 2024-09-10T11:34:52.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.242+0000 7feed375f640 1 -- 172.21.15.99:0/2532166629 wait complete. 2024-09-10T11:34:52.247 INFO:tasks.ceph.mds.b.smithi196.stderr: -9267> 2024-09-10T11:34:27.575+0000 7fcbd6d61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:34:52.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:34:52.280 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:34:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.353+0000 7f76ece4c640 1 Processor -- start 2024-09-10T11:34:52.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.353+0000 7f76ece4c640 1 -- start start 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76ece4c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76e80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76ece4c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76e80b3160 0x7f76e806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76ece4c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e80b3620 con 0x7f76e8163580 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76ece4c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f76e806d960 con 0x7f76e80b3160 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76ece4c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76e806dc40 con 0x7f76e80b3960 2024-09-10T11:34:52.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76e659c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f76e8163580 0x7f76e80d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36080/0 (socket says 172.21.15.99:36080) 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76e659c640 1 -- 172.21.15.99:0/3903375674 learned_addr learned my addr 172.21.15.99:0/3903375674 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76dffff640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76e80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76e6d9d640 1 --2- 172.21.15.99:0/3903375674 >> v2:172.21.15.196:3300/0 conn(0x7f76e80b3160 0x7f76e806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374663608 0 0) 0x7f76e80b3620 con 0x7f76e8163580 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.354+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76d4003520 con 0x7f76e8163580 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3543553750 0 0) 0x7f76d4003520 con 0x7f76e8163580 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 msgr2=0x7f76e80b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76e80b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 >> v2:172.21.15.196:3300/0 conn(0x7f76e80b3160 msgr2=0x7f76e806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 --2- 172.21.15.99:0/3903375674 >> v2:172.21.15.196:3300/0 conn(0x7f76e80b3160 0x7f76e806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e806df20 con 0x7f76e8163580 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f76d8001670 con 0x7f76e8163580 2024-09-10T11:34:52.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 >> v1:172.21.15.99:6789/0 conn(0x7f76e8163580 legacy=0x7f76e80d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d40039b0 0x7f76d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d4004800 0x7f76d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76e806d960 con 0x7f76d4004800 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76e806dc40 con 0x7f76d40039b0 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76d400ba40 con 0x7f76e80b3960 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d4004800 0x7f76d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e659c640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d40039b0 0x7f76d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76e6d9d640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 -- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 msgr2=0x7f76d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e80b3960 0x7f76d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 -- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d40039b0 msgr2=0x7f76d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:52.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d40039b0 0x7f76d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.355+0000 7f76dffff640 1 -- 172.21.15.99:0/3903375674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d4003520 con 0x7f76d4004800 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76dffff640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d4004800 0x7f76d4004c80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009ac0 tx=0x7f76d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9889205cc33559e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0036070 con 0x7f76d4004800 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d0031400 con 0x7f76d4004800 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76e5d9b640 1 -- 172.21.15.99:0/3903375674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0009690 con 0x7f76d4004800 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76ece4c640 1 -- 172.21.15.99:0/3903375674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d4004800 msgr2=0x7f76d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3903375674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76d4004800 0x7f76d4004c80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009ac0 tx=0x7f76d4005830 comp rx=0 tx=0).stop 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.356+0000 7f76e6d9d640 1 -- 172.21.15.99:0/3903375674 reap_dead start 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.357+0000 7f76ece4c640 1 -- 172.21.15.99:0/3903375674 shutdown_connections 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.357+0000 7f76ece4c640 1 -- 172.21.15.99:0/3903375674 >> 172.21.15.99:0/3903375674 conn(0x7f76e80b4570 msgr2=0x7f76e80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.357+0000 7f76ece4c640 1 -- 172.21.15.99:0/3903375674 shutdown_connections 2024-09-10T11:34:52.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.357+0000 7f76ece4c640 1 -- 172.21.15.99:0/3903375674 wait complete. 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.358+0000 7f76ece4c640 1 Processor -- start 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.358+0000 7f76ece4c640 1 -- start start 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d4004800 0x7f76e8147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76e80cc270 0x7f76e8147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80b5210 con 0x7f76e8157f90 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80b5090 con 0x7f76e80cc270 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76e80b5390 con 0x7f76d4004800 2024-09-10T11:34:52.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50538/0 (socket says 172.21.15.99:50538) 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 -- 172.21.15.99:0/3819522362 learned_addr learned my addr 172.21.15.99:0/3819522362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76e659c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d4004800 0x7f76e8147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76e6d9d640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76e80cc270 0x7f76e8147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 -- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d4004800 msgr2=0x7f76e8147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d4004800 0x7f76e8147520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 -- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76e80cc270 msgr2=0x7f76e8147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76e80cc270 0x7f76e8147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 -- 172.21.15.99:0/3819522362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80cb550 con 0x7f76e8157f90 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76dffff640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f76d0034040 tx=0x7f76d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0040020 con 0x7f76e8157f90 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d0003940 con 0x7f76e8157f90 2024-09-10T11:34:52.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0036040 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e80cd050 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.359+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e80cd610 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.360+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e80761a0 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.361+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76d0043020 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.361+0000 7f76deffd640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 0x7f76c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.361+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5795+0+0 (secure 0 0 0) 0x7f76d00ab280 con 0x7f76e8157f90 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.361+0000 7f76e659c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 0x7f76c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.362+0000 7f76e659c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 0x7f76c806e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76d8001d30 tx=0x7f76d8007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:52.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.365+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76d004a030 con 0x7f76e8157f90 2024-09-10T11:34:52.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:52.564+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f76e80b89a0 con 0x7f76e8157f90 2024-09-10T11:34:53.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.239+0000 7f76deffd640 1 -- 172.21.15.99:0/3819522362 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7f76d0079580 con 0x7f76e8157f90 2024-09-10T11:34:53.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.241+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 msgr2=0x7f76c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.241+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 0x7f76c806e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76d8001d30 tx=0x7f76d8007970 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.241+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 msgr2=0x7f76e80cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.241+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f76d0034040 tx=0x7f76d0004730 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 shutdown_connections 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f76c806be30 0x7f76c806e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76e80cc270 0x7f76e8147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e8157f90 0x7f76e80cbd30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 --2- 172.21.15.99:0/3819522362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d4004800 0x7f76e8147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 >> 172.21.15.99:0/3819522362 conn(0x7f76e80b4570 msgr2=0x7f76e80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 shutdown_connections 2024-09-10T11:34:53.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.242+0000 7f76ece4c640 1 -- 172.21.15.99:0/3819522362 wait complete. 2024-09-10T11:34:53.257 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:34:53.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.361+0000 7f15403c4640 1 Processor -- start 2024-09-10T11:34:53.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.361+0000 7f15403c4640 1 -- start start 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.362+0000 7f15403c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f153815ee10 0x7f15380af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.362+0000 7f15403c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f15380b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.362+0000 7f15403c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15381635c0 con 0x7f15381627e0 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.362+0000 7f15403c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f153806d550 con 0x7f153815ee10 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.362+0000 7f15403c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15380b7b50 con 0x7f15380b6fe0 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f153815ee10 0x7f15380af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f153815ee10 0x7f15380af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56572/0 (socket says 172.21.15.99:56572) 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 -- 172.21.15.99:0/724915407 learned_addr learned my addr 172.21.15.99:0/724915407 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153d95f640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f15380b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 -- 172.21.15.99:0/724915407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 msgr2=0x7f15380b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:53.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f15380b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 -- 172.21.15.99:0/724915407 >> v1:172.21.15.196:6789/0 conn(0x7f15381627e0 legacy=0x7f153806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:53.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153e160640 1 -- 172.21.15.99:0/724915407 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153806cf00 con 0x7f153815ee10 2024-09-10T11:34:53.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.363+0000 7f153d95f640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f15380b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f153e160640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f153815ee10 0x7f15380af930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1534009870 tx=0x7f153402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359b786a0d8fc6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f153d15e640 1 -- 172.21.15.99:0/724915407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534037070 con 0x7f153815ee10 2024-09-10T11:34:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f153d15e640 1 -- 172.21.15.99:0/724915407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1534032440 con 0x7f153815ee10 2024-09-10T11:34:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f153d15e640 1 -- 172.21.15.99:0/724915407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534032810 con 0x7f153815ee10 2024-09-10T11:34:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 -- 172.21.15.99:0/724915407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f153815ee10 msgr2=0x7f15380af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f153815ee10 0x7f15380af930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1534009870 tx=0x7f153402a200 comp rx=0 tx=0).stop 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 -- 172.21.15.99:0/724915407 shutdown_connections 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f15380b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 --2- 172.21.15.99:0/724915407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f153815ee10 0x7f15380af930 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 -- 172.21.15.99:0/724915407 >> 172.21.15.99:0/724915407 conn(0x7f1538073c60 msgr2=0x7f153809ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 -- 172.21.15.99:0/724915407 shutdown_connections 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.364+0000 7f15403c4640 1 -- 172.21.15.99:0/724915407 wait complete. 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.365+0000 7f15403c4640 1 Processor -- start 2024-09-10T11:34:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 -- start start 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f153814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f153815ee10 0x7f15381f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1538162e10 con 0x7f15381627e0 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1538162c90 con 0x7f153815ee10 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f15403c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1538162f90 con 0x7f15380b6fe0 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f153e961640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f153e961640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56574/0 (socket says 172.21.15.99:56574) 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f153e961640 1 -- 172.21.15.99:0/341810692 learned_addr learned my addr 172.21.15.99:0/341810692 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f153e160640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f153814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.366+0000 7f153d95f640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f153815ee10 0x7f15381f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 -- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 msgr2=0x7f153814f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f153814f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 -- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f153815ee10 msgr2=0x7f15381f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f153815ee10 0x7f15381f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 -- 172.21.15.99:0/341810692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15381e96c0 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f153e961640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f153000b9a0 tx=0x7f153000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530019070 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153000cba0 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153815f390 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530014440 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.367+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153814f8a0 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.368+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380af930 con 0x7f15381627e0 2024-09-10T11:34:53.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.368+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f153001c070 con 0x7f15381627e0 2024-09-10T11:34:53.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.369+0000 7f152effd640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 0x7f151c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:53.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.369+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5795+0+0 (secure 0 0 0) 0x7f153008aa50 con 0x7f15381627e0 2024-09-10T11:34:53.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.369+0000 7f153e160640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 0x7f151c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:53.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.369+0000 7f153e160640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 0x7f151c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1534034000 tx=0x7f153402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:53.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.373+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1530058d50 con 0x7f15381627e0 2024-09-10T11:34:53.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:53.536+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f15380d3790 con 0x7f15381627e0 2024-09-10T11:34:54.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.240+0000 7f152effd640 1 -- 172.21.15.99:0/341810692 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) ==== 181+0+0 (secure 0 0 0) 0x7f153005d050 con 0x7f15381627e0 2024-09-10T11:34:54.243 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:34:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 msgr2=0x7f151c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 0x7f151c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1534034000 tx=0x7f153402e000 comp rx=0 tx=0).stop 2024-09-10T11:34:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 msgr2=0x7f15381f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f153000b9a0 tx=0x7f153000be60 comp rx=0 tx=0).stop 2024-09-10T11:34:54.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 shutdown_connections 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f151c06be30 0x7f151c06e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15381627e0 0x7f15381f8ec0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f153815ee10 0x7f15381f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 --2- 172.21.15.99:0/341810692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15380b6fe0 0x7f153814f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.243+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 >> 172.21.15.99:0/341810692 conn(0x7f1538073c60 msgr2=0x7f153809ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:54.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.244+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 shutdown_connections 2024-09-10T11:34:54.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.244+0000 7f15403c4640 1 -- 172.21.15.99:0/341810692 wait complete. 2024-09-10T11:34:54.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:34:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.365+0000 7ff3f199f640 1 Processor -- start 2024-09-10T11:34:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.365+0000 7ff3f199f640 1 -- start start 2024-09-10T11:34:54.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3f199f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3ec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3f199f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec0afdc0 con 0x7ff3ec15ee10 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3f199f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec1635c0 con 0x7ff3ec0af930 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3f199f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec06d550 con 0x7ff3ec162ef0 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3eaffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff3ec15ee10 0x7ff3ec0b6fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40264/0 (socket says 172.21.15.99:40264) 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3eaffd640 1 -- 172.21.15.99:0/2037421739 learned_addr learned my addr 172.21.15.99:0/2037421739 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709330894 0 0) 0x7ff3ec0afdc0 con 0x7ff3ec15ee10 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3eb7fe640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3ec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.366+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3d8003520 con 0x7ff3ec15ee10 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2015273876 0 0) 0x7ff3d8003520 con 0x7ff3ec15ee10 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 msgr2=0x7ff3ec06d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3ec06d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 >> v1:172.21.15.196:6789/0 conn(0x7ff3ec0af930 legacy=0x7ff3ec1627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec0b7b50 con 0x7ff3ec15ee10 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7ff3dc001670 con 0x7ff3ec15ee10 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 >> v1:172.21.15.99:6789/0 conn(0x7ff3ec15ee10 legacy=0x7ff3ec0b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d80039e0 0x7ff3d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d8004800 0x7ff3d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec06d550 con 0x7ff3d80039e0 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d800baa0 con 0x7ff3d8004800 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d800bca0 con 0x7ff3ec162ef0 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d80039e0 0x7ff3d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3ea7fc640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d8004800 0x7ff3d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eb7fe640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 -- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 msgr2=0x7ff3d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:54.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec162ef0 0x7ff3d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 -- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d8004800 msgr2=0x7ff3d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d8004800 0x7ff3d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3eaffd640 1 -- 172.21.15.99:0/2037421739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8003520 con 0x7ff3d80039e0 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.367+0000 7ff3ea7fc640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d8004800 0x7ff3d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d80039e0 0x7ff3d8003e00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001eb0 tx=0x7ff3d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f67e9cdc28327 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007d30 con 0x7ff3d80039e0 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3dc005af0 con 0x7ff3d80039e0 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3e9ffb640 1 -- 172.21.15.99:0/2037421739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007070 con 0x7ff3d80039e0 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3f199f640 1 -- 172.21.15.99:0/2037421739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d80039e0 msgr2=0x7ff3d8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.368+0000 7ff3f199f640 1 --2- 172.21.15.99:0/2037421739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d80039e0 0x7ff3d8003e00 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001eb0 tx=0x7ff3d8005830 comp rx=0 tx=0).stop 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.369+0000 7ff3eb7fe640 1 -- 172.21.15.99:0/2037421739 reap_dead start 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.369+0000 7ff3f199f640 1 -- 172.21.15.99:0/2037421739 shutdown_connections 2024-09-10T11:34:54.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.369+0000 7ff3f199f640 1 -- 172.21.15.99:0/2037421739 >> 172.21.15.99:0/2037421739 conn(0x7ff3ec073c60 msgr2=0x7ff3ec09ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:54.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.369+0000 7ff3f199f640 1 -- 172.21.15.99:0/2037421739 shutdown_connections 2024-09-10T11:34:54.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.369+0000 7ff3f199f640 1 -- 172.21.15.99:0/2037421739 wait complete. 2024-09-10T11:34:54.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.370+0000 7ff3f199f640 1 Processor -- start 2024-09-10T11:34:54.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.370+0000 7ff3f199f640 1 -- start start 2024-09-10T11:34:54.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d80039e0 0x7ff3ec15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 0x7ff3ec161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec0b76f0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec06d1e0 con 0x7ff3d80039e0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec0b7870 con 0x7ff3ec1613c0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56594/0 (socket says 172.21.15.99:56594) 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 -- 172.21.15.99:0/6131713 learned_addr learned my addr 172.21.15.99:0/6131713 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d80039e0 0x7ff3ec15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3eb7fe640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 0x7ff3ec161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 -- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 msgr2=0x7ff3ec161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 0x7ff3ec161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 -- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d80039e0 msgr2=0x7ff3ec15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:54.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d80039e0 0x7ff3ec15d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 -- 172.21.15.99:0/6131713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec153970 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3eb7fe640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 0x7ff3ec161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3ea7fc640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400e4d0 tx=0x7ff3d400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d40052e0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3d4018070 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec15d7d0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4003810 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.371+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec15dd90 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.372+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ec09cab0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.373+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3d40039b0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.373+0000 7ff3e3fff640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 0x7ff3cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.373+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5357+0+0 (secure 0 0 0) 0x7ff3d408a920 con 0x7ff3ec1597d0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.373+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 0x7ff3cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:54.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.374+0000 7ff3eaffd640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 0x7ff3cc06e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc032000 tx=0x7ff3dc02d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:54.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.377+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3d40594c0 con 0x7ff3ec1597d0 2024-09-10T11:34:54.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:54.542+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff3ec0d3790 con 0x7ff3ec1597d0 2024-09-10T11:34:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.242+0000 7ff3e3fff640 1 -- 172.21.15.99:0/6131713 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) ==== 169+0+0 (secure 0 0 0) 0x7ff3d4058dc0 con 0x7ff3ec1597d0 2024-09-10T11:34:55.244 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:34:55.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.244+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 msgr2=0x7ff3cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.244+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 0x7ff3cc06e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc032000 tx=0x7ff3dc02d000 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.244+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 msgr2=0x7ff3ec153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.244+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400e4d0 tx=0x7ff3d400e990 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 shutdown_connections 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff3cc06be30 0x7ff3cc06e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3ec1613c0 0x7ff3ec161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec1597d0 0x7ff3ec153430 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 --2- 172.21.15.99:0/6131713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3d80039e0 0x7ff3ec15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 >> 172.21.15.99:0/6131713 conn(0x7ff3ec073c60 msgr2=0x7ff3ec06f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 shutdown_connections 2024-09-10T11:34:55.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.245+0000 7ff3f199f640 1 -- 172.21.15.99:0/6131713 wait complete. 2024-09-10T11:34:55.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:34:55.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.361+0000 7f6fe754b640 1 Processor -- start 2024-09-10T11:34:55.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.361+0000 7f6fe754b640 1 -- start start 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6fe00d1200 0x7f6fe0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 0x7f6fe006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 0x7f6fe015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe006ef80 con 0x7f6fe00d1200 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6fe0152e60 con 0x7f6fe006e5d0 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe754b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe00d15e0 con 0x7f6fe015c060 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6fe00d1200 0x7f6fe0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6fe00d1200 0x7f6fe0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56598/0 (socket says 172.21.15.99:56598) 2024-09-10T11:34:55.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 -- 172.21.15.99:0/2475463350 learned_addr learned my addr 172.21.15.99:0/2475463350 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe5ae8640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 0x7f6fe015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe4ae6640 1 --2- 172.21.15.99:0/2475463350 >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 0x7f6fe006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 -- 172.21.15.99:0/2475463350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 msgr2=0x7f6fe015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 0x7f6fe015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 -- 172.21.15.99:0/2475463350 >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 msgr2=0x7f6fe006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/2475463350 >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 0x7f6fe006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe52e7640 1 -- 172.21.15.99:0/2475463350 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe0072680 con 0x7f6fe00d1200 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe5ae8640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 0x7f6fe015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.362+0000 7f6fe4ae6640 1 --2- 172.21.15.99:0/2475463350 >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 0x7f6fe006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.363+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00d1200 0x7f6fe0073440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0071bb0 tx=0x7f6fdc02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb2f6c494039c18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.363+0000 7f6fd7fff640 1 -- 172.21.15.99:0/2475463350 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc032700 con 0x7f6fe00d1200 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.363+0000 7f6fd7fff640 1 -- 172.21.15.99:0/2475463350 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fdc0328a0 con 0x7f6fe00d1200 2024-09-10T11:34:55.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.363+0000 7f6fd7fff640 1 -- 172.21.15.99:0/2475463350 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc032c70 con 0x7f6fe00d1200 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 -- 172.21.15.99:0/2475463350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00d1200 msgr2=0x7f6fe0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00d1200 0x7f6fe0073440 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0071bb0 tx=0x7f6fdc02c800 comp rx=0 tx=0).stop 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 -- 172.21.15.99:0/2475463350 shutdown_connections 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe015c060 0x7f6fe015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 --2- 172.21.15.99:0/2475463350 >> v2:172.21.15.196:3300/0 conn(0x7f6fe006e5d0 0x7f6fe006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 --2- 172.21.15.99:0/2475463350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe00d1200 0x7f6fe0073440 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 -- 172.21.15.99:0/2475463350 >> 172.21.15.99:0/2475463350 conn(0x7f6fe006bc10 msgr2=0x7f6fe006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 -- 172.21.15.99:0/2475463350 shutdown_connections 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.364+0000 7f6fe754b640 1 -- 172.21.15.99:0/2475463350 wait complete. 2024-09-10T11:34:55.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.365+0000 7f6fe754b640 1 Processor -- start 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.365+0000 7f6fe754b640 1 -- start start 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 0x7f6fe014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe00d1200 0x7f6fe014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe0072f20 con 0x7f6fe015c060 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6fe0072c40 con 0x7f6fe006e5d0 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6fe015cd90 con 0x7f6fe00d1200 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56604/0 (socket says 172.21.15.99:56604) 2024-09-10T11:34:55.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 -- 172.21.15.99:0/3966590773 learned_addr learned my addr 172.21.15.99:0/3966590773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe4ae6640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe00d1200 0x7f6fe014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 0x7f6fe014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 -- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe00d1200 msgr2=0x7f6fe014ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe00d1200 0x7f6fe014ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 -- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 msgr2=0x7f6fe014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 0x7f6fe014fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 -- 172.21.15.99:0/3966590773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fe014eb90 con 0x7f6fe015c060 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 0x7f6fe014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe5ae8640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009fa0 tx=0x7f6fd800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd801a070 con 0x7f6fe015c060 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd8003bb0 con 0x7f6fe015c060 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.366+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fe014ee70 con 0x7f6fe015c060 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.367+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8004030 con 0x7f6fe015c060 2024-09-10T11:34:55.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.367+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fe014f430 con 0x7f6fe015c060 2024-09-10T11:34:55.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.368+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fe006ea40 con 0x7f6fe015c060 2024-09-10T11:34:55.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.368+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6fd801d070 con 0x7f6fe015c060 2024-09-10T11:34:55.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.368+0000 7f6fd5ffb640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 0x7f6fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.368+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f6fd808aef0 con 0x7f6fe015c060 2024-09-10T11:34:55.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.368+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 0x7f6fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.369+0000 7f6fe52e7640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 0x7f6fc406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc037000 tx=0x7f6fdc036040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.373+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6fd8010070 con 0x7f6fe015c060 2024-09-10T11:34:55.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.538+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6fe0154f00 con 0x7f6fe015c060 2024-09-10T11:34:55.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.541+0000 7f6fd5ffb640 1 -- 172.21.15.99:0/3966590773 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f6fd8059b10 con 0x7f6fe015c060 2024-09-10T11:34:55.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 msgr2=0x7f6fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 0x7f6fc406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc037000 tx=0x7f6fdc036040 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 msgr2=0x7f6fe014df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009fa0 tx=0x7f6fd800d560 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 shutdown_connections 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6fc406be30 0x7f6fc406e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fe015c060 0x7f6fe014df10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fe00d1200 0x7f6fe014ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 --2- 172.21.15.99:0/3966590773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fe006e5d0 0x7f6fe014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.543+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 >> 172.21.15.99:0/3966590773 conn(0x7f6fe006bc10 msgr2=0x7f6fe0154450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:55.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.544+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 shutdown_connections 2024-09-10T11:34:55.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.544+0000 7f6fe754b640 1 -- 172.21.15.99:0/3966590773 wait complete. 2024-09-10T11:34:55.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-09-10T11:34:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.654+0000 7f92f259f640 1 Processor -- start 2024-09-10T11:34:55.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.654+0000 7f92f259f640 1 -- start start 2024-09-10T11:34:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.654+0000 7f92f259f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92ec0d1200 0x7f92ec073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.654+0000 7f92f259f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec06e5d0 0x7f92ec06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92f259f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ec152e60 con 0x7f92ec15c060 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92f259f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f92ec06ef50 con 0x7f92ec0d1200 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92f259f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec0d15e0 con 0x7f92ec06e5d0 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92ec0d1200 0x7f92ec073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92ec0d1200 0x7f92ec073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56616/0 (socket says 172.21.15.99:56616) 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 -- 172.21.15.99:0/1048120920 learned_addr learned my addr 172.21.15.99:0/1048120920 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92f0b3c640 1 --1- 172.21.15.99:0/1048120920 >> v1:172.21.15.99:6789/0 conn(0x7f92ec15c060 0x7f92ec15c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40272/0 (socket says 172.21.15.99:40272) 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92eb7fe640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec06e5d0 0x7f92ec06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 -- 172.21.15.99:0/1048120920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec06e5d0 msgr2=0x7f92ec06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec06e5d0 0x7f92ec06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 -- 172.21.15.99:0/1048120920 >> v1:172.21.15.99:6789/0 conn(0x7f92ec15c060 legacy=0x7f92ec15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92eaffd640 1 -- 172.21.15.99:0/1048120920 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2583870195 0 0) 0x7f92ec152e60 con 0x7f92ec15c060 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.655+0000 7f92ebfff640 1 -- 172.21.15.99:0/1048120920 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec072680 con 0x7f92ec0d1200 2024-09-10T11:34:55.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92ebfff640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec0d1200 0x7f92ec073440 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1641a9ec0c7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92eaffd640 1 -- 172.21.15.99:0/1048120920 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc037070 con 0x7f92ec0d1200 2024-09-10T11:34:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92eaffd640 1 -- 172.21.15.99:0/1048120920 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92dc032440 con 0x7f92ec0d1200 2024-09-10T11:34:55.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92eaffd640 1 -- 172.21.15.99:0/1048120920 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc032810 con 0x7f92ec0d1200 2024-09-10T11:34:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 -- 172.21.15.99:0/1048120920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec0d1200 msgr2=0x7f92ec073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec0d1200 0x7f92ec073440 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009870 tx=0x7f92dc02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 -- 172.21.15.99:0/1048120920 shutdown_connections 2024-09-10T11:34:55.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec06e5d0 0x7f92ec06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 --2- 172.21.15.99:0/1048120920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec0d1200 0x7f92ec073440 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 -- 172.21.15.99:0/1048120920 >> 172.21.15.99:0/1048120920 conn(0x7f92ec06bc10 msgr2=0x7f92ec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 -- 172.21.15.99:0/1048120920 shutdown_connections 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.656+0000 7f92f259f640 1 -- 172.21.15.99:0/1048120920 wait complete. 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.657+0000 7f92f259f640 1 Processor -- start 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 -- start start 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92ec06e5d0 0x7f92ec14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec0d1200 0x7f92ec154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec072f20 con 0x7f92ec15c060 2024-09-10T11:34:55.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec072c40 con 0x7f92ec06e5d0 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.658+0000 7f92f259f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec15cd90 con 0x7f92ec0d1200 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56624/0 (socket says 172.21.15.99:56624) 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 -- 172.21.15.99:0/2354344612 learned_addr learned my addr 172.21.15.99:0/2354344612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92eb7fe640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec0d1200 0x7f92ec154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92ebfff640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92ec06e5d0 0x7f92ec14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 -- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec0d1200 msgr2=0x7f92ec154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec0d1200 0x7f92ec154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 -- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92ec06e5d0 msgr2=0x7f92ec14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92ec06e5d0 0x7f92ec14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 -- 172.21.15.99:0/2354344612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec14d630 con 0x7f92ec15c060 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f0b3c640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c7d0 tx=0x7f92e400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4018070 con 0x7f92ec15c060 2024-09-10T11:34:55.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e400a640 con 0x7f92ec15c060 2024-09-10T11:34:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec14d910 con 0x7f92ec15c060 2024-09-10T11:34:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400aa70 con 0x7f92ec15c060 2024-09-10T11:34:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.659+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec14ded0 con 0x7f92ec15c060 2024-09-10T11:34:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.660+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec0b22f0 con 0x7f92ec15c060 2024-09-10T11:34:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.661+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f92e401b070 con 0x7f92ec15c060 2024-09-10T11:34:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.661+0000 7f92e8ff9640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 0x7f92d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.661+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f92e408a0b0 con 0x7f92ec15c060 2024-09-10T11:34:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.661+0000 7f92ebfff640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 0x7f92d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.662+0000 7f92ebfff640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 0x7f92d806e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f92dc034000 tx=0x7f92dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:55.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.665+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92ec0b22f0 con 0x7f92ec15c060 2024-09-10T11:34:55.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:55.894+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7f92ec06ea10 con 0x7f92ec15c060 2024-09-10T11:34:56.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.246+0000 7f92e8ff9640 1 -- 172.21.15.99:0/2354344612 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v242) ==== 137+0+0 (secure 0 0 0) 0x7f92e4058710 con 0x7f92ec15c060 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 msgr2=0x7f92d806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 0x7f92d806e320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f92dc034000 tx=0x7f92dc02e000 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 msgr2=0x7f92ec155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f92e400c7d0 tx=0x7f92e400cc90 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 shutdown_connections 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f92d806be60 0x7f92d806e320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92ec15c060 0x7f92ec155340 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92ec0d1200 0x7f92ec154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 --2- 172.21.15.99:0/2354344612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92ec06e5d0 0x7f92ec14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 >> 172.21.15.99:0/2354344612 conn(0x7f92ec06bc10 msgr2=0x7f92ec06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 shutdown_connections 2024-09-10T11:34:56.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.248+0000 7f92f259f640 1 -- 172.21.15.99:0/2354344612 wait complete. 2024-09-10T11:34:56.261 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-09-10T11:34:56.262 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-09-10T11:34:56.262 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-09-10T11:34:56.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:34:56.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.358+0000 7f8ee8bfd640 1 Processor -- start 2024-09-10T11:34:56.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.358+0000 7f8ee8bfd640 1 -- start start 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.359+0000 7f8ee8bfd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ee41626c0 0x7f8ee40d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.359+0000 7f8ee8bfd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 0x7f8ee40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.359+0000 7f8ee8bfd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee415ff70 con 0x7f8ee415eb80 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.359+0000 7f8ee8bfd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee40d30c0 con 0x7f8ee41626c0 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.359+0000 7f8ee8bfd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8ee40d1620 con 0x7f8ee40d4e40 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ee41626c0 0x7f8ee40d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee259c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ee415eb80 0x7f8ee40d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40276/0 (socket says 172.21.15.99:40276) 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee259c640 1 -- 172.21.15.99:0/2327127856 learned_addr learned my addr 172.21.15.99:0/2327127856 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2327127856 >> v2:172.21.15.99:3300/0 conn(0x7f8ee41626c0 0x7f8ee40d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56630/0 (socket says 172.21.15.99:56630) 2024-09-10T11:34:56.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 -- 172.21.15.99:0/2327127856 >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 msgr2=0x7f8ee40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee2d9d640 1 --2- 172.21.15.99:0/2327127856 >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 0x7f8ee40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee159a640 1 -- 172.21.15.99:0/2327127856 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123310748 0 0) 0x7f8ee415ff70 con 0x7f8ee415eb80 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2327127856 >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 0x7f8ee40d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 -- 172.21.15.99:0/2327127856 >> v1:172.21.15.99:6789/0 conn(0x7f8ee415eb80 legacy=0x7f8ee40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee1d9b640 1 -- 172.21.15.99:0/2327127856 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee4162bc0 con 0x7f8ee41626c0 2024-09-10T11:34:56.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.360+0000 7f8ee2d9d640 1 --2- 172.21.15.99:0/2327127856 >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 0x7f8ee40d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:56.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2327127856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee41626c0 0x7f8ee40d4900 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0097a0 tx=0x7f8ecc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f29290a91e7e664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee159a640 1 -- 172.21.15.99:0/2327127856 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc037070 con 0x7f8ee41626c0 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee159a640 1 -- 172.21.15.99:0/2327127856 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc02ae50 con 0x7f8ee41626c0 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee159a640 1 -- 172.21.15.99:0/2327127856 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc032660 con 0x7f8ee41626c0 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2327127856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee41626c0 msgr2=0x7f8ee40d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2327127856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee41626c0 0x7f8ee40d4900 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0097a0 tx=0x7f8ecc02a140 comp rx=0 tx=0).stop 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2327127856 shutdown_connections 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2327127856 >> v2:172.21.15.196:3300/0 conn(0x7f8ee40d4e40 0x7f8ee40d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2327127856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee41626c0 0x7f8ee40d4900 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2327127856 >> 172.21.15.99:0/2327127856 conn(0x7f8ee408e680 msgr2=0x7f8ee409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2327127856 shutdown_connections 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.361+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2327127856 wait complete. 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 Processor -- start 2024-09-10T11:34:56.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 -- start start 2024-09-10T11:34:56.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 0x7f8ee415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 0x7f8ee415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4163460 con 0x7f8ee40d4e40 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee4163180 con 0x7f8ee41626c0 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.363+0000 7f8ee8bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee40d5290 con 0x7f8ee415eb80 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56638/0 (socket says 172.21.15.99:56638) 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 -- 172.21.15.99:0/2331275473 learned_addr learned my addr 172.21.15.99:0/2331275473 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee2d9d640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 0x7f8ee415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 0x7f8ee415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 -- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 msgr2=0x7f8ee415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 0x7f8ee415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 -- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 msgr2=0x7f8ee415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 0x7f8ee415c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 -- 172.21.15.99:0/2331275473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee41f8480 con 0x7f8ee40d4e40 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee2d9d640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 0x7f8ee415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:56.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 0x7f8ee415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee259c640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c430 tx=0x7f8ed400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed40074e0 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed40047e0 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41f8760 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4004bd0 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.364+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41f8d20 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.365+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee40d4900 con 0x7f8ee40d4e40 2024-09-10T11:34:56.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.366+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ed4011070 con 0x7f8ee40d4e40 2024-09-10T11:34:56.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.366+0000 7f8edaffd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.366+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f8ed408a800 con 0x7f8ee40d4e40 2024-09-10T11:34:56.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.366+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.367+0000 7f8ee1d9b640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 0x7f8ec406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.371+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ed40594b0 con 0x7f8ee40d4e40 2024-09-10T11:34:56.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.538+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ee406c460 con 0x7f8ee40d4e40 2024-09-10T11:34:56.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.539+0000 7f8edaffd640 1 -- 172.21.15.99:0/2331275473 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+13045 (secure 0 0 0) 0x7f8ed4058db0 con 0x7f8ee40d4e40 2024-09-10T11:34:56.541 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:56.541 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":114,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:34:55.242183+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/1425455454":"2024-09-11T11:34:52.237827+0000","172.21.15.196:6833/1425455454":"2024-09-11T11:34:52.237827+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:34:56.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 msgr2=0x7f8ec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 0x7f8ec406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc02e000 comp rx=0 tx=0).stop 2024-09-10T11:34:56.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 msgr2=0x7f8ee41e88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400c430 tx=0x7f8ed400c8f0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 shutdown_connections 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ec406be30 0x7f8ec406e2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee41626c0 0x7f8ee415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ee415eb80 0x7f8ee415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 --2- 172.21.15.99:0/2331275473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee40d4e40 0x7f8ee41e88f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 >> 172.21.15.99:0/2331275473 conn(0x7f8ee408e680 msgr2=0x7f8ee406ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 shutdown_connections 2024-09-10T11:34:56.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.541+0000 7f8ee8bfd640 1 -- 172.21.15.99:0/2331275473 wait complete. 2024-09-10T11:34:56.554 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:34:56.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:34:56.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.657+0000 7fb27664c640 1 Processor -- start 2024-09-10T11:34:56.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.657+0000 7fb27664c640 1 -- start start 2024-09-10T11:34:56.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb27664c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 0x7fb2700b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb27664c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2700b3160 0x7fb27006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb27664c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2700b3620 con 0x7fb2700b3960 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb27664c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb27006d960 con 0x7fb2700b3160 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb27664c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb27006dc40 con 0x7fb270163580 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2700b3160 0x7fb27006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2700b3160 0x7fb27006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56650/0 (socket says 172.21.15.99:56650) 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 -- 172.21.15.99:0/101913493 learned_addr learned my addr 172.21.15.99:0/101913493 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb26ffff640 1 --2- 172.21.15.99:0/101913493 >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 0x7fb2700b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 -- 172.21.15.99:0/101913493 >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 msgr2=0x7fb2700b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 --2- 172.21.15.99:0/101913493 >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 0x7fb2700b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 -- 172.21.15.99:0/101913493 >> v1:172.21.15.196:6789/0 conn(0x7fb2700b3960 legacy=0x7fb2700d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb274be9640 1 -- 172.21.15.99:0/101913493 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27006df20 con 0x7fb2700b3160 2024-09-10T11:34:56.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.658+0000 7fb26ffff640 1 --2- 172.21.15.99:0/101913493 >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 0x7fb2700b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.659+0000 7fb274be9640 1 --2- 172.21.15.99:0/101913493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2700b3160 0x7fb27006cd20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb270072600 tx=0x7fb26802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47121f09511fe319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.659+0000 7fb26effd640 1 -- 172.21.15.99:0/101913493 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268036070 con 0x7fb2700b3160 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.659+0000 7fb26effd640 1 -- 172.21.15.99:0/101913493 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268031600 con 0x7fb2700b3160 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.659+0000 7fb26effd640 1 -- 172.21.15.99:0/101913493 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2680319d0 con 0x7fb2700b3160 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 -- 172.21.15.99:0/101913493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2700b3160 msgr2=0x7fb27006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 --2- 172.21.15.99:0/101913493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2700b3160 0x7fb27006cd20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb270072600 tx=0x7fb26802a5d0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 -- 172.21.15.99:0/101913493 shutdown_connections 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 --2- 172.21.15.99:0/101913493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2700b3160 0x7fb27006cd20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 --2- 172.21.15.99:0/101913493 >> v2:172.21.15.196:3300/0 conn(0x7fb270163580 0x7fb2700b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 -- 172.21.15.99:0/101913493 >> 172.21.15.99:0/101913493 conn(0x7fb2700b4570 msgr2=0x7fb2700b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 -- 172.21.15.99:0/101913493 shutdown_connections 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.660+0000 7fb27664c640 1 -- 172.21.15.99:0/101913493 wait complete. 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.661+0000 7fb27664c640 1 Processor -- start 2024-09-10T11:34:56.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.661+0000 7fb27664c640 1 -- start start 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2700b3160 0x7fb2700b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2700b3960 0x7fb2700cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb27006d3e0 con 0x7fb270163580 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb27006d260 con 0x7fb2700b3960 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb27664c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb27006d560 con 0x7fb2700b3160 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56664/0 (socket says 172.21.15.99:56664) 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 -- 172.21.15.99:0/2565513596 learned_addr learned my addr 172.21.15.99:0/2565513596 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb26ffff640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2700b3160 0x7fb2700b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb26f7fe640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2700b3960 0x7fb2700cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 -- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2700b3160 msgr2=0x7fb2700b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2700b3160 0x7fb2700b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 -- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2700b3960 msgr2=0x7fb2700cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.662+0000 7fb274be9640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2700b3960 0x7fb2700cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb274be9640 1 -- 172.21.15.99:0/2565513596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2700b6c60 con 0x7fb270163580 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb274be9640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb26802abc0 tx=0x7fb268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268036050 con 0x7fb270163580 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb268042070 con 0x7fb270163580 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27015d4e0 con 0x7fb270163580 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268009510 con 0x7fb270163580 2024-09-10T11:34:56.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.663+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2700cd590 con 0x7fb270163580 2024-09-10T11:34:56.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.664+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27006cd20 con 0x7fb270163580 2024-09-10T11:34:56.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.664+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb268007bd0 con 0x7fb270163580 2024-09-10T11:34:56.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.665+0000 7fb26cff9640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 0x7fb25c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.665+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7fb2680aba60 con 0x7fb270163580 2024-09-10T11:34:56.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.665+0000 7fb26ffff640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 0x7fb25c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.665+0000 7fb26ffff640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 0x7fb25c06e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb270162c60 tx=0x7fb260008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.669+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb26807a7a0 con 0x7fb270163580 2024-09-10T11:34:56.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.834+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb270160720 con 0x7fb270163580 2024-09-10T11:34:56.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.835+0000 7fb26cff9640 1 -- 172.21.15.99:0/2565513596 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) ==== 93+0+10 (secure 0 0 0) 0x7fb26807a0a0 con 0x7fb270163580 2024-09-10T11:34:56.837 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 msgr2=0x7fb25c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 0x7fb25c06e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb270162c60 tx=0x7fb260008040 comp rx=0 tx=0).stop 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 msgr2=0x7fb270160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb26802abc0 tx=0x7fb268004a40 comp rx=0 tx=0).stop 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 shutdown_connections 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb25c06be30 0x7fb25c06e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb270163580 0x7fb270160da0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2700b3960 0x7fb2700cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 --2- 172.21.15.99:0/2565513596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2700b3160 0x7fb2700b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 >> 172.21.15.99:0/2565513596 conn(0x7fb2700b4570 msgr2=0x7fb2700d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 shutdown_connections 2024-09-10T11:34:56.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.837+0000 7fb27664c640 1 -- 172.21.15.99:0/2565513596 wait complete. 2024-09-10T11:34:56.852 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-09-10T11:34:56.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.956+0000 7fb7bb60a640 1 Processor -- start 2024-09-10T11:34:56.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.956+0000 7fb7bb60a640 1 -- start start 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7bb60a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b406bc10 0x7fb7b4163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7bb60a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7b4160030 0x7fb7b406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7bb60a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b406c530 con 0x7fb7b40b4930 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7bb60a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb7b40b5420 con 0x7fb7b4160030 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7bb60a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40b5140 con 0x7fb7b406bc10 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7b4160030 0x7fb7b406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b9ba7640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb7b40b4930 0x7fb7b406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40282/0 (socket says 172.21.15.99:40282) 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7b4160030 0x7fb7b406bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56674/0 (socket says 172.21.15.99:56674) 2024-09-10T11:34:56.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b9ba7640 1 -- 172.21.15.99:0/1373766441 learned_addr learned my addr 172.21.15.99:0/1373766441 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:56.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 -- 172.21.15.99:0/1373766441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b406bc10 msgr2=0x7fb7b4163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:56.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 --2- 172.21.15.99:0/1373766441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b406bc10 0x7fb7b4163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 -- 172.21.15.99:0/1373766441 >> v1:172.21.15.99:6789/0 conn(0x7fb7b40b4930 legacy=0x7fb7b406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b8ba5640 1 -- 172.21.15.99:0/1373766441 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b4162b40 con 0x7fb7b4160030 2024-09-10T11:34:56.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.957+0000 7fb7b3fff640 1 -- 172.21.15.99:0/1373766441 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265313633 0 0) 0x7fb7b406c530 con 0x7fb7b40b4930 2024-09-10T11:34:56.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7b8ba5640 1 --2- 172.21.15.99:0/1373766441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4160030 0x7fb7b406bff0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009d90 tx=0x7fb7a802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15483ddf1e8b56f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7b3fff640 1 -- 172.21.15.99:0/1373766441 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8031600 con 0x7fb7b4160030 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7b3fff640 1 -- 172.21.15.99:0/1373766441 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a80317a0 con 0x7fb7b4160030 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7b3fff640 1 -- 172.21.15.99:0/1373766441 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8031b70 con 0x7fb7b4160030 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1373766441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4160030 msgr2=0x7fb7b406bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1373766441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4160030 0x7fb7b406bff0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009d90 tx=0x7fb7a802a5d0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1373766441 shutdown_connections 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1373766441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b4160030 0x7fb7b406bff0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1373766441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b406bc10 0x7fb7b4163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1373766441 >> 172.21.15.99:0/1373766441 conn(0x7fb7b40b3630 msgr2=0x7fb7b40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.958+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1373766441 shutdown_connections 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.959+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1373766441 wait complete. 2024-09-10T11:34:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 Processor -- start 2024-09-10T11:34:56.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 -- start start 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7b40b4930 0x7fb7b41e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b4160030 0x7fb7b40c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b40b4db0 con 0x7fb7b406bc10 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b4163100 con 0x7fb7b40b4930 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7bb60a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b40d1560 con 0x7fb7b4160030 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7b93a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7b93a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56690/0 (socket says 172.21.15.99:56690) 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.960+0000 7fb7b93a6640 1 -- 172.21.15.99:0/1538620242 learned_addr learned my addr 172.21.15.99:0/1538620242 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 -- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b4160030 msgr2=0x7fb7b40c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b4160030 0x7fb7b40c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 -- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7b40b4930 msgr2=0x7fb7b41e8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7b40b4930 0x7fb7b41e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 -- 172.21.15.99:0/1538620242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b40c8c70 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b93a6640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b410 tx=0x7fb7ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac015020 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ac011850 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac011c40 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b40c8f50 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.961+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b40c9510 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.962+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b4163620 con 0x7fb7b406bc10 2024-09-10T11:34:56.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.962+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7ac018020 con 0x7fb7b406bc10 2024-09-10T11:34:56.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.963+0000 7fb7b1ffb640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 0x7fb7a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:56.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.963+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7fb7ac08aa80 con 0x7fb7b406bc10 2024-09-10T11:34:56.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.963+0000 7fb7b8ba5640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 0x7fb7a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:56.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.963+0000 7fb7b8ba5640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 0x7fb7a006e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8034000 tx=0x7fb7a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:56.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:56.967+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7ac01f020 con 0x7fb7b406bc10 2024-09-10T11:34:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.194+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7fb7b4073d40 con 0x7fb7b406bc10 2024-09-10T11:34:57.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.251+0000 7fb7b1ffb640 1 -- 172.21.15.99:0/1538620242 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v243) ==== 131+0+0 (secure 0 0 0) 0x7fb7ac0172c0 con 0x7fb7b406bc10 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.253+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 msgr2=0x7fb7a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.253+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 0x7fb7a006e2f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8034000 tx=0x7fb7a802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.253+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 msgr2=0x7fb7b41e89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.253+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00b410 tx=0x7fb7ac00b8d0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 shutdown_connections 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb7a006be30 0x7fb7a006e2f0 secure :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8034000 tx=0x7fb7a802e000 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7b4160030 0x7fb7b40c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7b40b4930 0x7fb7b41e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 --2- 172.21.15.99:0/1538620242 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7b406bc10 0x7fb7b41e89b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 >> 172.21.15.99:0/1538620242 conn(0x7fb7b40b3630 msgr2=0x7fb7b4147fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 shutdown_connections 2024-09-10T11:34:57.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.254+0000 7fb7bb60a640 1 -- 172.21.15.99:0/1538620242 wait complete. 2024-09-10T11:34:57.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:34:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.367+0000 7f557719f640 1 Processor -- start 2024-09-10T11:34:57.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.367+0000 7f557719f640 1 -- start start 2024-09-10T11:34:57.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557719f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55700cfbe0 0x7f5570162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557719f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5570162e50 0x7f557006e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557719f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55700b38a0 con 0x7f5570148620 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557719f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f557006dc30 con 0x7f5570162e50 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557719f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f55700d00c0 con 0x7f55700cfbe0 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557573c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5570162e50 0x7f557006e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f5574f3b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5570148620 0x7f55700b5ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40292/0 (socket says 172.21.15.99:40292) 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f5574f3b640 1 -- 172.21.15.99:0/906747062 learned_addr learned my addr 172.21.15.99:0/906747062 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f5567fff640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55700cfbe0 0x7f5570162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557573c640 1 -- 172.21.15.99:0/906747062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55700cfbe0 msgr2=0x7f5570162910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557573c640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55700cfbe0 0x7f5570162910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557573c640 1 -- 172.21.15.99:0/906747062 >> v1:172.21.15.99:6789/0 conn(0x7f5570148620 legacy=0x7f55700b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f557573c640 1 -- 172.21.15.99:0/906747062 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55700d03a0 con 0x7f5570162e50 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.368+0000 7f55677fe640 1 -- 172.21.15.99:0/906747062 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1331877043 0 0) 0x7f55700b38a0 con 0x7f5570148620 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.369+0000 7f557573c640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570162e50 0x7f557006e0c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f55680097a0 tx=0x7f556802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66c729799ff5e31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.369+0000 7f55677fe640 1 -- 172.21.15.99:0/906747062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568037070 con 0x7f5570162e50 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.369+0000 7f55677fe640 1 -- 172.21.15.99:0/906747062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5568032440 con 0x7f5570162e50 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.369+0000 7f55677fe640 1 -- 172.21.15.99:0/906747062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568032810 con 0x7f5570162e50 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 -- 172.21.15.99:0/906747062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570162e50 msgr2=0x7f557006e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570162e50 0x7f557006e0c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f55680097a0 tx=0x7f556802a180 comp rx=0 tx=0).stop 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 -- 172.21.15.99:0/906747062 shutdown_connections 2024-09-10T11:34:57.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570162e50 0x7f557006e0c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 --2- 172.21.15.99:0/906747062 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55700cfbe0 0x7f5570162910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 -- 172.21.15.99:0/906747062 >> 172.21.15.99:0/906747062 conn(0x7f55700b2a80 msgr2=0x7f55700b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:57.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 -- 172.21.15.99:0/906747062 shutdown_connections 2024-09-10T11:34:57.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.370+0000 7f557719f640 1 -- 172.21.15.99:0/906747062 wait complete. 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.371+0000 7f557719f640 1 Processor -- start 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.371+0000 7f557719f640 1 -- start start 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 0x7f5570137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5570162e50 0x7f55701465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5570148a00 con 0x7f5570148620 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f557006d8c0 con 0x7f55700cfbe0 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557719f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5570148b80 con 0x7f5570162e50 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56716/0 (socket says 172.21.15.99:56716) 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 -- 172.21.15.99:0/2477945220 learned_addr learned my addr 172.21.15.99:0/2477945220 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:57.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f557573c640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5570162e50 0x7f55701465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5574f3b640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 0x7f5570137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 -- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5570162e50 msgr2=0x7f55701465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5570162e50 0x7f55701465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 -- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 msgr2=0x7f5570137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 0x7f5570137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 -- 172.21.15.99:0/2477945220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55700cc420 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5574f3b640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 0x7f5570137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.372+0000 7f5567fff640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f556000d800 tx=0x7f556000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.373+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556000b660 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.373+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5560019070 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.373+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55700cc700 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.373+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556000ba20 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.373+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5570142090 con 0x7f5570148620 2024-09-10T11:34:57.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.374+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570162910 con 0x7f5570148620 2024-09-10T11:34:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.374+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f556000bbc0 con 0x7f5570148620 2024-09-10T11:34:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.374+0000 7f55657fa640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.375+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f556008ade0 con 0x7f5570148620 2024-09-10T11:34:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.375+0000 7f5574f3b640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.375+0000 7f5574f3b640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 0x7f555c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f556c0012e0 tx=0x7f556c007f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.379+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5560059b20 con 0x7f5570148620 2024-09-10T11:34:57.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.545+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f55700d0950 con 0x7f5570148620 2024-09-10T11:34:57.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.548+0000 7f55657fa640 1 -- 172.21.15.99:0/2477945220 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f5560059420 con 0x7f5570148620 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.550+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 msgr2=0x7f555c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.550+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 0x7f555c06e320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f556c0012e0 tx=0x7f556c007f90 comp rx=0 tx=0).stop 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 msgr2=0x7f55701f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f556000d800 tx=0x7f556000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 shutdown_connections 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f555c06be60 0x7f555c06e320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5570162e50 0x7f55701465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570148620 0x7f55701f4b40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 --2- 172.21.15.99:0/2477945220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55700cfbe0 0x7f5570137480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 >> 172.21.15.99:0/2477945220 conn(0x7f55700b2a80 msgr2=0x7f5570067c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:57.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 shutdown_connections 2024-09-10T11:34:57.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.551+0000 7f557719f640 1 -- 172.21.15.99:0/2477945220 wait complete. 2024-09-10T11:34:57.563 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:34:57.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:57.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:34:57.651 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:34:57.652 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:57.652 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:57.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:57.669 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:57.669 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:57.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:57.688 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:34:57.688 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:57.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:34:57.724 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:34:57.724 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:34:57.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:34:57.745 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:34:57.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:34:57.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.859+0000 7ff477531640 1 Processor -- start 2024-09-10T11:34:57.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.859+0000 7ff477531640 1 -- start start 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff477531640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 0x7ff470073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff477531640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff47015c770 0x7ff47006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff477531640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff470152e60 con 0x7ff47006e5d0 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff477531640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff47006ef80 con 0x7ff47015c770 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff477531640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff4700d15e0 con 0x7ff4700d1200 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff47015c770 0x7ff47006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff47015c770 0x7ff47006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56726/0 (socket says 172.21.15.99:56726) 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 -- 172.21.15.99:0/1778394275 learned_addr learned my addr 172.21.15.99:0/1778394275 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1778394275 >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 0x7ff470073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 -- 172.21.15.99:0/1778394275 >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 msgr2=0x7ff470073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 --2- 172.21.15.99:0/1778394275 >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 0x7ff470073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 -- 172.21.15.99:0/1778394275 >> v1:172.21.15.196:6789/0 conn(0x7ff47006e5d0 legacy=0x7ff47015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff475ace640 1 -- 172.21.15.99:0/1778394275 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff470072680 con 0x7ff47015c770 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.860+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1778394275 >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 0x7ff470073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.861+0000 7ff475ace640 1 --2- 172.21.15.99:0/1778394275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff47006ea40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff4700d1850 tx=0x7ff46802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c69518b3b65106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.861+0000 7ff467fff640 1 -- 172.21.15.99:0/1778394275 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468039070 con 0x7ff47015c770 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.861+0000 7ff467fff640 1 -- 172.21.15.99:0/1778394275 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46800b2e0 con 0x7ff47015c770 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.861+0000 7ff467fff640 1 -- 172.21.15.99:0/1778394275 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468009820 con 0x7ff47015c770 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 -- 172.21.15.99:0/1778394275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 msgr2=0x7ff47006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 --2- 172.21.15.99:0/1778394275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff47006ea40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff4700d1850 tx=0x7ff46802c860 comp rx=0 tx=0).stop 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 -- 172.21.15.99:0/1778394275 shutdown_connections 2024-09-10T11:34:57.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 --2- 172.21.15.99:0/1778394275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff47006ea40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 --2- 172.21.15.99:0/1778394275 >> v2:172.21.15.196:3300/0 conn(0x7ff4700d1200 0x7ff470073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 -- 172.21.15.99:0/1778394275 >> 172.21.15.99:0/1778394275 conn(0x7ff47006bc10 msgr2=0x7ff47006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 -- 172.21.15.99:0/1778394275 shutdown_connections 2024-09-10T11:34:57.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.862+0000 7ff477531640 1 -- 172.21.15.99:0/1778394275 wait complete. 2024-09-10T11:34:57.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.863+0000 7ff477531640 1 Processor -- start 2024-09-10T11:34:57.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.863+0000 7ff477531640 1 -- start start 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 0x7ff470158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4700d1200 0x7ff47015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff470072f20 con 0x7ff47015c770 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff470072c40 con 0x7ff4700d1200 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff47015cd90 con 0x7ff47006e5d0 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56728/0 (socket says 172.21.15.99:56728) 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 -- 172.21.15.99:0/1278019719 learned_addr learned my addr 172.21.15.99:0/1278019719 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 0x7ff470158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff474acc640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4700d1200 0x7ff47015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 -- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 msgr2=0x7ff470158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 0x7ff470158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 -- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4700d1200 msgr2=0x7ff47015b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4700d1200 0x7ff47015b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 -- 172.21.15.99:0/1278019719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff470158820 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 0x7ff470158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff475ace640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff468037040 tx=0x7ff468004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff468039050 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff468045070 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.864+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff470158b00 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.865+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46800b720 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.865+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4701590c0 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.866+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4700b49e0 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.866+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff46800b2e0 con 0x7ff47015c770 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.867+0000 7ff465ffb640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 0x7ff45406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.867+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 0x7ff45406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:57.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.867+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7ff4680ad2b0 con 0x7ff47015c770 2024-09-10T11:34:57.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.867+0000 7ff4752cd640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 0x7ff45406e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00a990 tx=0x7ff46c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:57.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:57.871+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff46804e020 con 0x7ff47015c770 2024-09-10T11:34:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.069+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4701e5ce0 con 0x7ff47015c770 2024-09-10T11:34:58.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.069+0000 7ff465ffb640 1 -- 172.21.15.99:0/1278019719 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+3917 (secure 0 0 0) 0x7ff46807bed0 con 0x7ff47015c770 2024-09-10T11:34:58.071 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:58.072 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":44,"btime":"2024-09-10T11:34:53:239338+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3953515234","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3953515234},{"type":"v1","addr":"172.21.15.196:6834","nonce":3953515234}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6088,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2469765540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2469765540},{"type":"v1","addr":"172.21.15.196:6835","nonce":2469765540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":6228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/4046864288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":4046864288},{"type":"v1","addr":"172.21.15.99:6836","nonce":4046864288}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1127412891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1127412891},{"type":"v1","addr":"172.21.15.99:6837","nonce":1127412891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-10T11:34:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 44 2024-09-10T11:34:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.071+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 msgr2=0x7ff45406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.071+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 0x7ff45406e320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00a990 tx=0x7ff46c008040 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.071+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 msgr2=0x7ff470155470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.071+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff468037040 tx=0x7ff468004a40 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 shutdown_connections 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff45406be60 0x7ff45406e320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff47015c770 0x7ff470155470 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4700d1200 0x7ff47015b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 --2- 172.21.15.99:0/1278019719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff47006e5d0 0x7ff470158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 >> 172.21.15.99:0/1278019719 conn(0x7ff47006bc10 msgr2=0x7ff47015f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 shutdown_connections 2024-09-10T11:34:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.072+0000 7ff477531640 1 -- 172.21.15.99:0/1278019719 wait complete. 2024-09-10T11:34:58.085 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:34:58.085 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:34:58.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:34:58.085 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:58.086 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:34:58.086 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:34:58.086 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:34:58.086 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:58.086 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:34:58.087 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:34:58.087 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:34:58.087 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:58.087 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:34:58.088 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:34:58.088 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:34:58.088 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:34:58.094 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:34:58.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:34:58.095 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:34:58.096 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:34:58.099 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:34:58.099 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:34:58.100 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:34:58.101 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:34:58.137 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:34:58.137 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:58.138 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:34:58.138 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:58.142 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:34:58.142 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:58.143 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:34:58.143 DEBUG:teuthology.parallel:result is None 2024-09-10T11:34:58.143 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:34:58.212 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:34:58.213 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:34:58.213 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:34:58.211+0000 7f3a3332e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-10T11:34:58.226 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:34:58.227 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:34:58.227 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-09-10T11:34:58.227+0000 7f9b85641640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T11:34:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.245+0000 7f8e97931640 1 Processor -- start 2024-09-10T11:34:58.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.245+0000 7f8e97931640 1 -- start start 2024-09-10T11:34:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e97931640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e90073300 0x7f8e900b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e97931640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e90070610 0x7f8e900709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e97931640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e90070f30 con 0x7f8e90146490 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e97931640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8e900b5d70 con 0x7f8e90070610 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e97931640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8e900b6050 con 0x7f8e90073300 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e90070610 0x7f8e900709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e90070610 0x7f8e900709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56744/0 (socket says 172.21.15.99:56744) 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 -- 172.21.15.99:0/1124960635 learned_addr learned my addr 172.21.15.99:0/1124960635 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e94ecc640 1 --2- 172.21.15.99:0/1124960635 >> v2:172.21.15.196:3300/0 conn(0x7f8e90073300 0x7f8e900b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 -- 172.21.15.99:0/1124960635 >> v2:172.21.15.196:3300/0 conn(0x7f8e90073300 msgr2=0x7f8e900b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 --2- 172.21.15.99:0/1124960635 >> v2:172.21.15.196:3300/0 conn(0x7f8e90073300 0x7f8e900b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 -- 172.21.15.99:0/1124960635 >> v1:172.21.15.196:6789/0 conn(0x7f8e90146490 legacy=0x7f8e900d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.246+0000 7f8e95ece640 1 -- 172.21.15.99:0/1124960635 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e900b6330 con 0x7f8e90070610 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.247+0000 7f8e95ece640 1 --2- 172.21.15.99:0/1124960635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e900709f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8e90072610 tx=0x7f8e8802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29949ccb91322346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.247+0000 7f8e83fff640 1 -- 172.21.15.99:0/1124960635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88036070 con 0x7f8e90070610 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.247+0000 7f8e83fff640 1 -- 172.21.15.99:0/1124960635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e88031600 con 0x7f8e90070610 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.247+0000 7f8e83fff640 1 -- 172.21.15.99:0/1124960635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e880319d0 con 0x7f8e90070610 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 -- 172.21.15.99:0/1124960635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 msgr2=0x7f8e900709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 --2- 172.21.15.99:0/1124960635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e900709f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8e90072610 tx=0x7f8e8802a6a0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 -- 172.21.15.99:0/1124960635 shutdown_connections 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 --2- 172.21.15.99:0/1124960635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e900709f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 --2- 172.21.15.99:0/1124960635 >> v2:172.21.15.196:3300/0 conn(0x7f8e90073300 0x7f8e900b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 -- 172.21.15.99:0/1124960635 >> 172.21.15.99:0/1124960635 conn(0x7f8e90072680 msgr2=0x7f8e90072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:58.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 -- 172.21.15.99:0/1124960635 shutdown_connections 2024-09-10T11:34:58.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.248+0000 7f8e97931640 1 -- 172.21.15.99:0/1124960635 wait complete. 2024-09-10T11:34:58.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.249+0000 7f8e97931640 1 Processor -- start 2024-09-10T11:34:58.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.249+0000 7f8e97931640 1 -- start start 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e90073300 0x7f8e901f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e90146490 0x7f8e90143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9015d3e0 con 0x7f8e90070610 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9015d260 con 0x7f8e90146490 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9015d560 con 0x7f8e90073300 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56752/0 (socket says 172.21.15.99:56752) 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 -- 172.21.15.99:0/2669397314 learned_addr learned my addr 172.21.15.99:0/2669397314 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e94ecc640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e90073300 0x7f8e901f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e95ece640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e90146490 0x7f8e90143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 -- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e90073300 msgr2=0x7f8e901f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e90073300 0x7f8e901f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 -- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e90146490 msgr2=0x7f8e90143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:58.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e90146490 0x7f8e90143390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 -- 172.21.15.99:0/2669397314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e901f2d90 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e956cd640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00c3c0 tx=0x7f8e8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c0052d0 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8c005490 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.250+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e901f3070 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.251+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c007260 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.251+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e90142e70 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.252+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e900709f0 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.252+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e8c007400 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.252+0000 7f8e81ffb640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 0x7f8e7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.253+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4943+0+0 (secure 0 0 0) 0x7f8e8c089f40 con 0x7f8e90070610 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.253+0000 7f8e94ecc640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 0x7f8e7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:58.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.253+0000 7f8e94ecc640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 0x7f8e7406e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8e90073e30 tx=0x7f8e84009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:58.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.262+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e8c058630 con 0x7f8e90070610 2024-09-10T11:34:58.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-10T11:34:58.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-10T11:34:58.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T11:34:58.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-10T11:34:58.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:58.445+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8e9015cc70 con 0x7f8e90070610 2024-09-10T11:34:58.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:34:58.785 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:34:59.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.278+0000 7f8e81ffb640 1 -- 172.21.15.99:0/2669397314 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v119) ==== 97+0+0 (secure 0 0 0) 0x7f8e8c05c930 con 0x7f8e90070610 2024-09-10T11:34:59.280 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 msgr2=0x7f8e7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 0x7f8e7406e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8e90073e30 tx=0x7f8e84009190 comp rx=0 tx=0).stop 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 msgr2=0x7f8e901403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00c3c0 tx=0x7f8e8c00c880 comp rx=0 tx=0).stop 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 shutdown_connections 2024-09-10T11:34:59.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e7406be30 0x7f8e7406e2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e90146490 0x7f8e90143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e90073300 0x7f8e901f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 --2- 172.21.15.99:0/2669397314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e90070610 0x7f8e901403f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 >> 172.21.15.99:0/2669397314 conn(0x7f8e90072680 msgr2=0x7f8e9015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 shutdown_connections 2024-09-10T11:34:59.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.281+0000 7f8e97931640 1 -- 172.21.15.99:0/2669397314 wait complete. 2024-09-10T11:34:59.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:34:59.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.393+0000 7fa4504e4640 1 Processor -- start 2024-09-10T11:34:59.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.393+0000 7fa4504e4640 1 -- start start 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.394+0000 7fa4504e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa4480d5010 0x7fa4480d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.394+0000 7fa4504e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44815ff70 con 0x7fa44815eb80 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.394+0000 7fa4504e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4480d1620 con 0x7fa4481626c0 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.394+0000 7fa4504e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa4480d30c0 con 0x7fa4480d5010 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa44815eb80 0x7fa4480d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40334/0 (socket says 172.21.15.99:40334) 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 -- 172.21.15.99:0/1730339109 learned_addr learned my addr 172.21.15.99:0/1730339109 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44ea81640 1 --2- 172.21.15.99:0/1730339109 >> v2:172.21.15.196:3300/0 conn(0x7fa4480d5010 0x7fa4480d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925047748 0 0) 0x7fa44815ff70 con 0x7fa44815eb80 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa434003520 con 0x7fa44815eb80 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856567781 0 0) 0x7fa434003520 con 0x7fa44815eb80 2024-09-10T11:34:59.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 >> v2:172.21.15.196:3300/0 conn(0x7fa4480d5010 msgr2=0x7fa4480d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 --2- 172.21.15.99:0/1730339109 >> v2:172.21.15.196:3300/0 conn(0x7fa4480d5010 0x7fa4480d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 >> v1:172.21.15.196:6789/0 conn(0x7fa4481626c0 legacy=0x7fa4480d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa448162bc0 con 0x7fa44815eb80 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa444001670 con 0x7fa44815eb80 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 >> v1:172.21.15.99:6789/0 conn(0x7fa44815eb80 legacy=0x7fa4480d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480d30c0 con 0x7fa4340039b0 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa43400c160 con 0x7fa434004800 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa43400c360 con 0x7fa4340051c0 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44da7f640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44ea81640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 -- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340051c0 msgr2=0x7fa43400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 -- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa434004800 msgr2=0x7fa434004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa434004800 0x7fa434004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44e280640 1 -- 172.21.15.99:0/1730339109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa434003520 con 0x7fa4340039b0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.395+0000 7fa44ea81640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340051c0 0x7fa43400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.396+0000 7fa44ea81640 1 -- 172.21.15.99:0/1730339109 reap_dead start 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.396+0000 7fa44e280640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 0x7fa434003dd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c210 tx=0x7fa434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d62c90d3b701c576 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.396+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444007d30 con 0x7fa4340039b0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.396+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444005ad0 con 0x7fa4340039b0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.396+0000 7fa44d27e640 1 -- 172.21.15.99:0/1730339109 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444007070 con 0x7fa4340039b0 2024-09-10T11:34:59.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 -- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 msgr2=0x7fa434003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 0x7fa434003dd0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa44400c210 tx=0x7fa434005830 comp rx=0 tx=0).stop 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 -- 172.21.15.99:0/1730339109 shutdown_connections 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 --2- 172.21.15.99:0/1730339109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4340039b0 0x7fa434003dd0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 -- 172.21.15.99:0/1730339109 >> 172.21.15.99:0/1730339109 conn(0x7fa44808e680 msgr2=0x7fa44809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 -- 172.21.15.99:0/1730339109 shutdown_connections 2024-09-10T11:34:59.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.397+0000 7fa4504e4640 1 -- 172.21.15.99:0/1730339109 wait complete. 2024-09-10T11:34:59.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.398+0000 7fa4504e4640 1 Processor -- start 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.398+0000 7fa4504e4640 1 -- start start 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340039b0 0x7fa4481f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 0x7fa448161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa448163460 con 0x7fa448160170 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa448163180 con 0x7fa448161790 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa4504e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480d29b0 con 0x7fa4340039b0 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56766/0 (socket says 172.21.15.99:56766) 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 -- 172.21.15.99:0/3593544310 learned_addr learned my addr 172.21.15.99:0/3593544310 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44e280640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340039b0 0x7fa4481f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44ea81640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 0x7fa448161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 -- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340039b0 msgr2=0x7fa4481f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340039b0 0x7fa4481f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 -- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 msgr2=0x7fa448161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 0x7fa448161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 -- 172.21.15.99:0/3593544310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481606f0 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44ea81640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 0x7fa448161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.399+0000 7fa44da7f640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa43800f620 tx=0x7fa43800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.400+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4380053a0 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.400+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa438018070 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.400+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481609d0 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.400+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438003810 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.400+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa448160f90 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.401+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4480d2d40 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.401+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4380039b0 con 0x7fa448160170 2024-09-10T11:34:59.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.402+0000 7fa43effd640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 0x7fa42c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.402+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4857+0+0 (secure 0 0 0) 0x7fa43808a7c0 con 0x7fa448160170 2024-09-10T11:34:59.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.402+0000 7fa44e280640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 0x7fa42c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.402+0000 7fa44e280640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 0x7fa42c06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa444032000 tx=0x7fa44402d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.406+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4380594c0 con 0x7fa448160170 2024-09-10T11:34:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.635+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa4481f5420 con 0x7fa448160170 2024-09-10T11:34:59.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.635+0000 7fa43effd640 1 -- 172.21.15.99:0/3593544310 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fa438058dc0 con 0x7fa448160170 2024-09-10T11:34:59.637 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:34:59.639 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:59.640 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.644 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.646 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:34:59.648 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.649 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:34:59.651 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 msgr2=0x7fa42c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 0x7fa42c06e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa444032000 tx=0x7fa44402d000 comp rx=0 tx=0).stop 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 msgr2=0x7fa44815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fa43800f620 tx=0x7fa43800fae0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 shutdown_connections 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa42c06be30 0x7fa42c06e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa448161790 0x7fa448161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa448160170 0x7fa44815a2b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 --2- 172.21.15.99:0/3593544310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4340039b0 0x7fa4481f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.637+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 >> 172.21.15.99:0/3593544310 conn(0x7fa44808e680 msgr2=0x7fa44809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:59.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.638+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 shutdown_connections 2024-09-10T11:34:59.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.638+0000 7fa4504e4640 1 -- 172.21.15.99:0/3593544310 wait complete. 2024-09-10T11:34:59.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:34:59.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.751+0000 7f38d5a53640 1 Processor -- start 2024-09-10T11:34:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.751+0000 7f38d5a53640 1 -- start start 2024-09-10T11:34:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38d5a53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38d00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38d5a53640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38d00d5010 0x7f38d00d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38d5a53640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d015ff70 con 0x7f38d01626c0 2024-09-10T11:34:59.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38d5a53640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f38d00d1620 con 0x7f38d00d5010 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38d5a53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f38d0162bc0 con 0x7f38d015eb80 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38ce7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f38d01626c0 0x7f38d00d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40350/0 (socket says 172.21.15.99:40350) 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38ce7fc640 1 -- 172.21.15.99:0/1328543043 learned_addr learned my addr 172.21.15.99:0/1328543043 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/1328543043 >> v2:172.21.15.196:3300/0 conn(0x7f38d00d5010 0x7f38d00d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38ceffd640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38d00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459054714 0 0) 0x7f38d015ff70 con 0x7f38d01626c0 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.752+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38b8003520 con 0x7f38d01626c0 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398939652 0 0) 0x7f38b8003520 con 0x7f38d01626c0 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 msgr2=0x7f38d00d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38d00d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 >> v2:172.21.15.196:3300/0 conn(0x7f38d00d5010 msgr2=0x7f38d00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 --2- 172.21.15.99:0/1328543043 >> v2:172.21.15.196:3300/0 conn(0x7f38d00d5010 0x7f38d00d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d0162ea0 con 0x7f38d01626c0 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f38c4001670 con 0x7f38d01626c0 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 >> v1:172.21.15.99:6789/0 conn(0x7f38d01626c0 legacy=0x7f38d00d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38b80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38b8003f30 0x7f38b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00d1620 con 0x7f38b8003f30 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0162bc0 con 0x7f38b8004d40 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f38b800ba10 con 0x7f38d015eb80 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38b8003f30 0x7f38b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ceffd640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38b80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 -- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 msgr2=0x7f38b80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d015eb80 0x7f38b80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 -- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8004d40 msgr2=0x7f38b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38ce7fc640 1 -- 172.21.15.99:0/1328543043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b8003520 con 0x7f38b8003f30 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.753+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8004d40 0x7f38b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38ce7fc640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38b8003f30 0x7f38b8004800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c240 tx=0x7f38b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643ae58f1c838ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4007d50 con 0x7f38b8003f30 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c402e070 con 0x7f38b8003f30 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38cdffb640 1 -- 172.21.15.99:0/1328543043 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4005d70 con 0x7f38b8003f30 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 -- 172.21.15.99:0/1328543043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38b8003f30 msgr2=0x7f38b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 --2- 172.21.15.99:0/1328543043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38b8003f30 0x7f38b8004800 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c240 tx=0x7f38b8005830 comp rx=0 tx=0).stop 2024-09-10T11:34:59.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38cf7fe640 1 -- 172.21.15.99:0/1328543043 reap_dead start 2024-09-10T11:34:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 -- 172.21.15.99:0/1328543043 shutdown_connections 2024-09-10T11:34:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 -- 172.21.15.99:0/1328543043 >> 172.21.15.99:0/1328543043 conn(0x7f38d008e680 msgr2=0x7f38d009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 -- 172.21.15.99:0/1328543043 shutdown_connections 2024-09-10T11:34:59.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.754+0000 7f38d5a53640 1 -- 172.21.15.99:0/1328543043 wait complete. 2024-09-10T11:34:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 Processor -- start 2024-09-10T11:34:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 -- start start 2024-09-10T11:34:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8003f30 0x7f38d00b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d01532f0 0x7f38d00b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00d2060 con 0x7f38d0152d30 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0163460 con 0x7f38b8003f30 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.756+0000 7f38d5a53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f38d00d21e0 con 0x7f38d01532f0 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56782/0 (socket says 172.21.15.99:56782) 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 -- 172.21.15.99:0/596910859 learned_addr learned my addr 172.21.15.99:0/596910859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38ce7fc640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d01532f0 0x7f38d00b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38ceffd640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8003f30 0x7f38d00b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 -- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d01532f0 msgr2=0x7f38d00b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d01532f0 0x7f38d00b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 -- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8003f30 msgr2=0x7f38d00b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8003f30 0x7f38d00b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 -- 172.21.15.99:0/596910859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00b85a0 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38cf7fe640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38c8007c80 tx=0x7f38c80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c801a020 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c8011c30 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d00ba7e0 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8016470 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.757+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d00b8b20 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.758+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d00d1b20 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.759+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38c8005d10 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.759+0000 7f38b7fff640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 0x7f38bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.759+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4857+0+0 (secure 0 0 0) 0x7f38c808b500 con 0x7f38d0152d30 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.759+0000 7f38ceffd640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 0x7f38bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.760+0000 7f38ceffd640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 0x7f38bc06e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38d015e330 tx=0x7f38c0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.763+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38c8024030 con 0x7f38d0152d30 2024-09-10T11:34:59.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.932+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38d00d0b00 con 0x7f38d0152d30 2024-09-10T11:34:59.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.932+0000 7f38b7fff640 1 -- 172.21.15.99:0/596910859 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) ==== 84+0+65 (secure 0 0 0) 0x7f38c805a290 con 0x7f38d0152d30 2024-09-10T11:34:59.934 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:34:59.934 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:34:59.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:34:59.935 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:34:59.935 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:34:59.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:34:59.935 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:34:59.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 msgr2=0x7f38bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 0x7f38bc06e320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38d015e330 tx=0x7f38c0009190 comp rx=0 tx=0).stop 2024-09-10T11:34:59.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 msgr2=0x7f38d00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:59.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38c8007c80 tx=0x7f38c80115a0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 shutdown_connections 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f38bc06be60 0x7f38bc06e320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38d0152d30 0x7f38d00b2f00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38d01532f0 0x7f38d00b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 --2- 172.21.15.99:0/596910859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38b8003f30 0x7f38d00b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 >> 172.21.15.99:0/596910859 conn(0x7f38d008e680 msgr2=0x7f38d01612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 shutdown_connections 2024-09-10T11:34:59.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:34:59.934+0000 7f38d5a53640 1 -- 172.21.15.99:0/596910859 wait complete. 2024-09-10T11:34:59.947 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:34:59.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:35:00.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.048+0000 7f5e374d0640 1 Processor -- start 2024-09-10T11:35:00.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.048+0000 7f5e374d0640 1 -- start start 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e374d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e300d1200 0x7f5e30073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e374d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 0x7f5e3006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e374d0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e30152e60 con 0x7f5e3015c060 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e374d0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5e3006ef50 con 0x7f5e300d1200 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e374d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e300d15e0 con 0x7f5e3006e5d0 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e3526c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e300d1200 0x7f5e30073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e35a6d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5e3015c060 0x7f5e3015c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40356/0 (socket says 172.21.15.99:40356) 2024-09-10T11:35:00.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.049+0000 7f5e35a6d640 1 -- 172.21.15.99:0/191598489 learned_addr learned my addr 172.21.15.99:0/191598489 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:00.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e3526c640 1 -- 172.21.15.99:0/191598489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 msgr2=0x7f5e3006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:00.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e34a6b640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 0x7f5e3006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e2ffff640 1 -- 172.21.15.99:0/191598489 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 648624289 0 0) 0x7f5e30152e60 con 0x7f5e3015c060 2024-09-10T11:35:00.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e3526c640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 0x7f5e3006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e3526c640 1 -- 172.21.15.99:0/191598489 >> v1:172.21.15.99:6789/0 conn(0x7f5e3015c060 legacy=0x7f5e3015c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e3526c640 1 -- 172.21.15.99:0/191598489 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e30072680 con 0x7f5e300d1200 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e34a6b640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 0x7f5e3006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.050+0000 7f5e3526c640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e300d1200 0x7f5e30073440 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009d90 tx=0x7f5e2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b1dca4e15600c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e2ffff640 1 -- 172.21.15.99:0/191598489 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20031600 con 0x7f5e300d1200 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e2ffff640 1 -- 172.21.15.99:0/191598489 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e200317a0 con 0x7f5e300d1200 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e2ffff640 1 -- 172.21.15.99:0/191598489 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20031b70 con 0x7f5e300d1200 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 -- 172.21.15.99:0/191598489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e300d1200 msgr2=0x7f5e30073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e300d1200 0x7f5e30073440 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009d90 tx=0x7f5e2002a5d0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 -- 172.21.15.99:0/191598489 shutdown_connections 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e3006e5d0 0x7f5e3006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 --2- 172.21.15.99:0/191598489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e300d1200 0x7f5e30073440 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 -- 172.21.15.99:0/191598489 >> 172.21.15.99:0/191598489 conn(0x7f5e3006bc10 msgr2=0x7f5e3006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 -- 172.21.15.99:0/191598489 shutdown_connections 2024-09-10T11:35:00.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.051+0000 7f5e374d0640 1 -- 172.21.15.99:0/191598489 wait complete. 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.052+0000 7f5e374d0640 1 Processor -- start 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.052+0000 7f5e374d0640 1 -- start start 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e300d1200 0x7f5e300ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e3015c060 0x7f5e3015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30072f20 con 0x7f5e3006e5d0 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30072c40 con 0x7f5e3015c060 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e374d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3015cd90 con 0x7f5e300d1200 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e3526c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e3526c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56810/0 (socket says 172.21.15.99:56810) 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.053+0000 7f5e3526c640 1 -- 172.21.15.99:0/3080462264 learned_addr learned my addr 172.21.15.99:0/3080462264 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e34a6b640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e300d1200 0x7f5e300ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e35a6d640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e3015c060 0x7f5e3015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 -- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e300d1200 msgr2=0x7f5e300ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e300d1200 0x7f5e300ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 -- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e3015c060 msgr2=0x7f5e3015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e3015c060 0x7f5e3015e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 -- 172.21.15.99:0/3080462264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e300abbe0 con 0x7f5e3006e5d0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e3526c640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009d90 tx=0x7f5e20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20040070 con 0x7f5e3006e5d0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20030be0 con 0x7f5e3006e5d0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20009450 con 0x7f5e3006e5d0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e300cda30 con 0x7f5e3006e5d0 2024-09-10T11:35:00.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.054+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e300cdff0 con 0x7f5e3006e5d0 2024-09-10T11:35:00.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.055+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3006ea10 con 0x7f5e3006e5d0 2024-09-10T11:35:00.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.056+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e20039040 con 0x7f5e3006e5d0 2024-09-10T11:35:00.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.056+0000 7f5e2dffb640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 0x7f5e1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.056+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 4857+0+0 (secure 0 0 0) 0x7f5e200abc80 con 0x7f5e3006e5d0 2024-09-10T11:35:00.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.056+0000 7f5e34a6b640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 0x7f5e1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.057+0000 7f5e34a6b640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 0x7f5e1c06e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5e301f8cb0 tx=0x7f5e24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.060+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e2007a980 con 0x7f5e3006e5d0 2024-09-10T11:35:00.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.226+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5e300b13e0 con 0x7f5e3006e5d0 2024-09-10T11:35:00.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.553+0000 7f5e2dffb640 1 -- 172.21.15.99:0/3080462264 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) ==== 138+0+0 (secure 0 0 0) 0x7f5e2007a280 con 0x7f5e3006e5d0 2024-09-10T11:35:00.555 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:35:00.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 msgr2=0x7f5e1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 0x7f5e1c06e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5e301f8cb0 tx=0x7f5e24008040 comp rx=0 tx=0).stop 2024-09-10T11:35:00.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 msgr2=0x7f5e301506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009d90 tx=0x7f5e20004a40 comp rx=0 tx=0).stop 2024-09-10T11:35:00.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 shutdown_connections 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e1c06be30 0x7f5e1c06e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e3015c060 0x7f5e3015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e300d1200 0x7f5e300ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 --2- 172.21.15.99:0/3080462264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e3006e5d0 0x7f5e301506f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 >> 172.21.15.99:0/3080462264 conn(0x7f5e3006bc10 msgr2=0x7f5e300b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 shutdown_connections 2024-09-10T11:35:00.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.556+0000 7f5e374d0640 1 -- 172.21.15.99:0/3080462264 wait complete. 2024-09-10T11:35:00.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:35:00.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.675+0000 7f5eb05b7640 1 Processor -- start 2024-09-10T11:35:00.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.675+0000 7f5eb05b7640 1 -- start start 2024-09-10T11:35:00.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eb05b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5ea80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eb05b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea80b40e0 con 0x7f5ea80b0090 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eb05b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ea80b5ab0 con 0x7f5ea815f1e0 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eb05b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea8073c50 con 0x7f5ea815ee00 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eaeb54640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5ea80b0090 0x7f5ea80b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40364/0 (socket says 172.21.15.99:40364) 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eaeb54640 1 -- 172.21.15.99:0/616754666 learned_addr learned my addr 172.21.15.99:0/616754666 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483104255 0 0) 0x7f5ea80b40e0 con 0x7f5ea80b0090 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.676+0000 7f5eae353640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5ea80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e94003520 con 0x7f5ea80b0090 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 791139743 0 0) 0x7f5e94003520 con 0x7f5ea80b0090 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 msgr2=0x7f5ea80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5ea80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 >> v1:172.21.15.196:6789/0 conn(0x7f5ea815f1e0 legacy=0x7f5ea80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea80b4380 con 0x7f5ea80b0090 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5ea00019f0 con 0x7f5ea80b0090 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 >> v1:172.21.15.99:6789/0 conn(0x7f5ea80b0090 legacy=0x7f5ea80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940039e0 0x7f5e94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5e94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e94004d40 0x7f5e940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8073c50 con 0x7f5e94004d40 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9400ba70 con 0x7f5e940039e0 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9400bc70 con 0x7f5ea815ee00 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e94004d40 0x7f5e940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eae353640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940039e0 0x7f5e94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eadb52640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5e94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 -- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 msgr2=0x7f5e94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ea815ee00 0x7f5e94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 -- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940039e0 msgr2=0x7f5e94003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e940039e0 0x7f5e94003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.677+0000 7f5eaeb54640 1 -- 172.21.15.99:0/616754666 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94003520 con 0x7f5e94004d40 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.678+0000 7f5eaeb54640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e94004d40 0x7f5e940051c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0002230 tx=0x7f5e94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfd823da4ef2967 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.678+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000a4a0 con 0x7f5e94004d40 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.678+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea002f070 con 0x7f5e94004d40 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.678+0000 7f5ead351640 1 -- 172.21.15.99:0/616754666 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea002bcb0 con 0x7f5e94004d40 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 -- 172.21.15.99:0/616754666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e94004d40 msgr2=0x7f5e940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/616754666 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e94004d40 0x7f5e940051c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0002230 tx=0x7f5e94005830 comp rx=0 tx=0).stop 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eaeb54640 1 -- 172.21.15.99:0/616754666 reap_dead start 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 -- 172.21.15.99:0/616754666 shutdown_connections 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 -- 172.21.15.99:0/616754666 >> 172.21.15.99:0/616754666 conn(0x7f5ea80730c0 msgr2=0x7f5ea80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 -- 172.21.15.99:0/616754666 shutdown_connections 2024-09-10T11:35:00.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.679+0000 7f5eb05b7640 1 -- 172.21.15.99:0/616754666 wait complete. 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.680+0000 7f5eb05b7640 1 Processor -- start 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.680+0000 7f5eb05b7640 1 -- start start 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e94004d40 0x7f5ea814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea809eb30 0x7f5ea81e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea80b0600 con 0x7f5ea81e8190 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea80afd60 con 0x7f5ea809eb30 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea80b0780 con 0x7f5e94004d40 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56836/0 (socket says 172.21.15.99:56836) 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 -- 172.21.15.99:0/2940510077 learned_addr learned my addr 172.21.15.99:0/2940510077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eae353640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e94004d40 0x7f5ea814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eaeb54640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea809eb30 0x7f5ea81e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 -- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e94004d40 msgr2=0x7f5ea814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e94004d40 0x7f5ea814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 -- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea809eb30 msgr2=0x7f5ea81e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea809eb30 0x7f5ea81e97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 -- 172.21.15.99:0/2940510077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea80cd4e0 con 0x7f5ea81e8190 2024-09-10T11:35:00.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eadb52640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800f610 tx=0x7f5e9800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980052e0 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e98018070 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea81e8700 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98003830 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.681+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81e8cc0 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.682+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea80b5f90 con 0x7f5ea81e8190 2024-09-10T11:35:00.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.683+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e980039d0 con 0x7f5ea81e8190 2024-09-10T11:35:00.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.683+0000 7f5e9effd640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 0x7f5e8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:00.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.684+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5227+0+0 (secure 0 0 0) 0x7f5e9808a920 con 0x7f5ea81e8190 2024-09-10T11:35:00.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.684+0000 7f5eae353640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 0x7f5e8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:00.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.684+0000 7f5eae353640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 0x7f5e8c06e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009bc0 tx=0x7f5ea4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:00.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.688+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e980594b0 con 0x7f5ea81e8190 2024-09-10T11:35:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:00.855+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5ea815c890 con 0x7f5ea81e8190 2024-09-10T11:35:01.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.558+0000 7f5e9effd640 1 -- 172.21.15.99:0/2940510077 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) ==== 170+0+0 (secure 0 0 0) 0x7f5e98058db0 con 0x7f5ea81e8190 2024-09-10T11:35:01.560 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 msgr2=0x7f5e8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 0x7f5e8c06e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009bc0 tx=0x7f5ea4009190 comp rx=0 tx=0).stop 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 msgr2=0x7f5ea80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800f610 tx=0x7f5e9800fa10 comp rx=0 tx=0).stop 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 shutdown_connections 2024-09-10T11:35:01.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e8c06be30 0x7f5e8c06e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ea809eb30 0x7f5ea81e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea81e8190 0x7f5ea80ccfa0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 --2- 172.21.15.99:0/2940510077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e94004d40 0x7f5ea814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 >> 172.21.15.99:0/2940510077 conn(0x7f5ea80730c0 msgr2=0x7f5ea80aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 shutdown_connections 2024-09-10T11:35:01.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.561+0000 7f5eb05b7640 1 -- 172.21.15.99:0/2940510077 wait complete. 2024-09-10T11:35:01.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:35:01.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.683+0000 7fc0406a4640 1 Processor -- start 2024-09-10T11:35:01.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.684+0000 7fc0406a4640 1 -- start start 2024-09-10T11:35:01.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc0406a4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 0x7fc038073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc0406a4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc03806e5d0 0x7fc03806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc0406a4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc038152e60 con 0x7fc03815c060 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc0406a4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc03806ef80 con 0x7fc03806e5d0 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc0406a4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc0380d15e0 con 0x7fc0380d1200 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc03806e5d0 0x7fc03806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc03806e5d0 0x7fc03806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56840/0 (socket says 172.21.15.99:56840) 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 -- 172.21.15.99:0/1677360055 learned_addr learned my addr 172.21.15.99:0/1677360055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03e440640 1 --2- 172.21.15.99:0/1677360055 >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 0x7fc038073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 -- 172.21.15.99:0/1677360055 >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 msgr2=0x7fc038073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 --2- 172.21.15.99:0/1677360055 >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 0x7fc038073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 -- 172.21.15.99:0/1677360055 >> v1:172.21.15.196:6789/0 conn(0x7fc03815c060 legacy=0x7fc03815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03dc3f640 1 -- 172.21.15.99:0/1677360055 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc038072680 con 0x7fc03806e5d0 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.685+0000 7fc03e440640 1 --2- 172.21.15.99:0/1677360055 >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 0x7fc038073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:01.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc03dc3f640 1 --2- 172.21.15.99:0/1677360055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03806e5d0 0x7fc03806ea40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc0380d1850 tx=0x7fc02802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19edbcd3a6596d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:01.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc03d43e640 1 -- 172.21.15.99:0/1677360055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028032710 con 0x7fc03806e5d0 2024-09-10T11:35:01.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc03d43e640 1 -- 172.21.15.99:0/1677360055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0280328b0 con 0x7fc03806e5d0 2024-09-10T11:35:01.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc03d43e640 1 -- 172.21.15.99:0/1677360055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028032ca0 con 0x7fc03806e5d0 2024-09-10T11:35:01.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 -- 172.21.15.99:0/1677360055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03806e5d0 msgr2=0x7fc03806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:01.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 --2- 172.21.15.99:0/1677360055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03806e5d0 0x7fc03806ea40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc0380d1850 tx=0x7fc02802c850 comp rx=0 tx=0).stop 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 -- 172.21.15.99:0/1677360055 shutdown_connections 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 --2- 172.21.15.99:0/1677360055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc03806e5d0 0x7fc03806ea40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 --2- 172.21.15.99:0/1677360055 >> v2:172.21.15.196:3300/0 conn(0x7fc0380d1200 0x7fc038073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 -- 172.21.15.99:0/1677360055 >> 172.21.15.99:0/1677360055 conn(0x7fc03806bc10 msgr2=0x7fc03806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 -- 172.21.15.99:0/1677360055 shutdown_connections 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.686+0000 7fc0406a4640 1 -- 172.21.15.99:0/1677360055 wait complete. 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 Processor -- start 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 -- start start 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc03806e5d0 0x7fc03814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc03815c060 0x7fc038156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc038072f20 con 0x7fc0380d1200 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc038072c40 con 0x7fc03806e5d0 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc0406a4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc03815cd90 con 0x7fc03815c060 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc03dc3f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc03dc3f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56842/0 (socket says 172.21.15.99:56842) 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.688+0000 7fc03dc3f640 1 -- 172.21.15.99:0/562043123 learned_addr learned my addr 172.21.15.99:0/562043123 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03e440640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc03806e5d0 0x7fc03814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03ec41640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc03815c060 0x7fc038156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 -- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc03815c060 msgr2=0x7fc038156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:01.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc03815c060 0x7fc038156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 -- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc03806e5d0 msgr2=0x7fc03814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc03806e5d0 0x7fc03814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 -- 172.21.15.99:0/562043123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc038154960 con 0x7fc0380d1200 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc03dc3f640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c890 tx=0x7fc0280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028042070 con 0x7fc0380d1200 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc028003850 con 0x7fc0380d1200 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0381523f0 con 0x7fc0380d1200 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028003c40 con 0x7fc0380d1200 2024-09-10T11:35:01.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.689+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc038153660 con 0x7fc0380d1200 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.690+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03806ea40 con 0x7fc0380d1200 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.691+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc028045070 con 0x7fc0380d1200 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.691+0000 7fc02effd640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.691+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5609+0+0 (secure 0 0 0) 0x7fc0280adc40 con 0x7fc0380d1200 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.691+0000 7fc03e440640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:01.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.691+0000 7fc03e440640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc03400a950 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:01.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.696+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc02807c650 con 0x7fc0380d1200 2024-09-10T11:35:01.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:01.897+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc03815d950 con 0x7fc0380d1200 2024-09-10T11:35:02.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.560+0000 7fc02effd640 1 -- 172.21.15.99:0/562043123 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v46) ==== 174+0+0 (secure 0 0 0) 0x7fc02807bf50 con 0x7fc0380d1200 2024-09-10T11:35:02.562 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 14 and data pool 15 2024-09-10T11:35:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.565+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 msgr2=0x7fc01c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.565+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 0x7fc01c06e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc03400a950 tx=0x7fc034008040 comp rx=0 tx=0).stop 2024-09-10T11:35:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.565+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 msgr2=0x7fc038154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.565+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c890 tx=0x7fc0280046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 shutdown_connections 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc01c06be30 0x7fc01c06e2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc03815c060 0x7fc038156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0380d1200 0x7fc038154420 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 --2- 172.21.15.99:0/562043123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc03806e5d0 0x7fc03814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 >> 172.21.15.99:0/562043123 conn(0x7fc03806bc10 msgr2=0x7fc03806d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 shutdown_connections 2024-09-10T11:35:02.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.566+0000 7fc0406a4640 1 -- 172.21.15.99:0/562043123 wait complete. 2024-09-10T11:35:02.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:02.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.691+0000 7f4122b86640 1 Processor -- start 2024-09-10T11:35:02.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.691+0000 7f4122b86640 1 -- start start 2024-09-10T11:35:02.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f4122b86640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f4122b86640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f411c15cc30 0x7f411c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f4122b86640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f411c0cea00 con 0x7f411c0cf7f0 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f4122b86640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f411c0d1540 con 0x7f411c15cc30 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f4122b86640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0d1770 con 0x7f411c0cf410 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f411bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f411c15cc30 0x7f411c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f411bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f411c15cc30 0x7f411c0d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44340/0 (socket says 172.21.15.99:44340) 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.692+0000 7f411bfff640 1 -- 172.21.15.99:0/1707931553 learned_addr learned my addr 172.21.15.99:0/1707931553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411bfff640 1 -- 172.21.15.99:0/1707931553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 msgr2=0x7f411c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411bfff640 1 --2- 172.21.15.99:0/1707931553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411bfff640 1 -- 172.21.15.99:0/1707931553 >> v1:172.21.15.196:6789/0 conn(0x7f411c0cf7f0 legacy=0x7f411c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:02.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411bfff640 1 -- 172.21.15.99:0/1707931553 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c0cd810 con 0x7f411c15cc30 2024-09-10T11:35:02.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411bfff640 1 --2- 172.21.15.99:0/1707931553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c0d1000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943d091e15516045 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411b7fe640 1 -- 172.21.15.99:0/1707931553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110037070 con 0x7f411c15cc30 2024-09-10T11:35:02.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411b7fe640 1 -- 172.21.15.99:0/1707931553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411002ae10 con 0x7f411c15cc30 2024-09-10T11:35:02.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.693+0000 7f411b7fe640 1 -- 172.21.15.99:0/1707931553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110032660 con 0x7f411c15cc30 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 -- 172.21.15.99:0/1707931553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 msgr2=0x7f411c0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 --2- 172.21.15.99:0/1707931553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c0d1000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4110009870 tx=0x7f411002a100 comp rx=0 tx=0).stop 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 -- 172.21.15.99:0/1707931553 shutdown_connections 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 --2- 172.21.15.99:0/1707931553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c0d1000 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 --2- 172.21.15.99:0/1707931553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 -- 172.21.15.99:0/1707931553 >> 172.21.15.99:0/1707931553 conn(0x7f411c0b4570 msgr2=0x7f411c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 -- 172.21.15.99:0/1707931553 shutdown_connections 2024-09-10T11:35:02.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.694+0000 7f4122b86640 1 -- 172.21.15.99:0/1707931553 wait complete. 2024-09-10T11:35:02.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.695+0000 7f4122b86640 1 Processor -- start 2024-09-10T11:35:02.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.695+0000 7f4122b86640 1 -- start start 2024-09-10T11:35:02.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f411c0cf7f0 0x7f411c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0cfc20 con 0x7f411c15cc30 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f411c0cd4c0 con 0x7f411c0cf7f0 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4122b86640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f411c0cfda0 con 0x7f411c0cf410 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44352/0 (socket says 172.21.15.99:44352) 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 -- 172.21.15.99:0/1842533543 learned_addr learned my addr 172.21.15.99:0/1842533543 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f411bfff640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f411c0cf7f0 0x7f411c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4120922640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 -- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 msgr2=0x7f411c1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c1f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 -- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f411c0cf7f0 msgr2=0x7f411c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f411c0cf7f0 0x7f411c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4121123640 1 -- 172.21.15.99:0/1842533543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c159df0 con 0x7f411c15cc30 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.696+0000 7f4120922640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f4121123640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f411400b970 tx=0x7f411400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114019070 con 0x7f411c15cc30 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411400cb40 con 0x7f411c15cc30 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c0c6fd0 con 0x7f411c15cc30 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4114014440 con 0x7f411c15cc30 2024-09-10T11:35:02.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.697+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c0a0320 con 0x7f411c15cc30 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.698+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c0d1000 con 0x7f411c15cc30 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.698+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f411401c070 con 0x7f411c15cc30 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.699+0000 7f41197fa640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 0x7f410806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.699+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f411408a940 con 0x7f411c15cc30 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.699+0000 7f4120922640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 0x7f410806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.699+0000 7f4120922640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 0x7f410806e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f410c009890 tx=0x7f410c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.703+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4114058d30 con 0x7f411c15cc30 2024-09-10T11:35:02.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.869+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f411c0b89a0 con 0x7f411c15cc30 2024-09-10T11:35:02.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.870+0000 7f41197fa640 1 -- 172.21.15.99:0/1842533543 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16693 (secure 0 0 0) 0x7f411405d030 con 0x7f411c15cc30 2024-09-10T11:35:02.872 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:02.873 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":122,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:01.899709+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 msgr2=0x7f410806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 0x7f410806e2f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f410c009890 tx=0x7f410c008040 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 msgr2=0x7f411c1e65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f411400b970 tx=0x7f411400be30 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 shutdown_connections 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f410806be30 0x7f410806e2f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f411c15cc30 0x7f411c1e65d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f411c0cf7f0 0x7f411c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 --2- 172.21.15.99:0/1842533543 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f411c0cf410 0x7f411c1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 >> 172.21.15.99:0/1842533543 conn(0x7f411c0b4570 msgr2=0x7f411c0b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:02.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 shutdown_connections 2024-09-10T11:35:02.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.873+0000 7f4122b86640 1 -- 172.21.15.99:0/1842533543 wait complete. 2024-09-10T11:35:02.886 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:02.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.984+0000 7f8042dd8640 1 Processor -- start 2024-09-10T11:35:02.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.984+0000 7f8042dd8640 1 -- start start 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8042dd8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f803c163140 0x7f803c163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8042dd8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c0b5d30 con 0x7f803c0d0740 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8042dd8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f803c071980 con 0x7f803c0714a0 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8042dd8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f803c071c60 con 0x7f803c163140 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8040b74640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f803c163140 0x7f803c163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8041375640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f803c0d0740 0x7f803c072a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45200/0 (socket says 172.21.15.99:45200) 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8040b74640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f803c163140 0x7f803c163520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44366/0 (socket says 172.21.15.99:44366) 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8041375640 1 -- 172.21.15.99:0/2230127649 learned_addr learned my addr 172.21.15.99:0/2230127649 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8040b74640 1 -- 172.21.15.99:0/2230127649 >> v1:172.21.15.196:6789/0 conn(0x7f803c0714a0 legacy=0x7f803c0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8040b74640 1 -- 172.21.15.99:0/2230127649 >> v1:172.21.15.99:6789/0 conn(0x7f803c0d0740 legacy=0x7f803c072a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f8040b74640 1 -- 172.21.15.99:0/2230127649 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c071f40 con 0x7f803c163140 2024-09-10T11:35:02.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.985+0000 7f80337fe640 1 -- 172.21.15.99:0/2230127649 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1739893863 0 0) 0x7f803c0b5d30 con 0x7f803c0d0740 2024-09-10T11:35:02.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f8040b74640 1 --2- 172.21.15.99:0/2230127649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c163520 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f80380097a0 tx=0x7f803802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3acab1cabb2e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f80337fe640 1 -- 172.21.15.99:0/2230127649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038037070 con 0x7f803c163140 2024-09-10T11:35:02.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f80337fe640 1 -- 172.21.15.99:0/2230127649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038032440 con 0x7f803c163140 2024-09-10T11:35:02.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f80337fe640 1 -- 172.21.15.99:0/2230127649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038032810 con 0x7f803c163140 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f8042dd8640 1 -- 172.21.15.99:0/2230127649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 msgr2=0x7f803c163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.986+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2230127649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c163520 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f80380097a0 tx=0x7f803802a230 comp rx=0 tx=0).stop 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.987+0000 7f8042dd8640 1 -- 172.21.15.99:0/2230127649 shutdown_connections 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.987+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2230127649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c163520 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.987+0000 7f8042dd8640 1 -- 172.21.15.99:0/2230127649 >> 172.21.15.99:0/2230127649 conn(0x7f803c06bc10 msgr2=0x7f803c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.987+0000 7f8042dd8640 1 -- 172.21.15.99:0/2230127649 shutdown_connections 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.987+0000 7f8042dd8640 1 -- 172.21.15.99:0/2230127649 wait complete. 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.988+0000 7f8042dd8640 1 Processor -- start 2024-09-10T11:35:02.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.988+0000 7f8042dd8640 1 -- start start 2024-09-10T11:35:02.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f803c0714a0 0x7f803c0ad180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 0x7f803c0af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f803c0d0b90 con 0x7f803c163140 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f803c072500 con 0x7f803c0d0740 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f803c0d0d10 con 0x7f803c0714a0 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44368/0 (socket says 172.21.15.99:44368) 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 -- 172.21.15.99:0/2582657798 learned_addr learned my addr 172.21.15.99:0/2582657798 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8040b74640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f803c0714a0 0x7f803c0ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8033fff640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 0x7f803c0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 -- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f803c0714a0 msgr2=0x7f803c0ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f803c0714a0 0x7f803c0ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 -- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 msgr2=0x7f803c0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 0x7f803c0af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 -- 172.21.15.99:0/2582657798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803c1f9fe0 con 0x7f803c163140 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8033fff640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 0x7f803c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:02.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8041375640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f803400c7a0 tx=0x7f803400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8034018070 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803400ce50 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803c1fa2c0 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803400a820 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.989+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803c1fa880 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.990+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f803c0d1010 con 0x7f803c163140 2024-09-10T11:35:02.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.991+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f803401b070 con 0x7f803c163140 2024-09-10T11:35:02.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.991+0000 7f80317fa640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 0x7f802806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:02.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.991+0000 7f8040b74640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 0x7f802806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:02.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.991+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f803408a380 con 0x7f803c163140 2024-09-10T11:35:02.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.992+0000 7f8040b74640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 0x7f802806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f803802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:02.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:02.996+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8034058dc0 con 0x7f803c163140 2024-09-10T11:35:03.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.160+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f803c0b13c0 con 0x7f803c163140 2024-09-10T11:35:03.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.161+0000 7f80317fa640 1 -- 172.21.15.99:0/2582657798 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16693 (secure 0 0 0) 0x7f80340586c0 con 0x7f803c163140 2024-09-10T11:35:03.163 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:03.163 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":122,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:01.899709+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 msgr2=0x7f802806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 0x7f802806e2f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f803802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 msgr2=0x7f803c0abef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f803400c7a0 tx=0x7f803400cc60 comp rx=0 tx=0).stop 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 shutdown_connections 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f802806be30 0x7f802806e2f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f803c163140 0x7f803c0abef0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f803c0d0740 0x7f803c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 --2- 172.21.15.99:0/2582657798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f803c0714a0 0x7f803c0ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 >> 172.21.15.99:0/2582657798 conn(0x7f803c06bc10 msgr2=0x7f803c165530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 shutdown_connections 2024-09-10T11:35:03.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.163+0000 7f8042dd8640 1 -- 172.21.15.99:0/2582657798 wait complete. 2024-09-10T11:35:03.176 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:35:03.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.276+0000 7ff42ece4640 1 Processor -- start 2024-09-10T11:35:03.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.276+0000 7ff42ece4640 1 -- start start 2024-09-10T11:35:03.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.276+0000 7ff42ece4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 0x7ff4280b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ece4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 0x7ff4280b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ece4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4280b0090 0x7ff4280b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ece4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280b40e0 con 0x7ff4280b0090 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ece4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff4280b5ab0 con 0x7ff42815ee00 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ece4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff428073c50 con 0x7ff42815f1e0 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4280b0090 0x7ff4280b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4280b0090 0x7ff4280b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44370/0 (socket says 172.21.15.99:44370) 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 -- 172.21.15.99:0/1240761463 learned_addr learned my addr 172.21.15.99:0/1240761463 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ca80640 1 --2- 172.21.15.99:0/1240761463 >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 0x7ff4280b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff427fff640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 0x7ff4280b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 -- 172.21.15.99:0/1240761463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 msgr2=0x7ff4280b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 0x7ff4280b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 -- 172.21.15.99:0/1240761463 >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 msgr2=0x7ff4280b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 --2- 172.21.15.99:0/1240761463 >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 0x7ff4280b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42d281640 1 -- 172.21.15.99:0/1240761463 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280b4380 con 0x7ff4280b0090 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff42ca80640 1 --2- 172.21.15.99:0/1240761463 >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 0x7ff4280b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.277+0000 7ff427fff640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 0x7ff4280b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42d281640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4280b0090 0x7ff4280b1aa0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff4280709a0 tx=0x7ff42002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5586c190ae3c79d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff4277fe640 1 -- 172.21.15.99:0/1240761463 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420036070 con 0x7ff4280b0090 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff4277fe640 1 -- 172.21.15.99:0/1240761463 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff420031600 con 0x7ff4280b0090 2024-09-10T11:35:03.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff4277fe640 1 -- 172.21.15.99:0/1240761463 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200319d0 con 0x7ff4280b0090 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 -- 172.21.15.99:0/1240761463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4280b0090 msgr2=0x7ff4280b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4280b0090 0x7ff4280b1aa0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff4280709a0 tx=0x7ff42002a6a0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 -- 172.21.15.99:0/1240761463 shutdown_connections 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4280b0090 0x7ff4280b1aa0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 --2- 172.21.15.99:0/1240761463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff42815f1e0 0x7ff4280b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 --2- 172.21.15.99:0/1240761463 >> v2:172.21.15.196:3300/0 conn(0x7ff42815ee00 0x7ff4280b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 -- 172.21.15.99:0/1240761463 >> 172.21.15.99:0/1240761463 conn(0x7ff4280730c0 msgr2=0x7ff4280734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.278+0000 7ff42ece4640 1 -- 172.21.15.99:0/1240761463 shutdown_connections 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.279+0000 7ff42ece4640 1 -- 172.21.15.99:0/1240761463 wait complete. 2024-09-10T11:35:03.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.280+0000 7ff42ece4640 1 Processor -- start 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.280+0000 7ff42ece4640 1 -- start start 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4280b0090 0x7ff4280caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42815f1e0 0x7ff4280ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280b1fe0 con 0x7ff42815ee00 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280afd60 con 0x7ff42815f1e0 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280b2160 con 0x7ff4280b0090 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44372/0 (socket says 172.21.15.99:44372) 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 -- 172.21.15.99:0/3186143902 learned_addr learned my addr 172.21.15.99:0/3186143902 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 -- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4280b0090 msgr2=0x7ff4280caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4280b0090 0x7ff4280caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42d281640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42815f1e0 0x7ff4280ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 -- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42815f1e0 msgr2=0x7ff4280ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42815f1e0 0x7ff4280ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 -- 172.21.15.99:0/3186143902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4280cc8b0 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff427fff640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c7f0 tx=0x7ff41400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414015070 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff414007820 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4280cc5d0 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414007bf0 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.281+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4280ca740 con 0x7ff42815ee00 2024-09-10T11:35:03.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.282+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4280d1ff0 con 0x7ff42815ee00 2024-09-10T11:35:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.283+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff414018070 con 0x7ff42815ee00 2024-09-10T11:35:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.283+0000 7ff4257fa640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.283+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7ff41408ae00 con 0x7ff42815ee00 2024-09-10T11:35:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.283+0000 7ff42ca80640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.284+0000 7ff42ca80640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 0x7ff40c06e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006fd0 tx=0x7ff41c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.287+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff414059840 con 0x7ff42815ee00 2024-09-10T11:35:03.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.486+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff42815c040 con 0x7ff42815ee00 2024-09-10T11:35:03.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.578+0000 7ff4257fa640 1 -- 172.21.15.99:0/3186143902 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) ==== 118+0+0 (secure 0 0 0) 0x7ff414059140 con 0x7ff42815ee00 2024-09-10T11:35:03.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 msgr2=0x7ff40c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 0x7ff40c06e2f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006fd0 tx=0x7ff41c008040 comp rx=0 tx=0).stop 2024-09-10T11:35:03.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 msgr2=0x7ff4280cbba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff41400c7f0 tx=0x7ff41400ccb0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 shutdown_connections 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff40c06be30 0x7ff40c06e2f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff42815f1e0 0x7ff4280ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff42815ee00 0x7ff4280cbba0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 --2- 172.21.15.99:0/3186143902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4280b0090 0x7ff4280caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 >> 172.21.15.99:0/3186143902 conn(0x7ff4280730c0 msgr2=0x7ff4280ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 shutdown_connections 2024-09-10T11:35:03.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.580+0000 7ff42ece4640 1 -- 172.21.15.99:0/3186143902 wait complete. 2024-09-10T11:35:03.593 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:35:03.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:35:03.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.693+0000 7fe76036d640 1 Processor -- start 2024-09-10T11:35:03.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.693+0000 7fe76036d640 1 -- start start 2024-09-10T11:35:03.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe76036d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe7580d1200 0x7fe758073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe76036d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe758152e60 con 0x7fe75815c770 2024-09-10T11:35:03.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe76036d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7580d15e0 con 0x7fe75806e5d0 2024-09-10T11:35:03.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe76036d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe758072680 con 0x7fe7580d1200 2024-09-10T11:35:03.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75e90a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe75815c770 0x7fe758159b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45204/0 (socket says 172.21.15.99:45204) 2024-09-10T11:35:03.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75e90a640 1 -- 172.21.15.99:0/3616957149 learned_addr learned my addr 172.21.15.99:0/3616957149 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:03.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75e109640 1 --2- 172.21.15.99:0/3616957149 >> v2:172.21.15.196:3300/0 conn(0x7fe7580d1200 0x7fe758073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279861191 0 0) 0x7fe758152e60 con 0x7fe75815c770 2024-09-10T11:35:03.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe744003520 con 0x7fe75815c770 2024-09-10T11:35:03.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3593704511 0 0) 0x7fe744003520 con 0x7fe75815c770 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 >> v2:172.21.15.196:3300/0 conn(0x7fe7580d1200 msgr2=0x7fe758073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 --2- 172.21.15.99:0/3616957149 >> v2:172.21.15.196:3300/0 conn(0x7fe7580d1200 0x7fe758073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 >> v1:172.21.15.196:6789/0 conn(0x7fe75806e5d0 legacy=0x7fe75815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe758072960 con 0x7fe75815c770 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fe7500019f0 con 0x7fe75815c770 2024-09-10T11:35:03.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 >> v1:172.21.15.99:6789/0 conn(0x7fe75815c770 legacy=0x7fe758159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.694+0000 7fe75d107640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440039b0 0x7fe744003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d107640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe744004800 0x7fe744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d107640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 0x7fe74400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe758072680 con 0x7fe7440051c0 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe74400c160 con 0x7fe7440039b0 2024-09-10T11:35:03.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe74400c360 con 0x7fe744004800 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 0x7fe74400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e109640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440039b0 0x7fe744003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75d908640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe744004800 0x7fe744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 -- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe744004800 msgr2=0x7fe744004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe744004800 0x7fe744004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 -- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440039b0 msgr2=0x7fe744003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:03.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7440039b0 0x7fe744003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.695+0000 7fe75e90a640 1 -- 172.21.15.99:0/3616957149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe744003520 con 0x7fe7440051c0 2024-09-10T11:35:03.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe75e90a640 1 -- 172.21.15.99:0/3616957149 reap_dead start 2024-09-10T11:35:03.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe75e90a640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 0x7fe74400b9c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a850 tx=0x7fe744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e920efb7be33f2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000a4a0 con 0x7fe7440051c0 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75002f070 con 0x7fe7440051c0 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe75d107640 1 -- 172.21.15.99:0/3616957149 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75002bce0 con 0x7fe7440051c0 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 -- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 msgr2=0x7fe74400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 0x7fe74400b9c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a850 tx=0x7fe744005830 comp rx=0 tx=0).stop 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 -- 172.21.15.99:0/3616957149 shutdown_connections 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 --2- 172.21.15.99:0/3616957149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7440051c0 0x7fe74400b9c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 -- 172.21.15.99:0/3616957149 >> 172.21.15.99:0/3616957149 conn(0x7fe75806bc10 msgr2=0x7fe75806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:03.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 -- 172.21.15.99:0/3616957149 shutdown_connections 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.696+0000 7fe76036d640 1 -- 172.21.15.99:0/3616957149 wait complete. 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 Processor -- start 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 -- start start 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7440051c0 0x7fe758152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 0x7fe7581f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe75815cd90 con 0x7fe7580aa490 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe758072f20 con 0x7fe7580cc370 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe76036d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe758073440 con 0x7fe7440051c0 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe75e90a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe75e90a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44382/0 (socket says 172.21.15.99:44382) 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.698+0000 7fe75e90a640 1 -- 172.21.15.99:0/1292127976 learned_addr learned my addr 172.21.15.99:0/1292127976 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:03.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e109640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7440051c0 0x7fe758152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75d908640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 0x7fe7581f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 -- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7440051c0 msgr2=0x7fe758152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7440051c0 0x7fe758152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 -- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 msgr2=0x7fe7581f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 0x7fe7581f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 -- 172.21.15.99:0/1292127976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7580b1680 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75d908640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 0x7fe7581f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe75e90a640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a970 tx=0x7fe75002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750045070 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7500406a0 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7580b13a0 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750040a90 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.699+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7580c9370 con 0x7fe7580aa490 2024-09-10T11:35:03.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.700+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75806ea40 con 0x7fe7580aa490 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.701+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe750048070 con 0x7fe7580aa490 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.701+0000 7fe74effd640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 0x7fe73c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.701+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7fe7500ab5a0 con 0x7fe7580aa490 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.701+0000 7fe75e109640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 0x7fe73c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.702+0000 7fe75e109640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 0x7fe73c06e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe7580cae20 tx=0x7fe754008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:03.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.705+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe75007a070 con 0x7fe7580aa490 2024-09-10T11:35:03.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:03.904+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe7580b43d0 con 0x7fe7580aa490 2024-09-10T11:35:04.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.580+0000 7fe74effd640 1 -- 172.21.15.99:0/1292127976 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fe750079970 con 0x7fe7580aa490 2024-09-10T11:35:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 msgr2=0x7fe73c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 0x7fe73c06e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe7580cae20 tx=0x7fe754008040 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 msgr2=0x7fe7580ac300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a970 tx=0x7fe75002d070 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 shutdown_connections 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe73c06be30 0x7fe73c06e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7580aa490 0x7fe7580ac300 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7580cc370 0x7fe7581f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 --2- 172.21.15.99:0/1292127976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7440051c0 0x7fe758152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 >> 172.21.15.99:0/1292127976 conn(0x7fe75806bc10 msgr2=0x7fe7580ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 shutdown_connections 2024-09-10T11:35:04.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.582+0000 7fe76036d640 1 -- 172.21.15.99:0/1292127976 wait complete. 2024-09-10T11:35:04.597 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:35:04.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:04.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.698+0000 7f5d6fe8d640 1 Processor -- start 2024-09-10T11:35:04.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.699+0000 7f5d6fe8d640 1 -- start start 2024-09-10T11:35:04.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.699+0000 7f5d6fe8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d680d0130 0x7f5d680cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.699+0000 7f5d6fe8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d680ce430 con 0x7f5d68160030 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.699+0000 7f5d6fe8d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d680ce710 con 0x7f5d680d1760 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.699+0000 7f5d6fe8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5d680ce200 con 0x7f5d680d0130 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6e42a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5d680d0130 0x7f5d680cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6dc29640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5d68160030 0x7f5d680b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45218/0 (socket says 172.21.15.99:45218) 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6dc29640 1 -- 172.21.15.99:0/1139501618 learned_addr learned my addr 172.21.15.99:0/1139501618 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1139501618 >> v1:172.21.15.196:6789/0 conn(0x7f5d680d1760 legacy=0x7f5d680cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1139501618 >> v1:172.21.15.99:6789/0 conn(0x7f5d68160030 legacy=0x7f5d680b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1139501618 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d680d1240 con 0x7f5d680d0130 2024-09-10T11:35:04.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.700+0000 7f5d6cc27640 1 -- 172.21.15.99:0/1139501618 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899098209 0 0) 0x7f5d680ce430 con 0x7f5d68160030 2024-09-10T11:35:04.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6e42a640 1 --2- 172.21.15.99:0/1139501618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d680d0130 0x7f5d680cdcc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009d90 tx=0x7f5d6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2a2ac30c163ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6cc27640 1 -- 172.21.15.99:0/1139501618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60007740 con 0x7f5d680d0130 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6cc27640 1 -- 172.21.15.99:0/1139501618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d600078e0 con 0x7f5d680d0130 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6cc27640 1 -- 172.21.15.99:0/1139501618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60007cb0 con 0x7f5d680d0130 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1139501618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d680d0130 msgr2=0x7f5d680cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1139501618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d680d0130 0x7f5d680cdcc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009d90 tx=0x7f5d6002a6a0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1139501618 shutdown_connections 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1139501618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d680d0130 0x7f5d680cdcc0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1139501618 >> 172.21.15.99:0/1139501618 conn(0x7f5d680b4650 msgr2=0x7f5d680b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1139501618 shutdown_connections 2024-09-10T11:35:04.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.701+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1139501618 wait complete. 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.702+0000 7f5d6fe8d640 1 Processor -- start 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 -- start start 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 0x7f5d68154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d680d1760 0x7f5d68166610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68167970 con 0x7f5d68160030 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d681606a0 con 0x7f5d680d1760 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6fe8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5d68167af0 con 0x7f5d680d0130 2024-09-10T11:35:04.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6e42a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6e42a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44410/0 (socket says 172.21.15.99:44410) 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1548087870 learned_addr learned my addr 172.21.15.99:0/1548087870 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6dc29640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 0x7f5d68154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.703+0000 7f5d6d428640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d680d1760 0x7f5d68166610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 msgr2=0x7f5d68154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 0x7f5d68154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d680d1760 msgr2=0x7f5d68166610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d680d1760 0x7f5d68166610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 -- 172.21.15.99:0/1548087870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d680b5220 con 0x7f5d68160030 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6dc29640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 0x7f5d68154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6e42a640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5d6002ac90 tx=0x7f5d60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60040070 con 0x7f5d68160030 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d60031c90 con 0x7f5d68160030 2024-09-10T11:35:04.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d68164cd0 con 0x7f5d68160030 2024-09-10T11:35:04.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60045680 con 0x7f5d68160030 2024-09-10T11:35:04.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.704+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6815a0c0 con 0x7f5d68160030 2024-09-10T11:35:04.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.705+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6815a2c0 con 0x7f5d68160030 2024-09-10T11:35:04.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.705+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d60039040 con 0x7f5d68160030 2024-09-10T11:35:04.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.706+0000 7f5d667fc640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:04.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.706+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f5d600ac860 con 0x7f5d68160030 2024-09-10T11:35:04.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.706+0000 7f5d6dc29640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:04.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.706+0000 7f5d6dc29640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000f60 tx=0x7f5d58007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:04.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.710+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d6815a2c0 con 0x7f5d68160030 2024-09-10T11:35:04.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.906+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d680b38f0 con 0x7f5d68160030 2024-09-10T11:35:04.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.907+0000 7f5d667fc640 1 -- 172.21.15.99:0/1548087870 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f5d600ac020 con 0x7f5d68160030 2024-09-10T11:35:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:04.909 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:04.909 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 msgr2=0x7f5d5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 0x7f5d5406e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000f60 tx=0x7f5d58007c10 comp rx=0 tx=0).stop 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 msgr2=0x7f5d68157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5d6002ac90 tx=0x7f5d60004a40 comp rx=0 tx=0).stop 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 shutdown_connections 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5d5406be30 0x7f5d5406e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d68160030 0x7f5d68157f40 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d680d1760 0x7f5d68166610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 --2- 172.21.15.99:0/1548087870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5d680d0130 0x7f5d68154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:04.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 >> 172.21.15.99:0/1548087870 conn(0x7f5d680b4650 msgr2=0x7f5d6806ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:04.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 shutdown_connections 2024-09-10T11:35:04.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:04.909+0000 7f5d6fe8d640 1 -- 172.21.15.99:0/1548087870 wait complete. 2024-09-10T11:35:04.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:05.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.019+0000 7f403b728640 1 Processor -- start 2024-09-10T11:35:05.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.019+0000 7f403b728640 1 -- start start 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f403b728640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40340d1200 0x7f4034073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f403b728640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f403415c770 0x7f403406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f403b728640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4034152e60 con 0x7f403406e5d0 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f403b728640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f403406ef80 con 0x7f40340d1200 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f403b728640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f40340d15e0 con 0x7f403415c770 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40340d1200 0x7f4034073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f40340d1200 0x7f4034073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44422/0 (socket says 172.21.15.99:44422) 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 -- 172.21.15.99:0/3441123397 learned_addr learned my addr 172.21.15.99:0/3441123397 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f4039cc5640 1 --2- 172.21.15.99:0/3441123397 >> v2:172.21.15.196:3300/0 conn(0x7f403415c770 0x7f403406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 -- 172.21.15.99:0/3441123397 >> v2:172.21.15.196:3300/0 conn(0x7f403415c770 msgr2=0x7f403406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 --2- 172.21.15.99:0/3441123397 >> v2:172.21.15.196:3300/0 conn(0x7f403415c770 0x7f403406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 -- 172.21.15.99:0/3441123397 >> v1:172.21.15.196:6789/0 conn(0x7f403406e5d0 legacy=0x7f403415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.020+0000 7f40394c4640 1 -- 172.21.15.99:0/3441123397 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4034072680 con 0x7f40340d1200 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f40394c4640 1 --2- 172.21.15.99:0/3441123397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40340d1200 0x7f4034073440 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40340720c0 tx=0x7f403002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eec12885e02ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f402bfff640 1 -- 172.21.15.99:0/3441123397 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030037070 con 0x7f40340d1200 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f402bfff640 1 -- 172.21.15.99:0/3441123397 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4030032440 con 0x7f40340d1200 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f402bfff640 1 -- 172.21.15.99:0/3441123397 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030032810 con 0x7f40340d1200 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f403b728640 1 -- 172.21.15.99:0/3441123397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40340d1200 msgr2=0x7f4034073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.021+0000 7f403b728640 1 --2- 172.21.15.99:0/3441123397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40340d1200 0x7f4034073440 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40340720c0 tx=0x7f403002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 -- 172.21.15.99:0/3441123397 shutdown_connections 2024-09-10T11:35:05.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 --2- 172.21.15.99:0/3441123397 >> v2:172.21.15.196:3300/0 conn(0x7f403415c770 0x7f403406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 --2- 172.21.15.99:0/3441123397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f40340d1200 0x7f4034073440 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 -- 172.21.15.99:0/3441123397 >> 172.21.15.99:0/3441123397 conn(0x7f403406bc10 msgr2=0x7f403406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 -- 172.21.15.99:0/3441123397 shutdown_connections 2024-09-10T11:35:05.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.022+0000 7f403b728640 1 -- 172.21.15.99:0/3441123397 wait complete. 2024-09-10T11:35:05.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-10T11:35:05.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-10T11:35:05.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T11:35:05.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.023+0000 7f403b728640 1 Processor -- start 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.023+0000 7f403b728640 1 -- start start 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.023+0000 7f403b728640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 0x7f4034155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 0x7f4034157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4034072f20 con 0x7f403406e5d0 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4034072c40 con 0x7f40340d1200 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f403415cd90 con 0x7f403415c770 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44434/0 (socket says 172.21.15.99:44434) 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 -- 172.21.15.99:0/3895138024 learned_addr learned my addr 172.21.15.99:0/3895138024 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4038cc3640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 0x7f4034155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4039cc5640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 0x7f4034157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 -- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 msgr2=0x7f4034157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 0x7f4034157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 -- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 msgr2=0x7f4034155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 0x7f4034155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 -- 172.21.15.99:0/3895138024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4034157ef0 con 0x7f403406e5d0 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4038cc3640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 0x7f4034155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4039cc5640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 0x7f4034157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f40394c4640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f403002a220 tx=0x7f40300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030041020 con 0x7f403406e5d0 2024-09-10T11:35:05.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40300306c0 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030037040 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40341581d0 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.024+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4034158790 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.025+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4034069640 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.026+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4030044020 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.026+0000 7f4029ffb640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 0x7f401806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.027+0000 7f4038cc3640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 0x7f401806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.027+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f40300ab5d0 con 0x7f403406e5d0 2024-09-10T11:35:05.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.027+0000 7f4038cc3640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 0x7f401806e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4034155de0 tx=0x7f4024009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.032+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4034069640 con 0x7f403406e5d0 2024-09-10T11:35:05.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.197+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4034073440 con 0x7f403406e5d0 2024-09-10T11:35:05.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.198+0000 7f4029ffb640 1 -- 172.21.15.99:0/3895138024 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+16680 (secure 0 0 0) 0x7f403004b030 con 0x7f403406e5d0 2024-09-10T11:35:05.200 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:05.201 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":123,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:03.561892+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 msgr2=0x7f401806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 0x7f401806e320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4034155de0 tx=0x7f4024009040 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 msgr2=0x7f4034155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f403002a220 tx=0x7f40300046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 shutdown_connections 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f401806be60 0x7f401806e320 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f403415c770 0x7f4034157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40340d1200 0x7f4034155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 --2- 172.21.15.99:0/3895138024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f403406e5d0 0x7f4034155240 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 >> 172.21.15.99:0/3895138024 conn(0x7f403406bc10 msgr2=0x7f403415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 shutdown_connections 2024-09-10T11:35:05.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.201+0000 7f403b728640 1 -- 172.21.15.99:0/3895138024 wait complete. 2024-09-10T11:35:05.215 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:35:05.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:05.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 Processor -- start 2024-09-10T11:35:05.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 -- start start 2024-09-10T11:35:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f289015eb80 0x7f28900d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f28901626c0 0x7f28900d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28900d4e40 0x7f28900d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f28900d52d0 con 0x7f289015eb80 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f28900d1620 con 0x7f28901626c0 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.311+0000 7f28966c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2890162bc0 con 0x7f28900d4e40 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f289015eb80 0x7f28900d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f289015eb80 0x7f28900d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44450/0 (socket says 172.21.15.99:44450) 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 -- 172.21.15.99:0/393680269 learned_addr learned my addr 172.21.15.99:0/393680269 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f2894c66640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28900d4e40 0x7f28900d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 -- 172.21.15.99:0/393680269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28900d4e40 msgr2=0x7f28900d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288f7fe640 1 --2- 172.21.15.99:0/393680269 >> v2:172.21.15.196:3300/0 conn(0x7f28901626c0 0x7f28900d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28900d4e40 0x7f28900d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 -- 172.21.15.99:0/393680269 >> v2:172.21.15.196:3300/0 conn(0x7f28901626c0 msgr2=0x7f28900d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 --2- 172.21.15.99:0/393680269 >> v2:172.21.15.196:3300/0 conn(0x7f28901626c0 0x7f28900d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.312+0000 7f288ffff640 1 -- 172.21.15.99:0/393680269 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2890162ea0 con 0x7f289015eb80 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f288ffff640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289015eb80 0x7f28900d29b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2880009870 tx=0x7f288002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a44ddb4355c84cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f288effd640 1 -- 172.21.15.99:0/393680269 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880037070 con 0x7f289015eb80 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f288effd640 1 -- 172.21.15.99:0/393680269 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2880032440 con 0x7f289015eb80 2024-09-10T11:35:05.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f288effd640 1 -- 172.21.15.99:0/393680269 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880032810 con 0x7f289015eb80 2024-09-10T11:35:05.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 -- 172.21.15.99:0/393680269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289015eb80 msgr2=0x7f28900d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289015eb80 0x7f28900d29b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2880009870 tx=0x7f288002a180 comp rx=0 tx=0).stop 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 -- 172.21.15.99:0/393680269 shutdown_connections 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28900d4e40 0x7f28900d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 --2- 172.21.15.99:0/393680269 >> v2:172.21.15.196:3300/0 conn(0x7f28901626c0 0x7f28900d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 --2- 172.21.15.99:0/393680269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f289015eb80 0x7f28900d29b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 -- 172.21.15.99:0/393680269 >> 172.21.15.99:0/393680269 conn(0x7f289008e680 msgr2=0x7f289009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 -- 172.21.15.99:0/393680269 shutdown_connections 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.313+0000 7f28966c9640 1 -- 172.21.15.99:0/393680269 wait complete. 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.314+0000 7f28966c9640 1 Processor -- start 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 -- start start 2024-09-10T11:35:05.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289015eb80 0x7f2890157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28901626c0 0x7f28900d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f28900d2ef0 con 0x7f28900d4e40 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2890163460 con 0x7f289015eb80 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f28966c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f28900d3070 con 0x7f28901626c0 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f288ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f288ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44458/0 (socket says 172.21.15.99:44458) 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.315+0000 7f288ffff640 1 -- 172.21.15.99:0/3696991328 learned_addr learned my addr 172.21.15.99:0/3696991328 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 -- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28901626c0 msgr2=0x7f28900d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28901626c0 0x7f28900d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 -- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289015eb80 msgr2=0x7f2890157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289015eb80 0x7f2890157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 -- 172.21.15.99:0/3696991328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289015cdd0 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288ffff640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f28800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880037040 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2880042070 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289015bad0 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2880030bd0 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.316+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289015c090 con 0x7f28900d4e40 2024-09-10T11:35:05.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.317+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28900d29b0 con 0x7f28900d4e40 2024-09-10T11:35:05.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.317+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2880047460 con 0x7f28900d4e40 2024-09-10T11:35:05.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.318+0000 7f288cff9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 0x7f287006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.318+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f28800abb20 con 0x7f28900d4e40 2024-09-10T11:35:05.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.318+0000 7f288f7fe640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 0x7f287006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.318+0000 7f288f7fe640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 0x7f287006e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f289006da20 tx=0x7f2878008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.322+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2880045360 con 0x7f28900d4e40 2024-09-10T11:35:05.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.484+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28900cf900 con 0x7f287006be30 2024-09-10T11:35:05.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.490+0000 7f288cff9640 1 -- 172.21.15.99:0/3696991328 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242639 (secure 0 0 0) 0x7f28900cf900 con 0x7f287006be30 2024-09-10T11:35:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:05.496 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 msgr2=0x7f287006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 0x7f287006e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f289006da20 tx=0x7f2878008040 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 msgr2=0x7f289015c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2880000c00 tx=0x7f28800046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 shutdown_connections 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f287006be30 0x7f287006e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28901626c0 0x7f28900d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f289015eb80 0x7f2890157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 --2- 172.21.15.99:0/3696991328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28900d4e40 0x7f289015c890 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 >> 172.21.15.99:0/3696991328 conn(0x7f289008e680 msgr2=0x7f289006d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 shutdown_connections 2024-09-10T11:35:05.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.493+0000 7f28966c9640 1 -- 172.21.15.99:0/3696991328 wait complete. 2024-09-10T11:35:05.505 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T11:35:04.967283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556198,"statfs":{"total":1600000000,"available":1593547008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.411713"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565919+0000","last_change":"2024-09-10T11:35:01.565919+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565919+0000","last_undegraded":"2024-09-10T11:35:01.565919+0000","last_fullsized":"2024-09-10T11:35:01.565919+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568411+0000","last_change":"2024-09-10T11:35:01.568411+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568411+0000","last_undegraded":"2024-09-10T11:35:01.568411+0000","last_fullsized":"2024-09-10T11:35:01.568411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569657+0000","last_change":"2024-09-10T11:35:01.569657+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569657+0000","last_undegraded":"2024-09-10T11:35:01.569657+0000","last_fullsized":"2024-09-10T11:35:01.569657+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565761+0000","last_change":"2024-09-10T11:35:01.565761+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565761+0000","last_undegraded":"2024-09-10T11:35:01.565761+0000","last_fullsized":"2024-09-10T11:35:01.565761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566643+0000","last_change":"2024-09-10T11:35:01.566643+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566643+0000","last_undegraded":"2024-09-10T11:35:01.566643+0000","last_fullsized":"2024-09-10T11:35:01.566643+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569064+0000","last_change":"2024-09-10T11:35:01.569064+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569064+0000","last_undegraded":"2024-09-10T11:35:01.569064+0000","last_fullsized":"2024-09-10T11:35:01.569064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567844+0000","last_change":"2024-09-10T11:35:01.567844+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567844+0000","last_undegraded":"2024-09-10T11:35:01.567844+0000","last_fullsized":"2024-09-10T11:35:01.567844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569277+0000","last_change":"2024-09-10T11:35:01.569277+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569277+0000","last_undegraded":"2024-09-10T11:35:01.569277+0000","last_fullsized":"2024-09-10T11:35:01.569277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568457+0000","last_change":"2024-09-10T11:35:01.568457+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568457+0000","last_undegraded":"2024-09-10T11:35:01.568457+0000","last_fullsized":"2024-09-10T11:35:01.568457+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566245+0000","last_change":"2024-09-10T11:35:01.566245+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566245+0000","last_undegraded":"2024-09-10T11:35:01.566245+0000","last_fullsized":"2024-09-10T11:35:01.566245+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568738+0000","last_change":"2024-09-10T11:35:01.568738+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568738+0000","last_undegraded":"2024-09-10T11:35:01.568738+0000","last_fullsized":"2024-09-10T11:35:01.568738+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571097+0000","last_change":"2024-09-10T11:35:01.571097+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571097+0000","last_undegraded":"2024-09-10T11:35:01.571097+0000","last_fullsized":"2024-09-10T11:35:01.571097+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565950+0000","last_change":"2024-09-10T11:35:01.565950+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565950+0000","last_undegraded":"2024-09-10T11:35:01.565950+0000","last_fullsized":"2024-09-10T11:35:01.565950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565425+0000","last_change":"2024-09-10T11:35:01.565425+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565425+0000","last_undegraded":"2024-09-10T11:35:01.565425+0000","last_fullsized":"2024-09-10T11:35:01.565425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566167+0000","last_change":"2024-09-10T11:35:01.566167+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566167+0000","last_undegraded":"2024-09-10T11:35:01.566167+0000","last_fullsized":"2024-09-10T11:35:01.566167+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567990+0000","last_change":"2024-09-10T11:35:01.567990+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567990+0000","last_undegraded":"2024-09-10T11:35:01.567990+0000","last_fullsized":"2024-09-10T11:35:01.567990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569715+0000","last_change":"2024-09-10T11:35:01.569715+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569715+0000","last_undegraded":"2024-09-10T11:35:01.569715+0000","last_fullsized":"2024-09-10T11:35:01.569715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566911+0000","last_change":"2024-09-10T11:35:01.566911+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566911+0000","last_undegraded":"2024-09-10T11:35:01.566911+0000","last_fullsized":"2024-09-10T11:35:01.566911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572880+0000","last_change":"2024-09-10T11:35:01.572880+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572880+0000","last_undegraded":"2024-09-10T11:35:01.572880+0000","last_fullsized":"2024-09-10T11:35:01.572880+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571066+0000","last_change":"2024-09-10T11:35:01.571066+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571066+0000","last_undegraded":"2024-09-10T11:35:01.571066+0000","last_fullsized":"2024-09-10T11:35:01.571066+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573732+0000","last_change":"2024-09-10T11:35:01.573732+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573732+0000","last_undegraded":"2024-09-10T11:35:01.573732+0000","last_fullsized":"2024-09-10T11:35:01.573732+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569408+0000","last_change":"2024-09-10T11:35:01.569408+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569408+0000","last_undegraded":"2024-09-10T11:35:01.569408+0000","last_fullsized":"2024-09-10T11:35:01.569408+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566541+0000","last_change":"2024-09-10T11:35:01.566541+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566541+0000","last_undegraded":"2024-09-10T11:35:01.566541+0000","last_fullsized":"2024-09-10T11:35:01.566541+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567219+0000","last_change":"2024-09-10T11:35:01.567219+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567219+0000","last_undegraded":"2024-09-10T11:35:01.567219+0000","last_fullsized":"2024-09-10T11:35:01.567219+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565927+0000","last_change":"2024-09-10T11:35:01.565927+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565927+0000","last_undegraded":"2024-09-10T11:35:01.565927+0000","last_fullsized":"2024-09-10T11:35:01.565927+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564831+0000","last_change":"2024-09-10T11:35:01.564831+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564831+0000","last_undegraded":"2024-09-10T11:35:01.564831+0000","last_fullsized":"2024-09-10T11:35:01.564831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568745+0000","last_change":"2024-09-10T11:35:01.568745+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568745+0000","last_undegraded":"2024-09-10T11:35:01.568745+0000","last_fullsized":"2024-09-10T11:35:01.568745+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567687+0000","last_change":"2024-09-10T11:35:01.567687+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567687+0000","last_undegraded":"2024-09-10T11:35:01.567687+0000","last_fullsized":"2024-09-10T11:35:01.567687+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564599+0000","last_change":"2024-09-10T11:35:01.564599+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564599+0000","last_undegraded":"2024-09-10T11:35:01.564599+0000","last_fullsized":"2024-09-10T11:35:01.564599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565127+0000","last_change":"2024-09-10T11:35:01.565127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565127+0000","last_undegraded":"2024-09-10T11:35:01.565127+0000","last_fullsized":"2024-09-10T11:35:01.565127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572127+0000","last_change":"2024-09-10T11:35:01.572127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572127+0000","last_undegraded":"2024-09-10T11:35:01.572127+0000","last_fullsized":"2024-09-10T11:35:01.572127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564934+0000","last_change":"2024-09-10T11:35:01.564934+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564934+0000","last_undegraded":"2024-09-10T11:35:01.564934+0000","last_fullsized":"2024-09-10T11:35:01.564934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572594+0000","last_change":"2024-09-10T11:35:01.572594+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572594+0000","last_undegraded":"2024-09-10T11:35:01.572594+0000","last_fullsized":"2024-09-10T11:35:01.572594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574983+0000","last_change":"2024-09-10T11:35:01.574138+0000","last_active":"2024-09-10T11:35:01.574983+0000","last_peered":"2024-09-10T11:35:01.574983+0000","last_clean":"2024-09-10T11:35:01.574983+0000","last_became_active":"2024-09-10T11:35:01.573905+0000","last_became_peered":"2024-09-10T11:35:01.573905+0000","last_unstale":"2024-09-10T11:35:01.574983+0000","last_undegraded":"2024-09-10T11:35:01.574983+0000","last_fullsized":"2024-09-10T11:35:01.574983+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:13.165426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564878+0000","last_change":"2024-09-10T11:35:01.564878+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564878+0000","last_undegraded":"2024-09-10T11:35:01.564878+0000","last_fullsized":"2024-09-10T11:35:01.564878+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577412+0000","last_change":"2024-09-10T11:35:01.577311+0000","last_active":"2024-09-10T11:35:01.577412+0000","last_peered":"2024-09-10T11:35:01.577412+0000","last_clean":"2024-09-10T11:35:01.577412+0000","last_became_active":"2024-09-10T11:35:01.576942+0000","last_became_peered":"2024-09-10T11:35:01.576942+0000","last_unstale":"2024-09-10T11:35:01.577412+0000","last_undegraded":"2024-09-10T11:35:01.577412+0000","last_fullsized":"2024-09-10T11:35:01.577412+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:36.991927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565534+0000","last_change":"2024-09-10T11:35:01.565534+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565534+0000","last_undegraded":"2024-09-10T11:35:01.565534+0000","last_fullsized":"2024-09-10T11:35:01.565534+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579753+0000","last_change":"2024-09-10T11:35:01.579621+0000","last_active":"2024-09-10T11:35:01.579753+0000","last_peered":"2024-09-10T11:35:01.579753+0000","last_clean":"2024-09-10T11:35:01.579753+0000","last_became_active":"2024-09-10T11:35:01.579238+0000","last_became_peered":"2024-09-10T11:35:01.579238+0000","last_unstale":"2024-09-10T11:35:01.579753+0000","last_undegraded":"2024-09-10T11:35:01.579753+0000","last_fullsized":"2024-09-10T11:35:01.579753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:52.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565557+0000","last_change":"2024-09-10T11:35:01.565557+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565557+0000","last_undegraded":"2024-09-10T11:35:01.565557+0000","last_fullsized":"2024-09-10T11:35:01.565557+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.578494+0000","last_change":"2024-09-10T11:35:01.578383+0000","last_active":"2024-09-10T11:35:01.578494+0000","last_peered":"2024-09-10T11:35:01.578494+0000","last_clean":"2024-09-10T11:35:01.578494+0000","last_became_active":"2024-09-10T11:35:01.578109+0000","last_became_peered":"2024-09-10T11:35:01.578109+0000","last_unstale":"2024-09-10T11:35:01.578494+0000","last_undegraded":"2024-09-10T11:35:01.578494+0000","last_fullsized":"2024-09-10T11:35:01.578494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:18.486537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572758+0000","last_change":"2024-09-10T11:35:01.572758+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572758+0000","last_undegraded":"2024-09-10T11:35:01.572758+0000","last_fullsized":"2024-09-10T11:35:01.572758+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572226+0000","last_change":"2024-09-10T11:35:01.572113+0000","last_active":"2024-09-10T11:35:01.572226+0000","last_peered":"2024-09-10T11:35:01.572226+0000","last_clean":"2024-09-10T11:35:01.572226+0000","last_became_active":"2024-09-10T11:35:01.571874+0000","last_became_peered":"2024-09-10T11:35:01.571874+0000","last_unstale":"2024-09-10T11:35:01.572226+0000","last_undegraded":"2024-09-10T11:35:01.572226+0000","last_fullsized":"2024-09-10T11:35:01.572226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:29.218475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568170+0000","last_change":"2024-09-10T11:35:01.568170+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568170+0000","last_undegraded":"2024-09-10T11:35:01.568170+0000","last_fullsized":"2024-09-10T11:35:01.568170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576235+0000","last_change":"2024-09-10T11:35:01.576109+0000","last_active":"2024-09-10T11:35:01.576235+0000","last_peered":"2024-09-10T11:35:01.576235+0000","last_clean":"2024-09-10T11:35:01.576235+0000","last_became_active":"2024-09-10T11:35:01.575849+0000","last_became_peered":"2024-09-10T11:35:01.575849+0000","last_unstale":"2024-09-10T11:35:01.576235+0000","last_undegraded":"2024-09-10T11:35:01.576235+0000","last_fullsized":"2024-09-10T11:35:01.576235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:54.020713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564259+0000","last_change":"2024-09-10T11:35:01.564259+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564259+0000","last_undegraded":"2024-09-10T11:35:01.564259+0000","last_fullsized":"2024-09-10T11:35:01.564259+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575753+0000","last_change":"2024-09-10T11:35:01.575658+0000","last_active":"2024-09-10T11:35:01.575753+0000","last_peered":"2024-09-10T11:35:01.575753+0000","last_clean":"2024-09-10T11:35:01.575753+0000","last_became_active":"2024-09-10T11:35:01.575428+0000","last_became_peered":"2024-09-10T11:35:01.575428+0000","last_unstale":"2024-09-10T11:35:01.575753+0000","last_undegraded":"2024-09-10T11:35:01.575753+0000","last_fullsized":"2024-09-10T11:35:01.575753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:42.923450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571841+0000","last_change":"2024-09-10T11:35:01.571841+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571841+0000","last_undegraded":"2024-09-10T11:35:01.571841+0000","last_fullsized":"2024-09-10T11:35:01.571841+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579686+0000","last_change":"2024-09-10T11:35:01.579550+0000","last_active":"2024-09-10T11:35:01.579686+0000","last_peered":"2024-09-10T11:35:01.579686+0000","last_clean":"2024-09-10T11:35:01.579686+0000","last_became_active":"2024-09-10T11:35:01.579226+0000","last_became_peered":"2024-09-10T11:35:01.579226+0000","last_unstale":"2024-09-10T11:35:01.579686+0000","last_undegraded":"2024-09-10T11:35:01.579686+0000","last_fullsized":"2024-09-10T11:35:01.579686+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:02.892037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564550+0000","last_change":"2024-09-10T11:35:01.564550+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564550+0000","last_undegraded":"2024-09-10T11:35:01.564550+0000","last_fullsized":"2024-09-10T11:35:01.564550+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577256+0000","last_change":"2024-09-10T11:35:01.577132+0000","last_active":"2024-09-10T11:35:01.577256+0000","last_peered":"2024-09-10T11:35:01.577256+0000","last_clean":"2024-09-10T11:35:01.577256+0000","last_became_active":"2024-09-10T11:35:01.576572+0000","last_became_peered":"2024-09-10T11:35:01.576572+0000","last_unstale":"2024-09-10T11:35:01.577256+0000","last_undegraded":"2024-09-10T11:35:01.577256+0000","last_fullsized":"2024-09-10T11:35:01.577256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:31.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569057+0000","last_change":"2024-09-10T11:35:01.569057+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569057+0000","last_undegraded":"2024-09-10T11:35:01.569057+0000","last_fullsized":"2024-09-10T11:35:01.569057+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577971+0000","last_change":"2024-09-10T11:35:01.577634+0000","last_active":"2024-09-10T11:35:01.577971+0000","last_peered":"2024-09-10T11:35:01.577971+0000","last_clean":"2024-09-10T11:35:01.577971+0000","last_became_active":"2024-09-10T11:35:01.577375+0000","last_became_peered":"2024-09-10T11:35:01.577375+0000","last_unstale":"2024-09-10T11:35:01.577971+0000","last_undegraded":"2024-09-10T11:35:01.577971+0000","last_fullsized":"2024-09-10T11:35:01.577971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:59.758708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568150+0000","last_change":"2024-09-10T11:35:01.568150+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568150+0000","last_undegraded":"2024-09-10T11:35:01.568150+0000","last_fullsized":"2024-09-10T11:35:01.568150+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573571+0000","last_change":"2024-09-10T11:35:01.573449+0000","last_active":"2024-09-10T11:35:01.573571+0000","last_peered":"2024-09-10T11:35:01.573571+0000","last_clean":"2024-09-10T11:35:01.573571+0000","last_became_active":"2024-09-10T11:35:01.573208+0000","last_became_peered":"2024-09-10T11:35:01.573208+0000","last_unstale":"2024-09-10T11:35:01.573571+0000","last_undegraded":"2024-09-10T11:35:01.573571+0000","last_fullsized":"2024-09-10T11:35:01.573571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:55.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567343+0000","last_change":"2024-09-10T11:35:01.567343+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567343+0000","last_undegraded":"2024-09-10T11:35:01.567343+0000","last_fullsized":"2024-09-10T11:35:01.567343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575166+0000","last_change":"2024-09-10T11:35:01.575061+0000","last_active":"2024-09-10T11:35:01.575166+0000","last_peered":"2024-09-10T11:35:01.575166+0000","last_clean":"2024-09-10T11:35:01.575166+0000","last_became_active":"2024-09-10T11:35:01.574722+0000","last_became_peered":"2024-09-10T11:35:01.574722+0000","last_unstale":"2024-09-10T11:35:01.575166+0000","last_undegraded":"2024-09-10T11:35:01.575166+0000","last_fullsized":"2024-09-10T11:35:01.575166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:45.426600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565092+0000","last_change":"2024-09-10T11:35:01.565092+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565092+0000","last_undegraded":"2024-09-10T11:35:01.565092+0000","last_fullsized":"2024-09-10T11:35:01.565092+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577253+0000","last_change":"2024-09-10T11:35:01.576337+0000","last_active":"2024-09-10T11:35:01.577253+0000","last_peered":"2024-09-10T11:35:01.577253+0000","last_clean":"2024-09-10T11:35:01.577253+0000","last_became_active":"2024-09-10T11:35:01.576094+0000","last_became_peered":"2024-09-10T11:35:01.576094+0000","last_unstale":"2024-09-10T11:35:01.577253+0000","last_undegraded":"2024-09-10T11:35:01.577253+0000","last_fullsized":"2024-09-10T11:35:01.577253+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:18.030605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573292+0000","last_change":"2024-09-10T11:35:01.573292+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573292+0000","last_undegraded":"2024-09-10T11:35:01.573292+0000","last_fullsized":"2024-09-10T11:35:01.573292+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567570+0000","last_change":"2024-09-10T11:35:01.567436+0000","last_active":"2024-09-10T11:35:01.567570+0000","last_peered":"2024-09-10T11:35:01.567570+0000","last_clean":"2024-09-10T11:35:01.567570+0000","last_became_active":"2024-09-10T11:35:01.566612+0000","last_became_peered":"2024-09-10T11:35:01.566612+0000","last_unstale":"2024-09-10T11:35:01.567570+0000","last_undegraded":"2024-09-10T11:35:01.567570+0000","last_fullsized":"2024-09-10T11:35:01.567570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:12.054238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568196+0000","last_change":"2024-09-10T11:35:01.568196+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568196+0000","last_undegraded":"2024-09-10T11:35:01.568196+0000","last_fullsized":"2024-09-10T11:35:01.568196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571385+0000","last_change":"2024-09-10T11:35:01.571276+0000","last_active":"2024-09-10T11:35:01.571385+0000","last_peered":"2024-09-10T11:35:01.571385+0000","last_clean":"2024-09-10T11:35:01.571385+0000","last_became_active":"2024-09-10T11:35:01.570511+0000","last_became_peered":"2024-09-10T11:35:01.570511+0000","last_unstale":"2024-09-10T11:35:01.571385+0000","last_undegraded":"2024-09-10T11:35:01.571385+0000","last_fullsized":"2024-09-10T11:35:01.571385+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:07.622876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569769+0000","last_change":"2024-09-10T11:35:01.569769+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569769+0000","last_undegraded":"2024-09-10T11:35:01.569769+0000","last_fullsized":"2024-09-10T11:35:01.569769+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.569217+0000","last_change":"2024-09-10T11:35:01.569105+0000","last_active":"2024-09-10T11:35:01.569217+0000","last_peered":"2024-09-10T11:35:01.569217+0000","last_clean":"2024-09-10T11:35:01.569217+0000","last_became_active":"2024-09-10T11:35:01.568619+0000","last_became_peered":"2024-09-10T11:35:01.568619+0000","last_unstale":"2024-09-10T11:35:01.569217+0000","last_undegraded":"2024-09-10T11:35:01.569217+0000","last_fullsized":"2024-09-10T11:35:01.569217+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:41.912527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566883+0000","last_change":"2024-09-10T11:35:01.566883+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566883+0000","last_undegraded":"2024-09-10T11:35:01.566883+0000","last_fullsized":"2024-09-10T11:35:01.566883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574687+0000","last_change":"2024-09-10T11:35:01.574578+0000","last_active":"2024-09-10T11:35:01.574687+0000","last_peered":"2024-09-10T11:35:01.574687+0000","last_clean":"2024-09-10T11:35:01.574687+0000","last_became_active":"2024-09-10T11:35:01.574313+0000","last_became_peered":"2024-09-10T11:35:01.574313+0000","last_unstale":"2024-09-10T11:35:01.574687+0000","last_undegraded":"2024-09-10T11:35:01.574687+0000","last_fullsized":"2024-09-10T11:35:01.574687+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:33.710736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568446+0000","last_change":"2024-09-10T11:35:01.568446+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568446+0000","last_undegraded":"2024-09-10T11:35:01.568446+0000","last_fullsized":"2024-09-10T11:35:01.568446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.570614+0000","last_change":"2024-09-10T11:35:01.570511+0000","last_active":"2024-09-10T11:35:01.570614+0000","last_peered":"2024-09-10T11:35:01.570614+0000","last_clean":"2024-09-10T11:35:01.570614+0000","last_became_active":"2024-09-10T11:35:01.570268+0000","last_became_peered":"2024-09-10T11:35:01.570268+0000","last_unstale":"2024-09-10T11:35:01.570614+0000","last_undegraded":"2024-09-10T11:35:01.570614+0000","last_fullsized":"2024-09-10T11:35:01.570614+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:23.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571499+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571499+0000","last_undegraded":"2024-09-10T11:35:01.571499+0000","last_fullsized":"2024-09-10T11:35:01.571499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575650+0000","last_change":"2024-09-10T11:35:01.575527+0000","last_active":"2024-09-10T11:35:01.575650+0000","last_peered":"2024-09-10T11:35:01.575650+0000","last_clean":"2024-09-10T11:35:01.575650+0000","last_became_active":"2024-09-10T11:35:01.575285+0000","last_became_peered":"2024-09-10T11:35:01.575285+0000","last_unstale":"2024-09-10T11:35:01.575650+0000","last_undegraded":"2024-09-10T11:35:01.575650+0000","last_fullsized":"2024-09-10T11:35:01.575650+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:50.445819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571652+0000","last_change":"2024-09-10T11:35:01.571652+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571652+0000","last_undegraded":"2024-09-10T11:35:01.571652+0000","last_fullsized":"2024-09-10T11:35:01.571652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577157+0000","last_change":"2024-09-10T11:35:01.577040+0000","last_active":"2024-09-10T11:35:01.577157+0000","last_peered":"2024-09-10T11:35:01.577157+0000","last_clean":"2024-09-10T11:35:01.577157+0000","last_became_active":"2024-09-10T11:35:01.576667+0000","last_became_peered":"2024-09-10T11:35:01.576667+0000","last_unstale":"2024-09-10T11:35:01.577157+0000","last_undegraded":"2024-09-10T11:35:01.577157+0000","last_fullsized":"2024-09-10T11:35:01.577157+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:55.579500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571257+0000","last_change":"2024-09-10T11:35:01.571257+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571257+0000","last_undegraded":"2024-09-10T11:35:01.571257+0000","last_fullsized":"2024-09-10T11:35:01.571257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574162+0000","last_change":"2024-09-10T11:35:01.573987+0000","last_active":"2024-09-10T11:35:01.574162+0000","last_peered":"2024-09-10T11:35:01.574162+0000","last_clean":"2024-09-10T11:35:01.574162+0000","last_became_active":"2024-09-10T11:35:01.573756+0000","last_became_peered":"2024-09-10T11:35:01.573756+0000","last_unstale":"2024-09-10T11:35:01.574162+0000","last_undegraded":"2024-09-10T11:35:01.574162+0000","last_fullsized":"2024-09-10T11:35:01.574162+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:11.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564649+0000","last_change":"2024-09-10T11:35:01.564649+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564649+0000","last_undegraded":"2024-09-10T11:35:01.564649+0000","last_fullsized":"2024-09-10T11:35:01.564649+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567997+0000","last_change":"2024-09-10T11:35:01.567874+0000","last_active":"2024-09-10T11:35:01.567997+0000","last_peered":"2024-09-10T11:35:01.567997+0000","last_clean":"2024-09-10T11:35:01.567997+0000","last_became_active":"2024-09-10T11:35:01.567649+0000","last_became_peered":"2024-09-10T11:35:01.567649+0000","last_unstale":"2024-09-10T11:35:01.567997+0000","last_undegraded":"2024-09-10T11:35:01.567997+0000","last_fullsized":"2024-09-10T11:35:01.567997+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:58.144492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566977+0000","last_change":"2024-09-10T11:35:01.566977+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566977+0000","last_undegraded":"2024-09-10T11:35:01.566977+0000","last_fullsized":"2024-09-10T11:35:01.566977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576998+0000","last_change":"2024-09-10T11:35:01.576843+0000","last_active":"2024-09-10T11:35:01.576998+0000","last_peered":"2024-09-10T11:35:01.576998+0000","last_clean":"2024-09-10T11:35:01.576998+0000","last_became_active":"2024-09-10T11:35:01.576103+0000","last_became_peered":"2024-09-10T11:35:01.576103+0000","last_unstale":"2024-09-10T11:35:01.576998+0000","last_undegraded":"2024-09-10T11:35:01.576998+0000","last_fullsized":"2024-09-10T11:35:01.576998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.717940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.570853+0000","last_change":"2024-09-10T11:35:01.570853+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.570853+0000","last_undegraded":"2024-09-10T11:35:01.570853+0000","last_fullsized":"2024-09-10T11:35:01.570853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571060+0000","last_change":"2024-09-10T11:35:01.570746+0000","last_active":"2024-09-10T11:35:01.571060+0000","last_peered":"2024-09-10T11:35:01.571060+0000","last_clean":"2024-09-10T11:35:01.571060+0000","last_became_active":"2024-09-10T11:35:01.570187+0000","last_became_peered":"2024-09-10T11:35:01.570187+0000","last_unstale":"2024-09-10T11:35:01.571060+0000","last_undegraded":"2024-09-10T11:35:01.571060+0000","last_fullsized":"2024-09-10T11:35:01.571060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:08.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569349+0000","last_change":"2024-09-10T11:35:01.569349+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569349+0000","last_undegraded":"2024-09-10T11:35:01.569349+0000","last_fullsized":"2024-09-10T11:35:01.569349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573953+0000","last_change":"2024-09-10T11:35:01.573744+0000","last_active":"2024-09-10T11:35:01.573953+0000","last_peered":"2024-09-10T11:35:01.573953+0000","last_clean":"2024-09-10T11:35:01.573953+0000","last_became_active":"2024-09-10T11:35:01.573463+0000","last_became_peered":"2024-09-10T11:35:01.573463+0000","last_unstale":"2024-09-10T11:35:01.573953+0000","last_undegraded":"2024-09-10T11:35:01.573953+0000","last_fullsized":"2024-09-10T11:35:01.573953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.610059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572681+0000","last_change":"2024-09-10T11:35:01.572681+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572681+0000","last_undegraded":"2024-09-10T11:35:01.572681+0000","last_fullsized":"2024-09-10T11:35:01.572681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579934+0000","last_change":"2024-09-10T11:35:01.579801+0000","last_active":"2024-09-10T11:35:01.579934+0000","last_peered":"2024-09-10T11:35:01.579934+0000","last_clean":"2024-09-10T11:35:01.579934+0000","last_became_active":"2024-09-10T11:35:01.578207+0000","last_became_peered":"2024-09-10T11:35:01.578207+0000","last_unstale":"2024-09-10T11:35:01.579934+0000","last_undegraded":"2024-09-10T11:35:01.579934+0000","last_fullsized":"2024-09-10T11:35:01.579934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:36.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567016+0000","last_change":"2024-09-10T11:35:01.567016+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567016+0000","last_undegraded":"2024-09-10T11:35:01.567016+0000","last_fullsized":"2024-09-10T11:35:01.567016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572601+0000","last_change":"2024-09-10T11:35:01.572125+0000","last_active":"2024-09-10T11:35:01.572601+0000","last_peered":"2024-09-10T11:35:01.572601+0000","last_clean":"2024-09-10T11:35:01.572601+0000","last_became_active":"2024-09-10T11:35:01.571887+0000","last_became_peered":"2024-09-10T11:35:01.571887+0000","last_unstale":"2024-09-10T11:35:01.572601+0000","last_undegraded":"2024-09-10T11:35:01.572601+0000","last_fullsized":"2024-09-10T11:35:01.572601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:09.224575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568970+0000","last_change":"2024-09-10T11:35:01.568970+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568970+0000","last_undegraded":"2024-09-10T11:35:01.568970+0000","last_fullsized":"2024-09-10T11:35:01.568970+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579086+0000","last_change":"2024-09-10T11:35:01.578976+0000","last_active":"2024-09-10T11:35:01.579086+0000","last_peered":"2024-09-10T11:35:01.579086+0000","last_clean":"2024-09-10T11:35:01.579086+0000","last_became_active":"2024-09-10T11:35:01.578755+0000","last_became_peered":"2024-09-10T11:35:01.578755+0000","last_unstale":"2024-09-10T11:35:01.579086+0000","last_undegraded":"2024-09-10T11:35:01.579086+0000","last_fullsized":"2024-09-10T11:35:01.579086+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:48.512747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567653+0000","last_change":"2024-09-10T11:35:01.567653+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567653+0000","last_undegraded":"2024-09-10T11:35:01.567653+0000","last_fullsized":"2024-09-10T11:35:01.567653+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571622+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.571622+0000","last_peered":"2024-09-10T11:35:01.571622+0000","last_clean":"2024-09-10T11:35:01.571622+0000","last_became_active":"2024-09-10T11:35:01.571267+0000","last_became_peered":"2024-09-10T11:35:01.571267+0000","last_unstale":"2024-09-10T11:35:01.571622+0000","last_undegraded":"2024-09-10T11:35:01.571622+0000","last_fullsized":"2024-09-10T11:35:01.571622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:47.651348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571015+0000","last_change":"2024-09-10T11:35:01.571015+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571015+0000","last_undegraded":"2024-09-10T11:35:01.571015+0000","last_fullsized":"2024-09-10T11:35:01.571015+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572739+0000","last_change":"2024-09-10T11:35:01.570370+0000","last_active":"2024-09-10T11:35:01.572739+0000","last_peered":"2024-09-10T11:35:01.572739+0000","last_clean":"2024-09-10T11:35:01.572739+0000","last_became_active":"2024-09-10T11:35:01.570120+0000","last_became_peered":"2024-09-10T11:35:01.570120+0000","last_unstale":"2024-09-10T11:35:01.572739+0000","last_undegraded":"2024-09-10T11:35:01.572739+0000","last_fullsized":"2024-09-10T11:35:01.572739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:10.126918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572471+0000","last_change":"2024-09-10T11:35:01.572471+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572471+0000","last_undegraded":"2024-09-10T11:35:01.572471+0000","last_fullsized":"2024-09-10T11:35:01.572471+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576660+0000","last_change":"2024-09-10T11:35:01.576531+0000","last_active":"2024-09-10T11:35:01.576660+0000","last_peered":"2024-09-10T11:35:01.576660+0000","last_clean":"2024-09-10T11:35:01.576660+0000","last_became_active":"2024-09-10T11:35:01.575776+0000","last_became_peered":"2024-09-10T11:35:01.575776+0000","last_unstale":"2024-09-10T11:35:01.576660+0000","last_undegraded":"2024-09-10T11:35:01.576660+0000","last_fullsized":"2024-09-10T11:35:01.576660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:54.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572997+0000","last_change":"2024-09-10T11:35:01.572997+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572997+0000","last_undegraded":"2024-09-10T11:35:01.572997+0000","last_fullsized":"2024-09-10T11:35:01.572997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575004+0000","last_change":"2024-09-10T11:35:01.574933+0000","last_active":"2024-09-10T11:35:01.575004+0000","last_peered":"2024-09-10T11:35:01.575004+0000","last_clean":"2024-09-10T11:35:01.575004+0000","last_became_active":"2024-09-10T11:35:01.574408+0000","last_became_peered":"2024-09-10T11:35:01.574408+0000","last_unstale":"2024-09-10T11:35:01.575004+0000","last_undegraded":"2024-09-10T11:35:01.575004+0000","last_fullsized":"2024-09-10T11:35:01.575004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:45.538369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.565044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:01.565044+0000","last_peered":"2024-09-10T11:35:01.565044+0000","last_clean":"2024-09-10T11:35:01.565044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:01.565044+0000","last_undegraded":"2024-09-10T11:35:01.565044+0000","last_fullsized":"2024-09-10T11:35:01.565044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":636,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194676,"statfs":{"total":200000000,"available":199348411,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:05.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:05.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.643+0000 7f485b391640 1 Processor -- start 2024-09-10T11:35:05.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.643+0000 7f485b391640 1 -- start start 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.643+0000 7f485b391640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4854073440 0x7f485415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.643+0000 7f485b391640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4854152e60 con 0x7f48540d1200 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485b391640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48540d15e0 con 0x7f485406ece0 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485b391640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4854072680 con 0x7f4854073440 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485912d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48540d1200 0x7f485406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45232/0 (socket says 172.21.15.99:45232) 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485912d640 1 -- 172.21.15.99:0/2647962776 learned_addr learned my addr 172.21.15.99:0/2647962776 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:05.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586616623 0 0) 0x7f4854152e60 con 0x7f48540d1200 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4840003520 con 0x7f48540d1200 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485992e640 1 --2- 172.21.15.99:0/2647962776 >> v2:172.21.15.196:3300/0 conn(0x7f4854073440 0x7f485415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 356179541 0 0) 0x7f4840003520 con 0x7f48540d1200 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 >> v2:172.21.15.196:3300/0 conn(0x7f4854073440 msgr2=0x7f485415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 --2- 172.21.15.99:0/2647962776 >> v2:172.21.15.196:3300/0 conn(0x7f4854073440 0x7f485415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 >> v1:172.21.15.196:6789/0 conn(0x7f485406ece0 legacy=0x7f485415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4854072960 con 0x7f48540d1200 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f484c0019f0 con 0x7f48540d1200 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485992e640 1 --2- 172.21.15.99:0/2647962776 >> v2:172.21.15.196:3300/0 conn(0x7f4854073440 0x7f485415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 >> v1:172.21.15.99:6789/0 conn(0x7f48540d1200 legacy=0x7f485406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 0x7f4840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400052e0 0x7f484000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4854072680 con 0x7f48400039b0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f484000c1b0 con 0x7f48400052e0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f484000c3b0 con 0x7f4840004800 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485912d640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 0x7f4840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485892c640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.644+0000 7f485992e640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400052e0 0x7f484000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 -- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4840004800 msgr2=0x7f4840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4840004800 0x7f4840004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 -- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400052e0 msgr2=0x7f484000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400052e0 0x7f484000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 -- 172.21.15.99:0/2647962776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4840003520 con 0x7f48400039b0 2024-09-10T11:35:05.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485992e640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400052e0 0x7f484000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485992e640 1 -- 172.21.15.99:0/2647962776 reap_dead start 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f485912d640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 0x7f4840003dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f484c002230 tx=0x7f4840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa8d097d78bda00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00a4a0 con 0x7f48400039b0 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f484c02f070 con 0x7f48400039b0 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.645+0000 7f484bfff640 1 -- 172.21.15.99:0/2647962776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c02bcf0 con 0x7f48400039b0 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 -- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 msgr2=0x7f4840003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 0x7f4840003dd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f484c002230 tx=0x7f4840005830 comp rx=0 tx=0).stop 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 -- 172.21.15.99:0/2647962776 shutdown_connections 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 --2- 172.21.15.99:0/2647962776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48400039b0 0x7f4840003dd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 -- 172.21.15.99:0/2647962776 >> 172.21.15.99:0/2647962776 conn(0x7f485406bc10 msgr2=0x7f485406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 -- 172.21.15.99:0/2647962776 shutdown_connections 2024-09-10T11:35:05.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.646+0000 7f485b391640 1 -- 172.21.15.99:0/2647962776 wait complete. 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.647+0000 7f485b391640 1 Processor -- start 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.647+0000 7f485b391640 1 -- start start 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400039b0 0x7f485415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4854158d30 0x7f48541591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f485415cd90 con 0x7f4854151d00 2024-09-10T11:35:05.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4854072f20 con 0x7f48400039b0 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485b391640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f485415c060 con 0x7f4854158d30 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44468/0 (socket says 172.21.15.99:44468) 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 -- 172.21.15.99:0/2118068846 learned_addr learned my addr 172.21.15.99:0/2118068846 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485992e640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4854158d30 0x7f48541591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485912d640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400039b0 0x7f485415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 -- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4854158d30 msgr2=0x7f48541591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4854158d30 0x7f48541591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 -- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400039b0 msgr2=0x7f485415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400039b0 0x7f485415aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 -- 172.21.15.99:0/2118068846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4854157c90 con 0x7f4854151d00 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.648+0000 7f485892c640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f484400e490 tx=0x7f484400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.649+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844019070 con 0x7f4854151d00 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.649+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48440053a0 con 0x7f4854151d00 2024-09-10T11:35:05.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.649+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4854157f70 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.649+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844003810 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.649+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4854158530 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.650+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f485415c3f0 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.650+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f484401c070 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.650+0000 7f4849ffb640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 0x7f483806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.650+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f484408a300 con 0x7f4854151d00 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.650+0000 7f485912d640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 0x7f483806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.651+0000 7f485912d640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 0x7f483806e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f484c033000 tx=0x7f484c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.655+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f48540d15e0 con 0x7f4854151d00 2024-09-10T11:35:05.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.817+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f485406c190 con 0x7f483806be60 2024-09-10T11:35:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.822+0000 7f4849ffb640 1 -- 172.21.15.99:0/2118068846 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242639 (secure 0 0 0) 0x7f485406c190 con 0x7f483806be60 2024-09-10T11:35:05.826 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 msgr2=0x7f483806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 0x7f483806e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f484c033000 tx=0x7f484c009030 comp rx=0 tx=0).stop 2024-09-10T11:35:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 msgr2=0x7f4854151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f484400e490 tx=0x7f484400e890 comp rx=0 tx=0).stop 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 shutdown_connections 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f483806be60 0x7f483806e320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4854158d30 0x7f48541591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4854151d00 0x7f4854151500 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 --2- 172.21.15.99:0/2118068846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48400039b0 0x7f485415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 >> 172.21.15.99:0/2118068846 conn(0x7f485406bc10 msgr2=0x7f48540d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 shutdown_connections 2024-09-10T11:35:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.825+0000 7f485b391640 1 -- 172.21.15.99:0/2118068846 wait complete. 2024-09-10T11:35:05.838 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T11:35:04.967283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556198,"statfs":{"total":1600000000,"available":1593547008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.411713"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565919+0000","last_change":"2024-09-10T11:35:01.565919+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565919+0000","last_undegraded":"2024-09-10T11:35:01.565919+0000","last_fullsized":"2024-09-10T11:35:01.565919+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568411+0000","last_change":"2024-09-10T11:35:01.568411+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568411+0000","last_undegraded":"2024-09-10T11:35:01.568411+0000","last_fullsized":"2024-09-10T11:35:01.568411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569657+0000","last_change":"2024-09-10T11:35:01.569657+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569657+0000","last_undegraded":"2024-09-10T11:35:01.569657+0000","last_fullsized":"2024-09-10T11:35:01.569657+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565761+0000","last_change":"2024-09-10T11:35:01.565761+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565761+0000","last_undegraded":"2024-09-10T11:35:01.565761+0000","last_fullsized":"2024-09-10T11:35:01.565761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566643+0000","last_change":"2024-09-10T11:35:01.566643+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566643+0000","last_undegraded":"2024-09-10T11:35:01.566643+0000","last_fullsized":"2024-09-10T11:35:01.566643+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569064+0000","last_change":"2024-09-10T11:35:01.569064+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569064+0000","last_undegraded":"2024-09-10T11:35:01.569064+0000","last_fullsized":"2024-09-10T11:35:01.569064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567844+0000","last_change":"2024-09-10T11:35:01.567844+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567844+0000","last_undegraded":"2024-09-10T11:35:01.567844+0000","last_fullsized":"2024-09-10T11:35:01.567844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569277+0000","last_change":"2024-09-10T11:35:01.569277+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569277+0000","last_undegraded":"2024-09-10T11:35:01.569277+0000","last_fullsized":"2024-09-10T11:35:01.569277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568457+0000","last_change":"2024-09-10T11:35:01.568457+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568457+0000","last_undegraded":"2024-09-10T11:35:01.568457+0000","last_fullsized":"2024-09-10T11:35:01.568457+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566245+0000","last_change":"2024-09-10T11:35:01.566245+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566245+0000","last_undegraded":"2024-09-10T11:35:01.566245+0000","last_fullsized":"2024-09-10T11:35:01.566245+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568738+0000","last_change":"2024-09-10T11:35:01.568738+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568738+0000","last_undegraded":"2024-09-10T11:35:01.568738+0000","last_fullsized":"2024-09-10T11:35:01.568738+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571097+0000","last_change":"2024-09-10T11:35:01.571097+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571097+0000","last_undegraded":"2024-09-10T11:35:01.571097+0000","last_fullsized":"2024-09-10T11:35:01.571097+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565950+0000","last_change":"2024-09-10T11:35:01.565950+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565950+0000","last_undegraded":"2024-09-10T11:35:01.565950+0000","last_fullsized":"2024-09-10T11:35:01.565950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565425+0000","last_change":"2024-09-10T11:35:01.565425+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565425+0000","last_undegraded":"2024-09-10T11:35:01.565425+0000","last_fullsized":"2024-09-10T11:35:01.565425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566167+0000","last_change":"2024-09-10T11:35:01.566167+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566167+0000","last_undegraded":"2024-09-10T11:35:01.566167+0000","last_fullsized":"2024-09-10T11:35:01.566167+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567990+0000","last_change":"2024-09-10T11:35:01.567990+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567990+0000","last_undegraded":"2024-09-10T11:35:01.567990+0000","last_fullsized":"2024-09-10T11:35:01.567990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569715+0000","last_change":"2024-09-10T11:35:01.569715+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569715+0000","last_undegraded":"2024-09-10T11:35:01.569715+0000","last_fullsized":"2024-09-10T11:35:01.569715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566911+0000","last_change":"2024-09-10T11:35:01.566911+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566911+0000","last_undegraded":"2024-09-10T11:35:01.566911+0000","last_fullsized":"2024-09-10T11:35:01.566911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572880+0000","last_change":"2024-09-10T11:35:01.572880+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572880+0000","last_undegraded":"2024-09-10T11:35:01.572880+0000","last_fullsized":"2024-09-10T11:35:01.572880+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571066+0000","last_change":"2024-09-10T11:35:01.571066+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571066+0000","last_undegraded":"2024-09-10T11:35:01.571066+0000","last_fullsized":"2024-09-10T11:35:01.571066+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573732+0000","last_change":"2024-09-10T11:35:01.573732+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573732+0000","last_undegraded":"2024-09-10T11:35:01.573732+0000","last_fullsized":"2024-09-10T11:35:01.573732+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569408+0000","last_change":"2024-09-10T11:35:01.569408+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569408+0000","last_undegraded":"2024-09-10T11:35:01.569408+0000","last_fullsized":"2024-09-10T11:35:01.569408+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566541+0000","last_change":"2024-09-10T11:35:01.566541+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566541+0000","last_undegraded":"2024-09-10T11:35:01.566541+0000","last_fullsized":"2024-09-10T11:35:01.566541+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567219+0000","last_change":"2024-09-10T11:35:01.567219+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567219+0000","last_undegraded":"2024-09-10T11:35:01.567219+0000","last_fullsized":"2024-09-10T11:35:01.567219+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565927+0000","last_change":"2024-09-10T11:35:01.565927+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565927+0000","last_undegraded":"2024-09-10T11:35:01.565927+0000","last_fullsized":"2024-09-10T11:35:01.565927+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564831+0000","last_change":"2024-09-10T11:35:01.564831+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564831+0000","last_undegraded":"2024-09-10T11:35:01.564831+0000","last_fullsized":"2024-09-10T11:35:01.564831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568745+0000","last_change":"2024-09-10T11:35:01.568745+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568745+0000","last_undegraded":"2024-09-10T11:35:01.568745+0000","last_fullsized":"2024-09-10T11:35:01.568745+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567687+0000","last_change":"2024-09-10T11:35:01.567687+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567687+0000","last_undegraded":"2024-09-10T11:35:01.567687+0000","last_fullsized":"2024-09-10T11:35:01.567687+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564599+0000","last_change":"2024-09-10T11:35:01.564599+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564599+0000","last_undegraded":"2024-09-10T11:35:01.564599+0000","last_fullsized":"2024-09-10T11:35:01.564599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565127+0000","last_change":"2024-09-10T11:35:01.565127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565127+0000","last_undegraded":"2024-09-10T11:35:01.565127+0000","last_fullsized":"2024-09-10T11:35:01.565127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572127+0000","last_change":"2024-09-10T11:35:01.572127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572127+0000","last_undegraded":"2024-09-10T11:35:01.572127+0000","last_fullsized":"2024-09-10T11:35:01.572127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564934+0000","last_change":"2024-09-10T11:35:01.564934+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564934+0000","last_undegraded":"2024-09-10T11:35:01.564934+0000","last_fullsized":"2024-09-10T11:35:01.564934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572594+0000","last_change":"2024-09-10T11:35:01.572594+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572594+0000","last_undegraded":"2024-09-10T11:35:01.572594+0000","last_fullsized":"2024-09-10T11:35:01.572594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574983+0000","last_change":"2024-09-10T11:35:01.574138+0000","last_active":"2024-09-10T11:35:01.574983+0000","last_peered":"2024-09-10T11:35:01.574983+0000","last_clean":"2024-09-10T11:35:01.574983+0000","last_became_active":"2024-09-10T11:35:01.573905+0000","last_became_peered":"2024-09-10T11:35:01.573905+0000","last_unstale":"2024-09-10T11:35:01.574983+0000","last_undegraded":"2024-09-10T11:35:01.574983+0000","last_fullsized":"2024-09-10T11:35:01.574983+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:13.165426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564878+0000","last_change":"2024-09-10T11:35:01.564878+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564878+0000","last_undegraded":"2024-09-10T11:35:01.564878+0000","last_fullsized":"2024-09-10T11:35:01.564878+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577412+0000","last_change":"2024-09-10T11:35:01.577311+0000","last_active":"2024-09-10T11:35:01.577412+0000","last_peered":"2024-09-10T11:35:01.577412+0000","last_clean":"2024-09-10T11:35:01.577412+0000","last_became_active":"2024-09-10T11:35:01.576942+0000","last_became_peered":"2024-09-10T11:35:01.576942+0000","last_unstale":"2024-09-10T11:35:01.577412+0000","last_undegraded":"2024-09-10T11:35:01.577412+0000","last_fullsized":"2024-09-10T11:35:01.577412+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:36.991927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565534+0000","last_change":"2024-09-10T11:35:01.565534+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565534+0000","last_undegraded":"2024-09-10T11:35:01.565534+0000","last_fullsized":"2024-09-10T11:35:01.565534+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579753+0000","last_change":"2024-09-10T11:35:01.579621+0000","last_active":"2024-09-10T11:35:01.579753+0000","last_peered":"2024-09-10T11:35:01.579753+0000","last_clean":"2024-09-10T11:35:01.579753+0000","last_became_active":"2024-09-10T11:35:01.579238+0000","last_became_peered":"2024-09-10T11:35:01.579238+0000","last_unstale":"2024-09-10T11:35:01.579753+0000","last_undegraded":"2024-09-10T11:35:01.579753+0000","last_fullsized":"2024-09-10T11:35:01.579753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:52.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565557+0000","last_change":"2024-09-10T11:35:01.565557+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565557+0000","last_undegraded":"2024-09-10T11:35:01.565557+0000","last_fullsized":"2024-09-10T11:35:01.565557+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.578494+0000","last_change":"2024-09-10T11:35:01.578383+0000","last_active":"2024-09-10T11:35:01.578494+0000","last_peered":"2024-09-10T11:35:01.578494+0000","last_clean":"2024-09-10T11:35:01.578494+0000","last_became_active":"2024-09-10T11:35:01.578109+0000","last_became_peered":"2024-09-10T11:35:01.578109+0000","last_unstale":"2024-09-10T11:35:01.578494+0000","last_undegraded":"2024-09-10T11:35:01.578494+0000","last_fullsized":"2024-09-10T11:35:01.578494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:18.486537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572758+0000","last_change":"2024-09-10T11:35:01.572758+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572758+0000","last_undegraded":"2024-09-10T11:35:01.572758+0000","last_fullsized":"2024-09-10T11:35:01.572758+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572226+0000","last_change":"2024-09-10T11:35:01.572113+0000","last_active":"2024-09-10T11:35:01.572226+0000","last_peered":"2024-09-10T11:35:01.572226+0000","last_clean":"2024-09-10T11:35:01.572226+0000","last_became_active":"2024-09-10T11:35:01.571874+0000","last_became_peered":"2024-09-10T11:35:01.571874+0000","last_unstale":"2024-09-10T11:35:01.572226+0000","last_undegraded":"2024-09-10T11:35:01.572226+0000","last_fullsized":"2024-09-10T11:35:01.572226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:29.218475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568170+0000","last_change":"2024-09-10T11:35:01.568170+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568170+0000","last_undegraded":"2024-09-10T11:35:01.568170+0000","last_fullsized":"2024-09-10T11:35:01.568170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576235+0000","last_change":"2024-09-10T11:35:01.576109+0000","last_active":"2024-09-10T11:35:01.576235+0000","last_peered":"2024-09-10T11:35:01.576235+0000","last_clean":"2024-09-10T11:35:01.576235+0000","last_became_active":"2024-09-10T11:35:01.575849+0000","last_became_peered":"2024-09-10T11:35:01.575849+0000","last_unstale":"2024-09-10T11:35:01.576235+0000","last_undegraded":"2024-09-10T11:35:01.576235+0000","last_fullsized":"2024-09-10T11:35:01.576235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:54.020713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564259+0000","last_change":"2024-09-10T11:35:01.564259+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564259+0000","last_undegraded":"2024-09-10T11:35:01.564259+0000","last_fullsized":"2024-09-10T11:35:01.564259+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575753+0000","last_change":"2024-09-10T11:35:01.575658+0000","last_active":"2024-09-10T11:35:01.575753+0000","last_peered":"2024-09-10T11:35:01.575753+0000","last_clean":"2024-09-10T11:35:01.575753+0000","last_became_active":"2024-09-10T11:35:01.575428+0000","last_became_peered":"2024-09-10T11:35:01.575428+0000","last_unstale":"2024-09-10T11:35:01.575753+0000","last_undegraded":"2024-09-10T11:35:01.575753+0000","last_fullsized":"2024-09-10T11:35:01.575753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:42.923450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571841+0000","last_change":"2024-09-10T11:35:01.571841+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571841+0000","last_undegraded":"2024-09-10T11:35:01.571841+0000","last_fullsized":"2024-09-10T11:35:01.571841+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579686+0000","last_change":"2024-09-10T11:35:01.579550+0000","last_active":"2024-09-10T11:35:01.579686+0000","last_peered":"2024-09-10T11:35:01.579686+0000","last_clean":"2024-09-10T11:35:01.579686+0000","last_became_active":"2024-09-10T11:35:01.579226+0000","last_became_peered":"2024-09-10T11:35:01.579226+0000","last_unstale":"2024-09-10T11:35:01.579686+0000","last_undegraded":"2024-09-10T11:35:01.579686+0000","last_fullsized":"2024-09-10T11:35:01.579686+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:02.892037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564550+0000","last_change":"2024-09-10T11:35:01.564550+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564550+0000","last_undegraded":"2024-09-10T11:35:01.564550+0000","last_fullsized":"2024-09-10T11:35:01.564550+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577256+0000","last_change":"2024-09-10T11:35:01.577132+0000","last_active":"2024-09-10T11:35:01.577256+0000","last_peered":"2024-09-10T11:35:01.577256+0000","last_clean":"2024-09-10T11:35:01.577256+0000","last_became_active":"2024-09-10T11:35:01.576572+0000","last_became_peered":"2024-09-10T11:35:01.576572+0000","last_unstale":"2024-09-10T11:35:01.577256+0000","last_undegraded":"2024-09-10T11:35:01.577256+0000","last_fullsized":"2024-09-10T11:35:01.577256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:31.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569057+0000","last_change":"2024-09-10T11:35:01.569057+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569057+0000","last_undegraded":"2024-09-10T11:35:01.569057+0000","last_fullsized":"2024-09-10T11:35:01.569057+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577971+0000","last_change":"2024-09-10T11:35:01.577634+0000","last_active":"2024-09-10T11:35:01.577971+0000","last_peered":"2024-09-10T11:35:01.577971+0000","last_clean":"2024-09-10T11:35:01.577971+0000","last_became_active":"2024-09-10T11:35:01.577375+0000","last_became_peered":"2024-09-10T11:35:01.577375+0000","last_unstale":"2024-09-10T11:35:01.577971+0000","last_undegraded":"2024-09-10T11:35:01.577971+0000","last_fullsized":"2024-09-10T11:35:01.577971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:59.758708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568150+0000","last_change":"2024-09-10T11:35:01.568150+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568150+0000","last_undegraded":"2024-09-10T11:35:01.568150+0000","last_fullsized":"2024-09-10T11:35:01.568150+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573571+0000","last_change":"2024-09-10T11:35:01.573449+0000","last_active":"2024-09-10T11:35:01.573571+0000","last_peered":"2024-09-10T11:35:01.573571+0000","last_clean":"2024-09-10T11:35:01.573571+0000","last_became_active":"2024-09-10T11:35:01.573208+0000","last_became_peered":"2024-09-10T11:35:01.573208+0000","last_unstale":"2024-09-10T11:35:01.573571+0000","last_undegraded":"2024-09-10T11:35:01.573571+0000","last_fullsized":"2024-09-10T11:35:01.573571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:55.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567343+0000","last_change":"2024-09-10T11:35:01.567343+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567343+0000","last_undegraded":"2024-09-10T11:35:01.567343+0000","last_fullsized":"2024-09-10T11:35:01.567343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575166+0000","last_change":"2024-09-10T11:35:01.575061+0000","last_active":"2024-09-10T11:35:01.575166+0000","last_peered":"2024-09-10T11:35:01.575166+0000","last_clean":"2024-09-10T11:35:01.575166+0000","last_became_active":"2024-09-10T11:35:01.574722+0000","last_became_peered":"2024-09-10T11:35:01.574722+0000","last_unstale":"2024-09-10T11:35:01.575166+0000","last_undegraded":"2024-09-10T11:35:01.575166+0000","last_fullsized":"2024-09-10T11:35:01.575166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:45.426600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565092+0000","last_change":"2024-09-10T11:35:01.565092+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565092+0000","last_undegraded":"2024-09-10T11:35:01.565092+0000","last_fullsized":"2024-09-10T11:35:01.565092+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577253+0000","last_change":"2024-09-10T11:35:01.576337+0000","last_active":"2024-09-10T11:35:01.577253+0000","last_peered":"2024-09-10T11:35:01.577253+0000","last_clean":"2024-09-10T11:35:01.577253+0000","last_became_active":"2024-09-10T11:35:01.576094+0000","last_became_peered":"2024-09-10T11:35:01.576094+0000","last_unstale":"2024-09-10T11:35:01.577253+0000","last_undegraded":"2024-09-10T11:35:01.577253+0000","last_fullsized":"2024-09-10T11:35:01.577253+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:18.030605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573292+0000","last_change":"2024-09-10T11:35:01.573292+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573292+0000","last_undegraded":"2024-09-10T11:35:01.573292+0000","last_fullsized":"2024-09-10T11:35:01.573292+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567570+0000","last_change":"2024-09-10T11:35:01.567436+0000","last_active":"2024-09-10T11:35:01.567570+0000","last_peered":"2024-09-10T11:35:01.567570+0000","last_clean":"2024-09-10T11:35:01.567570+0000","last_became_active":"2024-09-10T11:35:01.566612+0000","last_became_peered":"2024-09-10T11:35:01.566612+0000","last_unstale":"2024-09-10T11:35:01.567570+0000","last_undegraded":"2024-09-10T11:35:01.567570+0000","last_fullsized":"2024-09-10T11:35:01.567570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:12.054238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568196+0000","last_change":"2024-09-10T11:35:01.568196+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568196+0000","last_undegraded":"2024-09-10T11:35:01.568196+0000","last_fullsized":"2024-09-10T11:35:01.568196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571385+0000","last_change":"2024-09-10T11:35:01.571276+0000","last_active":"2024-09-10T11:35:01.571385+0000","last_peered":"2024-09-10T11:35:01.571385+0000","last_clean":"2024-09-10T11:35:01.571385+0000","last_became_active":"2024-09-10T11:35:01.570511+0000","last_became_peered":"2024-09-10T11:35:01.570511+0000","last_unstale":"2024-09-10T11:35:01.571385+0000","last_undegraded":"2024-09-10T11:35:01.571385+0000","last_fullsized":"2024-09-10T11:35:01.571385+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:07.622876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569769+0000","last_change":"2024-09-10T11:35:01.569769+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569769+0000","last_undegraded":"2024-09-10T11:35:01.569769+0000","last_fullsized":"2024-09-10T11:35:01.569769+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.569217+0000","last_change":"2024-09-10T11:35:01.569105+0000","last_active":"2024-09-10T11:35:01.569217+0000","last_peered":"2024-09-10T11:35:01.569217+0000","last_clean":"2024-09-10T11:35:01.569217+0000","last_became_active":"2024-09-10T11:35:01.568619+0000","last_became_peered":"2024-09-10T11:35:01.568619+0000","last_unstale":"2024-09-10T11:35:01.569217+0000","last_undegraded":"2024-09-10T11:35:01.569217+0000","last_fullsized":"2024-09-10T11:35:01.569217+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:41.912527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566883+0000","last_change":"2024-09-10T11:35:01.566883+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566883+0000","last_undegraded":"2024-09-10T11:35:01.566883+0000","last_fullsized":"2024-09-10T11:35:01.566883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574687+0000","last_change":"2024-09-10T11:35:01.574578+0000","last_active":"2024-09-10T11:35:01.574687+0000","last_peered":"2024-09-10T11:35:01.574687+0000","last_clean":"2024-09-10T11:35:01.574687+0000","last_became_active":"2024-09-10T11:35:01.574313+0000","last_became_peered":"2024-09-10T11:35:01.574313+0000","last_unstale":"2024-09-10T11:35:01.574687+0000","last_undegraded":"2024-09-10T11:35:01.574687+0000","last_fullsized":"2024-09-10T11:35:01.574687+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:33.710736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568446+0000","last_change":"2024-09-10T11:35:01.568446+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568446+0000","last_undegraded":"2024-09-10T11:35:01.568446+0000","last_fullsized":"2024-09-10T11:35:01.568446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.570614+0000","last_change":"2024-09-10T11:35:01.570511+0000","last_active":"2024-09-10T11:35:01.570614+0000","last_peered":"2024-09-10T11:35:01.570614+0000","last_clean":"2024-09-10T11:35:01.570614+0000","last_became_active":"2024-09-10T11:35:01.570268+0000","last_became_peered":"2024-09-10T11:35:01.570268+0000","last_unstale":"2024-09-10T11:35:01.570614+0000","last_undegraded":"2024-09-10T11:35:01.570614+0000","last_fullsized":"2024-09-10T11:35:01.570614+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:23.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571499+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571499+0000","last_undegraded":"2024-09-10T11:35:01.571499+0000","last_fullsized":"2024-09-10T11:35:01.571499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575650+0000","last_change":"2024-09-10T11:35:01.575527+0000","last_active":"2024-09-10T11:35:01.575650+0000","last_peered":"2024-09-10T11:35:01.575650+0000","last_clean":"2024-09-10T11:35:01.575650+0000","last_became_active":"2024-09-10T11:35:01.575285+0000","last_became_peered":"2024-09-10T11:35:01.575285+0000","last_unstale":"2024-09-10T11:35:01.575650+0000","last_undegraded":"2024-09-10T11:35:01.575650+0000","last_fullsized":"2024-09-10T11:35:01.575650+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:50.445819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571652+0000","last_change":"2024-09-10T11:35:01.571652+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571652+0000","last_undegraded":"2024-09-10T11:35:01.571652+0000","last_fullsized":"2024-09-10T11:35:01.571652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577157+0000","last_change":"2024-09-10T11:35:01.577040+0000","last_active":"2024-09-10T11:35:01.577157+0000","last_peered":"2024-09-10T11:35:01.577157+0000","last_clean":"2024-09-10T11:35:01.577157+0000","last_became_active":"2024-09-10T11:35:01.576667+0000","last_became_peered":"2024-09-10T11:35:01.576667+0000","last_unstale":"2024-09-10T11:35:01.577157+0000","last_undegraded":"2024-09-10T11:35:01.577157+0000","last_fullsized":"2024-09-10T11:35:01.577157+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:55.579500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571257+0000","last_change":"2024-09-10T11:35:01.571257+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571257+0000","last_undegraded":"2024-09-10T11:35:01.571257+0000","last_fullsized":"2024-09-10T11:35:01.571257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574162+0000","last_change":"2024-09-10T11:35:01.573987+0000","last_active":"2024-09-10T11:35:01.574162+0000","last_peered":"2024-09-10T11:35:01.574162+0000","last_clean":"2024-09-10T11:35:01.574162+0000","last_became_active":"2024-09-10T11:35:01.573756+0000","last_became_peered":"2024-09-10T11:35:01.573756+0000","last_unstale":"2024-09-10T11:35:01.574162+0000","last_undegraded":"2024-09-10T11:35:01.574162+0000","last_fullsized":"2024-09-10T11:35:01.574162+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:11.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564649+0000","last_change":"2024-09-10T11:35:01.564649+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564649+0000","last_undegraded":"2024-09-10T11:35:01.564649+0000","last_fullsized":"2024-09-10T11:35:01.564649+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567997+0000","last_change":"2024-09-10T11:35:01.567874+0000","last_active":"2024-09-10T11:35:01.567997+0000","last_peered":"2024-09-10T11:35:01.567997+0000","last_clean":"2024-09-10T11:35:01.567997+0000","last_became_active":"2024-09-10T11:35:01.567649+0000","last_became_peered":"2024-09-10T11:35:01.567649+0000","last_unstale":"2024-09-10T11:35:01.567997+0000","last_undegraded":"2024-09-10T11:35:01.567997+0000","last_fullsized":"2024-09-10T11:35:01.567997+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:58.144492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566977+0000","last_change":"2024-09-10T11:35:01.566977+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566977+0000","last_undegraded":"2024-09-10T11:35:01.566977+0000","last_fullsized":"2024-09-10T11:35:01.566977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576998+0000","last_change":"2024-09-10T11:35:01.576843+0000","last_active":"2024-09-10T11:35:01.576998+0000","last_peered":"2024-09-10T11:35:01.576998+0000","last_clean":"2024-09-10T11:35:01.576998+0000","last_became_active":"2024-09-10T11:35:01.576103+0000","last_became_peered":"2024-09-10T11:35:01.576103+0000","last_unstale":"2024-09-10T11:35:01.576998+0000","last_undegraded":"2024-09-10T11:35:01.576998+0000","last_fullsized":"2024-09-10T11:35:01.576998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.717940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.570853+0000","last_change":"2024-09-10T11:35:01.570853+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.570853+0000","last_undegraded":"2024-09-10T11:35:01.570853+0000","last_fullsized":"2024-09-10T11:35:01.570853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571060+0000","last_change":"2024-09-10T11:35:01.570746+0000","last_active":"2024-09-10T11:35:01.571060+0000","last_peered":"2024-09-10T11:35:01.571060+0000","last_clean":"2024-09-10T11:35:01.571060+0000","last_became_active":"2024-09-10T11:35:01.570187+0000","last_became_peered":"2024-09-10T11:35:01.570187+0000","last_unstale":"2024-09-10T11:35:01.571060+0000","last_undegraded":"2024-09-10T11:35:01.571060+0000","last_fullsized":"2024-09-10T11:35:01.571060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:08.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569349+0000","last_change":"2024-09-10T11:35:01.569349+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569349+0000","last_undegraded":"2024-09-10T11:35:01.569349+0000","last_fullsized":"2024-09-10T11:35:01.569349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573953+0000","last_change":"2024-09-10T11:35:01.573744+0000","last_active":"2024-09-10T11:35:01.573953+0000","last_peered":"2024-09-10T11:35:01.573953+0000","last_clean":"2024-09-10T11:35:01.573953+0000","last_became_active":"2024-09-10T11:35:01.573463+0000","last_became_peered":"2024-09-10T11:35:01.573463+0000","last_unstale":"2024-09-10T11:35:01.573953+0000","last_undegraded":"2024-09-10T11:35:01.573953+0000","last_fullsized":"2024-09-10T11:35:01.573953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.610059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572681+0000","last_change":"2024-09-10T11:35:01.572681+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572681+0000","last_undegraded":"2024-09-10T11:35:01.572681+0000","last_fullsized":"2024-09-10T11:35:01.572681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579934+0000","last_change":"2024-09-10T11:35:01.579801+0000","last_active":"2024-09-10T11:35:01.579934+0000","last_peered":"2024-09-10T11:35:01.579934+0000","last_clean":"2024-09-10T11:35:01.579934+0000","last_became_active":"2024-09-10T11:35:01.578207+0000","last_became_peered":"2024-09-10T11:35:01.578207+0000","last_unstale":"2024-09-10T11:35:01.579934+0000","last_undegraded":"2024-09-10T11:35:01.579934+0000","last_fullsized":"2024-09-10T11:35:01.579934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:36.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567016+0000","last_change":"2024-09-10T11:35:01.567016+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567016+0000","last_undegraded":"2024-09-10T11:35:01.567016+0000","last_fullsized":"2024-09-10T11:35:01.567016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572601+0000","last_change":"2024-09-10T11:35:01.572125+0000","last_active":"2024-09-10T11:35:01.572601+0000","last_peered":"2024-09-10T11:35:01.572601+0000","last_clean":"2024-09-10T11:35:01.572601+0000","last_became_active":"2024-09-10T11:35:01.571887+0000","last_became_peered":"2024-09-10T11:35:01.571887+0000","last_unstale":"2024-09-10T11:35:01.572601+0000","last_undegraded":"2024-09-10T11:35:01.572601+0000","last_fullsized":"2024-09-10T11:35:01.572601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:09.224575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568970+0000","last_change":"2024-09-10T11:35:01.568970+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568970+0000","last_undegraded":"2024-09-10T11:35:01.568970+0000","last_fullsized":"2024-09-10T11:35:01.568970+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579086+0000","last_change":"2024-09-10T11:35:01.578976+0000","last_active":"2024-09-10T11:35:01.579086+0000","last_peered":"2024-09-10T11:35:01.579086+0000","last_clean":"2024-09-10T11:35:01.579086+0000","last_became_active":"2024-09-10T11:35:01.578755+0000","last_became_peered":"2024-09-10T11:35:01.578755+0000","last_unstale":"2024-09-10T11:35:01.579086+0000","last_undegraded":"2024-09-10T11:35:01.579086+0000","last_fullsized":"2024-09-10T11:35:01.579086+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:48.512747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567653+0000","last_change":"2024-09-10T11:35:01.567653+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567653+0000","last_undegraded":"2024-09-10T11:35:01.567653+0000","last_fullsized":"2024-09-10T11:35:01.567653+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571622+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.571622+0000","last_peered":"2024-09-10T11:35:01.571622+0000","last_clean":"2024-09-10T11:35:01.571622+0000","last_became_active":"2024-09-10T11:35:01.571267+0000","last_became_peered":"2024-09-10T11:35:01.571267+0000","last_unstale":"2024-09-10T11:35:01.571622+0000","last_undegraded":"2024-09-10T11:35:01.571622+0000","last_fullsized":"2024-09-10T11:35:01.571622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:47.651348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571015+0000","last_change":"2024-09-10T11:35:01.571015+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571015+0000","last_undegraded":"2024-09-10T11:35:01.571015+0000","last_fullsized":"2024-09-10T11:35:01.571015+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572739+0000","last_change":"2024-09-10T11:35:01.570370+0000","last_active":"2024-09-10T11:35:01.572739+0000","last_peered":"2024-09-10T11:35:01.572739+0000","last_clean":"2024-09-10T11:35:01.572739+0000","last_became_active":"2024-09-10T11:35:01.570120+0000","last_became_peered":"2024-09-10T11:35:01.570120+0000","last_unstale":"2024-09-10T11:35:01.572739+0000","last_undegraded":"2024-09-10T11:35:01.572739+0000","last_fullsized":"2024-09-10T11:35:01.572739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:10.126918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572471+0000","last_change":"2024-09-10T11:35:01.572471+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572471+0000","last_undegraded":"2024-09-10T11:35:01.572471+0000","last_fullsized":"2024-09-10T11:35:01.572471+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576660+0000","last_change":"2024-09-10T11:35:01.576531+0000","last_active":"2024-09-10T11:35:01.576660+0000","last_peered":"2024-09-10T11:35:01.576660+0000","last_clean":"2024-09-10T11:35:01.576660+0000","last_became_active":"2024-09-10T11:35:01.575776+0000","last_became_peered":"2024-09-10T11:35:01.575776+0000","last_unstale":"2024-09-10T11:35:01.576660+0000","last_undegraded":"2024-09-10T11:35:01.576660+0000","last_fullsized":"2024-09-10T11:35:01.576660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:54.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572997+0000","last_change":"2024-09-10T11:35:01.572997+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572997+0000","last_undegraded":"2024-09-10T11:35:01.572997+0000","last_fullsized":"2024-09-10T11:35:01.572997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575004+0000","last_change":"2024-09-10T11:35:01.574933+0000","last_active":"2024-09-10T11:35:01.575004+0000","last_peered":"2024-09-10T11:35:01.575004+0000","last_clean":"2024-09-10T11:35:01.575004+0000","last_became_active":"2024-09-10T11:35:01.574408+0000","last_became_peered":"2024-09-10T11:35:01.574408+0000","last_unstale":"2024-09-10T11:35:01.575004+0000","last_undegraded":"2024-09-10T11:35:01.575004+0000","last_fullsized":"2024-09-10T11:35:01.575004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:45.538369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.565044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:01.565044+0000","last_peered":"2024-09-10T11:35:01.565044+0000","last_clean":"2024-09-10T11:35:01.565044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:01.565044+0000","last_undegraded":"2024-09-10T11:35:01.565044+0000","last_fullsized":"2024-09-10T11:35:01.565044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":636,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194676,"statfs":{"total":200000000,"available":199348411,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:05.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:05.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.975+0000 7f264d8ba640 1 Processor -- start 2024-09-10T11:35:05.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.975+0000 7f264d8ba640 1 -- start start 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f264d8ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26481626c0 0x7f26480d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f264d8ba640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264815ff70 con 0x7f264815eb80 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f264d8ba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26480d1620 con 0x7f26480d4e40 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f264d8ba640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f26480d30c0 con 0x7f26481626c0 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f263e7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26481626c0 0x7f26480d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f2646ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f264815eb80 0x7f26480d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45244/0 (socket says 172.21.15.99:45244) 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f263e7fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f26481626c0 0x7f26480d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44470/0 (socket says 172.21.15.99:44470) 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f2646ffd640 1 -- 172.21.15.99:0/3817699410 learned_addr learned my addr 172.21.15.99:0/3817699410 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f263e7fc640 1 -- 172.21.15.99:0/3817699410 >> v1:172.21.15.196:6789/0 conn(0x7f26480d4e40 legacy=0x7f26480d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:05.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f263e7fc640 1 -- 172.21.15.99:0/3817699410 >> v1:172.21.15.99:6789/0 conn(0x7f264815eb80 legacy=0x7f26480d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.976+0000 7f263e7fc640 1 -- 172.21.15.99:0/3817699410 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648162bc0 con 0x7f26481626c0 2024-09-10T11:35:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.977+0000 7f263e7fc640 1 --2- 172.21.15.99:0/3817699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26481626c0 0x7f26480d4900 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f26300097a0 tx=0x7f263002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1add65f70f2d7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.977+0000 7f26467fc640 1 -- 172.21.15.99:0/3817699410 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630037070 con 0x7f26481626c0 2024-09-10T11:35:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.977+0000 7f26467fc640 1 -- 172.21.15.99:0/3817699410 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263002ae50 con 0x7f26481626c0 2024-09-10T11:35:05.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.977+0000 7f26467fc640 1 -- 172.21.15.99:0/3817699410 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630032660 con 0x7f26481626c0 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 -- 172.21.15.99:0/3817699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26481626c0 msgr2=0x7f26480d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3817699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26481626c0 0x7f26480d4900 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f26300097a0 tx=0x7f263002a140 comp rx=0 tx=0).stop 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 -- 172.21.15.99:0/3817699410 shutdown_connections 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3817699410 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26481626c0 0x7f26480d4900 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 -- 172.21.15.99:0/3817699410 >> 172.21.15.99:0/3817699410 conn(0x7f264808e680 msgr2=0x7f264809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 -- 172.21.15.99:0/3817699410 shutdown_connections 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.978+0000 7f264d8ba640 1 -- 172.21.15.99:0/3817699410 wait complete. 2024-09-10T11:35:05.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.979+0000 7f264d8ba640 1 Processor -- start 2024-09-10T11:35:05.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.979+0000 7f264d8ba640 1 -- start start 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 0x7f2648157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26481626c0 0x7f26480ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2648163460 con 0x7f264815eb80 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2648163180 con 0x7f26481626c0 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f264d8ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f26480d5290 con 0x7f26480d4e40 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44480/0 (socket says 172.21.15.99:44480) 2024-09-10T11:35:05.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 -- 172.21.15.99:0/3633533061 learned_addr learned my addr 172.21.15.99:0/3633533061 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f2646ffd640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 0x7f2648157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f26477fe640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26481626c0 0x7f26480ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 -- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 msgr2=0x7f2648157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 0x7f2648157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 -- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26481626c0 msgr2=0x7f26480ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26481626c0 0x7f26480ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 -- 172.21.15.99:0/3633533061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26480b2db0 con 0x7f264815eb80 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f2646ffd640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 0x7f2648157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f263e7fc640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2630035040 tx=0x7f26300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.980+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630042070 con 0x7f264815eb80 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.981+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263003d650 con 0x7f264815eb80 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.981+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26480b1d10 con 0x7f264815eb80 2024-09-10T11:35:05.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.981+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630037040 con 0x7f264815eb80 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.981+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26480b22d0 con 0x7f264815eb80 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.982+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26480d0540 con 0x7f264815eb80 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.982+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2630045070 con 0x7f264815eb80 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.982+0000 7f2644ff9640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 0x7f262806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.983+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f26300ab560 con 0x7f264815eb80 2024-09-10T11:35:05.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.983+0000 7f2646ffd640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 0x7f262806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:05.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.983+0000 7f2646ffd640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 0x7f262806e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2638000f60 tx=0x7f2638009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:05.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:05.988+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26480d0540 con 0x7f264815eb80 2024-09-10T11:35:06.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.149+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26481577c0 con 0x7f262806be30 2024-09-10T11:35:06.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.154+0000 7f2644ff9640 1 -- 172.21.15.99:0/3633533061 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242639 (secure 0 0 0) 0x7f2638005890 con 0x7f262806be30 2024-09-10T11:35:06.159 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:06.160 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 msgr2=0x7f262806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 0x7f262806e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2638000f60 tx=0x7f2638009040 comp rx=0 tx=0).stop 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 msgr2=0x7f26480b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2630035040 tx=0x7f26300046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 shutdown_connections 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262806be30 0x7f262806e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26481626c0 0x7f26480ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f264815eb80 0x7f26480b3620 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 --2- 172.21.15.99:0/3633533061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f26480d4e40 0x7f2648157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 >> 172.21.15.99:0/3633533061 conn(0x7f264808e680 msgr2=0x7f26480b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:06.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 shutdown_connections 2024-09-10T11:35:06.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.157+0000 7f264d8ba640 1 -- 172.21.15.99:0/3633533061 wait complete. 2024-09-10T11:35:06.170 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T11:35:04.967283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556198,"statfs":{"total":1600000000,"available":1593547008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.411713"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565919+0000","last_change":"2024-09-10T11:35:01.565919+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565919+0000","last_undegraded":"2024-09-10T11:35:01.565919+0000","last_fullsized":"2024-09-10T11:35:01.565919+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568411+0000","last_change":"2024-09-10T11:35:01.568411+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568411+0000","last_undegraded":"2024-09-10T11:35:01.568411+0000","last_fullsized":"2024-09-10T11:35:01.568411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569657+0000","last_change":"2024-09-10T11:35:01.569657+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569657+0000","last_undegraded":"2024-09-10T11:35:01.569657+0000","last_fullsized":"2024-09-10T11:35:01.569657+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565761+0000","last_change":"2024-09-10T11:35:01.565761+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565761+0000","last_undegraded":"2024-09-10T11:35:01.565761+0000","last_fullsized":"2024-09-10T11:35:01.565761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566643+0000","last_change":"2024-09-10T11:35:01.566643+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566643+0000","last_undegraded":"2024-09-10T11:35:01.566643+0000","last_fullsized":"2024-09-10T11:35:01.566643+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569064+0000","last_change":"2024-09-10T11:35:01.569064+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569064+0000","last_undegraded":"2024-09-10T11:35:01.569064+0000","last_fullsized":"2024-09-10T11:35:01.569064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567844+0000","last_change":"2024-09-10T11:35:01.567844+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567844+0000","last_undegraded":"2024-09-10T11:35:01.567844+0000","last_fullsized":"2024-09-10T11:35:01.567844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569277+0000","last_change":"2024-09-10T11:35:01.569277+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569277+0000","last_undegraded":"2024-09-10T11:35:01.569277+0000","last_fullsized":"2024-09-10T11:35:01.569277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568457+0000","last_change":"2024-09-10T11:35:01.568457+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568457+0000","last_undegraded":"2024-09-10T11:35:01.568457+0000","last_fullsized":"2024-09-10T11:35:01.568457+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566245+0000","last_change":"2024-09-10T11:35:01.566245+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566245+0000","last_undegraded":"2024-09-10T11:35:01.566245+0000","last_fullsized":"2024-09-10T11:35:01.566245+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568738+0000","last_change":"2024-09-10T11:35:01.568738+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568738+0000","last_undegraded":"2024-09-10T11:35:01.568738+0000","last_fullsized":"2024-09-10T11:35:01.568738+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571097+0000","last_change":"2024-09-10T11:35:01.571097+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571097+0000","last_undegraded":"2024-09-10T11:35:01.571097+0000","last_fullsized":"2024-09-10T11:35:01.571097+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565950+0000","last_change":"2024-09-10T11:35:01.565950+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565950+0000","last_undegraded":"2024-09-10T11:35:01.565950+0000","last_fullsized":"2024-09-10T11:35:01.565950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565425+0000","last_change":"2024-09-10T11:35:01.565425+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565425+0000","last_undegraded":"2024-09-10T11:35:01.565425+0000","last_fullsized":"2024-09-10T11:35:01.565425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566167+0000","last_change":"2024-09-10T11:35:01.566167+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566167+0000","last_undegraded":"2024-09-10T11:35:01.566167+0000","last_fullsized":"2024-09-10T11:35:01.566167+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567990+0000","last_change":"2024-09-10T11:35:01.567990+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567990+0000","last_undegraded":"2024-09-10T11:35:01.567990+0000","last_fullsized":"2024-09-10T11:35:01.567990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569715+0000","last_change":"2024-09-10T11:35:01.569715+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569715+0000","last_undegraded":"2024-09-10T11:35:01.569715+0000","last_fullsized":"2024-09-10T11:35:01.569715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566911+0000","last_change":"2024-09-10T11:35:01.566911+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566911+0000","last_undegraded":"2024-09-10T11:35:01.566911+0000","last_fullsized":"2024-09-10T11:35:01.566911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572880+0000","last_change":"2024-09-10T11:35:01.572880+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572880+0000","last_undegraded":"2024-09-10T11:35:01.572880+0000","last_fullsized":"2024-09-10T11:35:01.572880+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571066+0000","last_change":"2024-09-10T11:35:01.571066+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571066+0000","last_undegraded":"2024-09-10T11:35:01.571066+0000","last_fullsized":"2024-09-10T11:35:01.571066+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573732+0000","last_change":"2024-09-10T11:35:01.573732+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573732+0000","last_undegraded":"2024-09-10T11:35:01.573732+0000","last_fullsized":"2024-09-10T11:35:01.573732+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569408+0000","last_change":"2024-09-10T11:35:01.569408+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569408+0000","last_undegraded":"2024-09-10T11:35:01.569408+0000","last_fullsized":"2024-09-10T11:35:01.569408+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566541+0000","last_change":"2024-09-10T11:35:01.566541+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566541+0000","last_undegraded":"2024-09-10T11:35:01.566541+0000","last_fullsized":"2024-09-10T11:35:01.566541+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567219+0000","last_change":"2024-09-10T11:35:01.567219+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567219+0000","last_undegraded":"2024-09-10T11:35:01.567219+0000","last_fullsized":"2024-09-10T11:35:01.567219+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565927+0000","last_change":"2024-09-10T11:35:01.565927+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565927+0000","last_undegraded":"2024-09-10T11:35:01.565927+0000","last_fullsized":"2024-09-10T11:35:01.565927+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564831+0000","last_change":"2024-09-10T11:35:01.564831+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564831+0000","last_undegraded":"2024-09-10T11:35:01.564831+0000","last_fullsized":"2024-09-10T11:35:01.564831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568745+0000","last_change":"2024-09-10T11:35:01.568745+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568745+0000","last_undegraded":"2024-09-10T11:35:01.568745+0000","last_fullsized":"2024-09-10T11:35:01.568745+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567687+0000","last_change":"2024-09-10T11:35:01.567687+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567687+0000","last_undegraded":"2024-09-10T11:35:01.567687+0000","last_fullsized":"2024-09-10T11:35:01.567687+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564599+0000","last_change":"2024-09-10T11:35:01.564599+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564599+0000","last_undegraded":"2024-09-10T11:35:01.564599+0000","last_fullsized":"2024-09-10T11:35:01.564599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565127+0000","last_change":"2024-09-10T11:35:01.565127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565127+0000","last_undegraded":"2024-09-10T11:35:01.565127+0000","last_fullsized":"2024-09-10T11:35:01.565127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572127+0000","last_change":"2024-09-10T11:35:01.572127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572127+0000","last_undegraded":"2024-09-10T11:35:01.572127+0000","last_fullsized":"2024-09-10T11:35:01.572127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564934+0000","last_change":"2024-09-10T11:35:01.564934+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564934+0000","last_undegraded":"2024-09-10T11:35:01.564934+0000","last_fullsized":"2024-09-10T11:35:01.564934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572594+0000","last_change":"2024-09-10T11:35:01.572594+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572594+0000","last_undegraded":"2024-09-10T11:35:01.572594+0000","last_fullsized":"2024-09-10T11:35:01.572594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574983+0000","last_change":"2024-09-10T11:35:01.574138+0000","last_active":"2024-09-10T11:35:01.574983+0000","last_peered":"2024-09-10T11:35:01.574983+0000","last_clean":"2024-09-10T11:35:01.574983+0000","last_became_active":"2024-09-10T11:35:01.573905+0000","last_became_peered":"2024-09-10T11:35:01.573905+0000","last_unstale":"2024-09-10T11:35:01.574983+0000","last_undegraded":"2024-09-10T11:35:01.574983+0000","last_fullsized":"2024-09-10T11:35:01.574983+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:13.165426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564878+0000","last_change":"2024-09-10T11:35:01.564878+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564878+0000","last_undegraded":"2024-09-10T11:35:01.564878+0000","last_fullsized":"2024-09-10T11:35:01.564878+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577412+0000","last_change":"2024-09-10T11:35:01.577311+0000","last_active":"2024-09-10T11:35:01.577412+0000","last_peered":"2024-09-10T11:35:01.577412+0000","last_clean":"2024-09-10T11:35:01.577412+0000","last_became_active":"2024-09-10T11:35:01.576942+0000","last_became_peered":"2024-09-10T11:35:01.576942+0000","last_unstale":"2024-09-10T11:35:01.577412+0000","last_undegraded":"2024-09-10T11:35:01.577412+0000","last_fullsized":"2024-09-10T11:35:01.577412+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:36.991927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565534+0000","last_change":"2024-09-10T11:35:01.565534+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565534+0000","last_undegraded":"2024-09-10T11:35:01.565534+0000","last_fullsized":"2024-09-10T11:35:01.565534+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579753+0000","last_change":"2024-09-10T11:35:01.579621+0000","last_active":"2024-09-10T11:35:01.579753+0000","last_peered":"2024-09-10T11:35:01.579753+0000","last_clean":"2024-09-10T11:35:01.579753+0000","last_became_active":"2024-09-10T11:35:01.579238+0000","last_became_peered":"2024-09-10T11:35:01.579238+0000","last_unstale":"2024-09-10T11:35:01.579753+0000","last_undegraded":"2024-09-10T11:35:01.579753+0000","last_fullsized":"2024-09-10T11:35:01.579753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:52.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565557+0000","last_change":"2024-09-10T11:35:01.565557+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565557+0000","last_undegraded":"2024-09-10T11:35:01.565557+0000","last_fullsized":"2024-09-10T11:35:01.565557+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.578494+0000","last_change":"2024-09-10T11:35:01.578383+0000","last_active":"2024-09-10T11:35:01.578494+0000","last_peered":"2024-09-10T11:35:01.578494+0000","last_clean":"2024-09-10T11:35:01.578494+0000","last_became_active":"2024-09-10T11:35:01.578109+0000","last_became_peered":"2024-09-10T11:35:01.578109+0000","last_unstale":"2024-09-10T11:35:01.578494+0000","last_undegraded":"2024-09-10T11:35:01.578494+0000","last_fullsized":"2024-09-10T11:35:01.578494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:18.486537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572758+0000","last_change":"2024-09-10T11:35:01.572758+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572758+0000","last_undegraded":"2024-09-10T11:35:01.572758+0000","last_fullsized":"2024-09-10T11:35:01.572758+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572226+0000","last_change":"2024-09-10T11:35:01.572113+0000","last_active":"2024-09-10T11:35:01.572226+0000","last_peered":"2024-09-10T11:35:01.572226+0000","last_clean":"2024-09-10T11:35:01.572226+0000","last_became_active":"2024-09-10T11:35:01.571874+0000","last_became_peered":"2024-09-10T11:35:01.571874+0000","last_unstale":"2024-09-10T11:35:01.572226+0000","last_undegraded":"2024-09-10T11:35:01.572226+0000","last_fullsized":"2024-09-10T11:35:01.572226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:29.218475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568170+0000","last_change":"2024-09-10T11:35:01.568170+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568170+0000","last_undegraded":"2024-09-10T11:35:01.568170+0000","last_fullsized":"2024-09-10T11:35:01.568170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576235+0000","last_change":"2024-09-10T11:35:01.576109+0000","last_active":"2024-09-10T11:35:01.576235+0000","last_peered":"2024-09-10T11:35:01.576235+0000","last_clean":"2024-09-10T11:35:01.576235+0000","last_became_active":"2024-09-10T11:35:01.575849+0000","last_became_peered":"2024-09-10T11:35:01.575849+0000","last_unstale":"2024-09-10T11:35:01.576235+0000","last_undegraded":"2024-09-10T11:35:01.576235+0000","last_fullsized":"2024-09-10T11:35:01.576235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:54.020713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564259+0000","last_change":"2024-09-10T11:35:01.564259+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564259+0000","last_undegraded":"2024-09-10T11:35:01.564259+0000","last_fullsized":"2024-09-10T11:35:01.564259+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575753+0000","last_change":"2024-09-10T11:35:01.575658+0000","last_active":"2024-09-10T11:35:01.575753+0000","last_peered":"2024-09-10T11:35:01.575753+0000","last_clean":"2024-09-10T11:35:01.575753+0000","last_became_active":"2024-09-10T11:35:01.575428+0000","last_became_peered":"2024-09-10T11:35:01.575428+0000","last_unstale":"2024-09-10T11:35:01.575753+0000","last_undegraded":"2024-09-10T11:35:01.575753+0000","last_fullsized":"2024-09-10T11:35:01.575753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:42.923450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571841+0000","last_change":"2024-09-10T11:35:01.571841+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571841+0000","last_undegraded":"2024-09-10T11:35:01.571841+0000","last_fullsized":"2024-09-10T11:35:01.571841+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579686+0000","last_change":"2024-09-10T11:35:01.579550+0000","last_active":"2024-09-10T11:35:01.579686+0000","last_peered":"2024-09-10T11:35:01.579686+0000","last_clean":"2024-09-10T11:35:01.579686+0000","last_became_active":"2024-09-10T11:35:01.579226+0000","last_became_peered":"2024-09-10T11:35:01.579226+0000","last_unstale":"2024-09-10T11:35:01.579686+0000","last_undegraded":"2024-09-10T11:35:01.579686+0000","last_fullsized":"2024-09-10T11:35:01.579686+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:02.892037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564550+0000","last_change":"2024-09-10T11:35:01.564550+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564550+0000","last_undegraded":"2024-09-10T11:35:01.564550+0000","last_fullsized":"2024-09-10T11:35:01.564550+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577256+0000","last_change":"2024-09-10T11:35:01.577132+0000","last_active":"2024-09-10T11:35:01.577256+0000","last_peered":"2024-09-10T11:35:01.577256+0000","last_clean":"2024-09-10T11:35:01.577256+0000","last_became_active":"2024-09-10T11:35:01.576572+0000","last_became_peered":"2024-09-10T11:35:01.576572+0000","last_unstale":"2024-09-10T11:35:01.577256+0000","last_undegraded":"2024-09-10T11:35:01.577256+0000","last_fullsized":"2024-09-10T11:35:01.577256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:31.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569057+0000","last_change":"2024-09-10T11:35:01.569057+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569057+0000","last_undegraded":"2024-09-10T11:35:01.569057+0000","last_fullsized":"2024-09-10T11:35:01.569057+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577971+0000","last_change":"2024-09-10T11:35:01.577634+0000","last_active":"2024-09-10T11:35:01.577971+0000","last_peered":"2024-09-10T11:35:01.577971+0000","last_clean":"2024-09-10T11:35:01.577971+0000","last_became_active":"2024-09-10T11:35:01.577375+0000","last_became_peered":"2024-09-10T11:35:01.577375+0000","last_unstale":"2024-09-10T11:35:01.577971+0000","last_undegraded":"2024-09-10T11:35:01.577971+0000","last_fullsized":"2024-09-10T11:35:01.577971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:59.758708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568150+0000","last_change":"2024-09-10T11:35:01.568150+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568150+0000","last_undegraded":"2024-09-10T11:35:01.568150+0000","last_fullsized":"2024-09-10T11:35:01.568150+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573571+0000","last_change":"2024-09-10T11:35:01.573449+0000","last_active":"2024-09-10T11:35:01.573571+0000","last_peered":"2024-09-10T11:35:01.573571+0000","last_clean":"2024-09-10T11:35:01.573571+0000","last_became_active":"2024-09-10T11:35:01.573208+0000","last_became_peered":"2024-09-10T11:35:01.573208+0000","last_unstale":"2024-09-10T11:35:01.573571+0000","last_undegraded":"2024-09-10T11:35:01.573571+0000","last_fullsized":"2024-09-10T11:35:01.573571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:55.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567343+0000","last_change":"2024-09-10T11:35:01.567343+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567343+0000","last_undegraded":"2024-09-10T11:35:01.567343+0000","last_fullsized":"2024-09-10T11:35:01.567343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575166+0000","last_change":"2024-09-10T11:35:01.575061+0000","last_active":"2024-09-10T11:35:01.575166+0000","last_peered":"2024-09-10T11:35:01.575166+0000","last_clean":"2024-09-10T11:35:01.575166+0000","last_became_active":"2024-09-10T11:35:01.574722+0000","last_became_peered":"2024-09-10T11:35:01.574722+0000","last_unstale":"2024-09-10T11:35:01.575166+0000","last_undegraded":"2024-09-10T11:35:01.575166+0000","last_fullsized":"2024-09-10T11:35:01.575166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:45.426600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565092+0000","last_change":"2024-09-10T11:35:01.565092+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565092+0000","last_undegraded":"2024-09-10T11:35:01.565092+0000","last_fullsized":"2024-09-10T11:35:01.565092+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577253+0000","last_change":"2024-09-10T11:35:01.576337+0000","last_active":"2024-09-10T11:35:01.577253+0000","last_peered":"2024-09-10T11:35:01.577253+0000","last_clean":"2024-09-10T11:35:01.577253+0000","last_became_active":"2024-09-10T11:35:01.576094+0000","last_became_peered":"2024-09-10T11:35:01.576094+0000","last_unstale":"2024-09-10T11:35:01.577253+0000","last_undegraded":"2024-09-10T11:35:01.577253+0000","last_fullsized":"2024-09-10T11:35:01.577253+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:18.030605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573292+0000","last_change":"2024-09-10T11:35:01.573292+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573292+0000","last_undegraded":"2024-09-10T11:35:01.573292+0000","last_fullsized":"2024-09-10T11:35:01.573292+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567570+0000","last_change":"2024-09-10T11:35:01.567436+0000","last_active":"2024-09-10T11:35:01.567570+0000","last_peered":"2024-09-10T11:35:01.567570+0000","last_clean":"2024-09-10T11:35:01.567570+0000","last_became_active":"2024-09-10T11:35:01.566612+0000","last_became_peered":"2024-09-10T11:35:01.566612+0000","last_unstale":"2024-09-10T11:35:01.567570+0000","last_undegraded":"2024-09-10T11:35:01.567570+0000","last_fullsized":"2024-09-10T11:35:01.567570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:12.054238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568196+0000","last_change":"2024-09-10T11:35:01.568196+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568196+0000","last_undegraded":"2024-09-10T11:35:01.568196+0000","last_fullsized":"2024-09-10T11:35:01.568196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571385+0000","last_change":"2024-09-10T11:35:01.571276+0000","last_active":"2024-09-10T11:35:01.571385+0000","last_peered":"2024-09-10T11:35:01.571385+0000","last_clean":"2024-09-10T11:35:01.571385+0000","last_became_active":"2024-09-10T11:35:01.570511+0000","last_became_peered":"2024-09-10T11:35:01.570511+0000","last_unstale":"2024-09-10T11:35:01.571385+0000","last_undegraded":"2024-09-10T11:35:01.571385+0000","last_fullsized":"2024-09-10T11:35:01.571385+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:07.622876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569769+0000","last_change":"2024-09-10T11:35:01.569769+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569769+0000","last_undegraded":"2024-09-10T11:35:01.569769+0000","last_fullsized":"2024-09-10T11:35:01.569769+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.569217+0000","last_change":"2024-09-10T11:35:01.569105+0000","last_active":"2024-09-10T11:35:01.569217+0000","last_peered":"2024-09-10T11:35:01.569217+0000","last_clean":"2024-09-10T11:35:01.569217+0000","last_became_active":"2024-09-10T11:35:01.568619+0000","last_became_peered":"2024-09-10T11:35:01.568619+0000","last_unstale":"2024-09-10T11:35:01.569217+0000","last_undegraded":"2024-09-10T11:35:01.569217+0000","last_fullsized":"2024-09-10T11:35:01.569217+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:41.912527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566883+0000","last_change":"2024-09-10T11:35:01.566883+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566883+0000","last_undegraded":"2024-09-10T11:35:01.566883+0000","last_fullsized":"2024-09-10T11:35:01.566883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574687+0000","last_change":"2024-09-10T11:35:01.574578+0000","last_active":"2024-09-10T11:35:01.574687+0000","last_peered":"2024-09-10T11:35:01.574687+0000","last_clean":"2024-09-10T11:35:01.574687+0000","last_became_active":"2024-09-10T11:35:01.574313+0000","last_became_peered":"2024-09-10T11:35:01.574313+0000","last_unstale":"2024-09-10T11:35:01.574687+0000","last_undegraded":"2024-09-10T11:35:01.574687+0000","last_fullsized":"2024-09-10T11:35:01.574687+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:33.710736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568446+0000","last_change":"2024-09-10T11:35:01.568446+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568446+0000","last_undegraded":"2024-09-10T11:35:01.568446+0000","last_fullsized":"2024-09-10T11:35:01.568446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.570614+0000","last_change":"2024-09-10T11:35:01.570511+0000","last_active":"2024-09-10T11:35:01.570614+0000","last_peered":"2024-09-10T11:35:01.570614+0000","last_clean":"2024-09-10T11:35:01.570614+0000","last_became_active":"2024-09-10T11:35:01.570268+0000","last_became_peered":"2024-09-10T11:35:01.570268+0000","last_unstale":"2024-09-10T11:35:01.570614+0000","last_undegraded":"2024-09-10T11:35:01.570614+0000","last_fullsized":"2024-09-10T11:35:01.570614+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:23.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571499+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571499+0000","last_undegraded":"2024-09-10T11:35:01.571499+0000","last_fullsized":"2024-09-10T11:35:01.571499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575650+0000","last_change":"2024-09-10T11:35:01.575527+0000","last_active":"2024-09-10T11:35:01.575650+0000","last_peered":"2024-09-10T11:35:01.575650+0000","last_clean":"2024-09-10T11:35:01.575650+0000","last_became_active":"2024-09-10T11:35:01.575285+0000","last_became_peered":"2024-09-10T11:35:01.575285+0000","last_unstale":"2024-09-10T11:35:01.575650+0000","last_undegraded":"2024-09-10T11:35:01.575650+0000","last_fullsized":"2024-09-10T11:35:01.575650+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:50.445819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571652+0000","last_change":"2024-09-10T11:35:01.571652+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571652+0000","last_undegraded":"2024-09-10T11:35:01.571652+0000","last_fullsized":"2024-09-10T11:35:01.571652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577157+0000","last_change":"2024-09-10T11:35:01.577040+0000","last_active":"2024-09-10T11:35:01.577157+0000","last_peered":"2024-09-10T11:35:01.577157+0000","last_clean":"2024-09-10T11:35:01.577157+0000","last_became_active":"2024-09-10T11:35:01.576667+0000","last_became_peered":"2024-09-10T11:35:01.576667+0000","last_unstale":"2024-09-10T11:35:01.577157+0000","last_undegraded":"2024-09-10T11:35:01.577157+0000","last_fullsized":"2024-09-10T11:35:01.577157+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:55.579500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571257+0000","last_change":"2024-09-10T11:35:01.571257+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571257+0000","last_undegraded":"2024-09-10T11:35:01.571257+0000","last_fullsized":"2024-09-10T11:35:01.571257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574162+0000","last_change":"2024-09-10T11:35:01.573987+0000","last_active":"2024-09-10T11:35:01.574162+0000","last_peered":"2024-09-10T11:35:01.574162+0000","last_clean":"2024-09-10T11:35:01.574162+0000","last_became_active":"2024-09-10T11:35:01.573756+0000","last_became_peered":"2024-09-10T11:35:01.573756+0000","last_unstale":"2024-09-10T11:35:01.574162+0000","last_undegraded":"2024-09-10T11:35:01.574162+0000","last_fullsized":"2024-09-10T11:35:01.574162+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:11.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564649+0000","last_change":"2024-09-10T11:35:01.564649+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564649+0000","last_undegraded":"2024-09-10T11:35:01.564649+0000","last_fullsized":"2024-09-10T11:35:01.564649+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567997+0000","last_change":"2024-09-10T11:35:01.567874+0000","last_active":"2024-09-10T11:35:01.567997+0000","last_peered":"2024-09-10T11:35:01.567997+0000","last_clean":"2024-09-10T11:35:01.567997+0000","last_became_active":"2024-09-10T11:35:01.567649+0000","last_became_peered":"2024-09-10T11:35:01.567649+0000","last_unstale":"2024-09-10T11:35:01.567997+0000","last_undegraded":"2024-09-10T11:35:01.567997+0000","last_fullsized":"2024-09-10T11:35:01.567997+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:58.144492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566977+0000","last_change":"2024-09-10T11:35:01.566977+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566977+0000","last_undegraded":"2024-09-10T11:35:01.566977+0000","last_fullsized":"2024-09-10T11:35:01.566977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576998+0000","last_change":"2024-09-10T11:35:01.576843+0000","last_active":"2024-09-10T11:35:01.576998+0000","last_peered":"2024-09-10T11:35:01.576998+0000","last_clean":"2024-09-10T11:35:01.576998+0000","last_became_active":"2024-09-10T11:35:01.576103+0000","last_became_peered":"2024-09-10T11:35:01.576103+0000","last_unstale":"2024-09-10T11:35:01.576998+0000","last_undegraded":"2024-09-10T11:35:01.576998+0000","last_fullsized":"2024-09-10T11:35:01.576998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.717940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.570853+0000","last_change":"2024-09-10T11:35:01.570853+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.570853+0000","last_undegraded":"2024-09-10T11:35:01.570853+0000","last_fullsized":"2024-09-10T11:35:01.570853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571060+0000","last_change":"2024-09-10T11:35:01.570746+0000","last_active":"2024-09-10T11:35:01.571060+0000","last_peered":"2024-09-10T11:35:01.571060+0000","last_clean":"2024-09-10T11:35:01.571060+0000","last_became_active":"2024-09-10T11:35:01.570187+0000","last_became_peered":"2024-09-10T11:35:01.570187+0000","last_unstale":"2024-09-10T11:35:01.571060+0000","last_undegraded":"2024-09-10T11:35:01.571060+0000","last_fullsized":"2024-09-10T11:35:01.571060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:08.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569349+0000","last_change":"2024-09-10T11:35:01.569349+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569349+0000","last_undegraded":"2024-09-10T11:35:01.569349+0000","last_fullsized":"2024-09-10T11:35:01.569349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573953+0000","last_change":"2024-09-10T11:35:01.573744+0000","last_active":"2024-09-10T11:35:01.573953+0000","last_peered":"2024-09-10T11:35:01.573953+0000","last_clean":"2024-09-10T11:35:01.573953+0000","last_became_active":"2024-09-10T11:35:01.573463+0000","last_became_peered":"2024-09-10T11:35:01.573463+0000","last_unstale":"2024-09-10T11:35:01.573953+0000","last_undegraded":"2024-09-10T11:35:01.573953+0000","last_fullsized":"2024-09-10T11:35:01.573953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.610059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572681+0000","last_change":"2024-09-10T11:35:01.572681+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572681+0000","last_undegraded":"2024-09-10T11:35:01.572681+0000","last_fullsized":"2024-09-10T11:35:01.572681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579934+0000","last_change":"2024-09-10T11:35:01.579801+0000","last_active":"2024-09-10T11:35:01.579934+0000","last_peered":"2024-09-10T11:35:01.579934+0000","last_clean":"2024-09-10T11:35:01.579934+0000","last_became_active":"2024-09-10T11:35:01.578207+0000","last_became_peered":"2024-09-10T11:35:01.578207+0000","last_unstale":"2024-09-10T11:35:01.579934+0000","last_undegraded":"2024-09-10T11:35:01.579934+0000","last_fullsized":"2024-09-10T11:35:01.579934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:36.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567016+0000","last_change":"2024-09-10T11:35:01.567016+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567016+0000","last_undegraded":"2024-09-10T11:35:01.567016+0000","last_fullsized":"2024-09-10T11:35:01.567016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572601+0000","last_change":"2024-09-10T11:35:01.572125+0000","last_active":"2024-09-10T11:35:01.572601+0000","last_peered":"2024-09-10T11:35:01.572601+0000","last_clean":"2024-09-10T11:35:01.572601+0000","last_became_active":"2024-09-10T11:35:01.571887+0000","last_became_peered":"2024-09-10T11:35:01.571887+0000","last_unstale":"2024-09-10T11:35:01.572601+0000","last_undegraded":"2024-09-10T11:35:01.572601+0000","last_fullsized":"2024-09-10T11:35:01.572601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:09.224575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568970+0000","last_change":"2024-09-10T11:35:01.568970+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568970+0000","last_undegraded":"2024-09-10T11:35:01.568970+0000","last_fullsized":"2024-09-10T11:35:01.568970+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579086+0000","last_change":"2024-09-10T11:35:01.578976+0000","last_active":"2024-09-10T11:35:01.579086+0000","last_peered":"2024-09-10T11:35:01.579086+0000","last_clean":"2024-09-10T11:35:01.579086+0000","last_became_active":"2024-09-10T11:35:01.578755+0000","last_became_peered":"2024-09-10T11:35:01.578755+0000","last_unstale":"2024-09-10T11:35:01.579086+0000","last_undegraded":"2024-09-10T11:35:01.579086+0000","last_fullsized":"2024-09-10T11:35:01.579086+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:48.512747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567653+0000","last_change":"2024-09-10T11:35:01.567653+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567653+0000","last_undegraded":"2024-09-10T11:35:01.567653+0000","last_fullsized":"2024-09-10T11:35:01.567653+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571622+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.571622+0000","last_peered":"2024-09-10T11:35:01.571622+0000","last_clean":"2024-09-10T11:35:01.571622+0000","last_became_active":"2024-09-10T11:35:01.571267+0000","last_became_peered":"2024-09-10T11:35:01.571267+0000","last_unstale":"2024-09-10T11:35:01.571622+0000","last_undegraded":"2024-09-10T11:35:01.571622+0000","last_fullsized":"2024-09-10T11:35:01.571622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:47.651348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571015+0000","last_change":"2024-09-10T11:35:01.571015+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571015+0000","last_undegraded":"2024-09-10T11:35:01.571015+0000","last_fullsized":"2024-09-10T11:35:01.571015+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572739+0000","last_change":"2024-09-10T11:35:01.570370+0000","last_active":"2024-09-10T11:35:01.572739+0000","last_peered":"2024-09-10T11:35:01.572739+0000","last_clean":"2024-09-10T11:35:01.572739+0000","last_became_active":"2024-09-10T11:35:01.570120+0000","last_became_peered":"2024-09-10T11:35:01.570120+0000","last_unstale":"2024-09-10T11:35:01.572739+0000","last_undegraded":"2024-09-10T11:35:01.572739+0000","last_fullsized":"2024-09-10T11:35:01.572739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:10.126918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572471+0000","last_change":"2024-09-10T11:35:01.572471+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572471+0000","last_undegraded":"2024-09-10T11:35:01.572471+0000","last_fullsized":"2024-09-10T11:35:01.572471+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576660+0000","last_change":"2024-09-10T11:35:01.576531+0000","last_active":"2024-09-10T11:35:01.576660+0000","last_peered":"2024-09-10T11:35:01.576660+0000","last_clean":"2024-09-10T11:35:01.576660+0000","last_became_active":"2024-09-10T11:35:01.575776+0000","last_became_peered":"2024-09-10T11:35:01.575776+0000","last_unstale":"2024-09-10T11:35:01.576660+0000","last_undegraded":"2024-09-10T11:35:01.576660+0000","last_fullsized":"2024-09-10T11:35:01.576660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:54.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572997+0000","last_change":"2024-09-10T11:35:01.572997+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572997+0000","last_undegraded":"2024-09-10T11:35:01.572997+0000","last_fullsized":"2024-09-10T11:35:01.572997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575004+0000","last_change":"2024-09-10T11:35:01.574933+0000","last_active":"2024-09-10T11:35:01.575004+0000","last_peered":"2024-09-10T11:35:01.575004+0000","last_clean":"2024-09-10T11:35:01.575004+0000","last_became_active":"2024-09-10T11:35:01.574408+0000","last_became_peered":"2024-09-10T11:35:01.574408+0000","last_unstale":"2024-09-10T11:35:01.575004+0000","last_undegraded":"2024-09-10T11:35:01.575004+0000","last_fullsized":"2024-09-10T11:35:01.575004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:45.538369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.565044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:01.565044+0000","last_peered":"2024-09-10T11:35:01.565044+0000","last_clean":"2024-09-10T11:35:01.565044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:01.565044+0000","last_undegraded":"2024-09-10T11:35:01.565044+0000","last_fullsized":"2024-09-10T11:35:01.565044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":636,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194676,"statfs":{"total":200000000,"available":199348411,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:06.173 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3f is not active+clean 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565919+0000', 'last_change': '2024-09-10T11:35:01.565919+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565919+0000', 'last_undegraded': '2024-09-10T11:35:01.565919+0000', 'last_fullsized': '2024-09-10T11:35:01.565919+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3e is not active+clean 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568411+0000', 'last_change': '2024-09-10T11:35:01.568411+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568411+0000', 'last_undegraded': '2024-09-10T11:35:01.568411+0000', 'last_fullsized': '2024-09-10T11:35:01.568411+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3d is not active+clean 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569657+0000', 'last_change': '2024-09-10T11:35:01.569657+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569657+0000', 'last_undegraded': '2024-09-10T11:35:01.569657+0000', 'last_fullsized': '2024-09-10T11:35:01.569657+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3c is not active+clean 2024-09-10T11:35:06.174 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565761+0000', 'last_change': '2024-09-10T11:35:01.565761+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565761+0000', 'last_undegraded': '2024-09-10T11:35:01.565761+0000', 'last_fullsized': '2024-09-10T11:35:01.565761+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3b is not active+clean 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566643+0000', 'last_change': '2024-09-10T11:35:01.566643+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566643+0000', 'last_undegraded': '2024-09-10T11:35:01.566643+0000', 'last_fullsized': '2024-09-10T11:35:01.566643+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3a is not active+clean 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569064+0000', 'last_change': '2024-09-10T11:35:01.569064+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569064+0000', 'last_undegraded': '2024-09-10T11:35:01.569064+0000', 'last_fullsized': '2024-09-10T11:35:01.569064+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.39 is not active+clean 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567844+0000', 'last_change': '2024-09-10T11:35:01.567844+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567844+0000', 'last_undegraded': '2024-09-10T11:35:01.567844+0000', 'last_fullsized': '2024-09-10T11:35:01.567844+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.38 is not active+clean 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569277+0000', 'last_change': '2024-09-10T11:35:01.569277+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569277+0000', 'last_undegraded': '2024-09-10T11:35:01.569277+0000', 'last_fullsized': '2024-09-10T11:35:01.569277+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.37 is not active+clean 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568457+0000', 'last_change': '2024-09-10T11:35:01.568457+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568457+0000', 'last_undegraded': '2024-09-10T11:35:01.568457+0000', 'last_fullsized': '2024-09-10T11:35:01.568457+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.175 INFO:tasks.ceph.ceph_manager.ceph:PG 15.36 is not active+clean 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566245+0000', 'last_change': '2024-09-10T11:35:01.566245+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566245+0000', 'last_undegraded': '2024-09-10T11:35:01.566245+0000', 'last_fullsized': '2024-09-10T11:35:01.566245+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:PG 15.35 is not active+clean 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568738+0000', 'last_change': '2024-09-10T11:35:01.568738+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568738+0000', 'last_undegraded': '2024-09-10T11:35:01.568738+0000', 'last_fullsized': '2024-09-10T11:35:01.568738+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:PG 15.34 is not active+clean 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571097+0000', 'last_change': '2024-09-10T11:35:01.571097+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571097+0000', 'last_undegraded': '2024-09-10T11:35:01.571097+0000', 'last_fullsized': '2024-09-10T11:35:01.571097+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:PG 15.33 is not active+clean 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565950+0000', 'last_change': '2024-09-10T11:35:01.565950+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565950+0000', 'last_undegraded': '2024-09-10T11:35:01.565950+0000', 'last_fullsized': '2024-09-10T11:35:01.565950+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:PG 15.32 is not active+clean 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565425+0000', 'last_change': '2024-09-10T11:35:01.565425+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565425+0000', 'last_undegraded': '2024-09-10T11:35:01.565425+0000', 'last_fullsized': '2024-09-10T11:35:01.565425+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.176 INFO:tasks.ceph.ceph_manager.ceph:PG 15.31 is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566167+0000', 'last_change': '2024-09-10T11:35:01.566167+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566167+0000', 'last_undegraded': '2024-09-10T11:35:01.566167+0000', 'last_fullsized': '2024-09-10T11:35:01.566167+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:PG 15.30 is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567990+0000', 'last_change': '2024-09-10T11:35:01.567990+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567990+0000', 'last_undegraded': '2024-09-10T11:35:01.567990+0000', 'last_fullsized': '2024-09-10T11:35:01.567990+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2f is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569715+0000', 'last_change': '2024-09-10T11:35:01.569715+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569715+0000', 'last_undegraded': '2024-09-10T11:35:01.569715+0000', 'last_fullsized': '2024-09-10T11:35:01.569715+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2e is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566911+0000', 'last_change': '2024-09-10T11:35:01.566911+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566911+0000', 'last_undegraded': '2024-09-10T11:35:01.566911+0000', 'last_fullsized': '2024-09-10T11:35:01.566911+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572880+0000', 'last_change': '2024-09-10T11:35:01.572880+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572880+0000', 'last_undegraded': '2024-09-10T11:35:01.572880+0000', 'last_fullsized': '2024-09-10T11:35:01.572880+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2c is not active+clean 2024-09-10T11:35:06.177 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571066+0000', 'last_change': '2024-09-10T11:35:01.571066+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571066+0000', 'last_undegraded': '2024-09-10T11:35:01.571066+0000', 'last_fullsized': '2024-09-10T11:35:01.571066+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.573732+0000', 'last_change': '2024-09-10T11:35:01.573732+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.573732+0000', 'last_undegraded': '2024-09-10T11:35:01.573732+0000', 'last_fullsized': '2024-09-10T11:35:01.573732+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2a is not active+clean 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569408+0000', 'last_change': '2024-09-10T11:35:01.569408+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569408+0000', 'last_undegraded': '2024-09-10T11:35:01.569408+0000', 'last_fullsized': '2024-09-10T11:35:01.569408+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:PG 15.29 is not active+clean 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566541+0000', 'last_change': '2024-09-10T11:35:01.566541+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566541+0000', 'last_undegraded': '2024-09-10T11:35:01.566541+0000', 'last_fullsized': '2024-09-10T11:35:01.566541+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:PG 15.28 is not active+clean 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567219+0000', 'last_change': '2024-09-10T11:35:01.567219+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567219+0000', 'last_undegraded': '2024-09-10T11:35:01.567219+0000', 'last_fullsized': '2024-09-10T11:35:01.567219+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:PG 15.27 is not active+clean 2024-09-10T11:35:06.178 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565927+0000', 'last_change': '2024-09-10T11:35:01.565927+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565927+0000', 'last_undegraded': '2024-09-10T11:35:01.565927+0000', 'last_fullsized': '2024-09-10T11:35:01.565927+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:PG 15.26 is not active+clean 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564831+0000', 'last_change': '2024-09-10T11:35:01.564831+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564831+0000', 'last_undegraded': '2024-09-10T11:35:01.564831+0000', 'last_fullsized': '2024-09-10T11:35:01.564831+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:PG 15.25 is not active+clean 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568745+0000', 'last_change': '2024-09-10T11:35:01.568745+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568745+0000', 'last_undegraded': '2024-09-10T11:35:01.568745+0000', 'last_fullsized': '2024-09-10T11:35:01.568745+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:PG 15.24 is not active+clean 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567687+0000', 'last_change': '2024-09-10T11:35:01.567687+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567687+0000', 'last_undegraded': '2024-09-10T11:35:01.567687+0000', 'last_fullsized': '2024-09-10T11:35:01.567687+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:PG 15.23 is not active+clean 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564599+0000', 'last_change': '2024-09-10T11:35:01.564599+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564599+0000', 'last_undegraded': '2024-09-10T11:35:01.564599+0000', 'last_fullsized': '2024-09-10T11:35:01.564599+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:PG 15.22 is not active+clean 2024-09-10T11:35:06.179 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565127+0000', 'last_change': '2024-09-10T11:35:01.565127+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565127+0000', 'last_undegraded': '2024-09-10T11:35:01.565127+0000', 'last_fullsized': '2024-09-10T11:35:01.565127+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.21 is not active+clean 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572127+0000', 'last_change': '2024-09-10T11:35:01.572127+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572127+0000', 'last_undegraded': '2024-09-10T11:35:01.572127+0000', 'last_fullsized': '2024-09-10T11:35:01.572127+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.20 is not active+clean 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564934+0000', 'last_change': '2024-09-10T11:35:01.564934+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564934+0000', 'last_undegraded': '2024-09-10T11:35:01.564934+0000', 'last_fullsized': '2024-09-10T11:35:01.564934+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572594+0000', 'last_change': '2024-09-10T11:35:01.572594+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572594+0000', 'last_undegraded': '2024-09-10T11:35:01.572594+0000', 'last_fullsized': '2024-09-10T11:35:01.572594+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1f is not active+clean 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564878+0000', 'last_change': '2024-09-10T11:35:01.564878+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564878+0000', 'last_undegraded': '2024-09-10T11:35:01.564878+0000', 'last_fullsized': '2024-09-10T11:35:01.564878+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1c is not active+clean 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565534+0000', 'last_change': '2024-09-10T11:35:01.565534+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565534+0000', 'last_undegraded': '2024-09-10T11:35:01.565534+0000', 'last_fullsized': '2024-09-10T11:35:01.565534+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.180 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1d is not active+clean 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565557+0000', 'last_change': '2024-09-10T11:35:01.565557+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565557+0000', 'last_undegraded': '2024-09-10T11:35:01.565557+0000', 'last_fullsized': '2024-09-10T11:35:01.565557+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1a is not active+clean 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572758+0000', 'last_change': '2024-09-10T11:35:01.572758+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572758+0000', 'last_undegraded': '2024-09-10T11:35:01.572758+0000', 'last_fullsized': '2024-09-10T11:35:01.572758+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1b is not active+clean 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568170+0000', 'last_change': '2024-09-10T11:35:01.568170+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568170+0000', 'last_undegraded': '2024-09-10T11:35:01.568170+0000', 'last_fullsized': '2024-09-10T11:35:01.568170+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:PG 15.18 is not active+clean 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564259+0000', 'last_change': '2024-09-10T11:35:01.564259+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564259+0000', 'last_undegraded': '2024-09-10T11:35:01.564259+0000', 'last_fullsized': '2024-09-10T11:35:01.564259+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:PG 15.19 is not active+clean 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571841+0000', 'last_change': '2024-09-10T11:35:01.571841+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571841+0000', 'last_undegraded': '2024-09-10T11:35:01.571841+0000', 'last_fullsized': '2024-09-10T11:35:01.571841+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.181 INFO:tasks.ceph.ceph_manager.ceph:PG 15.16 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564550+0000', 'last_change': '2024-09-10T11:35:01.564550+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564550+0000', 'last_undegraded': '2024-09-10T11:35:01.564550+0000', 'last_fullsized': '2024-09-10T11:35:01.564550+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:PG 15.17 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569057+0000', 'last_change': '2024-09-10T11:35:01.569057+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569057+0000', 'last_undegraded': '2024-09-10T11:35:01.569057+0000', 'last_fullsized': '2024-09-10T11:35:01.569057+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:PG 15.14 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568150+0000', 'last_change': '2024-09-10T11:35:01.568150+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568150+0000', 'last_undegraded': '2024-09-10T11:35:01.568150+0000', 'last_fullsized': '2024-09-10T11:35:01.568150+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:PG 15.15 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567343+0000', 'last_change': '2024-09-10T11:35:01.567343+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567343+0000', 'last_undegraded': '2024-09-10T11:35:01.567343+0000', 'last_fullsized': '2024-09-10T11:35:01.567343+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:PG 15.12 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.565092+0000', 'last_change': '2024-09-10T11:35:01.565092+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.565092+0000', 'last_undegraded': '2024-09-10T11:35:01.565092+0000', 'last_fullsized': '2024-09-10T11:35:01.565092+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-09-10T11:35:06.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.573292+0000', 'last_change': '2024-09-10T11:35:01.573292+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.573292+0000', 'last_undegraded': '2024-09-10T11:35:01.573292+0000', 'last_fullsized': '2024-09-10T11:35:01.573292+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.10 is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568196+0000', 'last_change': '2024-09-10T11:35:01.568196+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568196+0000', 'last_undegraded': '2024-09-10T11:35:01.568196+0000', 'last_fullsized': '2024-09-10T11:35:01.568196+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569769+0000', 'last_change': '2024-09-10T11:35:01.569769+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569769+0000', 'last_undegraded': '2024-09-10T11:35:01.569769+0000', 'last_fullsized': '2024-09-10T11:35:01.569769+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.f is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566883+0000', 'last_change': '2024-09-10T11:35:01.566883+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566883+0000', 'last_undegraded': '2024-09-10T11:35:01.566883+0000', 'last_fullsized': '2024-09-10T11:35:01.566883+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.c is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568446+0000', 'last_change': '2024-09-10T11:35:01.568446+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568446+0000', 'last_undegraded': '2024-09-10T11:35:01.568446+0000', 'last_fullsized': '2024-09-10T11:35:01.568446+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.d is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571499+0000', 'last_change': '2024-09-10T11:35:01.571499+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571499+0000', 'last_undegraded': '2024-09-10T11:35:01.571499+0000', 'last_fullsized': '2024-09-10T11:35:01.571499+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-09-10T11:35:06.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571652+0000', 'last_change': '2024-09-10T11:35:01.571652+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571652+0000', 'last_undegraded': '2024-09-10T11:35:01.571652+0000', 'last_fullsized': '2024-09-10T11:35:01.571652+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571257+0000', 'last_change': '2024-09-10T11:35:01.571257+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571257+0000', 'last_undegraded': '2024-09-10T11:35:01.571257+0000', 'last_fullsized': '2024-09-10T11:35:01.571257+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.8 is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.564649+0000', 'last_change': '2024-09-10T11:35:01.564649+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.564649+0000', 'last_undegraded': '2024-09-10T11:35:01.564649+0000', 'last_fullsized': '2024-09-10T11:35:01.564649+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.9 is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.566977+0000', 'last_change': '2024-09-10T11:35:01.566977+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.566977+0000', 'last_undegraded': '2024-09-10T11:35:01.566977+0000', 'last_fullsized': '2024-09-10T11:35:01.566977+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.6 is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.570853+0000', 'last_change': '2024-09-10T11:35:01.570853+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.570853+0000', 'last_undegraded': '2024-09-10T11:35:01.570853+0000', 'last_fullsized': '2024-09-10T11:35:01.570853+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.569349+0000', 'last_change': '2024-09-10T11:35:01.569349+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.569349+0000', 'last_undegraded': '2024-09-10T11:35:01.569349+0000', 'last_fullsized': '2024-09-10T11:35:01.569349+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-09-10T11:35:06.184 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572681+0000', 'last_change': '2024-09-10T11:35:01.572681+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572681+0000', 'last_undegraded': '2024-09-10T11:35:01.572681+0000', 'last_fullsized': '2024-09-10T11:35:01.572681+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:PG 15.5 is not active+clean 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567016+0000', 'last_change': '2024-09-10T11:35:01.567016+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567016+0000', 'last_undegraded': '2024-09-10T11:35:01.567016+0000', 'last_fullsized': '2024-09-10T11:35:01.567016+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.568970+0000', 'last_change': '2024-09-10T11:35:01.568970+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.568970+0000', 'last_undegraded': '2024-09-10T11:35:01.568970+0000', 'last_fullsized': '2024-09-10T11:35:01.568970+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3 is not active+clean 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.567653+0000', 'last_change': '2024-09-10T11:35:01.567653+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.567653+0000', 'last_undegraded': '2024-09-10T11:35:01.567653+0000', 'last_fullsized': '2024-09-10T11:35:01.567653+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.571015+0000', 'last_change': '2024-09-10T11:35:01.571015+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.571015+0000', 'last_undegraded': '2024-09-10T11:35:01.571015+0000', 'last_fullsized': '2024-09-10T11:35:01.571015+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1 is not active+clean 2024-09-10T11:35:06.185 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572471+0000', 'last_change': '2024-09-10T11:35:01.572471+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572471+0000', 'last_undegraded': '2024-09-10T11:35:01.572471+0000', 'last_fullsized': '2024-09-10T11:35:01.572471+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T11:35:06.186 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-09-10T11:35:06.186 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 121, 'state': 'creating+peering', 'last_fresh': '2024-09-10T11:35:01.572997+0000', 'last_change': '2024-09-10T11:35:01.572997+0000', 'last_active': '2024-09-10T11:35:01.555567+0000', 'last_peered': '2024-09-10T11:35:01.555567+0000', 'last_clean': '2024-09-10T11:35:01.555567+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T11:35:01.572997+0000', 'last_undegraded': '2024-09-10T11:35:01.572997+0000', 'last_fullsized': '2024-09-10T11:35:01.572997+0000', 'mapping_epoch': 121, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 121, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'last_clean_scrub_stamp': '2024-09-10T11:35:01.555567+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T11:35:06.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T11:35:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.306+0000 7f818b480640 1 Processor -- start 2024-09-10T11:35:06.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.306+0000 7f818b480640 1 -- start start 2024-09-10T11:35:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f818b480640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f818b480640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81840b3960 0x7f81840d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f818b480640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81840b3620 con 0x7f818406cd20 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f818b480640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f818406d960 con 0x7f81840b3960 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f818b480640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f818406dc40 con 0x7f8184163580 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f8188a1b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81840b3960 0x7f81840d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f8188a1b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81840b3960 0x7f81840d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44486/0 (socket says 172.21.15.99:44486) 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.307+0000 7f8188a1b640 1 -- 172.21.15.99:0/3389014927 learned_addr learned my addr 172.21.15.99:0/3389014927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f8188a1b640 1 -- 172.21.15.99:0/3389014927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 msgr2=0x7f81840b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f8188a1b640 1 --2- 172.21.15.99:0/3389014927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f8188a1b640 1 -- 172.21.15.99:0/3389014927 >> v1:172.21.15.196:6789/0 conn(0x7f818406cd20 legacy=0x7f818406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:06.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f8188a1b640 1 -- 172.21.15.99:0/3389014927 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818406df20 con 0x7f81840b3960 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f8188a1b640 1 --2- 172.21.15.99:0/3389014927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840d1be0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8174009870 tx=0x7f817402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce16b1b42f81ddea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f817bfff640 1 -- 172.21.15.99:0/3389014927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174037070 con 0x7f81840b3960 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f817bfff640 1 -- 172.21.15.99:0/3389014927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8174032440 con 0x7f81840b3960 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.308+0000 7f817bfff640 1 -- 172.21.15.99:0/3389014927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174032810 con 0x7f81840b3960 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 -- 172.21.15.99:0/3389014927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 msgr2=0x7f81840d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 --2- 172.21.15.99:0/3389014927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840d1be0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8174009870 tx=0x7f817402a180 comp rx=0 tx=0).stop 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 -- 172.21.15.99:0/3389014927 shutdown_connections 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 --2- 172.21.15.99:0/3389014927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840d1be0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 --2- 172.21.15.99:0/3389014927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 -- 172.21.15.99:0/3389014927 >> 172.21.15.99:0/3389014927 conn(0x7f81840b4570 msgr2=0x7f81840b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 -- 172.21.15.99:0/3389014927 shutdown_connections 2024-09-10T11:35:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.309+0000 7f818b480640 1 -- 172.21.15.99:0/3389014927 wait complete. 2024-09-10T11:35:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.310+0000 7f818b480640 1 Processor -- start 2024-09-10T11:35:06.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.310+0000 7f818b480640 1 -- start start 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818406cd20 0x7f81840cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81840b3160 con 0x7f81840b3960 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81840d2120 con 0x7f818406cd20 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818b480640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f81840b32e0 con 0x7f8184163580 2024-09-10T11:35:06.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44490/0 (socket says 172.21.15.99:44490) 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 -- 172.21.15.99:0/2804570869 learned_addr learned my addr 172.21.15.99:0/2804570869 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f818921c640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818406cd20 0x7f81840cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8189a1d640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 -- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 msgr2=0x7f81840b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 -- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818406cd20 msgr2=0x7f81840cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818406cd20 0x7f81840cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.311+0000 7f8188a1b640 1 -- 172.21.15.99:0/2804570869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81840b72e0 con 0x7f81840b3960 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f8188a1b640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8174000c00 tx=0x7f81740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174041070 con 0x7f81840b3960 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f817403c870 con 0x7f81840b3960 2024-09-10T11:35:06.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81840b75c0 con 0x7f81840b3960 2024-09-10T11:35:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174037040 con 0x7f81840b3960 2024-09-10T11:35:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.312+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81840b7b80 con 0x7f81840b3960 2024-09-10T11:35:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.313+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81840761a0 con 0x7f81840b3960 2024-09-10T11:35:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.313+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8174044070 con 0x7f81840b3960 2024-09-10T11:35:06.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.314+0000 7f8179ffb640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 0x7f816806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.314+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f81740ab970 con 0x7f81840b3960 2024-09-10T11:35:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.314+0000 7f818921c640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 0x7f816806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.314+0000 7f818921c640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 0x7f816806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8180009890 tx=0x7f8180008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.318+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f817407a440 con 0x7f81840b3960 2024-09-10T11:35:06.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.550+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f81840d1be0 con 0x7f81840b3960 2024-09-10T11:35:06.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.550+0000 7f8179ffb640 1 -- 172.21.15.99:0/2804570869 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1490 (secure 0 0 0) 0x7f8174079d40 con 0x7f81840b3960 2024-09-10T11:35:06.551 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:06.551 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":123,"num_osds":8,"num_up_osds":8,"osd_up_since":1725967815,"num_in_osds":8,"osd_in_since":1725967812,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":6452992,"bytes_avail":1593547008,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":6594}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T11:34:52.965309+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T11:35:06.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 msgr2=0x7f816806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 0x7f816806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8180009890 tx=0x7f8180008040 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 msgr2=0x7f81840b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8174000c00 tx=0x7f81740046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 shutdown_connections 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f816806be30 0x7f816806e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8184163580 0x7f81840b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81840b3960 0x7f81840b6bb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 --2- 172.21.15.99:0/2804570869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f818406cd20 0x7f81840cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.552+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 >> 172.21.15.99:0/2804570869 conn(0x7f81840b4570 msgr2=0x7f8184070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.553+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 shutdown_connections 2024-09-10T11:35:06.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.553+0000 7f818b480640 1 -- 172.21.15.99:0/2804570869 wait complete. 2024-09-10T11:35:06.564 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T11:35:06.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:06.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.662+0000 7f734bed7640 1 Processor -- start 2024-09-10T11:35:06.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.662+0000 7f734bed7640 1 -- start start 2024-09-10T11:35:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734bed7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7344163620 0x7f7344160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734bed7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73440b4930 0x7f734406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734bed7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73440b5140 con 0x7f734406bc10 2024-09-10T11:35:06.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734bed7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f73440b5420 con 0x7f73440b4930 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734bed7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7344162b40 con 0x7f7344163620 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734a474640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73440b4930 0x7f734406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f7349c73640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f734406bc10 0x7f734406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45252/0 (socket says 172.21.15.99:45252) 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f7349c73640 1 -- 172.21.15.99:0/1709671976 learned_addr learned my addr 172.21.15.99:0/1709671976 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734a474640 1 -- 172.21.15.99:0/1709671976 >> v2:172.21.15.196:3300/0 conn(0x7f7344163620 msgr2=0x7f7344160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734a474640 1 --2- 172.21.15.99:0/1709671976 >> v2:172.21.15.196:3300/0 conn(0x7f7344163620 0x7f7344160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734a474640 1 -- 172.21.15.99:0/1709671976 >> v1:172.21.15.99:6789/0 conn(0x7f734406bc10 legacy=0x7f734406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f7348c71640 1 -- 172.21.15.99:0/1709671976 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454178731 0 0) 0x7f73440b5140 con 0x7f734406bc10 2024-09-10T11:35:06.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.663+0000 7f734a474640 1 -- 172.21.15.99:0/1709671976 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344162e20 con 0x7f73440b4930 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.664+0000 7f734a474640 1 --2- 172.21.15.99:0/1709671976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734406e5d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7344148c00 tx=0x7f733c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60efa3cea619b0de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.664+0000 7f7348c71640 1 -- 172.21.15.99:0/1709671976 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c007740 con 0x7f73440b4930 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.664+0000 7f7348c71640 1 -- 172.21.15.99:0/1709671976 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c0078e0 con 0x7f73440b4930 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.664+0000 7f7348c71640 1 -- 172.21.15.99:0/1709671976 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c007cb0 con 0x7f73440b4930 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 -- 172.21.15.99:0/1709671976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 msgr2=0x7f734406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 --2- 172.21.15.99:0/1709671976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734406e5d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7344148c00 tx=0x7f733c02a5f0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 -- 172.21.15.99:0/1709671976 shutdown_connections 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 --2- 172.21.15.99:0/1709671976 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734406e5d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 --2- 172.21.15.99:0/1709671976 >> v2:172.21.15.196:3300/0 conn(0x7f7344163620 0x7f7344160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 -- 172.21.15.99:0/1709671976 >> 172.21.15.99:0/1709671976 conn(0x7f73440b3630 msgr2=0x7f73440b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 -- 172.21.15.99:0/1709671976 shutdown_connections 2024-09-10T11:35:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.665+0000 7f734bed7640 1 -- 172.21.15.99:0/1709671976 wait complete. 2024-09-10T11:35:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.666+0000 7f734bed7640 1 Processor -- start 2024-09-10T11:35:06.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.666+0000 7f734bed7640 1 -- start start 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f734406bc10 0x7f734415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7344163620 0x7f7344159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73440d1560 con 0x7f73440b4930 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73440b4dc0 con 0x7f734406bc10 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f73440d16e0 con 0x7f7344163620 2024-09-10T11:35:06.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44510/0 (socket says 172.21.15.99:44510) 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 -- 172.21.15.99:0/162743085 learned_addr learned my addr 172.21.15.99:0/162743085 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 -- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7344163620 msgr2=0x7f7344159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7344163620 0x7f7344159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 -- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f734406bc10 msgr2=0x7f734415b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f734406bc10 0x7f734415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 -- 172.21.15.99:0/162743085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734413b7c0 con 0x7f73440b4930 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f7349472640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f733400b440 tx=0x7f733400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334015020 con 0x7f73440b4930 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7334011890 con 0x7f73440b4930 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334011c80 con 0x7f73440b4930 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734415a900 con 0x7f73440b4930 2024-09-10T11:35:06.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.667+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734415aec0 con 0x7f73440b4930 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.668+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73440b5140 con 0x7f73440b4930 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.669+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7334018020 con 0x7f73440b4930 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.669+0000 7f733a7fc640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 0x7f732806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.669+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f733408af30 con 0x7f73440b4930 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.669+0000 7f7349c73640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 0x7f732806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:06.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.670+0000 7f7349c73640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 0x7f732806e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7340007fd0 tx=0x7f7340007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:06.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.674+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f733401f020 con 0x7f73440b4930 2024-09-10T11:35:06.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.835+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7344073210 con 0x7f732806be30 2024-09-10T11:35:06.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.841+0000 7f733a7fc640 1 -- 172.21.15.99:0/162743085 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242639 (secure 0 0 0) 0x7f7344073210 con 0x7f732806be30 2024-09-10T11:35:06.844 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:06.845 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:06.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.843+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 msgr2=0x7f732806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.843+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 0x7f732806e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7340007fd0 tx=0x7f7340007960 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.843+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 msgr2=0x7f734413b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.843+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f733400b440 tx=0x7f733400b900 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 shutdown_connections 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f732806be30 0x7f732806e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7344163620 0x7f7344159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73440b4930 0x7f734413b280 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 --2- 172.21.15.99:0/162743085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f734406bc10 0x7f734415b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 >> 172.21.15.99:0/162743085 conn(0x7f73440b3630 msgr2=0x7f734406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 shutdown_connections 2024-09-10T11:35:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:06.844+0000 7f734bed7640 1 -- 172.21.15.99:0/162743085 wait complete. 2024-09-10T11:35:06.855 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-10T11:35:04.967283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556198,"statfs":{"total":1600000000,"available":1593547008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.411713"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565919+0000","last_change":"2024-09-10T11:35:01.565919+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565919+0000","last_undegraded":"2024-09-10T11:35:01.565919+0000","last_fullsized":"2024-09-10T11:35:01.565919+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568411+0000","last_change":"2024-09-10T11:35:01.568411+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568411+0000","last_undegraded":"2024-09-10T11:35:01.568411+0000","last_fullsized":"2024-09-10T11:35:01.568411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569657+0000","last_change":"2024-09-10T11:35:01.569657+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569657+0000","last_undegraded":"2024-09-10T11:35:01.569657+0000","last_fullsized":"2024-09-10T11:35:01.569657+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565761+0000","last_change":"2024-09-10T11:35:01.565761+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565761+0000","last_undegraded":"2024-09-10T11:35:01.565761+0000","last_fullsized":"2024-09-10T11:35:01.565761+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566643+0000","last_change":"2024-09-10T11:35:01.566643+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566643+0000","last_undegraded":"2024-09-10T11:35:01.566643+0000","last_fullsized":"2024-09-10T11:35:01.566643+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569064+0000","last_change":"2024-09-10T11:35:01.569064+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569064+0000","last_undegraded":"2024-09-10T11:35:01.569064+0000","last_fullsized":"2024-09-10T11:35:01.569064+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567844+0000","last_change":"2024-09-10T11:35:01.567844+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567844+0000","last_undegraded":"2024-09-10T11:35:01.567844+0000","last_fullsized":"2024-09-10T11:35:01.567844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569277+0000","last_change":"2024-09-10T11:35:01.569277+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569277+0000","last_undegraded":"2024-09-10T11:35:01.569277+0000","last_fullsized":"2024-09-10T11:35:01.569277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568457+0000","last_change":"2024-09-10T11:35:01.568457+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568457+0000","last_undegraded":"2024-09-10T11:35:01.568457+0000","last_fullsized":"2024-09-10T11:35:01.568457+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566245+0000","last_change":"2024-09-10T11:35:01.566245+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566245+0000","last_undegraded":"2024-09-10T11:35:01.566245+0000","last_fullsized":"2024-09-10T11:35:01.566245+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568738+0000","last_change":"2024-09-10T11:35:01.568738+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568738+0000","last_undegraded":"2024-09-10T11:35:01.568738+0000","last_fullsized":"2024-09-10T11:35:01.568738+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571097+0000","last_change":"2024-09-10T11:35:01.571097+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571097+0000","last_undegraded":"2024-09-10T11:35:01.571097+0000","last_fullsized":"2024-09-10T11:35:01.571097+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565950+0000","last_change":"2024-09-10T11:35:01.565950+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565950+0000","last_undegraded":"2024-09-10T11:35:01.565950+0000","last_fullsized":"2024-09-10T11:35:01.565950+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565425+0000","last_change":"2024-09-10T11:35:01.565425+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565425+0000","last_undegraded":"2024-09-10T11:35:01.565425+0000","last_fullsized":"2024-09-10T11:35:01.565425+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566167+0000","last_change":"2024-09-10T11:35:01.566167+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566167+0000","last_undegraded":"2024-09-10T11:35:01.566167+0000","last_fullsized":"2024-09-10T11:35:01.566167+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567990+0000","last_change":"2024-09-10T11:35:01.567990+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567990+0000","last_undegraded":"2024-09-10T11:35:01.567990+0000","last_fullsized":"2024-09-10T11:35:01.567990+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569715+0000","last_change":"2024-09-10T11:35:01.569715+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569715+0000","last_undegraded":"2024-09-10T11:35:01.569715+0000","last_fullsized":"2024-09-10T11:35:01.569715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566911+0000","last_change":"2024-09-10T11:35:01.566911+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566911+0000","last_undegraded":"2024-09-10T11:35:01.566911+0000","last_fullsized":"2024-09-10T11:35:01.566911+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572880+0000","last_change":"2024-09-10T11:35:01.572880+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572880+0000","last_undegraded":"2024-09-10T11:35:01.572880+0000","last_fullsized":"2024-09-10T11:35:01.572880+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571066+0000","last_change":"2024-09-10T11:35:01.571066+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571066+0000","last_undegraded":"2024-09-10T11:35:01.571066+0000","last_fullsized":"2024-09-10T11:35:01.571066+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573732+0000","last_change":"2024-09-10T11:35:01.573732+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573732+0000","last_undegraded":"2024-09-10T11:35:01.573732+0000","last_fullsized":"2024-09-10T11:35:01.573732+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569408+0000","last_change":"2024-09-10T11:35:01.569408+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569408+0000","last_undegraded":"2024-09-10T11:35:01.569408+0000","last_fullsized":"2024-09-10T11:35:01.569408+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566541+0000","last_change":"2024-09-10T11:35:01.566541+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566541+0000","last_undegraded":"2024-09-10T11:35:01.566541+0000","last_fullsized":"2024-09-10T11:35:01.566541+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567219+0000","last_change":"2024-09-10T11:35:01.567219+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567219+0000","last_undegraded":"2024-09-10T11:35:01.567219+0000","last_fullsized":"2024-09-10T11:35:01.567219+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565927+0000","last_change":"2024-09-10T11:35:01.565927+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565927+0000","last_undegraded":"2024-09-10T11:35:01.565927+0000","last_fullsized":"2024-09-10T11:35:01.565927+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564831+0000","last_change":"2024-09-10T11:35:01.564831+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564831+0000","last_undegraded":"2024-09-10T11:35:01.564831+0000","last_fullsized":"2024-09-10T11:35:01.564831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568745+0000","last_change":"2024-09-10T11:35:01.568745+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568745+0000","last_undegraded":"2024-09-10T11:35:01.568745+0000","last_fullsized":"2024-09-10T11:35:01.568745+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567687+0000","last_change":"2024-09-10T11:35:01.567687+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567687+0000","last_undegraded":"2024-09-10T11:35:01.567687+0000","last_fullsized":"2024-09-10T11:35:01.567687+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564599+0000","last_change":"2024-09-10T11:35:01.564599+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564599+0000","last_undegraded":"2024-09-10T11:35:01.564599+0000","last_fullsized":"2024-09-10T11:35:01.564599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565127+0000","last_change":"2024-09-10T11:35:01.565127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565127+0000","last_undegraded":"2024-09-10T11:35:01.565127+0000","last_fullsized":"2024-09-10T11:35:01.565127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572127+0000","last_change":"2024-09-10T11:35:01.572127+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572127+0000","last_undegraded":"2024-09-10T11:35:01.572127+0000","last_fullsized":"2024-09-10T11:35:01.572127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564934+0000","last_change":"2024-09-10T11:35:01.564934+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564934+0000","last_undegraded":"2024-09-10T11:35:01.564934+0000","last_fullsized":"2024-09-10T11:35:01.564934+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572594+0000","last_change":"2024-09-10T11:35:01.572594+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572594+0000","last_undegraded":"2024-09-10T11:35:01.572594+0000","last_fullsized":"2024-09-10T11:35:01.572594+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574983+0000","last_change":"2024-09-10T11:35:01.574138+0000","last_active":"2024-09-10T11:35:01.574983+0000","last_peered":"2024-09-10T11:35:01.574983+0000","last_clean":"2024-09-10T11:35:01.574983+0000","last_became_active":"2024-09-10T11:35:01.573905+0000","last_became_peered":"2024-09-10T11:35:01.573905+0000","last_unstale":"2024-09-10T11:35:01.574983+0000","last_undegraded":"2024-09-10T11:35:01.574983+0000","last_fullsized":"2024-09-10T11:35:01.574983+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:06:13.165426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564878+0000","last_change":"2024-09-10T11:35:01.564878+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564878+0000","last_undegraded":"2024-09-10T11:35:01.564878+0000","last_fullsized":"2024-09-10T11:35:01.564878+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577412+0000","last_change":"2024-09-10T11:35:01.577311+0000","last_active":"2024-09-10T11:35:01.577412+0000","last_peered":"2024-09-10T11:35:01.577412+0000","last_clean":"2024-09-10T11:35:01.577412+0000","last_became_active":"2024-09-10T11:35:01.576942+0000","last_became_peered":"2024-09-10T11:35:01.576942+0000","last_unstale":"2024-09-10T11:35:01.577412+0000","last_undegraded":"2024-09-10T11:35:01.577412+0000","last_fullsized":"2024-09-10T11:35:01.577412+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:36.991927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565534+0000","last_change":"2024-09-10T11:35:01.565534+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565534+0000","last_undegraded":"2024-09-10T11:35:01.565534+0000","last_fullsized":"2024-09-10T11:35:01.565534+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579753+0000","last_change":"2024-09-10T11:35:01.579621+0000","last_active":"2024-09-10T11:35:01.579753+0000","last_peered":"2024-09-10T11:35:01.579753+0000","last_clean":"2024-09-10T11:35:01.579753+0000","last_became_active":"2024-09-10T11:35:01.579238+0000","last_became_peered":"2024-09-10T11:35:01.579238+0000","last_unstale":"2024-09-10T11:35:01.579753+0000","last_undegraded":"2024-09-10T11:35:01.579753+0000","last_fullsized":"2024-09-10T11:35:01.579753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:52.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565557+0000","last_change":"2024-09-10T11:35:01.565557+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565557+0000","last_undegraded":"2024-09-10T11:35:01.565557+0000","last_fullsized":"2024-09-10T11:35:01.565557+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.578494+0000","last_change":"2024-09-10T11:35:01.578383+0000","last_active":"2024-09-10T11:35:01.578494+0000","last_peered":"2024-09-10T11:35:01.578494+0000","last_clean":"2024-09-10T11:35:01.578494+0000","last_became_active":"2024-09-10T11:35:01.578109+0000","last_became_peered":"2024-09-10T11:35:01.578109+0000","last_unstale":"2024-09-10T11:35:01.578494+0000","last_undegraded":"2024-09-10T11:35:01.578494+0000","last_fullsized":"2024-09-10T11:35:01.578494+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:18.486537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572758+0000","last_change":"2024-09-10T11:35:01.572758+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572758+0000","last_undegraded":"2024-09-10T11:35:01.572758+0000","last_fullsized":"2024-09-10T11:35:01.572758+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572226+0000","last_change":"2024-09-10T11:35:01.572113+0000","last_active":"2024-09-10T11:35:01.572226+0000","last_peered":"2024-09-10T11:35:01.572226+0000","last_clean":"2024-09-10T11:35:01.572226+0000","last_became_active":"2024-09-10T11:35:01.571874+0000","last_became_peered":"2024-09-10T11:35:01.571874+0000","last_unstale":"2024-09-10T11:35:01.572226+0000","last_undegraded":"2024-09-10T11:35:01.572226+0000","last_fullsized":"2024-09-10T11:35:01.572226+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:29.218475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568170+0000","last_change":"2024-09-10T11:35:01.568170+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568170+0000","last_undegraded":"2024-09-10T11:35:01.568170+0000","last_fullsized":"2024-09-10T11:35:01.568170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576235+0000","last_change":"2024-09-10T11:35:01.576109+0000","last_active":"2024-09-10T11:35:01.576235+0000","last_peered":"2024-09-10T11:35:01.576235+0000","last_clean":"2024-09-10T11:35:01.576235+0000","last_became_active":"2024-09-10T11:35:01.575849+0000","last_became_peered":"2024-09-10T11:35:01.575849+0000","last_unstale":"2024-09-10T11:35:01.576235+0000","last_undegraded":"2024-09-10T11:35:01.576235+0000","last_fullsized":"2024-09-10T11:35:01.576235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:54.020713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564259+0000","last_change":"2024-09-10T11:35:01.564259+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564259+0000","last_undegraded":"2024-09-10T11:35:01.564259+0000","last_fullsized":"2024-09-10T11:35:01.564259+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575753+0000","last_change":"2024-09-10T11:35:01.575658+0000","last_active":"2024-09-10T11:35:01.575753+0000","last_peered":"2024-09-10T11:35:01.575753+0000","last_clean":"2024-09-10T11:35:01.575753+0000","last_became_active":"2024-09-10T11:35:01.575428+0000","last_became_peered":"2024-09-10T11:35:01.575428+0000","last_unstale":"2024-09-10T11:35:01.575753+0000","last_undegraded":"2024-09-10T11:35:01.575753+0000","last_fullsized":"2024-09-10T11:35:01.575753+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:42.923450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571841+0000","last_change":"2024-09-10T11:35:01.571841+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571841+0000","last_undegraded":"2024-09-10T11:35:01.571841+0000","last_fullsized":"2024-09-10T11:35:01.571841+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579686+0000","last_change":"2024-09-10T11:35:01.579550+0000","last_active":"2024-09-10T11:35:01.579686+0000","last_peered":"2024-09-10T11:35:01.579686+0000","last_clean":"2024-09-10T11:35:01.579686+0000","last_became_active":"2024-09-10T11:35:01.579226+0000","last_became_peered":"2024-09-10T11:35:01.579226+0000","last_unstale":"2024-09-10T11:35:01.579686+0000","last_undegraded":"2024-09-10T11:35:01.579686+0000","last_fullsized":"2024-09-10T11:35:01.579686+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:02.892037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564550+0000","last_change":"2024-09-10T11:35:01.564550+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564550+0000","last_undegraded":"2024-09-10T11:35:01.564550+0000","last_fullsized":"2024-09-10T11:35:01.564550+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577256+0000","last_change":"2024-09-10T11:35:01.577132+0000","last_active":"2024-09-10T11:35:01.577256+0000","last_peered":"2024-09-10T11:35:01.577256+0000","last_clean":"2024-09-10T11:35:01.577256+0000","last_became_active":"2024-09-10T11:35:01.576572+0000","last_became_peered":"2024-09-10T11:35:01.576572+0000","last_unstale":"2024-09-10T11:35:01.577256+0000","last_undegraded":"2024-09-10T11:35:01.577256+0000","last_fullsized":"2024-09-10T11:35:01.577256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:31.771305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569057+0000","last_change":"2024-09-10T11:35:01.569057+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569057+0000","last_undegraded":"2024-09-10T11:35:01.569057+0000","last_fullsized":"2024-09-10T11:35:01.569057+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577971+0000","last_change":"2024-09-10T11:35:01.577634+0000","last_active":"2024-09-10T11:35:01.577971+0000","last_peered":"2024-09-10T11:35:01.577971+0000","last_clean":"2024-09-10T11:35:01.577971+0000","last_became_active":"2024-09-10T11:35:01.577375+0000","last_became_peered":"2024-09-10T11:35:01.577375+0000","last_unstale":"2024-09-10T11:35:01.577971+0000","last_undegraded":"2024-09-10T11:35:01.577971+0000","last_fullsized":"2024-09-10T11:35:01.577971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:59.758708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568150+0000","last_change":"2024-09-10T11:35:01.568150+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568150+0000","last_undegraded":"2024-09-10T11:35:01.568150+0000","last_fullsized":"2024-09-10T11:35:01.568150+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573571+0000","last_change":"2024-09-10T11:35:01.573449+0000","last_active":"2024-09-10T11:35:01.573571+0000","last_peered":"2024-09-10T11:35:01.573571+0000","last_clean":"2024-09-10T11:35:01.573571+0000","last_became_active":"2024-09-10T11:35:01.573208+0000","last_became_peered":"2024-09-10T11:35:01.573208+0000","last_unstale":"2024-09-10T11:35:01.573571+0000","last_undegraded":"2024-09-10T11:35:01.573571+0000","last_fullsized":"2024-09-10T11:35:01.573571+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:55.342593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567343+0000","last_change":"2024-09-10T11:35:01.567343+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567343+0000","last_undegraded":"2024-09-10T11:35:01.567343+0000","last_fullsized":"2024-09-10T11:35:01.567343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575166+0000","last_change":"2024-09-10T11:35:01.575061+0000","last_active":"2024-09-10T11:35:01.575166+0000","last_peered":"2024-09-10T11:35:01.575166+0000","last_clean":"2024-09-10T11:35:01.575166+0000","last_became_active":"2024-09-10T11:35:01.574722+0000","last_became_peered":"2024-09-10T11:35:01.574722+0000","last_unstale":"2024-09-10T11:35:01.575166+0000","last_undegraded":"2024-09-10T11:35:01.575166+0000","last_fullsized":"2024-09-10T11:35:01.575166+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:45.426600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.565092+0000","last_change":"2024-09-10T11:35:01.565092+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.565092+0000","last_undegraded":"2024-09-10T11:35:01.565092+0000","last_fullsized":"2024-09-10T11:35:01.565092+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577253+0000","last_change":"2024-09-10T11:35:01.576337+0000","last_active":"2024-09-10T11:35:01.577253+0000","last_peered":"2024-09-10T11:35:01.577253+0000","last_clean":"2024-09-10T11:35:01.577253+0000","last_became_active":"2024-09-10T11:35:01.576094+0000","last_became_peered":"2024-09-10T11:35:01.576094+0000","last_unstale":"2024-09-10T11:35:01.577253+0000","last_undegraded":"2024-09-10T11:35:01.577253+0000","last_fullsized":"2024-09-10T11:35:01.577253+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:31:18.030605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.573292+0000","last_change":"2024-09-10T11:35:01.573292+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.573292+0000","last_undegraded":"2024-09-10T11:35:01.573292+0000","last_fullsized":"2024-09-10T11:35:01.573292+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567570+0000","last_change":"2024-09-10T11:35:01.567436+0000","last_active":"2024-09-10T11:35:01.567570+0000","last_peered":"2024-09-10T11:35:01.567570+0000","last_clean":"2024-09-10T11:35:01.567570+0000","last_became_active":"2024-09-10T11:35:01.566612+0000","last_became_peered":"2024-09-10T11:35:01.566612+0000","last_unstale":"2024-09-10T11:35:01.567570+0000","last_undegraded":"2024-09-10T11:35:01.567570+0000","last_fullsized":"2024-09-10T11:35:01.567570+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:12.054238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568196+0000","last_change":"2024-09-10T11:35:01.568196+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568196+0000","last_undegraded":"2024-09-10T11:35:01.568196+0000","last_fullsized":"2024-09-10T11:35:01.568196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571385+0000","last_change":"2024-09-10T11:35:01.571276+0000","last_active":"2024-09-10T11:35:01.571385+0000","last_peered":"2024-09-10T11:35:01.571385+0000","last_clean":"2024-09-10T11:35:01.571385+0000","last_became_active":"2024-09-10T11:35:01.570511+0000","last_became_peered":"2024-09-10T11:35:01.570511+0000","last_unstale":"2024-09-10T11:35:01.571385+0000","last_undegraded":"2024-09-10T11:35:01.571385+0000","last_fullsized":"2024-09-10T11:35:01.571385+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:07.622876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569769+0000","last_change":"2024-09-10T11:35:01.569769+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569769+0000","last_undegraded":"2024-09-10T11:35:01.569769+0000","last_fullsized":"2024-09-10T11:35:01.569769+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.569217+0000","last_change":"2024-09-10T11:35:01.569105+0000","last_active":"2024-09-10T11:35:01.569217+0000","last_peered":"2024-09-10T11:35:01.569217+0000","last_clean":"2024-09-10T11:35:01.569217+0000","last_became_active":"2024-09-10T11:35:01.568619+0000","last_became_peered":"2024-09-10T11:35:01.568619+0000","last_unstale":"2024-09-10T11:35:01.569217+0000","last_undegraded":"2024-09-10T11:35:01.569217+0000","last_fullsized":"2024-09-10T11:35:01.569217+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:41.912527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566883+0000","last_change":"2024-09-10T11:35:01.566883+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566883+0000","last_undegraded":"2024-09-10T11:35:01.566883+0000","last_fullsized":"2024-09-10T11:35:01.566883+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574687+0000","last_change":"2024-09-10T11:35:01.574578+0000","last_active":"2024-09-10T11:35:01.574687+0000","last_peered":"2024-09-10T11:35:01.574687+0000","last_clean":"2024-09-10T11:35:01.574687+0000","last_became_active":"2024-09-10T11:35:01.574313+0000","last_became_peered":"2024-09-10T11:35:01.574313+0000","last_unstale":"2024-09-10T11:35:01.574687+0000","last_undegraded":"2024-09-10T11:35:01.574687+0000","last_fullsized":"2024-09-10T11:35:01.574687+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:33.710736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568446+0000","last_change":"2024-09-10T11:35:01.568446+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568446+0000","last_undegraded":"2024-09-10T11:35:01.568446+0000","last_fullsized":"2024-09-10T11:35:01.568446+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.570614+0000","last_change":"2024-09-10T11:35:01.570511+0000","last_active":"2024-09-10T11:35:01.570614+0000","last_peered":"2024-09-10T11:35:01.570614+0000","last_clean":"2024-09-10T11:35:01.570614+0000","last_became_active":"2024-09-10T11:35:01.570268+0000","last_became_peered":"2024-09-10T11:35:01.570268+0000","last_unstale":"2024-09-10T11:35:01.570614+0000","last_undegraded":"2024-09-10T11:35:01.570614+0000","last_fullsized":"2024-09-10T11:35:01.570614+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:23.505565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571499+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571499+0000","last_undegraded":"2024-09-10T11:35:01.571499+0000","last_fullsized":"2024-09-10T11:35:01.571499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575650+0000","last_change":"2024-09-10T11:35:01.575527+0000","last_active":"2024-09-10T11:35:01.575650+0000","last_peered":"2024-09-10T11:35:01.575650+0000","last_clean":"2024-09-10T11:35:01.575650+0000","last_became_active":"2024-09-10T11:35:01.575285+0000","last_became_peered":"2024-09-10T11:35:01.575285+0000","last_unstale":"2024-09-10T11:35:01.575650+0000","last_undegraded":"2024-09-10T11:35:01.575650+0000","last_fullsized":"2024-09-10T11:35:01.575650+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:50.445819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571652+0000","last_change":"2024-09-10T11:35:01.571652+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571652+0000","last_undegraded":"2024-09-10T11:35:01.571652+0000","last_fullsized":"2024-09-10T11:35:01.571652+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.577157+0000","last_change":"2024-09-10T11:35:01.577040+0000","last_active":"2024-09-10T11:35:01.577157+0000","last_peered":"2024-09-10T11:35:01.577157+0000","last_clean":"2024-09-10T11:35:01.577157+0000","last_became_active":"2024-09-10T11:35:01.576667+0000","last_became_peered":"2024-09-10T11:35:01.576667+0000","last_unstale":"2024-09-10T11:35:01.577157+0000","last_undegraded":"2024-09-10T11:35:01.577157+0000","last_fullsized":"2024-09-10T11:35:01.577157+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:55.579500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571257+0000","last_change":"2024-09-10T11:35:01.571257+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571257+0000","last_undegraded":"2024-09-10T11:35:01.571257+0000","last_fullsized":"2024-09-10T11:35:01.571257+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.574162+0000","last_change":"2024-09-10T11:35:01.573987+0000","last_active":"2024-09-10T11:35:01.574162+0000","last_peered":"2024-09-10T11:35:01.574162+0000","last_clean":"2024-09-10T11:35:01.574162+0000","last_became_active":"2024-09-10T11:35:01.573756+0000","last_became_peered":"2024-09-10T11:35:01.573756+0000","last_unstale":"2024-09-10T11:35:01.574162+0000","last_undegraded":"2024-09-10T11:35:01.574162+0000","last_fullsized":"2024-09-10T11:35:01.574162+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:11.736603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.564649+0000","last_change":"2024-09-10T11:35:01.564649+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.564649+0000","last_undegraded":"2024-09-10T11:35:01.564649+0000","last_fullsized":"2024-09-10T11:35:01.564649+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.567997+0000","last_change":"2024-09-10T11:35:01.567874+0000","last_active":"2024-09-10T11:35:01.567997+0000","last_peered":"2024-09-10T11:35:01.567997+0000","last_clean":"2024-09-10T11:35:01.567997+0000","last_became_active":"2024-09-10T11:35:01.567649+0000","last_became_peered":"2024-09-10T11:35:01.567649+0000","last_unstale":"2024-09-10T11:35:01.567997+0000","last_undegraded":"2024-09-10T11:35:01.567997+0000","last_fullsized":"2024-09-10T11:35:01.567997+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:58.144492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.566977+0000","last_change":"2024-09-10T11:35:01.566977+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.566977+0000","last_undegraded":"2024-09-10T11:35:01.566977+0000","last_fullsized":"2024-09-10T11:35:01.566977+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576998+0000","last_change":"2024-09-10T11:35:01.576843+0000","last_active":"2024-09-10T11:35:01.576998+0000","last_peered":"2024-09-10T11:35:01.576998+0000","last_clean":"2024-09-10T11:35:01.576998+0000","last_became_active":"2024-09-10T11:35:01.576103+0000","last_became_peered":"2024-09-10T11:35:01.576103+0000","last_unstale":"2024-09-10T11:35:01.576998+0000","last_undegraded":"2024-09-10T11:35:01.576998+0000","last_fullsized":"2024-09-10T11:35:01.576998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.717940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.570853+0000","last_change":"2024-09-10T11:35:01.570853+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.570853+0000","last_undegraded":"2024-09-10T11:35:01.570853+0000","last_fullsized":"2024-09-10T11:35:01.570853+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571060+0000","last_change":"2024-09-10T11:35:01.570746+0000","last_active":"2024-09-10T11:35:01.571060+0000","last_peered":"2024-09-10T11:35:01.571060+0000","last_clean":"2024-09-10T11:35:01.571060+0000","last_became_active":"2024-09-10T11:35:01.570187+0000","last_became_peered":"2024-09-10T11:35:01.570187+0000","last_unstale":"2024-09-10T11:35:01.571060+0000","last_undegraded":"2024-09-10T11:35:01.571060+0000","last_fullsized":"2024-09-10T11:35:01.571060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:08.080682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.569349+0000","last_change":"2024-09-10T11:35:01.569349+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.569349+0000","last_undegraded":"2024-09-10T11:35:01.569349+0000","last_fullsized":"2024-09-10T11:35:01.569349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.573953+0000","last_change":"2024-09-10T11:35:01.573744+0000","last_active":"2024-09-10T11:35:01.573953+0000","last_peered":"2024-09-10T11:35:01.573953+0000","last_clean":"2024-09-10T11:35:01.573953+0000","last_became_active":"2024-09-10T11:35:01.573463+0000","last_became_peered":"2024-09-10T11:35:01.573463+0000","last_unstale":"2024-09-10T11:35:01.573953+0000","last_undegraded":"2024-09-10T11:35:01.573953+0000","last_fullsized":"2024-09-10T11:35:01.573953+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:43.610059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572681+0000","last_change":"2024-09-10T11:35:01.572681+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572681+0000","last_undegraded":"2024-09-10T11:35:01.572681+0000","last_fullsized":"2024-09-10T11:35:01.572681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579934+0000","last_change":"2024-09-10T11:35:01.579801+0000","last_active":"2024-09-10T11:35:01.579934+0000","last_peered":"2024-09-10T11:35:01.579934+0000","last_clean":"2024-09-10T11:35:01.579934+0000","last_became_active":"2024-09-10T11:35:01.578207+0000","last_became_peered":"2024-09-10T11:35:01.578207+0000","last_unstale":"2024-09-10T11:35:01.579934+0000","last_undegraded":"2024-09-10T11:35:01.579934+0000","last_fullsized":"2024-09-10T11:35:01.579934+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:18:36.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567016+0000","last_change":"2024-09-10T11:35:01.567016+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567016+0000","last_undegraded":"2024-09-10T11:35:01.567016+0000","last_fullsized":"2024-09-10T11:35:01.567016+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572601+0000","last_change":"2024-09-10T11:35:01.572125+0000","last_active":"2024-09-10T11:35:01.572601+0000","last_peered":"2024-09-10T11:35:01.572601+0000","last_clean":"2024-09-10T11:35:01.572601+0000","last_became_active":"2024-09-10T11:35:01.571887+0000","last_became_peered":"2024-09-10T11:35:01.571887+0000","last_unstale":"2024-09-10T11:35:01.572601+0000","last_undegraded":"2024-09-10T11:35:01.572601+0000","last_fullsized":"2024-09-10T11:35:01.572601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:09.224575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.568970+0000","last_change":"2024-09-10T11:35:01.568970+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.568970+0000","last_undegraded":"2024-09-10T11:35:01.568970+0000","last_fullsized":"2024-09-10T11:35:01.568970+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.579086+0000","last_change":"2024-09-10T11:35:01.578976+0000","last_active":"2024-09-10T11:35:01.579086+0000","last_peered":"2024-09-10T11:35:01.579086+0000","last_clean":"2024-09-10T11:35:01.579086+0000","last_became_active":"2024-09-10T11:35:01.578755+0000","last_became_peered":"2024-09-10T11:35:01.578755+0000","last_unstale":"2024-09-10T11:35:01.579086+0000","last_undegraded":"2024-09-10T11:35:01.579086+0000","last_fullsized":"2024-09-10T11:35:01.579086+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:48.512747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.567653+0000","last_change":"2024-09-10T11:35:01.567653+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.567653+0000","last_undegraded":"2024-09-10T11:35:01.567653+0000","last_fullsized":"2024-09-10T11:35:01.567653+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.571622+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:01.571622+0000","last_peered":"2024-09-10T11:35:01.571622+0000","last_clean":"2024-09-10T11:35:01.571622+0000","last_became_active":"2024-09-10T11:35:01.571267+0000","last_became_peered":"2024-09-10T11:35:01.571267+0000","last_unstale":"2024-09-10T11:35:01.571622+0000","last_undegraded":"2024-09-10T11:35:01.571622+0000","last_fullsized":"2024-09-10T11:35:01.571622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:47.651348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.571015+0000","last_change":"2024-09-10T11:35:01.571015+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.571015+0000","last_undegraded":"2024-09-10T11:35:01.571015+0000","last_fullsized":"2024-09-10T11:35:01.571015+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.572739+0000","last_change":"2024-09-10T11:35:01.570370+0000","last_active":"2024-09-10T11:35:01.572739+0000","last_peered":"2024-09-10T11:35:01.572739+0000","last_clean":"2024-09-10T11:35:01.572739+0000","last_became_active":"2024-09-10T11:35:01.570120+0000","last_became_peered":"2024-09-10T11:35:01.570120+0000","last_unstale":"2024-09-10T11:35:01.572739+0000","last_undegraded":"2024-09-10T11:35:01.572739+0000","last_fullsized":"2024-09-10T11:35:01.572739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:49:10.126918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572471+0000","last_change":"2024-09-10T11:35:01.572471+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572471+0000","last_undegraded":"2024-09-10T11:35:01.572471+0000","last_fullsized":"2024-09-10T11:35:01.572471+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.576660+0000","last_change":"2024-09-10T11:35:01.576531+0000","last_active":"2024-09-10T11:35:01.576660+0000","last_peered":"2024-09-10T11:35:01.576660+0000","last_clean":"2024-09-10T11:35:01.576660+0000","last_became_active":"2024-09-10T11:35:01.575776+0000","last_became_peered":"2024-09-10T11:35:01.575776+0000","last_unstale":"2024-09-10T11:35:01.576660+0000","last_undegraded":"2024-09-10T11:35:01.576660+0000","last_fullsized":"2024-09-10T11:35:01.576660+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:54.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":121,"state":"creating+peering","last_fresh":"2024-09-10T11:35:01.572997+0000","last_change":"2024-09-10T11:35:01.572997+0000","last_active":"2024-09-10T11:35:01.555567+0000","last_peered":"2024-09-10T11:35:01.555567+0000","last_clean":"2024-09-10T11:35:01.555567+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:01.572997+0000","last_undegraded":"2024-09-10T11:35:01.572997+0000","last_fullsized":"2024-09-10T11:35:01.572997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.575004+0000","last_change":"2024-09-10T11:35:01.574933+0000","last_active":"2024-09-10T11:35:01.575004+0000","last_peered":"2024-09-10T11:35:01.575004+0000","last_clean":"2024-09-10T11:35:01.575004+0000","last_became_active":"2024-09-10T11:35:01.574408+0000","last_became_peered":"2024-09-10T11:35:01.574408+0000","last_unstale":"2024-09-10T11:35:01.575004+0000","last_undegraded":"2024-09-10T11:35:01.575004+0000","last_fullsized":"2024-09-10T11:35:01.575004+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:45.538369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":376,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T11:35:01.565044+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:01.565044+0000","last_peered":"2024-09-10T11:35:01.565044+0000","last_clean":"2024-09-10T11:35:01.565044+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:01.565044+0000","last_undegraded":"2024-09-10T11:35:01.565044+0000","last_fullsized":"2024-09-10T11:35:01.565044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":641,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194670,"statfs":{"total":200000000,"available":199342594,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194664,"statfs":{"total":200000000,"available":199336319,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1218,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194093,"statfs":{"total":200000000,"available":198752226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":636,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194676,"statfs":{"total":200000000,"available":199348411,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:08.558 INFO:tasks.ceph.mds.c.smithi099.stderr:2024-09-10T11:35:08.557+0000 7f948d887640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:35:09.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 Processor -- start 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 -- start start 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c0b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f070c1465b0 0x7f070c0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f070c161100 con 0x7f070c146060 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f070c0b3c70 con 0x7f070c1465b0 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.971+0000 7f0713227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f070c075210 con 0x7f070c0731c0 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f070c1465b0 0x7f070c0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f070c1465b0 0x7f070c0af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44526/0 (socket says 172.21.15.99:44526) 2024-09-10T11:35:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 -- 172.21.15.99:0/928383153 learned_addr learned my addr 172.21.15.99:0/928383153 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 -- 172.21.15.99:0/928383153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 msgr2=0x7f070c0b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f0710fc3640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c0b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c0b3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 -- 172.21.15.99:0/928383153 >> v1:172.21.15.196:6789/0 conn(0x7f070c146060 legacy=0x7f070c0735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f07117c4640 1 -- 172.21.15.99:0/928383153 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1613e0 con 0x7f070c1465b0 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.972+0000 7f0710fc3640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f07117c4640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1465b0 0x7f070c0af5e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dda671603e2c0b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f070b7fe640 1 -- 172.21.15.99:0/928383153 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4037070 con 0x7f070c1465b0 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f070b7fe640 1 -- 172.21.15.99:0/928383153 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4032440 con 0x7f070c1465b0 2024-09-10T11:35:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f070b7fe640 1 -- 172.21.15.99:0/928383153 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4032810 con 0x7f070c1465b0 2024-09-10T11:35:09.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f0713227640 1 -- 172.21.15.99:0/928383153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1465b0 msgr2=0x7f070c0af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:09.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.973+0000 7f0713227640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1465b0 0x7f070c0af5e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 -- 172.21.15.99:0/928383153 shutdown_connections 2024-09-10T11:35:09.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c1465b0 0x7f070c0af5e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 --2- 172.21.15.99:0/928383153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c0b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 -- 172.21.15.99:0/928383153 >> 172.21.15.99:0/928383153 conn(0x7f070c072310 msgr2=0x7f070c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 -- 172.21.15.99:0/928383153 shutdown_connections 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.974+0000 7f0713227640 1 -- 172.21.15.99:0/928383153 wait complete. 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.975+0000 7f0713227640 1 Processor -- start 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.975+0000 7f0713227640 1 -- start start 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c09b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c1465b0 0x7f070c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15dec0 con 0x7f070c146060 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f070c075740 con 0x7f070c1465b0 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0713227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f070c0b6480 con 0x7f070c0731c0 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44536/0 (socket says 172.21.15.99:44536) 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 -- 172.21.15.99:0/1904716220 learned_addr learned my addr 172.21.15.99:0/1904716220 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f0710fc3640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c09b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f07117c4640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c1465b0 0x7f070c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 -- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 msgr2=0x7f070c09b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c09b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 -- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c1465b0 msgr2=0x7f070c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c1465b0 0x7f070c06e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 -- 172.21.15.99:0/1904716220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c0cff80 con 0x7f070c146060 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f070bfff640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c400 tx=0x7f06fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.976+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0052d0 con 0x7f070c146060 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.977+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc005490 con 0x7f070c146060 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.977+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c0d0220 con 0x7f070c146060 2024-09-10T11:35:09.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.977+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc007260 con 0x7f070c146060 2024-09-10T11:35:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.977+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c0d07e0 con 0x7f070c146060 2024-09-10T11:35:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.978+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0b3730 con 0x7f070c146060 2024-09-10T11:35:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.978+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06fc007400 con 0x7f070c146060 2024-09-10T11:35:09.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.979+0000 7f07097fa640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 0x7f06ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:09.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.979+0000 7f0710fc3640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 0x7f06ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:09.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.979+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f06fc08a240 con 0x7f070c146060 2024-09-10T11:35:09.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.979+0000 7f0710fc3640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 0x7f06ec06e320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f070400d620 tx=0x7f0704007810 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:09.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:09.983+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06fc058630 con 0x7f070c146060 2024-09-10T11:35:10.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.144+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f070c0cd8a0 con 0x7f06ec06be60 2024-09-10T11:35:10.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.149+0000 7f07097fa640 1 -- 172.21.15.99:0/1904716220 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247992 (secure 0 0 0) 0x7f070c0cd8a0 con 0x7f06ec06be60 2024-09-10T11:35:10.152 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:10.154 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 msgr2=0x7f06ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 0x7f06ec06e320 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f070400d620 tx=0x7f0704007810 comp rx=0 tx=0).stop 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 msgr2=0x7f070c157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00c400 tx=0x7f06fc00c8c0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 shutdown_connections 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f06ec06be60 0x7f06ec06e320 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070c1465b0 0x7f070c06e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f070c146060 0x7f070c157400 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 --2- 172.21.15.99:0/1904716220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f070c0731c0 0x7f070c09b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 >> 172.21.15.99:0/1904716220 conn(0x7f070c072310 msgr2=0x7f070c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:10.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 shutdown_connections 2024-09-10T11:35:10.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.152+0000 7f0713227640 1 -- 172.21.15.99:0/1904716220 wait complete. 2024-09-10T11:35:10.164 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-10T11:35:08.968173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6444,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556055,"statfs":{"total":1600000000,"available":1593400396,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.407489"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566181+0000","last_change":"2024-09-10T11:35:02.584239+0000","last_active":"2024-09-10T11:35:03.566181+0000","last_peered":"2024-09-10T11:35:03.566181+0000","last_clean":"2024-09-10T11:35:03.566181+0000","last_became_active":"2024-09-10T11:35:02.583608+0000","last_became_peered":"2024-09-10T11:35:02.583608+0000","last_unstale":"2024-09-10T11:35:03.566181+0000","last_undegraded":"2024-09-10T11:35:03.566181+0000","last_fullsized":"2024-09-10T11:35:03.566181+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:32.272105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565411+0000","last_change":"2024-09-10T11:35:02.571269+0000","last_active":"2024-09-10T11:35:03.565411+0000","last_peered":"2024-09-10T11:35:03.565411+0000","last_clean":"2024-09-10T11:35:03.565411+0000","last_became_active":"2024-09-10T11:35:02.570821+0000","last_became_peered":"2024-09-10T11:35:02.570821+0000","last_unstale":"2024-09-10T11:35:03.565411+0000","last_undegraded":"2024-09-10T11:35:03.565411+0000","last_fullsized":"2024-09-10T11:35:03.565411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:34.295361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565467+0000","last_change":"2024-09-10T11:35:02.577030+0000","last_active":"2024-09-10T11:35:03.565467+0000","last_peered":"2024-09-10T11:35:03.565467+0000","last_clean":"2024-09-10T11:35:03.565467+0000","last_became_active":"2024-09-10T11:35:02.575391+0000","last_became_peered":"2024-09-10T11:35:02.575391+0000","last_unstale":"2024-09-10T11:35:03.565467+0000","last_undegraded":"2024-09-10T11:35:03.565467+0000","last_fullsized":"2024-09-10T11:35:03.565467+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:21.097161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565334+0000","last_change":"2024-09-10T11:35:02.579176+0000","last_active":"2024-09-10T11:35:03.565334+0000","last_peered":"2024-09-10T11:35:03.565334+0000","last_clean":"2024-09-10T11:35:03.565334+0000","last_became_active":"2024-09-10T11:35:02.578937+0000","last_became_peered":"2024-09-10T11:35:02.578937+0000","last_unstale":"2024-09-10T11:35:03.565334+0000","last_undegraded":"2024-09-10T11:35:03.565334+0000","last_fullsized":"2024-09-10T11:35:03.565334+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:57.158429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105689+0000","last_change":"2024-09-10T11:35:02.576355+0000","last_active":"2024-09-10T11:35:05.105689+0000","last_peered":"2024-09-10T11:35:05.105689+0000","last_clean":"2024-09-10T11:35:05.105689+0000","last_became_active":"2024-09-10T11:35:02.575698+0000","last_became_peered":"2024-09-10T11:35:02.575698+0000","last_unstale":"2024-09-10T11:35:05.105689+0000","last_undegraded":"2024-09-10T11:35:05.105689+0000","last_fullsized":"2024-09-10T11:35:05.105689+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:44.584265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105611+0000","last_change":"2024-09-10T11:35:02.583337+0000","last_active":"2024-09-10T11:35:05.105611+0000","last_peered":"2024-09-10T11:35:05.105611+0000","last_clean":"2024-09-10T11:35:05.105611+0000","last_became_active":"2024-09-10T11:35:02.582083+0000","last_became_peered":"2024-09-10T11:35:02.582083+0000","last_unstale":"2024-09-10T11:35:05.105611+0000","last_undegraded":"2024-09-10T11:35:05.105611+0000","last_fullsized":"2024-09-10T11:35:05.105611+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:30.166060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105955+0000","last_change":"2024-09-10T11:35:02.576293+0000","last_active":"2024-09-10T11:35:05.105955+0000","last_peered":"2024-09-10T11:35:05.105955+0000","last_clean":"2024-09-10T11:35:05.105955+0000","last_became_active":"2024-09-10T11:35:02.575961+0000","last_became_peered":"2024-09-10T11:35:02.575961+0000","last_unstale":"2024-09-10T11:35:05.105955+0000","last_undegraded":"2024-09-10T11:35:05.105955+0000","last_fullsized":"2024-09-10T11:35:05.105955+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:05.375836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565633+0000","last_change":"2024-09-10T11:35:02.577222+0000","last_active":"2024-09-10T11:35:03.565633+0000","last_peered":"2024-09-10T11:35:03.565633+0000","last_clean":"2024-09-10T11:35:03.565633+0000","last_became_active":"2024-09-10T11:35:02.575780+0000","last_became_peered":"2024-09-10T11:35:02.575780+0000","last_unstale":"2024-09-10T11:35:03.565633+0000","last_undegraded":"2024-09-10T11:35:03.565633+0000","last_fullsized":"2024-09-10T11:35:03.565633+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:57.848877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105569+0000","last_change":"2024-09-10T11:35:02.581652+0000","last_active":"2024-09-10T11:35:05.105569+0000","last_peered":"2024-09-10T11:35:05.105569+0000","last_clean":"2024-09-10T11:35:05.105569+0000","last_became_active":"2024-09-10T11:35:02.581354+0000","last_became_peered":"2024-09-10T11:35:02.581354+0000","last_unstale":"2024-09-10T11:35:05.105569+0000","last_undegraded":"2024-09-10T11:35:05.105569+0000","last_fullsized":"2024-09-10T11:35:05.105569+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:30.624599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402794+0000","last_change":"2024-09-10T11:35:02.580562+0000","last_active":"2024-09-10T11:35:05.402794+0000","last_peered":"2024-09-10T11:35:05.402794+0000","last_clean":"2024-09-10T11:35:05.402794+0000","last_became_active":"2024-09-10T11:35:02.580277+0000","last_became_peered":"2024-09-10T11:35:02.580277+0000","last_unstale":"2024-09-10T11:35:05.402794+0000","last_undegraded":"2024-09-10T11:35:05.402794+0000","last_fullsized":"2024-09-10T11:35:05.402794+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:27.775314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105759+0000","last_change":"2024-09-10T11:35:02.583131+0000","last_active":"2024-09-10T11:35:05.105759+0000","last_peered":"2024-09-10T11:35:05.105759+0000","last_clean":"2024-09-10T11:35:05.105759+0000","last_became_active":"2024-09-10T11:35:02.581526+0000","last_became_peered":"2024-09-10T11:35:02.581526+0000","last_unstale":"2024-09-10T11:35:05.105759+0000","last_undegraded":"2024-09-10T11:35:05.105759+0000","last_fullsized":"2024-09-10T11:35:05.105759+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:58.025160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565071+0000","last_change":"2024-09-10T11:35:02.578256+0000","last_active":"2024-09-10T11:35:03.565071+0000","last_peered":"2024-09-10T11:35:03.565071+0000","last_clean":"2024-09-10T11:35:03.565071+0000","last_became_active":"2024-09-10T11:35:02.577963+0000","last_became_peered":"2024-09-10T11:35:02.577963+0000","last_unstale":"2024-09-10T11:35:03.565071+0000","last_undegraded":"2024-09-10T11:35:03.565071+0000","last_fullsized":"2024-09-10T11:35:03.565071+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:17.271231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198451+0000","last_change":"2024-09-10T11:35:02.574573+0000","last_active":"2024-09-10T11:35:05.198451+0000","last_peered":"2024-09-10T11:35:05.198451+0000","last_clean":"2024-09-10T11:35:05.198451+0000","last_became_active":"2024-09-10T11:35:02.574343+0000","last_became_peered":"2024-09-10T11:35:02.574343+0000","last_unstale":"2024-09-10T11:35:05.198451+0000","last_undegraded":"2024-09-10T11:35:05.198451+0000","last_fullsized":"2024-09-10T11:35:05.198451+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:47.293739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565522+0000","last_change":"2024-09-10T11:35:02.568164+0000","last_active":"2024-09-10T11:35:03.565522+0000","last_peered":"2024-09-10T11:35:03.565522+0000","last_clean":"2024-09-10T11:35:03.565522+0000","last_became_active":"2024-09-10T11:35:02.567932+0000","last_became_peered":"2024-09-10T11:35:02.567932+0000","last_unstale":"2024-09-10T11:35:03.565522+0000","last_undegraded":"2024-09-10T11:35:03.565522+0000","last_fullsized":"2024-09-10T11:35:03.565522+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:46:56.648056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565952+0000","last_change":"2024-09-10T11:35:02.572017+0000","last_active":"2024-09-10T11:35:03.565952+0000","last_peered":"2024-09-10T11:35:03.565952+0000","last_clean":"2024-09-10T11:35:03.565952+0000","last_became_active":"2024-09-10T11:35:02.570723+0000","last_became_peered":"2024-09-10T11:35:02.570723+0000","last_unstale":"2024-09-10T11:35:03.565952+0000","last_undegraded":"2024-09-10T11:35:03.565952+0000","last_fullsized":"2024-09-10T11:35:03.565952+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:37.019204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105902+0000","last_change":"2024-09-10T11:35:02.577650+0000","last_active":"2024-09-10T11:35:05.105902+0000","last_peered":"2024-09-10T11:35:05.105902+0000","last_clean":"2024-09-10T11:35:05.105902+0000","last_became_active":"2024-09-10T11:35:02.577314+0000","last_became_peered":"2024-09-10T11:35:02.577314+0000","last_unstale":"2024-09-10T11:35:05.105902+0000","last_undegraded":"2024-09-10T11:35:05.105902+0000","last_fullsized":"2024-09-10T11:35:05.105902+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:00.867457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.719777+0000","last_change":"2024-09-10T11:35:02.575585+0000","last_active":"2024-09-10T11:35:03.719777+0000","last_peered":"2024-09-10T11:35:03.719777+0000","last_clean":"2024-09-10T11:35:03.719777+0000","last_became_active":"2024-09-10T11:35:02.575326+0000","last_became_peered":"2024-09-10T11:35:02.575326+0000","last_unstale":"2024-09-10T11:35:03.719777+0000","last_undegraded":"2024-09-10T11:35:03.719777+0000","last_fullsized":"2024-09-10T11:35:03.719777+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:05.179122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566248+0000","last_change":"2024-09-10T11:35:02.571245+0000","last_active":"2024-09-10T11:35:03.566248+0000","last_peered":"2024-09-10T11:35:03.566248+0000","last_clean":"2024-09-10T11:35:03.566248+0000","last_became_active":"2024-09-10T11:35:02.570881+0000","last_became_peered":"2024-09-10T11:35:02.570881+0000","last_unstale":"2024-09-10T11:35:03.566248+0000","last_undegraded":"2024-09-10T11:35:03.566248+0000","last_fullsized":"2024-09-10T11:35:03.566248+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:34.866446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.719798+0000","last_change":"2024-09-10T11:35:02.582257+0000","last_active":"2024-09-10T11:35:03.719798+0000","last_peered":"2024-09-10T11:35:03.719798+0000","last_clean":"2024-09-10T11:35:03.719798+0000","last_became_active":"2024-09-10T11:35:02.581743+0000","last_became_peered":"2024-09-10T11:35:02.581743+0000","last_unstale":"2024-09-10T11:35:03.719798+0000","last_undegraded":"2024-09-10T11:35:03.719798+0000","last_fullsized":"2024-09-10T11:35:03.719798+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:29.929506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565084+0000","last_change":"2024-09-10T11:35:02.581969+0000","last_active":"2024-09-10T11:35:03.565084+0000","last_peered":"2024-09-10T11:35:03.565084+0000","last_clean":"2024-09-10T11:35:03.565084+0000","last_became_active":"2024-09-10T11:35:02.581388+0000","last_became_peered":"2024-09-10T11:35:02.581388+0000","last_unstale":"2024-09-10T11:35:03.565084+0000","last_undegraded":"2024-09-10T11:35:03.565084+0000","last_fullsized":"2024-09-10T11:35:03.565084+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:18.611005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565236+0000","last_change":"2024-09-10T11:35:02.580687+0000","last_active":"2024-09-10T11:35:03.565236+0000","last_peered":"2024-09-10T11:35:03.565236+0000","last_clean":"2024-09-10T11:35:03.565236+0000","last_became_active":"2024-09-10T11:35:02.580040+0000","last_became_peered":"2024-09-10T11:35:02.580040+0000","last_unstale":"2024-09-10T11:35:03.565236+0000","last_undegraded":"2024-09-10T11:35:03.565236+0000","last_fullsized":"2024-09-10T11:35:03.565236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:37.922052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565374+0000","last_change":"2024-09-10T11:35:02.581244+0000","last_active":"2024-09-10T11:35:03.565374+0000","last_peered":"2024-09-10T11:35:03.565374+0000","last_clean":"2024-09-10T11:35:03.565374+0000","last_became_active":"2024-09-10T11:35:02.581017+0000","last_became_peered":"2024-09-10T11:35:02.581017+0000","last_unstale":"2024-09-10T11:35:03.565374+0000","last_undegraded":"2024-09-10T11:35:03.565374+0000","last_fullsized":"2024-09-10T11:35:03.565374+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:32.005959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566405+0000","last_change":"2024-09-10T11:35:02.573768+0000","last_active":"2024-09-10T11:35:03.566405+0000","last_peered":"2024-09-10T11:35:03.566405+0000","last_clean":"2024-09-10T11:35:03.566405+0000","last_became_active":"2024-09-10T11:35:02.573307+0000","last_became_peered":"2024-09-10T11:35:02.573307+0000","last_unstale":"2024-09-10T11:35:03.566405+0000","last_undegraded":"2024-09-10T11:35:03.566405+0000","last_fullsized":"2024-09-10T11:35:03.566405+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:27.298294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402503+0000","last_change":"2024-09-10T11:35:02.580788+0000","last_active":"2024-09-10T11:35:05.402503+0000","last_peered":"2024-09-10T11:35:05.402503+0000","last_clean":"2024-09-10T11:35:05.402503+0000","last_became_active":"2024-09-10T11:35:02.580454+0000","last_became_peered":"2024-09-10T11:35:02.580454+0000","last_unstale":"2024-09-10T11:35:05.402503+0000","last_undegraded":"2024-09-10T11:35:05.402503+0000","last_fullsized":"2024-09-10T11:35:05.402503+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:41.557017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402657+0000","last_change":"2024-09-10T11:35:02.576850+0000","last_active":"2024-09-10T11:35:05.402657+0000","last_peered":"2024-09-10T11:35:05.402657+0000","last_clean":"2024-09-10T11:35:05.402657+0000","last_became_active":"2024-09-10T11:35:02.576615+0000","last_became_peered":"2024-09-10T11:35:02.576615+0000","last_unstale":"2024-09-10T11:35:05.402657+0000","last_undegraded":"2024-09-10T11:35:05.402657+0000","last_fullsized":"2024-09-10T11:35:05.402657+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:01.307976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198533+0000","last_change":"2024-09-10T11:35:02.574757+0000","last_active":"2024-09-10T11:35:05.198533+0000","last_peered":"2024-09-10T11:35:05.198533+0000","last_clean":"2024-09-10T11:35:05.198533+0000","last_became_active":"2024-09-10T11:35:02.574472+0000","last_became_peered":"2024-09-10T11:35:02.574472+0000","last_unstale":"2024-09-10T11:35:05.198533+0000","last_undegraded":"2024-09-10T11:35:05.198533+0000","last_fullsized":"2024-09-10T11:35:05.198533+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:24.918700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402602+0000","last_change":"2024-09-10T11:35:02.577692+0000","last_active":"2024-09-10T11:35:05.402602+0000","last_peered":"2024-09-10T11:35:05.402602+0000","last_clean":"2024-09-10T11:35:05.402602+0000","last_became_active":"2024-09-10T11:35:02.577464+0000","last_became_peered":"2024-09-10T11:35:02.577464+0000","last_unstale":"2024-09-10T11:35:05.402602+0000","last_undegraded":"2024-09-10T11:35:05.402602+0000","last_fullsized":"2024-09-10T11:35:05.402602+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:38.255466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402815+0000","last_change":"2024-09-10T11:35:02.580992+0000","last_active":"2024-09-10T11:35:05.402815+0000","last_peered":"2024-09-10T11:35:05.402815+0000","last_clean":"2024-09-10T11:35:05.402815+0000","last_became_active":"2024-09-10T11:35:02.580783+0000","last_became_peered":"2024-09-10T11:35:02.580783+0000","last_unstale":"2024-09-10T11:35:05.402815+0000","last_undegraded":"2024-09-10T11:35:05.402815+0000","last_fullsized":"2024-09-10T11:35:05.402815+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:21.003203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198480+0000","last_change":"2024-09-10T11:35:02.576113+0000","last_active":"2024-09-10T11:35:05.198480+0000","last_peered":"2024-09-10T11:35:05.198480+0000","last_clean":"2024-09-10T11:35:05.198480+0000","last_became_active":"2024-09-10T11:35:02.575822+0000","last_became_peered":"2024-09-10T11:35:02.575822+0000","last_unstale":"2024-09-10T11:35:05.198480+0000","last_undegraded":"2024-09-10T11:35:05.198480+0000","last_fullsized":"2024-09-10T11:35:05.198480+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:53.929971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198664+0000","last_change":"2024-09-10T11:35:02.576642+0000","last_active":"2024-09-10T11:35:05.198664+0000","last_peered":"2024-09-10T11:35:05.198664+0000","last_clean":"2024-09-10T11:35:05.198664+0000","last_became_active":"2024-09-10T11:35:02.576307+0000","last_became_peered":"2024-09-10T11:35:02.576307+0000","last_unstale":"2024-09-10T11:35:05.198664+0000","last_undegraded":"2024-09-10T11:35:05.198664+0000","last_fullsized":"2024-09-10T11:35:05.198664+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:33.426528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565546+0000","last_change":"2024-09-10T11:35:02.580882+0000","last_active":"2024-09-10T11:35:03.565546+0000","last_peered":"2024-09-10T11:35:03.565546+0000","last_clean":"2024-09-10T11:35:03.565546+0000","last_became_active":"2024-09-10T11:35:02.580364+0000","last_became_peered":"2024-09-10T11:35:02.580364+0000","last_unstale":"2024-09-10T11:35:03.565546+0000","last_undegraded":"2024-09-10T11:35:03.565546+0000","last_fullsized":"2024-09-10T11:35:03.565546+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:01.518660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565831+0000","last_change":"2024-09-10T11:35:02.577407+0000","last_active":"2024-09-10T11:35:03.565831+0000","last_peered":"2024-09-10T11:35:03.565831+0000","last_clean":"2024-09-10T11:35:03.565831+0000","last_became_active":"2024-09-10T11:35:02.576946+0000","last_became_peered":"2024-09-10T11:35:02.576946+0000","last_unstale":"2024-09-10T11:35:03.565831+0000","last_undegraded":"2024-09-10T11:35:03.565831+0000","last_fullsized":"2024-09-10T11:35:03.565831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:53.507251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.719974+0000","last_change":"2024-09-10T11:35:02.582806+0000","last_active":"2024-09-10T11:35:03.719974+0000","last_peered":"2024-09-10T11:35:03.719974+0000","last_clean":"2024-09-10T11:35:03.719974+0000","last_became_active":"2024-09-10T11:35:02.582544+0000","last_became_peered":"2024-09-10T11:35:02.582544+0000","last_unstale":"2024-09-10T11:35:03.719974+0000","last_undegraded":"2024-09-10T11:35:03.719974+0000","last_fullsized":"2024-09-10T11:35:03.719974+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:41.128685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"122'4","reported_seq":18,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198658+0000","last_change":"2024-09-10T11:35:01.574138+0000","last_active":"2024-09-10T11:35:05.198658+0000","last_peered":"2024-09-10T11:35:05.198658+0000","last_clean":"2024-09-10T11:35:05.198658+0000","last_became_active":"2024-09-10T11:35:01.573905+0000","last_became_peered":"2024-09-10T11:35:01.573905+0000","last_unstale":"2024-09-10T11:35:05.198658+0000","last_undegraded":"2024-09-10T11:35:05.198658+0000","last_fullsized":"2024-09-10T11:35:05.198658+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:54:49.214048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198589+0000","last_change":"2024-09-10T11:35:02.576845+0000","last_active":"2024-09-10T11:35:05.198589+0000","last_peered":"2024-09-10T11:35:05.198589+0000","last_clean":"2024-09-10T11:35:05.198589+0000","last_became_active":"2024-09-10T11:35:02.576572+0000","last_became_peered":"2024-09-10T11:35:02.576572+0000","last_unstale":"2024-09-10T11:35:05.198589+0000","last_undegraded":"2024-09-10T11:35:05.198589+0000","last_fullsized":"2024-09-10T11:35:05.198589+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:26.947925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105795+0000","last_change":"2024-09-10T11:35:01.577311+0000","last_active":"2024-09-10T11:35:05.105795+0000","last_peered":"2024-09-10T11:35:05.105795+0000","last_clean":"2024-09-10T11:35:05.105795+0000","last_became_active":"2024-09-10T11:35:01.576942+0000","last_became_peered":"2024-09-10T11:35:01.576942+0000","last_unstale":"2024-09-10T11:35:05.105795+0000","last_undegraded":"2024-09-10T11:35:05.105795+0000","last_fullsized":"2024-09-10T11:35:05.105795+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:09:19.833449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566323+0000","last_change":"2024-09-10T11:35:02.577890+0000","last_active":"2024-09-10T11:35:03.566323+0000","last_peered":"2024-09-10T11:35:03.566323+0000","last_clean":"2024-09-10T11:35:03.566323+0000","last_became_active":"2024-09-10T11:35:02.577259+0000","last_became_peered":"2024-09-10T11:35:02.577259+0000","last_unstale":"2024-09-10T11:35:03.566323+0000","last_undegraded":"2024-09-10T11:35:03.566323+0000","last_fullsized":"2024-09-10T11:35:03.566323+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:05.684347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566025+0000","last_change":"2024-09-10T11:35:01.579621+0000","last_active":"2024-09-10T11:35:03.566025+0000","last_peered":"2024-09-10T11:35:03.566025+0000","last_clean":"2024-09-10T11:35:03.566025+0000","last_became_active":"2024-09-10T11:35:01.579238+0000","last_became_peered":"2024-09-10T11:35:01.579238+0000","last_unstale":"2024-09-10T11:35:03.566025+0000","last_undegraded":"2024-09-10T11:35:03.566025+0000","last_fullsized":"2024-09-10T11:35:03.566025+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:54.258667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402926+0000","last_change":"2024-09-10T11:35:02.577678+0000","last_active":"2024-09-10T11:35:05.402926+0000","last_peered":"2024-09-10T11:35:05.402926+0000","last_clean":"2024-09-10T11:35:05.402926+0000","last_became_active":"2024-09-10T11:35:02.577443+0000","last_became_peered":"2024-09-10T11:35:02.577443+0000","last_unstale":"2024-09-10T11:35:05.402926+0000","last_undegraded":"2024-09-10T11:35:05.402926+0000","last_fullsized":"2024-09-10T11:35:05.402926+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:18.130502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566672+0000","last_change":"2024-09-10T11:35:01.578383+0000","last_active":"2024-09-10T11:35:03.566672+0000","last_peered":"2024-09-10T11:35:03.566672+0000","last_clean":"2024-09-10T11:35:03.566672+0000","last_became_active":"2024-09-10T11:35:01.578109+0000","last_became_peered":"2024-09-10T11:35:01.578109+0000","last_unstale":"2024-09-10T11:35:03.566672+0000","last_undegraded":"2024-09-10T11:35:03.566672+0000","last_fullsized":"2024-09-10T11:35:03.566672+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:03.177166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565078+0000","last_change":"2024-09-10T11:35:02.572449+0000","last_active":"2024-09-10T11:35:03.565078+0000","last_peered":"2024-09-10T11:35:03.565078+0000","last_clean":"2024-09-10T11:35:03.565078+0000","last_became_active":"2024-09-10T11:35:02.571420+0000","last_became_peered":"2024-09-10T11:35:02.571420+0000","last_unstale":"2024-09-10T11:35:03.565078+0000","last_undegraded":"2024-09-10T11:35:03.565078+0000","last_fullsized":"2024-09-10T11:35:03.565078+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:25.997112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565927+0000","last_change":"2024-09-10T11:35:01.572113+0000","last_active":"2024-09-10T11:35:03.565927+0000","last_peered":"2024-09-10T11:35:03.565927+0000","last_clean":"2024-09-10T11:35:03.565927+0000","last_became_active":"2024-09-10T11:35:01.571874+0000","last_became_peered":"2024-09-10T11:35:01.571874+0000","last_unstale":"2024-09-10T11:35:03.565927+0000","last_undegraded":"2024-09-10T11:35:03.565927+0000","last_fullsized":"2024-09-10T11:35:03.565927+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:22.131441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105779+0000","last_change":"2024-09-10T11:35:02.580656+0000","last_active":"2024-09-10T11:35:05.105779+0000","last_peered":"2024-09-10T11:35:05.105779+0000","last_clean":"2024-09-10T11:35:05.105779+0000","last_became_active":"2024-09-10T11:35:02.580425+0000","last_became_peered":"2024-09-10T11:35:02.580425+0000","last_unstale":"2024-09-10T11:35:05.105779+0000","last_undegraded":"2024-09-10T11:35:05.105779+0000","last_fullsized":"2024-09-10T11:35:05.105779+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:58.921002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566397+0000","last_change":"2024-09-10T11:35:01.576109+0000","last_active":"2024-09-10T11:35:03.566397+0000","last_peered":"2024-09-10T11:35:03.566397+0000","last_clean":"2024-09-10T11:35:03.566397+0000","last_became_active":"2024-09-10T11:35:01.575849+0000","last_became_peered":"2024-09-10T11:35:01.575849+0000","last_unstale":"2024-09-10T11:35:03.566397+0000","last_undegraded":"2024-09-10T11:35:03.566397+0000","last_fullsized":"2024-09-10T11:35:03.566397+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:02.991491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198813+0000","last_change":"2024-09-10T11:35:02.586035+0000","last_active":"2024-09-10T11:35:05.198813+0000","last_peered":"2024-09-10T11:35:05.198813+0000","last_clean":"2024-09-10T11:35:05.198813+0000","last_became_active":"2024-09-10T11:35:02.585554+0000","last_became_peered":"2024-09-10T11:35:02.585554+0000","last_unstale":"2024-09-10T11:35:05.198813+0000","last_undegraded":"2024-09-10T11:35:05.198813+0000","last_fullsized":"2024-09-10T11:35:05.198813+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:46.553871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.106006+0000","last_change":"2024-09-10T11:35:01.575658+0000","last_active":"2024-09-10T11:35:05.106006+0000","last_peered":"2024-09-10T11:35:05.106006+0000","last_clean":"2024-09-10T11:35:05.106006+0000","last_became_active":"2024-09-10T11:35:01.575428+0000","last_became_peered":"2024-09-10T11:35:01.575428+0000","last_unstale":"2024-09-10T11:35:05.106006+0000","last_undegraded":"2024-09-10T11:35:05.106006+0000","last_fullsized":"2024-09-10T11:35:05.106006+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:55:52.793320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.564973+0000","last_change":"2024-09-10T11:35:02.581485+0000","last_active":"2024-09-10T11:35:03.564973+0000","last_peered":"2024-09-10T11:35:03.564973+0000","last_clean":"2024-09-10T11:35:03.564973+0000","last_became_active":"2024-09-10T11:35:02.579403+0000","last_became_peered":"2024-09-10T11:35:02.579403+0000","last_unstale":"2024-09-10T11:35:03.564973+0000","last_undegraded":"2024-09-10T11:35:03.564973+0000","last_fullsized":"2024-09-10T11:35:03.564973+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:38.460668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720139+0000","last_change":"2024-09-10T11:35:01.579550+0000","last_active":"2024-09-10T11:35:03.720139+0000","last_peered":"2024-09-10T11:35:03.720139+0000","last_clean":"2024-09-10T11:35:03.720139+0000","last_became_active":"2024-09-10T11:35:01.579226+0000","last_became_peered":"2024-09-10T11:35:01.579226+0000","last_unstale":"2024-09-10T11:35:03.720139+0000","last_undegraded":"2024-09-10T11:35:03.720139+0000","last_fullsized":"2024-09-10T11:35:03.720139+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:21.774646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565591+0000","last_change":"2024-09-10T11:35:02.577213+0000","last_active":"2024-09-10T11:35:03.565591+0000","last_peered":"2024-09-10T11:35:03.565591+0000","last_clean":"2024-09-10T11:35:03.565591+0000","last_became_active":"2024-09-10T11:35:02.576627+0000","last_became_peered":"2024-09-10T11:35:02.576627+0000","last_unstale":"2024-09-10T11:35:03.565591+0000","last_undegraded":"2024-09-10T11:35:03.565591+0000","last_fullsized":"2024-09-10T11:35:03.565591+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:20.771179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565829+0000","last_change":"2024-09-10T11:35:01.577132+0000","last_active":"2024-09-10T11:35:03.565829+0000","last_peered":"2024-09-10T11:35:03.565829+0000","last_clean":"2024-09-10T11:35:03.565829+0000","last_became_active":"2024-09-10T11:35:01.576572+0000","last_became_peered":"2024-09-10T11:35:01.576572+0000","last_unstale":"2024-09-10T11:35:03.565829+0000","last_undegraded":"2024-09-10T11:35:03.565829+0000","last_fullsized":"2024-09-10T11:35:03.565829+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:11.246721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402708+0000","last_change":"2024-09-10T11:35:02.575094+0000","last_active":"2024-09-10T11:35:05.402708+0000","last_peered":"2024-09-10T11:35:05.402708+0000","last_clean":"2024-09-10T11:35:05.402708+0000","last_became_active":"2024-09-10T11:35:02.574868+0000","last_became_peered":"2024-09-10T11:35:02.574868+0000","last_unstale":"2024-09-10T11:35:05.402708+0000","last_undegraded":"2024-09-10T11:35:05.402708+0000","last_fullsized":"2024-09-10T11:35:05.402708+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:21.812178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566026+0000","last_change":"2024-09-10T11:35:01.577634+0000","last_active":"2024-09-10T11:35:03.566026+0000","last_peered":"2024-09-10T11:35:03.566026+0000","last_clean":"2024-09-10T11:35:03.566026+0000","last_became_active":"2024-09-10T11:35:01.577375+0000","last_became_peered":"2024-09-10T11:35:01.577375+0000","last_unstale":"2024-09-10T11:35:03.566026+0000","last_undegraded":"2024-09-10T11:35:03.566026+0000","last_fullsized":"2024-09-10T11:35:03.566026+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:01.371618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105706+0000","last_change":"2024-09-10T11:35:02.575341+0000","last_active":"2024-09-10T11:35:05.105706+0000","last_peered":"2024-09-10T11:35:05.105706+0000","last_clean":"2024-09-10T11:35:05.105706+0000","last_became_active":"2024-09-10T11:35:02.574914+0000","last_became_peered":"2024-09-10T11:35:02.574914+0000","last_unstale":"2024-09-10T11:35:05.105706+0000","last_undegraded":"2024-09-10T11:35:05.105706+0000","last_fullsized":"2024-09-10T11:35:05.105706+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:49:32.628032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"122'3","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105687+0000","last_change":"2024-09-10T11:35:01.573449+0000","last_active":"2024-09-10T11:35:05.105687+0000","last_peered":"2024-09-10T11:35:05.105687+0000","last_clean":"2024-09-10T11:35:05.105687+0000","last_became_active":"2024-09-10T11:35:01.573208+0000","last_became_peered":"2024-09-10T11:35:01.573208+0000","last_unstale":"2024-09-10T11:35:05.105687+0000","last_undegraded":"2024-09-10T11:35:05.105687+0000","last_fullsized":"2024-09-10T11:35:05.105687+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:34.904757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402942+0000","last_change":"2024-09-10T11:35:02.574158+0000","last_active":"2024-09-10T11:35:05.402942+0000","last_peered":"2024-09-10T11:35:05.402942+0000","last_clean":"2024-09-10T11:35:05.402942+0000","last_became_active":"2024-09-10T11:35:02.573936+0000","last_became_peered":"2024-09-10T11:35:02.573936+0000","last_unstale":"2024-09-10T11:35:05.402942+0000","last_undegraded":"2024-09-10T11:35:05.402942+0000","last_fullsized":"2024-09-10T11:35:05.402942+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:29.204013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"122'6","reported_seq":20,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565170+0000","last_change":"2024-09-10T11:35:01.575061+0000","last_active":"2024-09-10T11:35:03.565170+0000","last_peered":"2024-09-10T11:35:03.565170+0000","last_clean":"2024-09-10T11:35:03.565170+0000","last_became_active":"2024-09-10T11:35:01.574722+0000","last_became_peered":"2024-09-10T11:35:01.574722+0000","last_unstale":"2024-09-10T11:35:03.565170+0000","last_undegraded":"2024-09-10T11:35:03.565170+0000","last_fullsized":"2024-09-10T11:35:03.565170+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:16.138685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566482+0000","last_change":"2024-09-10T11:35:02.578084+0000","last_active":"2024-09-10T11:35:03.566482+0000","last_peered":"2024-09-10T11:35:03.566482+0000","last_clean":"2024-09-10T11:35:03.566482+0000","last_became_active":"2024-09-10T11:35:02.577632+0000","last_became_peered":"2024-09-10T11:35:02.577632+0000","last_unstale":"2024-09-10T11:35:03.566482+0000","last_undegraded":"2024-09-10T11:35:03.566482+0000","last_fullsized":"2024-09-10T11:35:03.566482+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:34.071862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720266+0000","last_change":"2024-09-10T11:35:01.576337+0000","last_active":"2024-09-10T11:35:03.720266+0000","last_peered":"2024-09-10T11:35:03.720266+0000","last_clean":"2024-09-10T11:35:03.720266+0000","last_became_active":"2024-09-10T11:35:01.576094+0000","last_became_peered":"2024-09-10T11:35:01.576094+0000","last_unstale":"2024-09-10T11:35:03.720266+0000","last_undegraded":"2024-09-10T11:35:03.720266+0000","last_fullsized":"2024-09-10T11:35:03.720266+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.574251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565555+0000","last_change":"2024-09-10T11:35:02.576271+0000","last_active":"2024-09-10T11:35:03.565555+0000","last_peered":"2024-09-10T11:35:03.565555+0000","last_clean":"2024-09-10T11:35:03.565555+0000","last_became_active":"2024-09-10T11:35:02.576043+0000","last_became_peered":"2024-09-10T11:35:02.576043+0000","last_unstale":"2024-09-10T11:35:03.565555+0000","last_undegraded":"2024-09-10T11:35:03.565555+0000","last_fullsized":"2024-09-10T11:35:03.565555+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:32.323103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566730+0000","last_change":"2024-09-10T11:35:01.567436+0000","last_active":"2024-09-10T11:35:03.566730+0000","last_peered":"2024-09-10T11:35:03.566730+0000","last_clean":"2024-09-10T11:35:03.566730+0000","last_became_active":"2024-09-10T11:35:01.566612+0000","last_became_peered":"2024-09-10T11:35:01.566612+0000","last_unstale":"2024-09-10T11:35:03.566730+0000","last_undegraded":"2024-09-10T11:35:03.566730+0000","last_fullsized":"2024-09-10T11:35:03.566730+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:17.381285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105585+0000","last_change":"2024-09-10T11:35:02.572297+0000","last_active":"2024-09-10T11:35:05.105585+0000","last_peered":"2024-09-10T11:35:05.105585+0000","last_clean":"2024-09-10T11:35:05.105585+0000","last_became_active":"2024-09-10T11:35:02.572060+0000","last_became_peered":"2024-09-10T11:35:02.572060+0000","last_unstale":"2024-09-10T11:35:05.105585+0000","last_undegraded":"2024-09-10T11:35:05.105585+0000","last_fullsized":"2024-09-10T11:35:05.105585+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:10:15.631342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565127+0000","last_change":"2024-09-10T11:35:01.571276+0000","last_active":"2024-09-10T11:35:03.565127+0000","last_peered":"2024-09-10T11:35:03.565127+0000","last_clean":"2024-09-10T11:35:03.565127+0000","last_became_active":"2024-09-10T11:35:01.570511+0000","last_became_peered":"2024-09-10T11:35:01.570511+0000","last_unstale":"2024-09-10T11:35:03.565127+0000","last_undegraded":"2024-09-10T11:35:03.565127+0000","last_fullsized":"2024-09-10T11:35:03.565127+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:17.721333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565206+0000","last_change":"2024-09-10T11:35:02.578425+0000","last_active":"2024-09-10T11:35:03.565206+0000","last_peered":"2024-09-10T11:35:03.565206+0000","last_clean":"2024-09-10T11:35:03.565206+0000","last_became_active":"2024-09-10T11:35:02.578162+0000","last_became_peered":"2024-09-10T11:35:02.578162+0000","last_unstale":"2024-09-10T11:35:03.565206+0000","last_undegraded":"2024-09-10T11:35:03.565206+0000","last_fullsized":"2024-09-10T11:35:03.565206+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:35.076690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"122'8","reported_seq":22,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566489+0000","last_change":"2024-09-10T11:35:01.569105+0000","last_active":"2024-09-10T11:35:03.566489+0000","last_peered":"2024-09-10T11:35:03.566489+0000","last_clean":"2024-09-10T11:35:03.566489+0000","last_became_active":"2024-09-10T11:35:01.568619+0000","last_became_peered":"2024-09-10T11:35:01.568619+0000","last_unstale":"2024-09-10T11:35:03.566489+0000","last_undegraded":"2024-09-10T11:35:03.566489+0000","last_fullsized":"2024-09-10T11:35:03.566489+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:20.792727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402765+0000","last_change":"2024-09-10T11:35:02.583522+0000","last_active":"2024-09-10T11:35:05.402765+0000","last_peered":"2024-09-10T11:35:05.402765+0000","last_clean":"2024-09-10T11:35:05.402765+0000","last_became_active":"2024-09-10T11:35:02.583281+0000","last_became_peered":"2024-09-10T11:35:02.583281+0000","last_unstale":"2024-09-10T11:35:05.402765+0000","last_undegraded":"2024-09-10T11:35:05.402765+0000","last_fullsized":"2024-09-10T11:35:05.402765+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:52.577098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565240+0000","last_change":"2024-09-10T11:35:01.574578+0000","last_active":"2024-09-10T11:35:03.565240+0000","last_peered":"2024-09-10T11:35:03.565240+0000","last_clean":"2024-09-10T11:35:03.565240+0000","last_became_active":"2024-09-10T11:35:01.574313+0000","last_became_peered":"2024-09-10T11:35:01.574313+0000","last_unstale":"2024-09-10T11:35:03.565240+0000","last_undegraded":"2024-09-10T11:35:03.565240+0000","last_fullsized":"2024-09-10T11:35:03.565240+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:01.087309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402750+0000","last_change":"2024-09-10T11:35:02.582924+0000","last_active":"2024-09-10T11:35:05.402750+0000","last_peered":"2024-09-10T11:35:05.402750+0000","last_clean":"2024-09-10T11:35:05.402750+0000","last_became_active":"2024-09-10T11:35:02.582695+0000","last_became_peered":"2024-09-10T11:35:02.582695+0000","last_unstale":"2024-09-10T11:35:05.402750+0000","last_undegraded":"2024-09-10T11:35:05.402750+0000","last_fullsized":"2024-09-10T11:35:05.402750+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:54.901185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.106172+0000","last_change":"2024-09-10T11:35:01.570511+0000","last_active":"2024-09-10T11:35:05.106172+0000","last_peered":"2024-09-10T11:35:05.106172+0000","last_clean":"2024-09-10T11:35:05.106172+0000","last_became_active":"2024-09-10T11:35:01.570268+0000","last_became_peered":"2024-09-10T11:35:01.570268+0000","last_unstale":"2024-09-10T11:35:05.106172+0000","last_undegraded":"2024-09-10T11:35:05.106172+0000","last_fullsized":"2024-09-10T11:35:05.106172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:18.970065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565108+0000","last_change":"2024-09-10T11:35:02.571370+0000","last_active":"2024-09-10T11:35:03.565108+0000","last_peered":"2024-09-10T11:35:03.565108+0000","last_clean":"2024-09-10T11:35:03.565108+0000","last_became_active":"2024-09-10T11:35:02.571137+0000","last_became_peered":"2024-09-10T11:35:02.571137+0000","last_unstale":"2024-09-10T11:35:03.565108+0000","last_undegraded":"2024-09-10T11:35:03.565108+0000","last_fullsized":"2024-09-10T11:35:03.565108+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:24.312949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720282+0000","last_change":"2024-09-10T11:35:01.575527+0000","last_active":"2024-09-10T11:35:03.720282+0000","last_peered":"2024-09-10T11:35:03.720282+0000","last_clean":"2024-09-10T11:35:03.720282+0000","last_became_active":"2024-09-10T11:35:01.575285+0000","last_became_peered":"2024-09-10T11:35:01.575285+0000","last_unstale":"2024-09-10T11:35:03.720282+0000","last_undegraded":"2024-09-10T11:35:03.720282+0000","last_fullsized":"2024-09-10T11:35:03.720282+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:33.374977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720147+0000","last_change":"2024-09-10T11:35:02.582058+0000","last_active":"2024-09-10T11:35:03.720147+0000","last_peered":"2024-09-10T11:35:03.720147+0000","last_clean":"2024-09-10T11:35:03.720147+0000","last_became_active":"2024-09-10T11:35:02.581373+0000","last_became_peered":"2024-09-10T11:35:02.581373+0000","last_unstale":"2024-09-10T11:35:03.720147+0000","last_undegraded":"2024-09-10T11:35:03.720147+0000","last_fullsized":"2024-09-10T11:35:03.720147+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:12.631892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566584+0000","last_change":"2024-09-10T11:35:01.577040+0000","last_active":"2024-09-10T11:35:03.566584+0000","last_peered":"2024-09-10T11:35:03.566584+0000","last_clean":"2024-09-10T11:35:03.566584+0000","last_became_active":"2024-09-10T11:35:01.576667+0000","last_became_peered":"2024-09-10T11:35:01.576667+0000","last_unstale":"2024-09-10T11:35:03.566584+0000","last_undegraded":"2024-09-10T11:35:03.566584+0000","last_fullsized":"2024-09-10T11:35:03.566584+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:38.740445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.719835+0000","last_change":"2024-09-10T11:35:02.581686+0000","last_active":"2024-09-10T11:35:03.719835+0000","last_peered":"2024-09-10T11:35:03.719835+0000","last_clean":"2024-09-10T11:35:03.719835+0000","last_became_active":"2024-09-10T11:35:02.581008+0000","last_became_peered":"2024-09-10T11:35:02.581008+0000","last_unstale":"2024-09-10T11:35:03.719835+0000","last_undegraded":"2024-09-10T11:35:03.719835+0000","last_fullsized":"2024-09-10T11:35:03.719835+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:52.099416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402820+0000","last_change":"2024-09-10T11:35:01.573987+0000","last_active":"2024-09-10T11:35:05.402820+0000","last_peered":"2024-09-10T11:35:05.402820+0000","last_clean":"2024-09-10T11:35:05.402820+0000","last_became_active":"2024-09-10T11:35:01.573756+0000","last_became_peered":"2024-09-10T11:35:01.573756+0000","last_unstale":"2024-09-10T11:35:05.402820+0000","last_undegraded":"2024-09-10T11:35:05.402820+0000","last_fullsized":"2024-09-10T11:35:05.402820+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:51.551721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566607+0000","last_change":"2024-09-10T11:35:02.571449+0000","last_active":"2024-09-10T11:35:03.566607+0000","last_peered":"2024-09-10T11:35:03.566607+0000","last_clean":"2024-09-10T11:35:03.566607+0000","last_became_active":"2024-09-10T11:35:02.570968+0000","last_became_peered":"2024-09-10T11:35:02.570968+0000","last_unstale":"2024-09-10T11:35:03.566607+0000","last_undegraded":"2024-09-10T11:35:03.566607+0000","last_fullsized":"2024-09-10T11:35:03.566607+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:23.617553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.403008+0000","last_change":"2024-09-10T11:35:01.567874+0000","last_active":"2024-09-10T11:35:05.403008+0000","last_peered":"2024-09-10T11:35:05.403008+0000","last_clean":"2024-09-10T11:35:05.403008+0000","last_became_active":"2024-09-10T11:35:01.567649+0000","last_became_peered":"2024-09-10T11:35:01.567649+0000","last_unstale":"2024-09-10T11:35:05.403008+0000","last_undegraded":"2024-09-10T11:35:05.403008+0000","last_fullsized":"2024-09-10T11:35:05.403008+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:31.008195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402499+0000","last_change":"2024-09-10T11:35:02.585083+0000","last_active":"2024-09-10T11:35:05.402499+0000","last_peered":"2024-09-10T11:35:05.402499+0000","last_clean":"2024-09-10T11:35:05.402499+0000","last_became_active":"2024-09-10T11:35:02.584843+0000","last_became_peered":"2024-09-10T11:35:02.584843+0000","last_unstale":"2024-09-10T11:35:05.402499+0000","last_undegraded":"2024-09-10T11:35:05.402499+0000","last_fullsized":"2024-09-10T11:35:05.402499+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:32.337300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565917+0000","last_change":"2024-09-10T11:35:01.576843+0000","last_active":"2024-09-10T11:35:03.565917+0000","last_peered":"2024-09-10T11:35:03.565917+0000","last_clean":"2024-09-10T11:35:03.565917+0000","last_became_active":"2024-09-10T11:35:01.576103+0000","last_became_peered":"2024-09-10T11:35:01.576103+0000","last_unstale":"2024-09-10T11:35:03.565917+0000","last_undegraded":"2024-09-10T11:35:03.565917+0000","last_fullsized":"2024-09-10T11:35:03.565917+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:22.559341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720017+0000","last_change":"2024-09-10T11:35:02.578949+0000","last_active":"2024-09-10T11:35:03.720017+0000","last_peered":"2024-09-10T11:35:03.720017+0000","last_clean":"2024-09-10T11:35:03.720017+0000","last_became_active":"2024-09-10T11:35:02.578076+0000","last_became_peered":"2024-09-10T11:35:02.578076+0000","last_unstale":"2024-09-10T11:35:03.720017+0000","last_undegraded":"2024-09-10T11:35:03.720017+0000","last_fullsized":"2024-09-10T11:35:03.720017+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:37.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"122'2","reported_seq":16,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565187+0000","last_change":"2024-09-10T11:35:01.570746+0000","last_active":"2024-09-10T11:35:03.565187+0000","last_peered":"2024-09-10T11:35:03.565187+0000","last_clean":"2024-09-10T11:35:03.565187+0000","last_became_active":"2024-09-10T11:35:01.570187+0000","last_became_peered":"2024-09-10T11:35:01.570187+0000","last_unstale":"2024-09-10T11:35:03.565187+0000","last_undegraded":"2024-09-10T11:35:03.565187+0000","last_fullsized":"2024-09-10T11:35:03.565187+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:39.251120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720012+0000","last_change":"2024-09-10T11:35:02.580849+0000","last_active":"2024-09-10T11:35:03.720012+0000","last_peered":"2024-09-10T11:35:03.720012+0000","last_clean":"2024-09-10T11:35:03.720012+0000","last_became_active":"2024-09-10T11:35:02.580604+0000","last_became_peered":"2024-09-10T11:35:02.580604+0000","last_unstale":"2024-09-10T11:35:03.720012+0000","last_undegraded":"2024-09-10T11:35:03.720012+0000","last_fullsized":"2024-09-10T11:35:03.720012+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:27.299821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"123'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720185+0000","last_change":"2024-09-10T11:35:01.573744+0000","last_active":"2024-09-10T11:35:03.720185+0000","last_peered":"2024-09-10T11:35:03.720185+0000","last_clean":"2024-09-10T11:35:03.720185+0000","last_became_active":"2024-09-10T11:35:01.573463+0000","last_became_peered":"2024-09-10T11:35:01.573463+0000","last_unstale":"2024-09-10T11:35:03.720185+0000","last_undegraded":"2024-09-10T11:35:03.720185+0000","last_fullsized":"2024-09-10T11:35:03.720185+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:47.687520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565854+0000","last_change":"2024-09-10T11:35:02.584659+0000","last_active":"2024-09-10T11:35:03.565854+0000","last_peered":"2024-09-10T11:35:03.565854+0000","last_clean":"2024-09-10T11:35:03.565854+0000","last_became_active":"2024-09-10T11:35:02.584315+0000","last_became_peered":"2024-09-10T11:35:02.584315+0000","last_unstale":"2024-09-10T11:35:03.565854+0000","last_undegraded":"2024-09-10T11:35:03.565854+0000","last_fullsized":"2024-09-10T11:35:03.565854+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:18.417573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720431+0000","last_change":"2024-09-10T11:35:01.579801+0000","last_active":"2024-09-10T11:35:03.720431+0000","last_peered":"2024-09-10T11:35:03.720431+0000","last_clean":"2024-09-10T11:35:03.720431+0000","last_became_active":"2024-09-10T11:35:01.578207+0000","last_became_peered":"2024-09-10T11:35:01.578207+0000","last_unstale":"2024-09-10T11:35:03.720431+0000","last_undegraded":"2024-09-10T11:35:03.720431+0000","last_fullsized":"2024-09-10T11:35:03.720431+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:08.379206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.105929+0000","last_change":"2024-09-10T11:35:02.578576+0000","last_active":"2024-09-10T11:35:05.105929+0000","last_peered":"2024-09-10T11:35:05.105929+0000","last_clean":"2024-09-10T11:35:05.105929+0000","last_became_active":"2024-09-10T11:35:02.578335+0000","last_became_peered":"2024-09-10T11:35:02.578335+0000","last_unstale":"2024-09-10T11:35:05.105929+0000","last_undegraded":"2024-09-10T11:35:05.105929+0000","last_fullsized":"2024-09-10T11:35:05.105929+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:26.441455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402601+0000","last_change":"2024-09-10T11:35:01.572125+0000","last_active":"2024-09-10T11:35:05.402601+0000","last_peered":"2024-09-10T11:35:05.402601+0000","last_clean":"2024-09-10T11:35:05.402601+0000","last_became_active":"2024-09-10T11:35:01.571887+0000","last_became_peered":"2024-09-10T11:35:01.571887+0000","last_unstale":"2024-09-10T11:35:05.402601+0000","last_undegraded":"2024-09-10T11:35:05.402601+0000","last_fullsized":"2024-09-10T11:35:05.402601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:32.081097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720130+0000","last_change":"2024-09-10T11:35:02.578640+0000","last_active":"2024-09-10T11:35:03.720130+0000","last_peered":"2024-09-10T11:35:03.720130+0000","last_clean":"2024-09-10T11:35:03.720130+0000","last_became_active":"2024-09-10T11:35:02.578241+0000","last_became_peered":"2024-09-10T11:35:02.578241+0000","last_unstale":"2024-09-10T11:35:03.720130+0000","last_undegraded":"2024-09-10T11:35:03.720130+0000","last_fullsized":"2024-09-10T11:35:03.720130+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:33.428436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.566017+0000","last_change":"2024-09-10T11:35:01.578976+0000","last_active":"2024-09-10T11:35:03.566017+0000","last_peered":"2024-09-10T11:35:03.566017+0000","last_clean":"2024-09-10T11:35:03.566017+0000","last_became_active":"2024-09-10T11:35:01.578755+0000","last_became_peered":"2024-09-10T11:35:01.578755+0000","last_unstale":"2024-09-10T11:35:03.566017+0000","last_undegraded":"2024-09-10T11:35:03.566017+0000","last_fullsized":"2024-09-10T11:35:03.566017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:27.496026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.106034+0000","last_change":"2024-09-10T11:35:02.583676+0000","last_active":"2024-09-10T11:35:05.106034+0000","last_peered":"2024-09-10T11:35:05.106034+0000","last_clean":"2024-09-10T11:35:05.106034+0000","last_became_active":"2024-09-10T11:35:02.582835+0000","last_became_peered":"2024-09-10T11:35:02.582835+0000","last_unstale":"2024-09-10T11:35:05.106034+0000","last_undegraded":"2024-09-10T11:35:05.106034+0000","last_fullsized":"2024-09-10T11:35:05.106034+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:23.819690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.402996+0000","last_change":"2024-09-10T11:35:01.571499+0000","last_active":"2024-09-10T11:35:05.402996+0000","last_peered":"2024-09-10T11:35:05.402996+0000","last_clean":"2024-09-10T11:35:05.402996+0000","last_became_active":"2024-09-10T11:35:01.571267+0000","last_became_peered":"2024-09-10T11:35:01.571267+0000","last_unstale":"2024-09-10T11:35:05.402996+0000","last_undegraded":"2024-09-10T11:35:05.402996+0000","last_fullsized":"2024-09-10T11:35:05.402996+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:50.360211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.720320+0000","last_change":"2024-09-10T11:35:02.578632+0000","last_active":"2024-09-10T11:35:03.720320+0000","last_peered":"2024-09-10T11:35:03.720320+0000","last_clean":"2024-09-10T11:35:03.720320+0000","last_became_active":"2024-09-10T11:35:02.578354+0000","last_became_peered":"2024-09-10T11:35:02.578354+0000","last_unstale":"2024-09-10T11:35:03.720320+0000","last_undegraded":"2024-09-10T11:35:03.720320+0000","last_fullsized":"2024-09-10T11:35:03.720320+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:38.957493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:05.198719+0000","last_change":"2024-09-10T11:35:01.570370+0000","last_active":"2024-09-10T11:35:05.198719+0000","last_peered":"2024-09-10T11:35:05.198719+0000","last_clean":"2024-09-10T11:35:05.198719+0000","last_became_active":"2024-09-10T11:35:01.570120+0000","last_became_peered":"2024-09-10T11:35:01.570120+0000","last_unstale":"2024-09-10T11:35:05.198719+0000","last_undegraded":"2024-09-10T11:35:05.198719+0000","last_fullsized":"2024-09-10T11:35:05.198719+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:24.142262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565206+0000","last_change":"2024-09-10T11:35:02.582679+0000","last_active":"2024-09-10T11:35:03.565206+0000","last_peered":"2024-09-10T11:35:03.565206+0000","last_clean":"2024-09-10T11:35:03.565206+0000","last_became_active":"2024-09-10T11:35:02.579068+0000","last_became_peered":"2024-09-10T11:35:02.579068+0000","last_unstale":"2024-09-10T11:35:03.565206+0000","last_undegraded":"2024-09-10T11:35:03.565206+0000","last_fullsized":"2024-09-10T11:35:03.565206+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:18.099421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565214+0000","last_change":"2024-09-10T11:35:01.576531+0000","last_active":"2024-09-10T11:35:03.565214+0000","last_peered":"2024-09-10T11:35:03.565214+0000","last_clean":"2024-09-10T11:35:03.565214+0000","last_became_active":"2024-09-10T11:35:01.575776+0000","last_became_peered":"2024-09-10T11:35:01.575776+0000","last_unstale":"2024-09-10T11:35:03.565214+0000","last_undegraded":"2024-09-10T11:35:03.565214+0000","last_fullsized":"2024-09-10T11:35:03.565214+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:40:09.017720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565716+0000","last_change":"2024-09-10T11:35:02.584027+0000","last_active":"2024-09-10T11:35:03.565716+0000","last_peered":"2024-09-10T11:35:03.565716+0000","last_clean":"2024-09-10T11:35:03.565716+0000","last_became_active":"2024-09-10T11:35:02.582716+0000","last_became_peered":"2024-09-10T11:35:02.582716+0000","last_unstale":"2024-09-10T11:35:03.565716+0000","last_undegraded":"2024-09-10T11:35:03.565716+0000","last_fullsized":"2024-09-10T11:35:03.565716+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:01.555567+0000","last_clean_scrub_stamp":"2024-09-10T11:35:01.555567+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:51.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"122'1","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565273+0000","last_change":"2024-09-10T11:35:01.574933+0000","last_active":"2024-09-10T11:35:03.565273+0000","last_peered":"2024-09-10T11:35:03.565273+0000","last_clean":"2024-09-10T11:35:03.565273+0000","last_became_active":"2024-09-10T11:35:01.574408+0000","last_became_peered":"2024-09-10T11:35:01.574408+0000","last_unstale":"2024-09-10T11:35:03.565273+0000","last_undegraded":"2024-09-10T11:35:03.565273+0000","last_fullsized":"2024-09-10T11:35:03.565273+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:00.553312+0000","last_clean_scrub_stamp":"2024-09-10T11:35:00.553312+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:57.011705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-09-10T11:35:03.565366+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:03.565366+0000","last_peered":"2024-09-10T11:35:03.565366+0000","last_clean":"2024-09-10T11:35:03.565366+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:03.565366+0000","last_undegraded":"2024-09-10T11:35:03.565366+0000","last_fullsized":"2024-09-10T11:35:03.565366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199322662,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":660,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199323244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1239,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194073,"statfs":{"total":200000000,"available":198731208,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199322662,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":661,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199323050,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":660,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199323174,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1239,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194073,"statfs":{"total":200000000,"available":198731152,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":660,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194651,"statfs":{"total":200000000,"available":199323244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:10.167 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:35:10.168 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:35:10.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.310+0000 7ff749d9f640 1 Processor -- start 2024-09-10T11:35:10.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff749d9f640 1 -- start start 2024-09-10T11:35:10.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff749d9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff74415eb70 0x7ff7440c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff749d9f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7440c7ef0 con 0x7ff7440cd5a0 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff749d9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7440c81d0 con 0x7ff74406d530 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff749d9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff7440c84b0 con 0x7ff74415eb70 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff74959e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff7440cd5a0 0x7ff7440b2cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45264/0 (socket says 172.21.15.99:45264) 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.311+0000 7ff748d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff74415eb70 0x7ff7440c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff74959e640 1 -- 172.21.15.99:0/821761812 learned_addr learned my addr 172.21.15.99:0/821761812 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff748d9d640 1 -- 172.21.15.99:0/821761812 >> v1:172.21.15.196:6789/0 conn(0x7ff74406d530 legacy=0x7ff7440cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff748d9d640 1 -- 172.21.15.99:0/821761812 >> v1:172.21.15.99:6789/0 conn(0x7ff7440cd5a0 legacy=0x7ff7440b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff7437fe640 1 -- 172.21.15.99:0/821761812 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1002139370 0 0) 0x7ff7440c7ef0 con 0x7ff7440cd5a0 2024-09-10T11:35:10.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.312+0000 7ff748d9d640 1 -- 172.21.15.99:0/821761812 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7440c8790 con 0x7ff74415eb70 2024-09-10T11:35:10.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff748d9d640 1 --2- 172.21.15.99:0/821761812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440c8f90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d90 tx=0x7ff73402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904496004c97fd2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff7437fe640 1 -- 172.21.15.99:0/821761812 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734007740 con 0x7ff74415eb70 2024-09-10T11:35:10.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff7437fe640 1 -- 172.21.15.99:0/821761812 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7340078e0 con 0x7ff74415eb70 2024-09-10T11:35:10.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff7437fe640 1 -- 172.21.15.99:0/821761812 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734007cb0 con 0x7ff74415eb70 2024-09-10T11:35:10.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 -- 172.21.15.99:0/821761812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 msgr2=0x7ff7440c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 --2- 172.21.15.99:0/821761812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440c8f90 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff734009d90 tx=0x7ff73402a660 comp rx=0 tx=0).stop 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 -- 172.21.15.99:0/821761812 shutdown_connections 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 --2- 172.21.15.99:0/821761812 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440c8f90 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 -- 172.21.15.99:0/821761812 >> 172.21.15.99:0/821761812 conn(0x7ff74409ba30 msgr2=0x7ff74409be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 -- 172.21.15.99:0/821761812 shutdown_connections 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.313+0000 7ff749d9f640 1 -- 172.21.15.99:0/821761812 wait complete. 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.314+0000 7ff749d9f640 1 Processor -- start 2024-09-10T11:35:10.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 -- start start 2024-09-10T11:35:10.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 0x7ff74414c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 0x7ff7440b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7440c9650 con 0x7ff74415eb70 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff7440c94d0 con 0x7ff74406d530 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff749d9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff7440b3400 con 0x7ff7440cd5a0 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.315+0000 7ff74959e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44564/0 (socket says 172.21.15.99:44564) 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 -- 172.21.15.99:0/1982957253 learned_addr learned my addr 172.21.15.99:0/1982957253 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff748d9d640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 0x7ff74414c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff743fff640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 0x7ff7440b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 -- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 msgr2=0x7ff7440b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 0x7ff7440b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 -- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 msgr2=0x7ff74414c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 0x7ff74414c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 -- 172.21.15.99:0/1982957253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7440b58a0 con 0x7ff74415eb70 2024-09-10T11:35:10.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff743fff640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 0x7ff7440b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff748d9d640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 0x7ff74414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff74959e640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c480 tx=0x7ff73c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c005ce0 con 0x7ff74415eb70 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c007070 con 0x7ff74415eb70 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7440b5b80 con 0x7ff74415eb70 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0074e0 con 0x7ff74415eb70 2024-09-10T11:35:10.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.316+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7440b6420 con 0x7ff74415eb70 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.317+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7440b5f70 con 0x7ff74415eb70 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.318+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff73c01b400 con 0x7ff74415eb70 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.318+0000 7ff7417fa640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.318+0000 7ff748d9d640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.318+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7ff73c08aa20 con 0x7ff74415eb70 2024-09-10T11:35:10.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.319+0000 7ff748d9d640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 0x7ff72406e320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.322+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff73c059460 con 0x7ff74415eb70 2024-09-10T11:35:10.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.550+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff7440b37c0 con 0x7ff74415eb70 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.552+0000 7ff7417fa640 1 -- 172.21.15.99:0/1982957253 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7ff73c058d60 con 0x7ff74415eb70 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:35:10.553 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:35:10.554 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:35:10.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 msgr2=0x7ff72406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 0x7ff72406e320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402f000 comp rx=0 tx=0).stop 2024-09-10T11:35:10.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 msgr2=0x7ff7440b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00c480 tx=0x7ff73c00c940 comp rx=0 tx=0).stop 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 shutdown_connections 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff74415eb70 0x7ff7440b0e10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7440cd5a0 0x7ff7440b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 --2- 172.21.15.99:0/1982957253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff74406d530 0x7ff74414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 >> 172.21.15.99:0/1982957253 conn(0x7ff74409ba30 msgr2=0x7ff744070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 shutdown_connections 2024-09-10T11:35:10.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.554+0000 7ff749d9f640 1 -- 172.21.15.99:0/1982957253 wait complete. 2024-09-10T11:35:10.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:10.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.665+0000 7f4ba765c640 1 Processor -- start 2024-09-10T11:35:10.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.665+0000 7f4ba765c640 1 -- start start 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba765c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4ba01605c0 0x7f4ba01609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba765c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4ba0160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba765c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ba00d1bd0 con 0x7f4ba015cfd0 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba765c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4ba0075620 con 0x7f4ba01605c0 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba765c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba00752b0 con 0x7f4ba00ae020 2024-09-10T11:35:10.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba5bf9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4ba015cfd0 0x7f4ba015d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45268/0 (socket says 172.21.15.99:45268) 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba5bf9640 1 -- 172.21.15.99:0/651962184 learned_addr learned my addr 172.21.15.99:0/651962184 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba4bf7640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4ba0160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba53f8640 1 --2- 172.21.15.99:0/651962184 >> v2:172.21.15.196:3300/0 conn(0x7f4ba01605c0 0x7f4ba01609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3257419027 0 0) 0x7f4ba00d1bd0 con 0x7f4ba015cfd0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b88003520 con 0x7f4ba015cfd0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3871285110 0 0) 0x7f4b88003520 con 0x7f4ba015cfd0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 msgr2=0x7f4ba0160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4ba0160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 >> v2:172.21.15.196:3300/0 conn(0x7f4ba01605c0 msgr2=0x7f4ba01609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 --2- 172.21.15.99:0/651962184 >> v2:172.21.15.196:3300/0 conn(0x7f4ba01605c0 0x7f4ba01609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba00d1eb0 con 0x7f4ba015cfd0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4b98001670 con 0x7f4ba015cfd0 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 >> v1:172.21.15.99:6789/0 conn(0x7f4ba015cfd0 legacy=0x7f4ba015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4b88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0075620 con 0x7f4b88004d40 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba00752b0 con 0x7f4b880039b0 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8800b9c0 con 0x7f4ba00ae020 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.666+0000 7f4ba5bf9640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba53f8640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba4bf7640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4b88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 -- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 msgr2=0x7f4b88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba00ae020 0x7f4b88004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 -- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b880039b0 msgr2=0x7f4b88003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b880039b0 0x7f4b88003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 -- 172.21.15.99:0/651962184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88003520 con 0x7f4b88004d40 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.667+0000 7f4ba5bf9640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c210 tx=0x7f4b88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc97e8cae049131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98007d50 con 0x7f4b88004d40 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b98005af0 con 0x7f4b88004d40 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4b9ffff640 1 -- 172.21.15.99:0/651962184 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98007070 con 0x7f4b88004d40 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 -- 172.21.15.99:0/651962184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b88004d40 msgr2=0x7f4b880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 --2- 172.21.15.99:0/651962184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b88004d40 0x7f4b880051c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c210 tx=0x7f4b88005830 comp rx=0 tx=0).stop 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba5bf9640 1 -- 172.21.15.99:0/651962184 reap_dead start 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 -- 172.21.15.99:0/651962184 shutdown_connections 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 -- 172.21.15.99:0/651962184 >> 172.21.15.99:0/651962184 conn(0x7f4ba00730c0 msgr2=0x7f4ba00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 -- 172.21.15.99:0/651962184 shutdown_connections 2024-09-10T11:35:10.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.668+0000 7f4ba765c640 1 -- 172.21.15.99:0/651962184 wait complete. 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.669+0000 7f4ba765c640 1 Processor -- start 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.669+0000 7f4ba765c640 1 -- start start 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba0150940 0x7f4ba0153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0151120 0x7f4ba0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba0073d00 con 0x7f4ba0150d20 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba015a9e0 con 0x7f4ba0151120 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba765c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba0073e80 con 0x7f4ba0150940 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba4bf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba4bf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44586/0 (socket says 172.21.15.99:44586) 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba4bf7640 1 -- 172.21.15.99:0/1009920618 learned_addr learned my addr 172.21.15.99:0/1009920618 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba53f8640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba0150940 0x7f4ba0153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba5bf9640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0151120 0x7f4ba0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba4bf7640 1 -- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba0150940 msgr2=0x7f4ba0153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.670+0000 7f4ba4bf7640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba0150940 0x7f4ba0153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba4bf7640 1 -- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0151120 msgr2=0x7f4ba0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba4bf7640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0151120 0x7f4ba0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba4bf7640 1 -- 172.21.15.99:0/1009920618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba0153ff0 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba4bf7640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4b94011910 tx=0x7f4b94011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400faf0 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9400fc90 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba01542d0 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94014440 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.671+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba00b1080 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.672+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba01609a0 con 0x7f4ba0150d20 2024-09-10T11:35:10.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.673+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b94012040 con 0x7f4ba0150d20 2024-09-10T11:35:10.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.673+0000 7f4b9dffb640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:10.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.673+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f4b9408b530 con 0x7f4ba0150d20 2024-09-10T11:35:10.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.673+0000 7f4ba53f8640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:10.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.673+0000 7f4ba53f8640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 0x7f4b8c06e320 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007520 tx=0x7f4b9000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:10.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.677+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b94019030 con 0x7f4ba0150d20 2024-09-10T11:35:10.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.875+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ba00d0e00 con 0x7f4ba0150d20 2024-09-10T11:35:10.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.875+0000 7f4b9dffb640 1 -- 172.21.15.99:0/1009920618 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f4b94059890 con 0x7f4ba0150d20 2024-09-10T11:35:10.876 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:10.876 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:10.877 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:10.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 msgr2=0x7f4b8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 0x7f4b8c06e320 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4b90007520 tx=0x7f4b9000b040 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 msgr2=0x7f4ba0151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4b94011910 tx=0x7f4b94011dd0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 shutdown_connections 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba0151120 0x7f4ba0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ba0150d20 0x7f4ba0151aa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 --2- 172.21.15.99:0/1009920618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba0150940 0x7f4ba0153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 >> 172.21.15.99:0/1009920618 conn(0x7f4ba00730c0 msgr2=0x7f4ba015f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 shutdown_connections 2024-09-10T11:35:10.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:10.877+0000 7f4ba765c640 1 -- 172.21.15.99:0/1009920618 wait complete. 2024-09-10T11:35:11.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:35:01.900031+0000', 'modified': '2024-09-10T11:35:03.905945+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6594}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6594': {'gid': 6594, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6836/1212129697', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 1212129697}, {'type': 'v1', 'addr': '172.21.15.99:6836', 'nonce': 1212129697}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6594, 'qdb_cluster': [6594]} 2024-09-10T11:35:11.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:35:11.292 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T11:35:11.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.394+0000 7f1575911640 1 Processor -- start 2024-09-10T11:35:11.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.394+0000 7f1575911640 1 -- start start 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f1575911640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f157015f020 0x7f15700d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f1575911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f15700b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f1575911640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15700d3b40 con 0x7f15700d32f0 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f1575911640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f15700d3e20 con 0x7f157015f020 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f1575911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15700b4810 con 0x7f15700d1d10 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156e7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15700d32f0 0x7f15700d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45280/0 (socket says 172.21.15.99:45280) 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156e7fc640 1 -- 172.21.15.99:0/13486638 learned_addr learned my addr 172.21.15.99:0/13486638 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f15700b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156effd640 1 --2- 172.21.15.99:0/13486638 >> v2:172.21.15.196:3300/0 conn(0x7f157015f020 0x7f15700d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374465918 0 0) 0x7f15700d3b40 con 0x7f15700d32f0 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.395+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1558003520 con 0x7f15700d32f0 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3146532953 0 0) 0x7f1558003520 con 0x7f15700d32f0 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 msgr2=0x7f15700b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f15700b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 >> v2:172.21.15.196:3300/0 conn(0x7f157015f020 msgr2=0x7f15700d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 --2- 172.21.15.99:0/13486638 >> v2:172.21.15.196:3300/0 conn(0x7f157015f020 0x7f15700d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15700b4af0 con 0x7f15700d32f0 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1564001670 con 0x7f15700d32f0 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f15700b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 >> v1:172.21.15.99:6789/0 conn(0x7f15700d32f0 legacy=0x7f15700d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15580039b0 0x7f1558003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f1558004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15700d3e20 con 0x7f1558004d40 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15700b4810 con 0x7f15580039b0 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f155800b9c0 con 0x7f15700d1d10 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156effd640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15580039b0 0x7f1558003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156e7fc640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f1558004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 -- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 msgr2=0x7f1558004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f1558004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 -- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15580039b0 msgr2=0x7f1558003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15580039b0 0x7f1558003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156f7fe640 1 -- 172.21.15.99:0/13486638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1558003520 con 0x7f1558004d40 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156effd640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15580039b0 0x7f1558003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.396+0000 7f156e7fc640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15700d1d10 0x7f1558004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f156f7fe640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15580051c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f15700c8420 tx=0x7f1558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18daaf6596f7fefc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568033070 con 0x7f1558004d40 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1568003b90 con 0x7f1558004d40 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f156dffb640 1 -- 172.21.15.99:0/13486638 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568009de0 con 0x7f1558004d40 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 -- 172.21.15.99:0/13486638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 msgr2=0x7f15580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 --2- 172.21.15.99:0/13486638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15580051c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f15700c8420 tx=0x7f1558005830 comp rx=0 tx=0).stop 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f156f7fe640 1 -- 172.21.15.99:0/13486638 reap_dead start 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 -- 172.21.15.99:0/13486638 shutdown_connections 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 -- 172.21.15.99:0/13486638 >> 172.21.15.99:0/13486638 conn(0x7f157009dfa0 msgr2=0x7f15700d4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 -- 172.21.15.99:0/13486638 shutdown_connections 2024-09-10T11:35:11.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.397+0000 7f1575911640 1 -- 172.21.15.99:0/13486638 wait complete. 2024-09-10T11:35:11.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 Processor -- start 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 -- start start 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 0x7f1570162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 0x7f1570160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f15700d38e0 con 0x7f1558004d40 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15700d3760 con 0x7f1570161b80 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.399+0000 7f1575911640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15700b52f0 con 0x7f15701603d0 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44606/0 (socket says 172.21.15.99:44606) 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 -- 172.21.15.99:0/4021900305 learned_addr learned my addr 172.21.15.99:0/4021900305 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156f7fe640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 0x7f1570160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156e7fc640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 0x7f1570162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 -- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 msgr2=0x7f1570160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 0x7f1570160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 -- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 msgr2=0x7f1570162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 0x7f1570162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 -- 172.21.15.99:0/4021900305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15700cc990 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156f7fe640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 0x7f1570160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156e7fc640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 0x7f1570162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f156effd640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f15600115d0 tx=0x7f1560011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f1557fff640 1 -- 172.21.15.99:0/4021900305 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156000f8f0 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f1557fff640 1 -- 172.21.15.99:0/4021900305 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156000fa90 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15700ccc70 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f1557fff640 1 -- 172.21.15.99:0/4021900305 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156000fe60 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.400+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15700cd230 con 0x7f1558004d40 2024-09-10T11:35:11.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.402+0000 7f1557fff640 1 -- 172.21.15.99:0/4021900305 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1560015460 con 0x7f1558004d40 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.402+0000 7f1557fff640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 0x7f155c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.402+0000 7f1557fff640 1 -- 172.21.15.99:0/4021900305 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f156008b790 con 0x7f1558004d40 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.402+0000 7f156e7fc640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 0x7f155c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.402+0000 7f156e7fc640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 0x7f155c06e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1564000fb0 tx=0x7f1564001e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.403+0000 7f15786a7740 1 Processor -- start 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.403+0000 7f15786a7740 1 -- start start 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 0x55f479b536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f479ab5520 con 0x55f479b532c0 2024-09-10T11:35:11.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f479a5d3e0 con 0x55f479b54a50 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f479ab14e0 con 0x55f479b50a80 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44618/0 (socket says 172.21.15.99:44618) 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 -- 172.21.15.99:0/2123765640 learned_addr learned my addr 172.21.15.99:0/2123765640 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156e7fc640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 0x55f479b536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156f7fe640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 -- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 msgr2=0x55f479b536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 0x55f479b536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 -- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 msgr2=0x55f479b4d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b4d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 -- 172.21.15.99:0/2123765640 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f479a782f0 con 0x55f479b532c0 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156e7fc640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 0x55f479b536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f156effd640 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1560062180 tx=0x7f1560059c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bffc12d37da0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15557fa640 1 -- 172.21.15.99:0/2123765640 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560057b70 con 0x55f479b532c0 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15557fa640 1 -- 172.21.15.99:0/2123765640 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1560057dd0 con 0x55f479b532c0 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.404+0000 7f15557fa640 1 -- 172.21.15.99:0/2123765640 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560058f20 con 0x55f479b532c0 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 -- 172.21.15.99:0/2123765640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 msgr2=0x55f479b98a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1560062180 tx=0x7f1560059c10 comp rx=0 tx=0).stop 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 -- 172.21.15.99:0/2123765640 shutdown_connections 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b4d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b50a80 0x55f479b536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 --2- 172.21.15.99:0/2123765640 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479b98a60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 -- 172.21.15.99:0/2123765640 >> 172.21.15.99:0/2123765640 conn(0x55f479b4fa80 msgr2=0x55f479b97f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 -- 172.21.15.99:0/2123765640 shutdown_connections 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.405+0000 7f15786a7740 1 -- 172.21.15.99:0/2123765640 wait complete. 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.406+0000 7f15786a7740 1 -- start start 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.406+0000 7f15786a7740 1 Processor -- start 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b7d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f479b4ef20 con 0x55f479b76bb0 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f479a3e450 con 0x55f479b54a50 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f479b99290 con 0x55f479b532c0 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 10 client.0 _ll_register_callbacks cb 0x55f4797c7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44626/0 (socket says 172.21.15.99:44626) 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 -- 172.21.15.99:0/788732962 learned_addr learned my addr 172.21.15.99:0/788732962 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156effd640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b7d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156e7fc640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 -- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 msgr2=0x55f479b7d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b7d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 -- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 msgr2=0x55f479b76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b76670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 -- 172.21.15.99:0/788732962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4797c82d0 con 0x55f479b76bb0 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f156f7fe640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f156802fa30 tx=0x7f1568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cff12bc68523f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f154e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568033040 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f1568043af0 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.407+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1560057dd0 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1568009a50 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 10 client.6759 fetch_fsmap learned FSMap version 48 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55f4798926b0 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1560057dd0 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7f1568003990 con 0x55f479b76bb0 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 10 client.6759 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 10 client.6759 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 20 client.6759 populate_metadata read hostname 'smithi099' 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479bb3b50 0x55f479bb5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 4 client.6759 mds_command: new command op to 6594 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 --> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f479ac2c50 con 0x55f479bb3b50 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f156effd640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479bb3b50 0x55f479bb5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f154e7fc640 10 client.6759 ms_handle_connect on v2:172.21.15.99:6834/1212129697 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.408+0000 7f156effd640 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479bb3b50 0x55f479bb5f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f3286533f6a56a server_cookie=9101a59e423c5b9a in_seq=0 out_seq=0 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f154e7fc640 1 -- 172.21.15.99:0/788732962 <== mds.0 v2:172.21.15.99:6834/1212129697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f479ac2c50 con 0x55f479bb3b50 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f154e7fc640 10 client.6759 handle_command_reply: tid=0 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 client.6759 shutdown 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 msgr2=0x55f479ba8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f156802fa30 tx=0x7f1568004a40 comp rx=0 tx=0).stop 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 shutdown_connections 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479bb3b50 0x55f479bb5f40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba8470 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479b76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f154e7fc640 0 client.6759 ms_handle_reset on v2:172.21.15.99:6834/1212129697 2024-09-10T11:35:11.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 --2- 172.21.15.99:0/788732962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b7d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 >> 172.21.15.99:0/788732962 conn(0x55f479b50a80 msgr2=0x55f479b51020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 shutdown_connections 2024-09-10T11:35:11.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 1 -- 172.21.15.99:0/788732962 wait complete. 2024-09-10T11:35:11.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.409+0000 7f15786a7740 20 client.6759 trim_cache size 0 max 16384 2024-09-10T11:35:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.427+0000 7f15786a7740 1 Processor -- start 2024-09-10T11:35:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.427+0000 7f15786a7740 1 -- start start 2024-09-10T11:35:11.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1570200af0 con 0x55f479b76bb0 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f15700b8c60 con 0x55f479b54a50 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f15702005f0 con 0x55f479b532c0 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44642/0 (socket says 172.21.15.99:44642) 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 -- 172.21.15.99:0/914296132 learned_addr learned my addr 172.21.15.99:0/914296132 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 -- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 msgr2=0x55f479b980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156e7fc640 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 -- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 msgr2=0x55f479ba75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 -- 172.21.15.99:0/914296132 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f479b97b10 con 0x55f479b76bb0 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f156f7fe640 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1568002230 tx=0x7f1568030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c002ef3f8149fc4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f1554ff9640 1 -- 172.21.15.99:0/914296132 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1568052040 con 0x55f479b76bb0 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f1554ff9640 1 -- 172.21.15.99:0/914296132 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1568009a50 con 0x55f479b76bb0 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.428+0000 7f1554ff9640 1 -- 172.21.15.99:0/914296132 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156802fca0 con 0x55f479b76bb0 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 -- 172.21.15.99:0/914296132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 msgr2=0x55f479ba7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1568002230 tx=0x7f1568030000 comp rx=0 tx=0).stop 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 -- 172.21.15.99:0/914296132 shutdown_connections 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b76bb0 0x55f479ba7b10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 --2- 172.21.15.99:0/914296132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b532c0 0x55f479b980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 -- 172.21.15.99:0/914296132 >> 172.21.15.99:0/914296132 conn(0x55f479b50a80 msgr2=0x55f479b541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 -- 172.21.15.99:0/914296132 shutdown_connections 2024-09-10T11:35:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.429+0000 7f15786a7740 1 -- 172.21.15.99:0/914296132 wait complete. 2024-09-10T11:35:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 -- start start 2024-09-10T11:35:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 Processor -- start 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b76bb0 0x55f479ba9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f479b7db30 con 0x55f479b532c0 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f479b7c730 con 0x55f479b54a50 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55f479b75770 con 0x55f479b76bb0 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f15786a7740 10 client.0 _ll_register_callbacks cb 0x7f15580036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:35:11.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f156effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44650/0 (socket says 172.21.15.99:44650) 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.430+0000 7f156effd640 1 -- 172.21.15.99:0/1767503857 learned_addr learned my addr 172.21.15.99:0/1767503857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156e7fc640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156f7fe640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b76bb0 0x55f479ba9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 -- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b76bb0 msgr2=0x55f479ba9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b76bb0 0x55f479ba9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 -- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 msgr2=0x55f479ba97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 -- 172.21.15.99:0/1767503857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f479b7d6c0 con 0x55f479b532c0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156e7fc640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f156effd640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1560055f30 tx=0x7f1560089c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e84c6a68434fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:35:11.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560056140 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f1560058f20 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1568009a50 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f156007d050 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 10 client.6765 fetch_fsmap learned FSMap version 48 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55f479bab360 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1568009a50 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7f1560080020 con 0x55f479b532c0 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 10 client.6765 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 10 client.6765 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T11:35:11.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 20 client.6765 populate_metadata read hostname 'smithi099' 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479b52190 0x55f479b52570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 4 client.6765 mds_command: new command op to 6594 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.431+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 --> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] -- command(tid 0: {"prefix": "status"}) -- 0x55f479abcfa0 con 0x55f479b52190 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f156e7fc640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479b52190 0x55f479b52570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f154e7fc640 10 client.6765 ms_handle_connect on v2:172.21.15.99:6834/1212129697 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f156e7fc640 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479b52190 0x55f479b52570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb75de0620b2a0ec server_cookie=1a0d8eb677f300a1 in_seq=0 out_seq=0 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f154e7fc640 1 -- 172.21.15.99:0/1767503857 <== mds.0 v2:172.21.15.99:6834/1212129697 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f479abcfa0 con 0x55f479b52190 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f154e7fc640 10 client.6765 handle_command_reply: tid=0 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 client.6765 shutdown 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 msgr2=0x55f479c184d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1560055f30 tx=0x7f1560089c20 comp rx=0 tx=0).stop 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 shutdown_connections 2024-09-10T11:35:11.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:6834/1212129697,v1:172.21.15.99:6836/1212129697] conn(0x55f479b52190 0x55f479b52570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f479b76bb0 0x55f479ba9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f479b54a50 0x55f479ba97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f154e7fc640 0 client.6765 ms_handle_reset on v2:172.21.15.99:6834/1212129697 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 --2- 172.21.15.99:0/1767503857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f479b532c0 0x55f479c184d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 >> 172.21.15.99:0/1767503857 conn(0x55f479b50a80 msgr2=0x55f479b9b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 shutdown_connections 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 1 -- 172.21.15.99:0/1767503857 wait complete. 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.432+0000 7f15786a7740 20 client.6765 trim_cache size 0 max 16384 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:11.441 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6594, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 8.8704115259999998, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 48, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 123, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 123, 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 13.219071562 2024-09-10T11:35:11.442 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:11.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 msgr2=0x7f155c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 0x7f155c06e2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1564000fb0 tx=0x7f1564001e10 comp rx=0 tx=0).stop 2024-09-10T11:35:11.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 msgr2=0x7f15700b9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f15600115d0 tx=0x7f1560011a90 comp rx=0 tx=0).stop 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 shutdown_connections 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f155c06be30 0x7f155c06e2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15701603d0 0x7f1570160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1570161b80 0x7f1570162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 --2- 172.21.15.99:0/4021900305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558004d40 0x7f15700b9c90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 >> 172.21.15.99:0/4021900305 conn(0x7f157009dfa0 msgr2=0x7f15700b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 shutdown_connections 2024-09-10T11:35:11.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.433+0000 7f1575911640 1 -- 172.21.15.99:0/4021900305 wait complete. 2024-09-10T11:35:11.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:35:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.544+0000 7f90ad5e4640 1 Processor -- start 2024-09-10T11:35:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.544+0000 7f90ad5e4640 1 -- start start 2024-09-10T11:35:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90ad5e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90a80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90ad5e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f90a81626c0 0x7f90a80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90ad5e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a80d52d0 con 0x7f90a80d4e40 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90ad5e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f90a815ff70 con 0x7f90a81626c0 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90ad5e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f90a80d1620 con 0x7f90a815eb80 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a77fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f90a80d4e40 0x7f90a80d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45284/0 (socket says 172.21.15.99:45284) 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a77fe640 1 -- 172.21.15.99:0/676487773 learned_addr learned my addr 172.21.15.99:0/676487773 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90a80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1917955249 0 0) 0x7f90a80d52d0 con 0x7f90a80d4e40 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f909e7fc640 1 --2- 172.21.15.99:0/676487773 >> v2:172.21.15.196:3300/0 conn(0x7f90a81626c0 0x7f90a80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9094003520 con 0x7f90a80d4e40 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216687279 0 0) 0x7f9094003520 con 0x7f90a80d4e40 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 msgr2=0x7f90a80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90a80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 >> v2:172.21.15.196:3300/0 conn(0x7f90a81626c0 msgr2=0x7f90a80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 --2- 172.21.15.99:0/676487773 >> v2:172.21.15.196:3300/0 conn(0x7f90a81626c0 0x7f90a80d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a8162bc0 con 0x7f90a80d4e40 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f90a00019f0 con 0x7f90a80d4e40 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 >> v1:172.21.15.99:6789/0 conn(0x7f90a80d4e40 legacy=0x7f90a80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90940039b0 0x7f9094003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9094004800 0x7f9094004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90a815ff70 con 0x7f90940039b0 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90a80d1620 con 0x7f9094004800 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.545+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f909400b9c0 con 0x7f90a815eb80 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90940039b0 0x7f9094003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f909e7fc640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9094004800 0x7f9094004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a77fe640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 -- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 msgr2=0x7f90940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90a815eb80 0x7f90940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 -- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9094004800 msgr2=0x7f9094004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9094004800 0x7f9094004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 -- 172.21.15.99:0/676487773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9094003520 con 0x7f90940039b0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.546+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90940039b0 0x7f9094003dd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9098009af0 tx=0x7f9094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af50e61d13261c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098036070 con 0x7f90940039b0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098031400 con 0x7f90940039b0 2024-09-10T11:35:11.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90a67fc640 1 -- 172.21.15.99:0/676487773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098009690 con 0x7f90940039b0 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 -- 172.21.15.99:0/676487773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90940039b0 msgr2=0x7f9094003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/676487773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90940039b0 0x7f9094003dd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9098009af0 tx=0x7f9094005830 comp rx=0 tx=0).stop 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90a77fe640 1 -- 172.21.15.99:0/676487773 reap_dead start 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 -- 172.21.15.99:0/676487773 shutdown_connections 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 -- 172.21.15.99:0/676487773 >> 172.21.15.99:0/676487773 conn(0x7f90a808e680 msgr2=0x7f90a809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 -- 172.21.15.99:0/676487773 shutdown_connections 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.547+0000 7f90ad5e4640 1 -- 172.21.15.99:0/676487773 wait complete. 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.548+0000 7f90ad5e4640 1 Processor -- start 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.548+0000 7f90ad5e4640 1 -- start start 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90940039b0 0x7f90a8153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90a80b31f0 0x7f90a80ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8163460 con 0x7f90a80ceb50 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90a8163180 con 0x7f90a80b31f0 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90ad5e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f90a80d2230 con 0x7f90940039b0 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90a77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44664/0 (socket says 172.21.15.99:44664) 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.549+0000 7f90a77fe640 1 -- 172.21.15.99:0/1085723318 learned_addr learned my addr 172.21.15.99:0/1085723318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f909e7fc640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90a80b31f0 0x7f90a80ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90940039b0 0x7f90a8153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 -- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90940039b0 msgr2=0x7f90a8153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90940039b0 0x7f90a8153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 -- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90a80b31f0 msgr2=0x7f90a80ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90a80b31f0 0x7f90a80ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 -- 172.21.15.99:0/1085723318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a80b2760 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90a77fe640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b44a0 tx=0x7f90a000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a000ebb0 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a0018070 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a80cd000 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0013420 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.550+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a80ba130 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.551+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a8161ab0 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.551+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90a00135c0 con 0x7f90a80ceb50 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.552+0000 7f909ffff640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 0x7f908806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:11.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.552+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f90a008b100 con 0x7f90a80ceb50 2024-09-10T11:35:11.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.552+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 0x7f908806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:11.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.552+0000 7f90a6ffd640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 0x7f908806e320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9098033000 tx=0x7f9098008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:11.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.556+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90a0059b40 con 0x7f90a80ceb50 2024-09-10T11:35:11.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.756+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f90a81617d0 con 0x7f90a80ceb50 2024-09-10T11:35:11.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.756+0000 7f909ffff640 1 -- 172.21.15.99:0/1085723318 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f90a0059440 con 0x7f90a80ceb50 2024-09-10T11:35:11.757 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 msgr2=0x7f908806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 0x7f908806e320 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9098033000 tx=0x7f9098008030 comp rx=0 tx=0).stop 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 msgr2=0x7f90a80cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b44a0 tx=0x7f90a000e530 comp rx=0 tx=0).stop 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 shutdown_connections 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f908806be60 0x7f908806e320 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80ceb50 0x7f90a80cc240 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90a80b31f0 0x7f90a80ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 --2- 172.21.15.99:0/1085723318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90940039b0 0x7f90a8153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:11.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 >> 172.21.15.99:0/1085723318 conn(0x7f90a808e680 msgr2=0x7f90a806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:11.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 shutdown_connections 2024-09-10T11:35:11.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11.758+0000 7f90ad5e4640 1 -- 172.21.15.99:0/1085723318 wait complete. 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:35:11.770 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:11.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:11.771 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.787 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42537sec preferred_lft 42537sec 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:35:11.788 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:11.789 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:35:11.790 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:11.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:11.790 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:35:11.850 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:35:11.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:11.852 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:35:11.909 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:35:11.909 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:35:11.909 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:35:11.909 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:35:11.909 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:35:11.909 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:35:11.910 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:35:11.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:35:11.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:35:11.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:35:11.910 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:35:11.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:35:12.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:12.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.036 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:35:12.051 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:35:12.051 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:35:12.051 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:12.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:35:12.116 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:35:12.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:12.117 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:12.172 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:12.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:12.173 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:12.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:35:12.293 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:35:12.349 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:35:12.350 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:12.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.352 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:35:12.404 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:35:12.405 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:35:12.447 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:35:12.471 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:35:12.528 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:12.529 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:12.530 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:12.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:35:12.527+0000 7f77d471d480 -1 init, newargv = 0x557170219fc0 newargc=15 2024-09-10T11:35:12.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[52432]: starting ceph client 2024-09-10T11:35:12.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.532 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:35:12.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[52432]: starting fuse 2024-09-10T11:35:12.587 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:35:12.587 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:12.588 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:12.589 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:12.705 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:12.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:12.716 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:12.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52432.asok status 2024-09-10T11:35:12.848 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "52432", 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:12.849 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6780, 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6780 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:12.850 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6780 192.168.0.1:0/3726021730", 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3726021730", 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 48, 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 123, 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:35:12.851 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:12.852 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:12.852 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:12.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.857 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:35:12.857 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:12.873 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:35:12.873 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:12.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:12.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:12.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:35:13.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.077+0000 7fdfbc581640 1 Processor -- start 2024-09-10T11:35:13.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.077+0000 7fdfbc581640 1 -- start start 2024-09-10T11:35:13.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbc581640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfb4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbc581640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdfb406e5d0 0x7fdfb406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbc581640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb4152e60 con 0x7fdfb415c060 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbc581640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb406ef50 con 0x7fdfb406e5d0 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbc581640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb40d15e0 con 0x7fdfb40d1200 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbab1e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdfb415c060 0x7fdfb415c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57088/0 (socket says 172.21.15.99:57088) 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfbab1e640 1 -- 172.21.15.99:0/4251458216 learned_addr learned my addr 172.21.15.99:0/4251458216 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594136992 0 0) 0x7fdfb4152e60 con 0x7fdfb415c060 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfa0003520 con 0x7fdfb415c060 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfba31d640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfb4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.078+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> v2:172.21.15.196:3300/0 conn(0x7fdfb406e5d0 0x7fdfb406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 525354065 0 0) 0x7fdfa0003520 con 0x7fdfb415c060 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 msgr2=0x7fdfb4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:13.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfb4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 >> v2:172.21.15.196:3300/0 conn(0x7fdfb406e5d0 msgr2=0x7fdfb406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 --2- 172.21.15.99:0/4251458216 >> v2:172.21.15.196:3300/0 conn(0x7fdfb406e5d0 0x7fdfb406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb4072680 con 0x7fdfb415c060 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fdfac0019f0 con 0x7fdfb415c060 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> v2:172.21.15.196:3300/0 conn(0x7fdfb406e5d0 0x7fdfb406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 >> v1:172.21.15.99:6789/0 conn(0x7fdfb415c060 legacy=0x7fdfb415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfa00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa0003f80 0x7fdfa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0004d40 0x7fdfa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb406ef50 con 0x7fdfa0003f80 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb40d15e0 con 0x7fdfa0004d40 2024-09-10T11:35:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa000bb20 con 0x7fdfb40d1200 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa0003f80 0x7fdfa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfba31d640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfa00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfbab1e640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0004d40 0x7fdfa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 msgr2=0x7fdfa00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfa00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0004d40 msgr2=0x7fdfa00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0004d40 0x7fdfa00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/4251458216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa0003520 con 0x7fdfa0003f80 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.079+0000 7fdfba31d640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40d1200 0x7fdfa00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa0003f80 0x7fdfa0004800 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40b4200 tx=0x7fdfa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c158e723e1fd5193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4037070 con 0x7fdfa0003f80 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa402ad00 con 0x7fdfa0003f80 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfb931b640 1 -- 172.21.15.99:0/4251458216 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400f040 con 0x7fdfa0003f80 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfbc581640 1 -- 172.21.15.99:0/4251458216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa0003f80 msgr2=0x7fdfa0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfbc581640 1 --2- 172.21.15.99:0/4251458216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfa0003f80 0x7fdfa0004800 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40b4200 tx=0x7fdfa0005830 comp rx=0 tx=0).stop 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.080+0000 7fdfbab1e640 1 -- 172.21.15.99:0/4251458216 reap_dead start 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.081+0000 7fdfbc581640 1 -- 172.21.15.99:0/4251458216 shutdown_connections 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.081+0000 7fdfbc581640 1 -- 172.21.15.99:0/4251458216 >> 172.21.15.99:0/4251458216 conn(0x7fdfb406bc10 msgr2=0x7fdfb406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.081+0000 7fdfbc581640 1 -- 172.21.15.99:0/4251458216 shutdown_connections 2024-09-10T11:35:13.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.081+0000 7fdfbc581640 1 -- 172.21.15.99:0/4251458216 wait complete. 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 Processor -- start 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 -- start start 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0003f80 0x7fdfb40ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40cbf30 0x7fdfb40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4072f20 con 0x7fdfb40ac270 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4072c40 con 0x7fdfa0003f80 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.082+0000 7fdfbc581640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb415cd90 con 0x7fdfb40cbf30 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39270/0 (socket says 172.21.15.99:39270) 2024-09-10T11:35:13.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/1643515059 learned_addr learned my addr 172.21.15.99:0/1643515059 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfbab1e640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40cbf30 0x7fdfb40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfba31d640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0003f80 0x7fdfb40ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40cbf30 msgr2=0x7fdfb40cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40cbf30 0x7fdfb40cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0003f80 msgr2=0x7fdfb40ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0003f80 0x7fdfb40ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 -- 172.21.15.99:0/1643515059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb40acac0 con 0x7fdfb40ac270 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfb9b1c640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402a7f0 tx=0x7fdfa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4040020 con 0x7fdfb40ac270 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4038040 con 0x7fdfb40ac270 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb40acda0 con 0x7fdfb40ac270 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4037070 con 0x7fdfb40ac270 2024-09-10T11:35:13.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.083+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb40c9ec0 con 0x7fdfb40ac270 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.084+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb406ea10 con 0x7fdfb40ac270 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.085+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfa4043020 con 0x7fdfb40ac270 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.085+0000 7fdfaaffd640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 0x7fdf9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.085+0000 7fdfba31d640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 0x7fdf9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.085+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7fdfa40ab0d0 con 0x7fdfb40ac270 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.086+0000 7fdfba31d640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 0x7fdf9806e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009a60 tx=0x7fdfb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.090+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfa404a020 con 0x7fdfb40ac270 2024-09-10T11:35:13.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.288+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdfb415e3f0 con 0x7fdfb40ac270 2024-09-10T11:35:13.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.288+0000 7fdfaaffd640 1 -- 172.21.15.99:0/1643515059 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7fdfa4079430 con 0x7fdfb40ac270 2024-09-10T11:35:13.289 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:35:13.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 msgr2=0x7fdf9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 0x7fdf9806e2f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009a60 tx=0x7fdfb0008040 comp rx=0 tx=0).stop 2024-09-10T11:35:13.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 msgr2=0x7fdfb414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdfa402a7f0 tx=0x7fdfa4004730 comp rx=0 tx=0).stop 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 shutdown_connections 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdf9806be30 0x7fdf9806e2f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdfb40cbf30 0x7fdfb40cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfb40ac270 0x7fdfb414ec00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 --2- 172.21.15.99:0/1643515059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfa0003f80 0x7fdfb40ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 >> 172.21.15.99:0/1643515059 conn(0x7fdfb406bc10 msgr2=0x7fdfb415d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 shutdown_connections 2024-09-10T11:35:13.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:13.290+0000 7fdfbc581640 1 -- 172.21.15.99:0/1643515059 wait complete. 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:35:13.302 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:13.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.303 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:13.323 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42551sec preferred_lft 42551sec 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.324 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:35:13.325 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:13.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.326 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:35:13.391 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:35:13.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.393 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:35:13.453 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:35:13.453 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:35:13.453 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:35:13.453 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:35:13.453 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:35:13.454 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:35:13.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:35:13.583 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:13.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.590 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:35:13.605 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:35:13.605 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:35:13.606 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:13.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:35:13.671 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:35:13.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:13.671 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:13.727 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:13.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:13.728 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:13.783 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:35:13.847 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:13.904 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:35:13.904 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:35:13.904 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:35:13.904 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:35:13.904 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:35:13.905 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:35:13.906 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:35:13.906 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:35:13.907 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:13.908 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:35:13.960 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:35:13.960 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:35:14.002 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:35:14.027 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:14.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:35:14.082+0000 7f1a886fb480 -1 init, newargv = 0x55a7d9314460 newargc=15 2024-09-10T11:35:14.084 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[37534]: starting ceph client 2024-09-10T11:35:14.084 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:35:14.085 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:35:14.086 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:14.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:14.087 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:35:14.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[37534]: starting fuse 2024-09-10T11:35:14.102 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:35:14.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:35:14.103 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:35:14.104 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:35:14.105 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:35:14.105 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:14.105 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:14.105 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:35:14.105 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:35:14.219 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.37534.asok 2024-09-10T11:35:14.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:14.230 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.37534.asok 2024-09-10T11:35:14.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:14.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.37534.asok status 2024-09-10T11:35:14.362 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:35:14.362 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "37534", 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 6259, 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:35:14.363 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 6259 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2689508024 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2689508024 2024-09-10T11:35:14.364 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.6259 192.168.0.1:0/2689508024", 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/2689508024", 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 48, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 123, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:35:14.365 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:35:14.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:14.371 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:35:14.371 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:14.387 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:35:14.387 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:14.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:14.387 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:14.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:14.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.562+0000 7f2ffe077640 1 Processor -- start 2024-09-10T11:35:14.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.562+0000 7f2ffe077640 1 -- start start 2024-09-10T11:35:14.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.562+0000 7f2ffe077640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ff80cfb90 0x7f2ff80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ffe077640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ff80d1410 0x7f2ff80cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ffe077640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ff806df80 con 0x7f2ff80b3660 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ffe077640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff80d18a0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ffe077640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ff80d2120 con 0x7f2ff80d1410 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ff80cfb90 0x7f2ff80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ff80cfb90 0x7f2ff80b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39280/0 (socket says 172.21.15.99:39280) 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 -- 172.21.15.99:0/137656378 learned_addr learned my addr 172.21.15.99:0/137656378 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:14.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff7fff640 1 --2- 172.21.15.99:0/137656378 >> v2:172.21.15.196:3300/0 conn(0x7f2ff80d1410 0x7f2ff80cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 -- 172.21.15.99:0/137656378 >> v2:172.21.15.196:3300/0 conn(0x7f2ff80d1410 msgr2=0x7f2ff80cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 --2- 172.21.15.99:0/137656378 >> v2:172.21.15.196:3300/0 conn(0x7f2ff80d1410 0x7f2ff80cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 -- 172.21.15.99:0/137656378 >> v1:172.21.15.196:6789/0 conn(0x7f2ff80b3660 legacy=0x7f2ff80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.563+0000 7f2ff77fe640 1 -- 172.21.15.99:0/137656378 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff80d1ad0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ff77fe640 1 --2- 172.21.15.99:0/137656378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff80b5170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8073970 tx=0x7f2fe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d9db1c401c0794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ff6ffd640 1 -- 172.21.15.99:0/137656378 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8037070 con 0x7f2ff80cfb90 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ff6ffd640 1 -- 172.21.15.99:0/137656378 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8032440 con 0x7f2ff80cfb90 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ff6ffd640 1 -- 172.21.15.99:0/137656378 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8032810 con 0x7f2ff80cfb90 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ffe077640 1 -- 172.21.15.99:0/137656378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 msgr2=0x7f2ff80b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.564+0000 7f2ffe077640 1 --2- 172.21.15.99:0/137656378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff80b5170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8073970 tx=0x7f2fe802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 -- 172.21.15.99:0/137656378 shutdown_connections 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 --2- 172.21.15.99:0/137656378 >> v2:172.21.15.196:3300/0 conn(0x7f2ff80d1410 0x7f2ff80cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 --2- 172.21.15.99:0/137656378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff80b5170 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 -- 172.21.15.99:0/137656378 >> 172.21.15.99:0/137656378 conn(0x7f2ff80b4650 msgr2=0x7f2ff80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 -- 172.21.15.99:0/137656378 shutdown_connections 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.565+0000 7f2ffe077640 1 -- 172.21.15.99:0/137656378 wait complete. 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.566+0000 7f2ffe077640 1 Processor -- start 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.566+0000 7f2ffe077640 1 -- start start 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff80b3660 0x7f2ff80b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80d1410 0x7f2ff8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff806dd30 con 0x7f2ff80cfb90 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff806e260 con 0x7f2ff80b3660 2024-09-10T11:35:14.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff80cff70 con 0x7f2ff80d1410 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39296/0 (socket says 172.21.15.99:39296) 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 -- 172.21.15.99:0/2303959346 learned_addr learned my addr 172.21.15.99:0/2303959346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ff77fe640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff80b3660 0x7f2ff80b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ff7fff640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80d1410 0x7f2ff8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 -- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80d1410 msgr2=0x7f2ff8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80d1410 0x7f2ff8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 -- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff80b3660 msgr2=0x7f2ff80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff80b3660 0x7f2ff80b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 -- 172.21.15.99:0/2303959346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8155710 con 0x7f2ff80cfb90 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2feffff640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3c0 tx=0x7f2fe000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00052d0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0005490 con 0x7f2ff80cfb90 2024-09-10T11:35:14.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0007260 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff8155430 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.567+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff80b19d0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.568+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff80b5170 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.569+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2fe0007400 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.569+0000 7f2ff4ff9640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 0x7f2fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.570+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f2fe008a200 con 0x7f2ff80cfb90 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.570+0000 7f2ff77fe640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 0x7f2fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.570+0000 7f2ff77fe640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 0x7f2fd806e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8034000 tx=0x7f2fe802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.574+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2fe00585f0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.774+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ff80b3420 con 0x7f2ff80cfb90 2024-09-10T11:35:14.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.774+0000 7f2ff4ff9640 1 -- 172.21.15.99:0/2303959346 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f2fe005c8f0 con 0x7f2ff80cfb90 2024-09-10T11:35:14.775 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:14.776 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:14.776 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:14.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 msgr2=0x7f2fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 0x7f2fd806e2f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8034000 tx=0x7f2fe802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 msgr2=0x7f2ff8152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c3c0 tx=0x7f2fe000c880 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 shutdown_connections 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2fd806be30 0x7f2fd806e2f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80d1410 0x7f2ff8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ff80cfb90 0x7f2ff8152770 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 --2- 172.21.15.99:0/2303959346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff80b3660 0x7f2ff80b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.776+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 >> 172.21.15.99:0/2303959346 conn(0x7f2ff80b4650 msgr2=0x7f2ff806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.777+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 shutdown_connections 2024-09-10T11:35:14.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:14.777+0000 7f2ffe077640 1 -- 172.21.15.99:0/2303959346 wait complete. 2024-09-10T11:35:14.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-09-10T11:35:14.905 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6594,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.333189469000001,"mdsmap_epoch":48,"osdmap_epoch":123,"osdmap_epoch_barrier":123,"uptime":16.681852687999999} 2024-09-10T11:35:14.906 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6594, "mdsmap_epoch": 48, "osdmap_epoch": 123, "osdmap_epoch_barrier": 123, "rank_uptime": 12.333189469, "state": "up:active", "uptime": 16.681852688, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:35:14.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-09-10T11:35:15.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.010+0000 7f4b2a75f640 1 Processor -- start 2024-09-10T11:35:15.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.010+0000 7f4b2a75f640 1 -- start start 2024-09-10T11:35:15.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b2a75f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b240d1200 0x7f4b24073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b2a75f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b2406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b2a75f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b24152e60 con 0x7f4b2406e5d0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b2a75f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b2406ef80 con 0x7f4b240d1200 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b2a75f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240d15e0 con 0x7f4b2415c770 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b237fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4b2406e5d0 0x7f4b2415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57092/0 (socket says 172.21.15.99:57092) 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b237fe640 1 -- 172.21.15.99:0/3167609556 learned_addr learned my addr 172.21.15.99:0/3167609556 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b23fff640 1 --2- 172.21.15.99:0/3167609556 >> v2:172.21.15.196:3300/0 conn(0x7f4b240d1200 0x7f4b24073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b28cfc640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b2406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1080099575 0 0) 0x7f4b24152e60 con 0x7f4b2406e5d0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b0c003520 con 0x7f4b2406e5d0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1489647662 0 0) 0x7f4b0c003520 con 0x7f4b2406e5d0 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 msgr2=0x7f4b2406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b2406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 >> v2:172.21.15.196:3300/0 conn(0x7f4b240d1200 msgr2=0x7f4b24073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:15.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 --2- 172.21.15.99:0/3167609556 >> v2:172.21.15.196:3300/0 conn(0x7f4b240d1200 0x7f4b24073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b24072680 con 0x7f4b2406e5d0 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4b18001670 con 0x7f4b2406e5d0 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.011+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 >> v1:172.21.15.99:6789/0 conn(0x7f4b2406e5d0 legacy=0x7f4b2415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b0c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b0c003f30 0x7f4b0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c004d40 0x7f4b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2406ef80 con 0x7f4b0c003f30 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240d15e0 con 0x7f4b0c004d40 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c00ba10 con 0x7f4b2415c770 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b0c003f30 0x7f4b0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b23fff640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b0c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b28cfc640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c004d40 0x7f4b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 -- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 msgr2=0x7f4b0c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:15.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2415c770 0x7f4b0c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 -- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c004d40 msgr2=0x7f4b0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b0c004d40 0x7f4b0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.012+0000 7f4b237fe640 1 -- 172.21.15.99:0/3167609556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c003520 con 0x7f4b0c003f30 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b237fe640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b0c003f30 0x7f4b0c004800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c210 tx=0x7f4b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e649ca5053f8096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18007d70 con 0x7f4b0c003f30 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b18005af0 con 0x7f4b0c003f30 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b22ffd640 1 -- 172.21.15.99:0/3167609556 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18007070 con 0x7f4b0c003f30 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3167609556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b0c003f30 msgr2=0x7f4b0c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3167609556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b0c003f30 0x7f4b0c004800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800c210 tx=0x7f4b0c005830 comp rx=0 tx=0).stop 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b28cfc640 1 -- 172.21.15.99:0/3167609556 reap_dead start 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3167609556 shutdown_connections 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3167609556 >> 172.21.15.99:0/3167609556 conn(0x7f4b2406bc10 msgr2=0x7f4b2406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:15.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3167609556 shutdown_connections 2024-09-10T11:35:15.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.013+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3167609556 wait complete. 2024-09-10T11:35:15.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.014+0000 7f4b2a75f640 1 Processor -- start 2024-09-10T11:35:15.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 -- start start 2024-09-10T11:35:15.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b0c003f30 0x7f4b2413d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b24140910 0x7f4b24155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24072f20 con 0x7f4b2415ae00 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24072c40 con 0x7f4b24140910 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b2a75f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2415cd90 con 0x7f4b0c003f30 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b28cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b28cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39312/0 (socket says 172.21.15.99:39312) 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.015+0000 7f4b28cfc640 1 -- 172.21.15.99:0/3762834210 learned_addr learned my addr 172.21.15.99:0/3762834210 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b237fe640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b24140910 0x7f4b24155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b23fff640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b0c003f30 0x7f4b2413d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 -- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b0c003f30 msgr2=0x7f4b2413d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b0c003f30 0x7f4b2413d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 -- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b24140910 msgr2=0x7f4b24155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b24140910 0x7f4b24155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 -- 172.21.15.99:0/3762834210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b24153e50 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b28cfc640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4b24071b80 tx=0x7f4b1c00f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00fcb0 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c01d070 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b241540f0 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c015670 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.016+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241546b0 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.017+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b24073440 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.018+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b1c015810 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.018+0000 7f4b20ff9640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 0x7f4b1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.018+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f4b1c08b4f0 con 0x7f4b2415ae00 2024-09-10T11:35:15.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.018+0000 7f4b23fff640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 0x7f4b1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.019+0000 7f4b23fff640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 0x7f4b1006e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009bc0 tx=0x7f4b14009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:15.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.022+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b1c010040 con 0x7f4b2415ae00 2024-09-10T11:35:15.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.187+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f4b240ce650 con 0x7f4b2415ae00 2024-09-10T11:35:15.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.618+0000 7f4b20ff9640 1 -- 172.21.15.99:0/3762834210 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v124) ==== 181+0+0 (secure 0 0 0) 0x7f4b1c059ea0 con 0x7f4b2415ae00 2024-09-10T11:35:15.619 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-09-10T11:35:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 msgr2=0x7f4b1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 0x7f4b1006e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4b14009bc0 tx=0x7f4b14009190 comp rx=0 tx=0).stop 2024-09-10T11:35:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 msgr2=0x7f4b2415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f4b24071b80 tx=0x7f4b1c00f1b0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 shutdown_connections 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b1006be30 0x7f4b1006e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2415ae00 0x7f4b2415b280 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b24140910 0x7f4b24155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 --2- 172.21.15.99:0/3762834210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b0c003f30 0x7f4b2413d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 >> 172.21.15.99:0/3762834210 conn(0x7f4b2406bc10 msgr2=0x7f4b240cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 shutdown_connections 2024-09-10T11:35:15.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:15.621+0000 7f4b2a75f640 1 -- 172.21.15.99:0/3762834210 wait complete. 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:35:15.635 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:16.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:16.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:16.644 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:16.644 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:16.644 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:16.644 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:16.645 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:16.646 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:16.764 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:16.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:16.774 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:16.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:16.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52432.asok status 2024-09-10T11:35:16.907 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:16.907 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:16.907 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:16.907 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "52432", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 1, 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6780, 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6780 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:16.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6780 192.168.0.1:0/3726021730", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3726021730", 2024-09-10T11:35:16.909 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 2, 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 48, 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 123, 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 123, 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:16.910 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:16.916 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-09-10T11:35:17.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.015+0000 7fbec808d640 1 Processor -- start 2024-09-10T11:35:17.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.015+0000 7fbec808d640 1 -- start start 2024-09-10T11:35:17.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec808d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbec01626c0 0x7fbec00d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec808d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec015ff70 con 0x7fbec00d4e40 2024-09-10T11:35:17.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec808d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec00d1620 con 0x7fbec015eb80 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec808d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbec00d30c0 con 0x7fbec01626c0 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec5628640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbec01626c0 0x7fbec00d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec662a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbec00d4e40 0x7fbec00d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57098/0 (socket says 172.21.15.99:57098) 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.016+0000 7fbec662a640 1 -- 172.21.15.99:0/1031034265 learned_addr learned my addr 172.21.15.99:0/1031034265 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.017+0000 7fbec5628640 1 -- 172.21.15.99:0/1031034265 >> v1:172.21.15.196:6789/0 conn(0x7fbec015eb80 legacy=0x7fbec00d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.017+0000 7fbec4e27640 1 -- 172.21.15.99:0/1031034265 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866624708 0 0) 0x7fbec015ff70 con 0x7fbec00d4e40 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.017+0000 7fbec5628640 1 -- 172.21.15.99:0/1031034265 >> v1:172.21.15.99:6789/0 conn(0x7fbec00d4e40 legacy=0x7fbec00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.017+0000 7fbec5628640 1 -- 172.21.15.99:0/1031034265 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec0162bc0 con 0x7fbec01626c0 2024-09-10T11:35:17.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.017+0000 7fbec5628640 1 --2- 172.21.15.99:0/1031034265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec01626c0 0x7fbec00d4900 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00097a0 tx=0x7fbeb002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e318c56c5233104c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec4e27640 1 -- 172.21.15.99:0/1031034265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0037070 con 0x7fbec01626c0 2024-09-10T11:35:17.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec4e27640 1 -- 172.21.15.99:0/1031034265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb002ae50 con 0x7fbec01626c0 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec4e27640 1 -- 172.21.15.99:0/1031034265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0032660 con 0x7fbec01626c0 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 -- 172.21.15.99:0/1031034265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec01626c0 msgr2=0x7fbec00d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 --2- 172.21.15.99:0/1031034265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec01626c0 0x7fbec00d4900 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00097a0 tx=0x7fbeb002a140 comp rx=0 tx=0).stop 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 -- 172.21.15.99:0/1031034265 shutdown_connections 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 --2- 172.21.15.99:0/1031034265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec01626c0 0x7fbec00d4900 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 -- 172.21.15.99:0/1031034265 >> 172.21.15.99:0/1031034265 conn(0x7fbec008e680 msgr2=0x7fbec009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 -- 172.21.15.99:0/1031034265 shutdown_connections 2024-09-10T11:35:17.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.018+0000 7fbec808d640 1 -- 172.21.15.99:0/1031034265 wait complete. 2024-09-10T11:35:17.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.019+0000 7fbec808d640 1 Processor -- start 2024-09-10T11:35:17.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.019+0000 7fbec808d640 1 -- start start 2024-09-10T11:35:17.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbec00d4e40 0x7fbec00c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbec01626c0 0x7fbec00b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0163460 con 0x7fbec015eb80 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbec0163180 con 0x7fbec00d4e40 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec808d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbec00d5290 con 0x7fbec01626c0 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39330/0 (socket says 172.21.15.99:39330) 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 -- 172.21.15.99:0/3099658683 learned_addr learned my addr 172.21.15.99:0/3099658683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5e29640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbec00d4e40 0x7fbec00c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec662a640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbec01626c0 0x7fbec00b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 -- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbec01626c0 msgr2=0x7fbec00b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbec01626c0 0x7fbec00b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 -- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbec00d4e40 msgr2=0x7fbec00c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbec00d4e40 0x7fbec00c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.020+0000 7fbec5628640 1 -- 172.21.15.99:0/3099658683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec00c8cf0 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbec5628640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0035040 tx=0x7fbeb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0037040 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb0004030 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec00c8fd0 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb0004440 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.021+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec00c9870 con 0x7fbec015eb80 2024-09-10T11:35:17.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.022+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec00c91a0 con 0x7fbec015eb80 2024-09-10T11:35:17.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.022+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeb003d650 con 0x7fbec015eb80 2024-09-10T11:35:17.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.023+0000 7fbeb67fc640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 0x7fbea406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.023+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5767+0+0 (secure 0 0 0) 0x7fbeb00ab510 con 0x7fbec015eb80 2024-09-10T11:35:17.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.023+0000 7fbec5e29640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 0x7fbea406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.023+0000 7fbec5e29640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 0x7fbea406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.027+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbeb0042020 con 0x7fbec015eb80 2024-09-10T11:35:17.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.200+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7fbec00c85b0 con 0x7fbec015eb80 2024-09-10T11:35:17.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.629+0000 7fbeb67fc640 1 -- 172.21.15.99:0/3099658683 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v126) ==== 93+0+0 (secure 0 0 0) 0x7fbec00c85b0 con 0x7fbec015eb80 2024-09-10T11:35:17.630 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is set 2024-09-10T11:35:17.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 msgr2=0x7fbea406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 0x7fbea406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbebc006fd0 tx=0x7fbebc008040 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 msgr2=0x7fbec00b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbeb0035040 tx=0x7fbeb00046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 shutdown_connections 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbea406be30 0x7fbea406e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbec01626c0 0x7fbec00b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbec015eb80 0x7fbec00b27b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 --2- 172.21.15.99:0/3099658683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbec00d4e40 0x7fbec00c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.631+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 >> 172.21.15.99:0/3099658683 conn(0x7fbec008e680 msgr2=0x7fbec00b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.632+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 shutdown_connections 2024-09-10T11:35:17.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.632+0000 7fbec808d640 1 -- 172.21.15.99:0/3099658683 wait complete. 2024-09-10T11:35:17.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-09-10T11:35:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 Processor -- start 2024-09-10T11:35:17.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 -- start start 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 0x7f8c7c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c0ceb10 con 0x7f8c7c0d0fb0 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c0d1420 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.745+0000 7f8c830f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c0b3840 con 0x7f8c7c06f000 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39346/0 (socket says 172.21.15.99:39346) 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 -- 172.21.15.99:0/2787495421 learned_addr learned my addr 172.21.15.99:0/2787495421 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c8168e640 1 --2- 172.21.15.99:0/2787495421 >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 0x7f8c7c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 -- 172.21.15.99:0/2787495421 >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 msgr2=0x7f8c7c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 --2- 172.21.15.99:0/2787495421 >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 0x7f8c7c06d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 -- 172.21.15.99:0/2787495421 >> v1:172.21.15.196:6789/0 conn(0x7f8c7c0d0fb0 legacy=0x7f8c7c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c80e8d640 1 -- 172.21.15.99:0/2787495421 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c0b3a90 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.746+0000 7f8c8168e640 1 --2- 172.21.15.99:0/2787495421 >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 0x7f8c7c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c80e8d640 1 --2- 172.21.15.99:0/2787495421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0d1890 tx=0x7f8c7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978b4688135eb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c7b7fe640 1 -- 172.21.15.99:0/2787495421 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70037070 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c7b7fe640 1 -- 172.21.15.99:0/2787495421 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c70032440 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c7b7fe640 1 -- 172.21.15.99:0/2787495421 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70032810 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 -- 172.21.15.99:0/2787495421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 msgr2=0x7f8c7c0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2787495421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0d1890 tx=0x7f8c7002a230 comp rx=0 tx=0).stop 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 -- 172.21.15.99:0/2787495421 shutdown_connections 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2787495421 >> v2:172.21.15.196:3300/0 conn(0x7f8c7c06f000 0x7f8c7c06d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2787495421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c0d0a70 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 -- 172.21.15.99:0/2787495421 >> 172.21.15.99:0/2787495421 conn(0x7f8c7c06bc10 msgr2=0x7f8c7c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 -- 172.21.15.99:0/2787495421 shutdown_connections 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.747+0000 7f8c830f1640 1 -- 172.21.15.99:0/2787495421 wait complete. 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.748+0000 7f8c830f1640 1 Processor -- start 2024-09-10T11:35:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 -- start start 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c7c06f000 0x7f8c7c1e8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 0x7f8c7c1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c0b4330 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c0b4050 con 0x7f8c7c06f000 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c830f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c0b4610 con 0x7f8c7c0d0fb0 2024-09-10T11:35:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.749+0000 7f8c8168e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39362/0 (socket says 172.21.15.99:39362) 2024-09-10T11:35:17.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 -- 172.21.15.99:0/2412505210 learned_addr learned my addr 172.21.15.99:0/2412505210 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:17.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c80e8d640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c7c06f000 0x7f8c7c1e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c7bfff640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 0x7f8c7c1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 -- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 msgr2=0x7f8c7c1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 0x7f8c7c1484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 -- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c7c06f000 msgr2=0x7f8c7c1e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:17.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c7c06f000 0x7f8c7c1e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c14a690 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c7bfff640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 0x7f8c7c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c8168e640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400cef0 tx=0x7f8c7400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7401a070 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7400d860 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400dc50 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c1e7510 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.750+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c1e7db0 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.751+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7c15cc20 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.752+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c7401d070 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.752+0000 7f8c797fa640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 0x7f8c6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:17.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.752+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5767+0+0 (secure 0 0 0) 0x7f8c7408b6d0 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.752+0000 7f8c80e8d640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 0x7f8c6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:17.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.752+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=127+}) -- 0x7f8c60073a20 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.753+0000 7f8c80e8d640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 0x7f8c6006e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8c70034000 tx=0x7f8c7002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:17.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.756+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c74059ab0 con 0x7f8c7c15c7b0 2024-09-10T11:35:17.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:17.929+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7f8c7c06d920 con 0x7f8c7c15c7b0 2024-09-10T11:35:18.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.631+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v127) ==== 97+0+0 (secure 0 0 0) 0x7f8c7405ddb0 con 0x7f8c7c15c7b0 2024-09-10T11:35:18.632 INFO:teuthology.orchestra.run.smithi099.stderr:pauserd,pausewr is unset 2024-09-10T11:35:18.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.631+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(127..127 src has 1..127) ==== 288+0+0 (secure 0 0 0) 0x7f8c740bfd50 con 0x7f8c7c15c7b0 2024-09-10T11:35:18.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.631+0000 7f8c797fa640 1 -- 172.21.15.99:0/2412505210 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=128}) -- 0x7f8c600743f0 con 0x7f8c7c15c7b0 2024-09-10T11:35:18.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 msgr2=0x7f8c6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 0x7f8c6006e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8c70034000 tx=0x7f8c7002e000 comp rx=0 tx=0).stop 2024-09-10T11:35:18.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 msgr2=0x7f8c7c14c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400cef0 tx=0x7f8c7400a710 comp rx=0 tx=0).stop 2024-09-10T11:35:18.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 shutdown_connections 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8c6006be30 0x7f8c6006e2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c7c15c7b0 0x7f8c7c14c210 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c7c0d0fb0 0x7f8c7c1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 --2- 172.21.15.99:0/2412505210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c7c06f000 0x7f8c7c1e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.634+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 >> 172.21.15.99:0/2412505210 conn(0x7f8c7c06bc10 msgr2=0x7f8c7c0b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.635+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 shutdown_connections 2024-09-10T11:35:18.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.635+0000 7f8c830f1640 1 -- 172.21.15.99:0/2412505210 wait complete. 2024-09-10T11:35:18.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.750+0000 7fb58e08d640 1 Processor -- start 2024-09-10T11:35:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 -- start start 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb5880b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 0x7fb5880cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5880d0130 0x7fb5880cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb5880ce430 con 0x7fb5880d0130 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb5880ce710 con 0x7fb5880d1760 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb58e08d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb5880d1240 con 0x7fb588160030 2024-09-10T11:35:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb587fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5880d0130 0x7fb5880cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb587fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5880d0130 0x7fb5880cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39364/0 (socket says 172.21.15.99:39364) 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.751+0000 7fb587fff640 1 -- 172.21.15.99:0/1281246845 learned_addr learned my addr 172.21.15.99:0/1281246845 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb5880b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb586ffd640 1 --2- 172.21.15.99:0/1281246845 >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 0x7fb5880cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 -- 172.21.15.99:0/1281246845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 msgr2=0x7fb5880b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb5880b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 -- 172.21.15.99:0/1281246845 >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 msgr2=0x7fb5880cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 --2- 172.21.15.99:0/1281246845 >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 0x7fb5880cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 -- 172.21.15.99:0/1281246845 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880b3e30 con 0x7fb5880d0130 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb5880b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb586ffd640 1 --2- 172.21.15.99:0/1281246845 >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 0x7fb5880cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.752+0000 7fb587fff640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d0130 0x7fb5880cdaf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb588072e90 tx=0x7fb58002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58faa8ce7cd7d90f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:18.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb5867fc640 1 -- 172.21.15.99:0/1281246845 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb5880d0130 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb5867fc640 1 -- 172.21.15.99:0/1281246845 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb580032440 con 0x7fb5880d0130 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb5867fc640 1 -- 172.21.15.99:0/1281246845 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580032810 con 0x7fb5880d0130 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 -- 172.21.15.99:0/1281246845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d0130 msgr2=0x7fb5880cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d0130 0x7fb5880cdaf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb588072e90 tx=0x7fb58002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 -- 172.21.15.99:0/1281246845 shutdown_connections 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d0130 0x7fb5880cdaf0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1281246845 >> v2:172.21.15.196:3300/0 conn(0x7fb5880d1760 0x7fb5880cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1281246845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb5880b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 -- 172.21.15.99:0/1281246845 >> 172.21.15.99:0/1281246845 conn(0x7fb5880b4650 msgr2=0x7fb5880b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 -- 172.21.15.99:0/1281246845 shutdown_connections 2024-09-10T11:35:18.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.753+0000 7fb58e08d640 1 -- 172.21.15.99:0/1281246845 wait complete. 2024-09-10T11:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.754+0000 7fb58e08d640 1 Processor -- start 2024-09-10T11:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 -- start start 2024-09-10T11:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 0x7fb5881491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb58814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb588160590 con 0x7fb5880d1760 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb588160410 con 0x7fb5880d0130 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb58e08d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb588160710 con 0x7fb588160030 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb586ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb586ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39378/0 (socket says 172.21.15.99:39378) 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.755+0000 7fb586ffd640 1 -- 172.21.15.99:0/1800359121 learned_addr learned my addr 172.21.15.99:0/1800359121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 0x7fb5881491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb587fff640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb58814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 -- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 msgr2=0x7fb58814bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb58814bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 -- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 msgr2=0x7fb5881491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 0x7fb5881491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 -- 172.21.15.99:0/1800359121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5880cc620 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb587fff640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb58814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 0x7fb5881491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb586ffd640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c7c0 tx=0x7fb57000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570015070 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570007820 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5880cc900 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb570007bf0 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.756+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5880b0b20 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.757+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5880cce20 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.758+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb570018070 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.758+0000 7fb57ffff640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 0x7fb56806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.758+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fb57008adc0 con 0x7fb5880d1760 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.758+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 0x7fb56806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.758+0000 7fb5877fe640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 0x7fb56806e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb578007f70 tx=0x7fb5780074c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:18.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.762+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5880cce20 con 0x7fb5880d1760 2024-09-10T11:35:18.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.928+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5880b38f0 con 0x7fb5880d1760 2024-09-10T11:35:18.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.929+0000 7fb57ffff640 1 -- 172.21.15.99:0/1800359121 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16784 (secure 0 0 0) 0x7fb570059730 con 0x7fb5880d1760 2024-09-10T11:35:18.930 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:18.931 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:18.629832+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.6","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 msgr2=0x7fb56806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 0x7fb56806e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb578007f70 tx=0x7fb5780074c0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 msgr2=0x7fb5880c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c7c0 tx=0x7fb57000cc80 comp rx=0 tx=0).stop 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 shutdown_connections 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb56806be30 0x7fb56806e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588160030 0x7fb58814bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5880d1760 0x7fb5880c9b90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 --2- 172.21.15.99:0/1800359121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5880d0130 0x7fb5881491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 >> 172.21.15.99:0/1800359121 conn(0x7fb5880b4650 msgr2=0x7fb5880b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 shutdown_connections 2024-09-10T11:35:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:18.931+0000 7fb58e08d640 1 -- 172.21.15.99:0/1800359121 wait complete. 2024-09-10T11:35:18.943 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:35:18.943 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:18.944 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:35:18.944 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:35:18.944 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:35:18.944 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:35:18.944 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:19.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:19.020 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.021 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:19.022 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:19.137 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:19.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:19.147 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:19.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:19.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52432.asok status 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "52432", 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 1, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6780, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6780 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:19.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6780 192.168.0.1:0/3726021730", 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3726021730", 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 2, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 48, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 123, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 123, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:19.282 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:19.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:19.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.385+0000 7f3e15cd0640 1 Processor -- start 2024-09-10T11:35:19.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.385+0000 7f3e15cd0640 1 -- start start 2024-09-10T11:35:19.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e15cd0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3e100d1200 0x7f3e10073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e15cd0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e1006e5d0 0x7f3e1006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e15cd0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e10152e60 con 0x7f3e1015c060 2024-09-10T11:35:19.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e15cd0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1006ef80 con 0x7f3e100d1200 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e15cd0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3e100d15e0 con 0x7f3e1006e5d0 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e0f7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3e100d1200 0x7f3e10073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e0ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3e1015c060 0x7f3e1015c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57100/0 (socket says 172.21.15.99:57100) 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.386+0000 7f3e0ffff640 1 -- 172.21.15.99:0/152207134 learned_addr learned my addr 172.21.15.99:0/152207134 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0effd640 1 --2- 172.21.15.99:0/152207134 >> v2:172.21.15.196:3300/0 conn(0x7f3e1006e5d0 0x7f3e1006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0f7fe640 1 -- 172.21.15.99:0/152207134 >> v2:172.21.15.196:3300/0 conn(0x7f3e1006e5d0 msgr2=0x7f3e1006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0f7fe640 1 --2- 172.21.15.99:0/152207134 >> v2:172.21.15.196:3300/0 conn(0x7f3e1006e5d0 0x7f3e1006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0e7fc640 1 -- 172.21.15.99:0/152207134 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432554638 0 0) 0x7f3e10152e60 con 0x7f3e1015c060 2024-09-10T11:35:19.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0f7fe640 1 -- 172.21.15.99:0/152207134 >> v1:172.21.15.99:6789/0 conn(0x7f3e1015c060 legacy=0x7f3e1015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0f7fe640 1 -- 172.21.15.99:0/152207134 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e10072680 con 0x7f3e100d1200 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0f7fe640 1 --2- 172.21.15.99:0/152207134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e100d1200 0x7f3e10073440 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3e10071bb0 tx=0x7f3e0002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59a32f846aa5abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0e7fc640 1 -- 172.21.15.99:0/152207134 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00007740 con 0x7f3e100d1200 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0e7fc640 1 -- 172.21.15.99:0/152207134 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e000078e0 con 0x7f3e100d1200 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.387+0000 7f3e0e7fc640 1 -- 172.21.15.99:0/152207134 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00007cb0 con 0x7f3e100d1200 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 -- 172.21.15.99:0/152207134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e100d1200 msgr2=0x7f3e10073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/152207134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e100d1200 0x7f3e10073440 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f3e10071bb0 tx=0x7f3e0002a5f0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 -- 172.21.15.99:0/152207134 shutdown_connections 2024-09-10T11:35:19.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/152207134 >> v2:172.21.15.196:3300/0 conn(0x7f3e1006e5d0 0x7f3e1006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/152207134 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e100d1200 0x7f3e10073440 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 -- 172.21.15.99:0/152207134 >> 172.21.15.99:0/152207134 conn(0x7f3e1006bc10 msgr2=0x7f3e1006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:19.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 -- 172.21.15.99:0/152207134 shutdown_connections 2024-09-10T11:35:19.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.388+0000 7f3e15cd0640 1 -- 172.21.15.99:0/152207134 wait complete. 2024-09-10T11:35:19.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.389+0000 7f3e15cd0640 1 Processor -- start 2024-09-10T11:35:19.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.389+0000 7f3e15cd0640 1 -- start start 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e1006e5d0 0x7f3e10142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e100d1200 0x7f3e101473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10072f20 con 0x7f3e1015c060 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e10072c40 con 0x7f3e1006e5d0 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e15cd0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1015cd90 con 0x7f3e100d1200 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39394/0 (socket says 172.21.15.99:39394) 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 -- 172.21.15.99:0/4231400439 learned_addr learned my addr 172.21.15.99:0/4231400439 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0f7fe640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e1006e5d0 0x7f3e10142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0effd640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e100d1200 0x7f3e101473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 -- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e100d1200 msgr2=0x7f3e101473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e100d1200 0x7f3e101473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 -- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e1006e5d0 msgr2=0x7f3e10142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e1006e5d0 0x7f3e10142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.390+0000 7f3e0ffff640 1 -- 172.21.15.99:0/4231400439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e10146350 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3e0ffff640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800c480 tx=0x7f3e0800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08018070 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e08007500 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e10146630 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08004a20 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.391+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e10146bf0 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.392+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1006ea40 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.392+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e0801b070 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.393+0000 7f3df7fff640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 0x7f3dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.393+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7f3e0808a180 con 0x7f3e1015c060 2024-09-10T11:35:19.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.393+0000 7f3e0f7fe640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 0x7f3dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:19.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.393+0000 7f3e0f7fe640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 0x7f3dfc06e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3e00034000 tx=0x7f3e00033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:19.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.397+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e08058b80 con 0x7f3e1015c060 2024-09-10T11:35:19.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.597+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e100b4d30 con 0x7f3e1015c060 2024-09-10T11:35:19.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.598+0000 7f3df7fff640 1 -- 172.21.15.99:0/4231400439 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f3e08058480 con 0x7f3e1015c060 2024-09-10T11:35:19.602 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:19.602 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.599+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 msgr2=0x7f3dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.599+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 0x7f3dfc06e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3e00034000 tx=0x7f3e00033040 comp rx=0 tx=0).stop 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 msgr2=0x7f3e10143370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800c480 tx=0x7f3e0800c940 comp rx=0 tx=0).stop 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 shutdown_connections 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3dfc06be30 0x7f3dfc06e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3e1015c060 0x7f3e10143370 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e100d1200 0x7f3e101473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 --2- 172.21.15.99:0/4231400439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e1006e5d0 0x7f3e10142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 >> 172.21.15.99:0/4231400439 conn(0x7f3e1006bc10 msgr2=0x7f3e1014c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 shutdown_connections 2024-09-10T11:35:19.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19.600+0000 7f3e15cd0640 1 -- 172.21.15.99:0/4231400439 wait complete. 2024-09-10T11:35:19.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 127 2024-09-10T11:35:19.727 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-09-10T11:35:19.727 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:35:19.727 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:19.728 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-09-10T11:35:19.728 DEBUG:teuthology.orchestra.run.smithi099:> os.fchmod(fd, 0O666) 2024-09-10T11:35:19.728 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(fd) 2024-09-10T11:35:19.728 DEBUG:teuthology.orchestra.run.smithi099:> os.close(fd) 2024-09-10T11:35:19.728 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:19.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:19.810 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:19.811 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:19.928 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:19.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:19.939 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:19.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:19.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52432.asok status 2024-09-10T11:35:20.072 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "52432", 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 2, 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6780, 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6780 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6780 192.168.0.1:0/3726021730", 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3726021730", 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 3, 2024-09-10T11:35:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 48, 2024-09-10T11:35:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 127, 2024-09-10T11:35:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 127, 2024-09-10T11:35:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:20.076 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:20.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:20.081 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:20.081 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:20.081 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:20.081 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:20.082 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:20.083 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:20.160 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:20.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52432.asok 2024-09-10T11:35:20.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:20.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52432.asok status 2024-09-10T11:35:20.303 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "52432", 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:20.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 2, 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6780, 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6780 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:20.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3726021730 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6780 192.168.0.1:0/3726021730", 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3726021730", 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 3, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 48, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 127, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 127, 2024-09-10T11:35:20.306 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:20.307 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:20.307 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:20.312 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:35:20.312 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:35:20.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:35:20.353 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:35:20.394 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:35:20.395 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:35:20.395 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:20.395 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:20.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.549+0000 7f2914bfd640 1 Processor -- start 2024-09-10T11:35:20.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.549+0000 7f2914bfd640 1 -- start start 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f2914bfd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f29101605c0 0x7f29101609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f2914bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f2910160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f2914bfd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29100d1bd0 con 0x7f29100ae020 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f2914bfd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2910075620 con 0x7f29101605c0 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f2914bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f29100752b0 con 0x7f291015f210 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f290dd9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f29100ae020 0x7f291015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57106/0 (socket says 172.21.15.99:57106) 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f290dd9b640 1 -- 172.21.15.99:0/2721566479 learned_addr learned my addr 172.21.15.99:0/2721566479 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.550+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790995096 0 0) 0x7f29100d1bd0 con 0x7f29100ae020 2024-09-10T11:35:20.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290e59c640 1 --2- 172.21.15.99:0/2721566479 >> v2:172.21.15.196:3300/0 conn(0x7f29101605c0 0x7f29101609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28f8003520 con 0x7f29100ae020 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f2910160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629385745 0 0) 0x7f28f8003520 con 0x7f29100ae020 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 msgr2=0x7f2910160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f2910160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 >> v2:172.21.15.196:3300/0 conn(0x7f29101605c0 msgr2=0x7f29101609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 --2- 172.21.15.99:0/2721566479 >> v2:172.21.15.196:3300/0 conn(0x7f29101605c0 0x7f29101609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29100d1eb0 con 0x7f29100ae020 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f2904001670 con 0x7f29100ae020 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 >> v1:172.21.15.99:6789/0 conn(0x7f29100ae020 legacy=0x7f291015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f28f80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f8003f30 0x7f28f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28f8004d40 0x7f28f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2910075620 con 0x7f28f8004d40 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29100752b0 con 0x7f28f8003f30 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f28f800b9c0 con 0x7f291015f210 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28f8004d40 0x7f28f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290dd9b640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f8003f30 0x7f28f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290e59c640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f28f80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 -- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 msgr2=0x7f28f80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f28f80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 -- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f8003f30 msgr2=0x7f28f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f8003f30 0x7f28f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290ed9d640 1 -- 172.21.15.99:0/2721566479 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f8003520 con 0x7f28f8004d40 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290dd9b640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28f8003f30 0x7f28f8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.551+0000 7f290e59c640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f291015f210 0x7f28f80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.552+0000 7f290ed9d640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28f8004d40 0x7f28f80051c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f90 tx=0x7f28f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611747ccfa7a1272 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.552+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908009c10 con 0x7f28f8004d40 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.552+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2908036070 con 0x7f28f8004d40 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.552+0000 7f290d59a640 1 -- 172.21.15.99:0/2721566479 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908031670 con 0x7f28f8004d40 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 -- 172.21.15.99:0/2721566479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28f8004d40 msgr2=0x7f28f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 --2- 172.21.15.99:0/2721566479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f28f8004d40 0x7f28f80051c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2908009f90 tx=0x7f28f8005830 comp rx=0 tx=0).stop 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f290ed9d640 1 -- 172.21.15.99:0/2721566479 reap_dead start 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 -- 172.21.15.99:0/2721566479 shutdown_connections 2024-09-10T11:35:20.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 -- 172.21.15.99:0/2721566479 >> 172.21.15.99:0/2721566479 conn(0x7f29100730c0 msgr2=0x7f29100734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:20.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 -- 172.21.15.99:0/2721566479 shutdown_connections 2024-09-10T11:35:20.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.553+0000 7f2914bfd640 1 -- 172.21.15.99:0/2721566479 wait complete. 2024-09-10T11:35:20.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.554+0000 7f2914bfd640 1 Processor -- start 2024-09-10T11:35:20.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.554+0000 7f2914bfd640 1 -- start start 2024-09-10T11:35:20.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 0x7f2910151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 0x7f2910158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2910073d00 con 0x7f291015d6e0 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f291015a9e0 con 0x7f29101523c0 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2910073e80 con 0x7f28f8004d40 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39414/0 (socket says 172.21.15.99:39414) 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 -- 172.21.15.99:0/4237374288 learned_addr learned my addr 172.21.15.99:0/4237374288 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290e59c640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 0x7f2910151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290ed9d640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 0x7f2910158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 -- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 msgr2=0x7f2910151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 0x7f2910151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 -- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 msgr2=0x7f2910158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 0x7f2910158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 -- 172.21.15.99:0/4237374288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291009c720 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290e59c640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 0x7f2910151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290ed9d640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 0x7f2910158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f290dd9b640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f290400e870 tx=0x7f290400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290400c870 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2904018070 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2910155a80 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290400cc30 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.555+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2910156040 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.557+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29100d1a30 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.557+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2904013420 con 0x7f291015d6e0 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.557+0000 7f28feffd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 0x7f28f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:20.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.557+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7f290408ab00 con 0x7f291015d6e0 2024-09-10T11:35:20.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.557+0000 7f290e59c640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 0x7f28f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:20.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.558+0000 7f290e59c640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 0x7f28f006e320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2900009a80 tx=0x7f2900008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:20.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.562+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2904022250 con 0x7f291015d6e0 2024-09-10T11:35:20.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.741+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29100d0e00 con 0x7f291015d6e0 2024-09-10T11:35:20.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.741+0000 7f28feffd640 1 -- 172.21.15.99:0/4237374288 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7f2904059530 con 0x7f291015d6e0 2024-09-10T11:35:20.742 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:35:20.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 msgr2=0x7f28f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 0x7f28f006e320 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2900009a80 tx=0x7f2900008040 comp rx=0 tx=0).stop 2024-09-10T11:35:20.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 msgr2=0x7f291009c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:20.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f290400e870 tx=0x7f290400ed30 comp rx=0 tx=0).stop 2024-09-10T11:35:20.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 shutdown_connections 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f28f006be60 0x7f28f006e320 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29101523c0 0x7f2910158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f291015d6e0 0x7f291009c1e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 --2- 172.21.15.99:0/4237374288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f28f8004d40 0x7f2910151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 >> 172.21.15.99:0/4237374288 conn(0x7f29100730c0 msgr2=0x7f29101609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 shutdown_connections 2024-09-10T11:35:20.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:20.743+0000 7f2914bfd640 1 -- 172.21.15.99:0/4237374288 wait complete. 2024-09-10T11:35:20.755 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:35:20.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:20.755 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:20.779 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[52432]: fuse finished with error 0 and tester_r 0 2024-09-10T11:35:20.780 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:35:20.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:20.780 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:35:20.926 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:35:20.926 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:35:20.926 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:20.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:35:21.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.044+0000 7fb233253640 1 Processor -- start 2024-09-10T11:35:21.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.044+0000 7fb233253640 1 -- start start 2024-09-10T11:35:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb233253640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb22c15aef0 0x7fb22c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb233253640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb22c15b2d0 0x7fb22c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb233253640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb22c06ee10 con 0x7fb22c0cd290 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb233253640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c0b0290 con 0x7fb22c15b2d0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb233253640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb22c06e570 con 0x7fb22c15aef0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb22c15b2d0 0x7fb22c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb22c15b2d0 0x7fb22c0afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39430/0 (socket says 172.21.15.99:39430) 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 -- 172.21.15.99:0/3539981338 learned_addr learned my addr 172.21.15.99:0/3539981338 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb230fef640 1 --2- 172.21.15.99:0/3539981338 >> v2:172.21.15.196:3300/0 conn(0x7fb22c15aef0 0x7fb22c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 -- 172.21.15.99:0/3539981338 >> v2:172.21.15.196:3300/0 conn(0x7fb22c15aef0 msgr2=0x7fb22c0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 --2- 172.21.15.99:0/3539981338 >> v2:172.21.15.196:3300/0 conn(0x7fb22c15aef0 0x7fb22c0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 -- 172.21.15.99:0/3539981338 >> v1:172.21.15.196:6789/0 conn(0x7fb22c0cd290 legacy=0x7fb22c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.045+0000 7fb22bfff640 1 -- 172.21.15.99:0/3539981338 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c06e850 con 0x7fb22c15b2d0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.046+0000 7fb22bfff640 1 --2- 172.21.15.99:0/3539981338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15b2d0 0x7fb22c0afbb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb218009870 tx=0x7fb21802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c64a4fa95863b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.046+0000 7fb22b7fe640 1 -- 172.21.15.99:0/3539981338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218037070 con 0x7fb22c15b2d0 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.046+0000 7fb22b7fe640 1 -- 172.21.15.99:0/3539981338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb218032440 con 0x7fb22c15b2d0 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.046+0000 7fb22b7fe640 1 -- 172.21.15.99:0/3539981338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218032810 con 0x7fb22c15b2d0 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 -- 172.21.15.99:0/3539981338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15b2d0 msgr2=0x7fb22c0afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 --2- 172.21.15.99:0/3539981338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15b2d0 0x7fb22c0afbb0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb218009870 tx=0x7fb21802a270 comp rx=0 tx=0).stop 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 -- 172.21.15.99:0/3539981338 shutdown_connections 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 --2- 172.21.15.99:0/3539981338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15b2d0 0x7fb22c0afbb0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 --2- 172.21.15.99:0/3539981338 >> v2:172.21.15.196:3300/0 conn(0x7fb22c15aef0 0x7fb22c0b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 -- 172.21.15.99:0/3539981338 >> 172.21.15.99:0/3539981338 conn(0x7fb22c06bc10 msgr2=0x7fb22c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 -- 172.21.15.99:0/3539981338 shutdown_connections 2024-09-10T11:35:21.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.047+0000 7fb233253640 1 -- 172.21.15.99:0/3539981338 wait complete. 2024-09-10T11:35:21.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.048+0000 7fb233253640 1 Processor -- start 2024-09-10T11:35:21.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.048+0000 7fb233253640 1 -- start start 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 0x7fb22c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22c15b2d0 0x7fb22c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c071d30 con 0x7fb22c15aef0 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb22c071bb0 con 0x7fb22c0cd290 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb22c071eb0 con 0x7fb22c15b2d0 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39438/0 (socket says 172.21.15.99:39438) 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 -- 172.21.15.99:0/2570707265 learned_addr learned my addr 172.21.15.99:0/2570707265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb230fef640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 0x7fb22c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb2317f0640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22c15b2d0 0x7fb22c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 -- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22c15b2d0 msgr2=0x7fb22c1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22c15b2d0 0x7fb22c1f5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 -- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 msgr2=0x7fb22c1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 0x7fb22c1f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 -- 172.21.15.99:0/2570707265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22c14e7d0 con 0x7fb22c15aef0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb230fef640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 0x7fb22c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb22bfff640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb218000c00 tx=0x7fb2180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218037040 con 0x7fb22c15aef0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb218004230 con 0x7fb22c15aef0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218003610 con 0x7fb22c15aef0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22c14eaa0 con 0x7fb22c15aef0 2024-09-10T11:35:21.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.049+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22c0cdc80 con 0x7fb22c15aef0 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.050+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22c0cd790 con 0x7fb22c15aef0 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.051+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2180037b0 con 0x7fb22c15aef0 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.051+0000 7fb2297fa640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.051+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fb2180abb80 con 0x7fb22c15aef0 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.051+0000 7fb230fef640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.052+0000 7fb230fef640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 0x7fb21006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb220006fd0 tx=0x7fb220008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.056+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb21807a580 con 0x7fb22c15aef0 2024-09-10T11:35:21.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.232+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb22c0761b0 con 0x7fb22c15aef0 2024-09-10T11:35:21.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.232+0000 7fb2297fa640 1 -- 172.21.15.99:0/2570707265 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) ==== 80+0+0 (secure 0 0 0) 0x7fb218079e80 con 0x7fb22c15aef0 2024-09-10T11:35:21.233 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:35:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 msgr2=0x7fb21006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 0x7fb21006e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb220006fd0 tx=0x7fb220008040 comp rx=0 tx=0).stop 2024-09-10T11:35:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 msgr2=0x7fb22c14e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb218000c00 tx=0x7fb2180046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 shutdown_connections 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb21006be30 0x7fb21006e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb22c15b2d0 0x7fb22c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb22c15aef0 0x7fb22c14e290 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 --2- 172.21.15.99:0/2570707265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb22c0cd290 0x7fb22c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 >> 172.21.15.99:0/2570707265 conn(0x7fb22c06bc10 msgr2=0x7fb22c15a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 shutdown_connections 2024-09-10T11:35:21.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.234+0000 7fb233253640 1 -- 172.21.15.99:0/2570707265 wait complete. 2024-09-10T11:35:21.249 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:35:21.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:21.249 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:21.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[37534]: fuse finished with error 0 and tester_r 0 2024-09-10T11:35:21.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:35:21.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:21.280 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:35:21.449 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:35:21.450 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:21.450 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:21.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:35:21.466 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:35:21.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:21.466 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:35:21.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:35:21.522 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:35:21.523 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:35:21.523 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:35:21.523 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:21.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:35:21.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:35:21.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:21.542 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:35:21.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:35:21.599 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:35:21.600 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:21.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.698+0000 7f6d2ee9b640 1 Processor -- start 2024-09-10T11:35:21.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 -- start start 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d2815eb80 0x7f6d280d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d281626c0 0x7f6d280d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6d280d52d0 con 0x7f6d2815eb80 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6d280d1620 con 0x7f6d281626c0 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2ee9b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6d28162bc0 con 0x7f6d280d4e40 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.699+0000 7f6d2cc37640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d2815eb80 0x7f6d280d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6d2815eb80 0x7f6d280d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39448/0 (socket says 172.21.15.99:39448) 2024-09-10T11:35:21.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 -- 172.21.15.99:0/2642345550 learned_addr learned my addr 172.21.15.99:0/2642345550 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 -- 172.21.15.99:0/2642345550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 msgr2=0x7f6d280d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/2642345550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 -- 172.21.15.99:0/2642345550 >> v2:172.21.15.196:3300/0 conn(0x7f6d281626c0 msgr2=0x7f6d280d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/2642345550 >> v2:172.21.15.196:3300/0 conn(0x7f6d281626c0 0x7f6d280d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 -- 172.21.15.99:0/2642345550 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d28162ea0 con 0x7f6d2815eb80 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.700+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/2642345550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d280d29b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009870 tx=0x7f6d1c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da6e651fa1cf6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d277fe640 1 -- 172.21.15.99:0/2642345550 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c037070 con 0x7f6d2815eb80 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d277fe640 1 -- 172.21.15.99:0/2642345550 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c032440 con 0x7f6d2815eb80 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d277fe640 1 -- 172.21.15.99:0/2642345550 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c032810 con 0x7f6d2815eb80 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/2642345550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 msgr2=0x7f6d280d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/2642345550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d280d29b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009870 tx=0x7f6d1c02a180 comp rx=0 tx=0).stop 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/2642345550 shutdown_connections 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/2642345550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/2642345550 >> v2:172.21.15.196:3300/0 conn(0x7f6d281626c0 0x7f6d280d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/2642345550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d280d29b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/2642345550 >> 172.21.15.99:0/2642345550 conn(0x7f6d2808e680 msgr2=0x7f6d2809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/2642345550 shutdown_connections 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.701+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/2642345550 wait complete. 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.702+0000 7f6d2ee9b640 1 Processor -- start 2024-09-10T11:35:21.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.702+0000 7f6d2ee9b640 1 -- start start 2024-09-10T11:35:21.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d281626c0 0x7f6d280ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d280d2ef0 con 0x7f6d2815eb80 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28163460 con 0x7f6d281626c0 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2ee9b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6d280d3070 con 0x7f6d280d4e40 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39458/0 (socket says 172.21.15.99:39458) 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 -- 172.21.15.99:0/744540618 learned_addr learned my addr 172.21.15.99:0/744540618 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d2d438640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d281626c0 0x7f6d280ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 -- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 msgr2=0x7f6d280c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 -- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d281626c0 msgr2=0x7f6d280ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d281626c0 0x7f6d280ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.703+0000 7f6d27fff640 1 -- 172.21.15.99:0/744540618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d280b8870 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d27fff640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400b410 tx=0x7f6d1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14015020 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d14011830 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d280b8b50 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d14011c20 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.704+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d280c8d30 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.705+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280d29b0 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.705+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d14018020 con 0x7f6d2815eb80 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.706+0000 7f6d257fa640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.706+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:21.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.706+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7f6d1408ada0 con 0x7f6d2815eb80 2024-09-10T11:35:21.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.706+0000 7f6d2cc37640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c034000 tx=0x7f6d1c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:21.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.710+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d1401f020 con 0x7f6d2815eb80 2024-09-10T11:35:21.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.907+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d280b2f90 con 0x7f6d2815eb80 2024-09-10T11:35:21.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.907+0000 7f6d257fa640 1 -- 172.21.15.99:0/744540618 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f6d14059710 con 0x7f6d2815eb80 2024-09-10T11:35:21.909 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:21.909 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:21.909 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 msgr2=0x7f6d0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c034000 tx=0x7f6d1c02e000 comp rx=0 tx=0).stop 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 msgr2=0x7f6d28150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400b410 tx=0x7f6d1400b8d0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 shutdown_connections 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6d0c06be30 0x7f6d0c06e2f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d281626c0 0x7f6d280ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d2815eb80 0x7f6d28150080 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 --2- 172.21.15.99:0/744540618 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d280d4e40 0x7f6d280c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:21.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.909+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 >> 172.21.15.99:0/744540618 conn(0x7f6d2808e680 msgr2=0x7f6d2806c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:21.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.910+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 shutdown_connections 2024-09-10T11:35:21.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:21.910+0000 7f6d2ee9b640 1 -- 172.21.15.99:0/744540618 wait complete. 2024-09-10T11:35:21.922 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:22.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.019+0000 7f015732a640 1 Processor -- start 2024-09-10T11:35:22.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.019+0000 7f015732a640 1 -- start start 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f015732a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 0x7f01500b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f015732a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 0x7f01500afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f015732a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015006ee10 con 0x7f01500cd290 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f015732a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f01500b0290 con 0x7f015015b2d0 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f015732a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f015006e570 con 0x7f015015aef0 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01558c7640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f01500cd290 0x7f01500714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57120/0 (socket says 172.21.15.99:57120) 2024-09-10T11:35:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01558c7640 1 -- 172.21.15.99:0/3575452225 learned_addr learned my addr 172.21.15.99:0/3575452225 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01548c5640 1 --2- 172.21.15.99:0/3575452225 >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 0x7f01500afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01550c6640 1 --2- 172.21.15.99:0/3575452225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 0x7f01500b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f0147fff640 1 -- 172.21.15.99:0/3575452225 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479884666 0 0) 0x7f015006ee10 con 0x7f01500cd290 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f0147fff640 1 -- 172.21.15.99:0/3575452225 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f013c003520 con 0x7f01500cd290 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01550c6640 1 -- 172.21.15.99:0/3575452225 >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 msgr2=0x7f01500afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01550c6640 1 --2- 172.21.15.99:0/3575452225 >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 0x7f01500afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01550c6640 1 -- 172.21.15.99:0/3575452225 >> v1:172.21.15.99:6789/0 conn(0x7f01500cd290 legacy=0x7f01500714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01550c6640 1 -- 172.21.15.99:0/3575452225 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015006e850 con 0x7f015015aef0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.020+0000 7f01548c5640 1 --2- 172.21.15.99:0/3575452225 >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 0x7f01500afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.021+0000 7f01550c6640 1 --2- 172.21.15.99:0/3575452225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 0x7f01500b4510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097a0 tx=0x7f014c02c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792b8ff9d14650cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.021+0000 7f0147fff640 1 -- 172.21.15.99:0/3575452225 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c039070 con 0x7f015015aef0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.021+0000 7f0147fff640 1 -- 172.21.15.99:0/3575452225 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014c034640 con 0x7f015015aef0 2024-09-10T11:35:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.021+0000 7f0147fff640 1 -- 172.21.15.99:0/3575452225 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0349f0 con 0x7f015015aef0 2024-09-10T11:35:22.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 -- 172.21.15.99:0/3575452225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 msgr2=0x7f01500b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 --2- 172.21.15.99:0/3575452225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 0x7f01500b4510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f014c0097a0 tx=0x7f014c02c390 comp rx=0 tx=0).stop 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 -- 172.21.15.99:0/3575452225 shutdown_connections 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 --2- 172.21.15.99:0/3575452225 >> v2:172.21.15.196:3300/0 conn(0x7f015015b2d0 0x7f01500afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 --2- 172.21.15.99:0/3575452225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015015aef0 0x7f01500b4510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 -- 172.21.15.99:0/3575452225 >> 172.21.15.99:0/3575452225 conn(0x7f015006bc10 msgr2=0x7f015006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 -- 172.21.15.99:0/3575452225 shutdown_connections 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.022+0000 7f015732a640 1 -- 172.21.15.99:0/3575452225 wait complete. 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.023+0000 7f015732a640 1 Processor -- start 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.023+0000 7f015732a640 1 -- start start 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01500cd290 0x7f01501440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015015aef0 0x7f0150140d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0150071d30 con 0x7f015015b2d0 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0150071bb0 con 0x7f015015aef0 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f015732a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0150071eb0 con 0x7f01500cd290 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39460/0 (socket says 172.21.15.99:39460) 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 -- 172.21.15.99:0/41310798 learned_addr learned my addr 172.21.15.99:0/41310798 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01550c6640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01500cd290 0x7f01501440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01548c5640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015015aef0 0x7f0150140d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 -- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01500cd290 msgr2=0x7f01501440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01500cd290 0x7f01501440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 -- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015015aef0 msgr2=0x7f0150140d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015015aef0 0x7f0150140d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 -- 172.21.15.99:0/41310798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0150153ac0 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.024+0000 7f01558c7640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f014800caa0 tx=0x7f014800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.025+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148017070 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.025+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f014800ac00 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.025+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148012400 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.025+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0150151d90 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.025+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01501f0a00 con 0x7f015015b2d0 2024-09-10T11:35:22.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.026+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01500cd760 con 0x7f015015b2d0 2024-09-10T11:35:22.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.026+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f014801a070 con 0x7f015015b2d0 2024-09-10T11:35:22.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.027+0000 7f0145ffb640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 0x7f013406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.027+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7f014808aaf0 con 0x7f015015b2d0 2024-09-10T11:35:22.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.027+0000 7f01550c6640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 0x7f013406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.027+0000 7f01550c6640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 0x7f013406e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.031+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0150153ac0 con 0x7f015015b2d0 2024-09-10T11:35:22.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.232+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01500b46c0 con 0x7f015015b2d0 2024-09-10T11:35:22.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.233+0000 7f0145ffb640 1 -- 172.21.15.99:0/41310798 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7f0148058ea0 con 0x7f015015b2d0 2024-09-10T11:35:22.234 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:22.235 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 msgr2=0x7f013406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 0x7f013406e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c030000 comp rx=0 tx=0).stop 2024-09-10T11:35:22.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 msgr2=0x7f0150072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f014800caa0 tx=0x7f014800cf60 comp rx=0 tx=0).stop 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 shutdown_connections 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f013406be30 0x7f013406e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015015b2d0 0x7f0150072100 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015015aef0 0x7f0150140d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 --2- 172.21.15.99:0/41310798 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01500cd290 0x7f01501440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 >> 172.21.15.99:0/41310798 conn(0x7f015006bc10 msgr2=0x7f015015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 shutdown_connections 2024-09-10T11:35:22.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.235+0000 7f015732a640 1 -- 172.21.15.99:0/41310798 wait complete. 2024-09-10T11:35:22.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:22.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.343+0000 7fcbe7cf1640 1 Processor -- start 2024-09-10T11:35:22.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.344+0000 7fcbe7cf1640 1 -- start start 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.344+0000 7fcbe7cf1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbe015eb80 0x7fcbe00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.344+0000 7fcbe7cf1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe015ff70 con 0x7fcbe01626c0 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.344+0000 7fcbe7cf1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe00d1620 con 0x7fcbe00d5010 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.344+0000 7fcbe7cf1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe0162bc0 con 0x7fcbe015eb80 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe528c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcbe01626c0 0x7fcbe00d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:57136/0 (socket says 172.21.15.99:57136) 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe528c640 1 -- 172.21.15.99:0/3397686927 learned_addr learned my addr 172.21.15.99:0/3397686927 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:22.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe5a8d640 1 --2- 172.21.15.99:0/3397686927 >> v2:172.21.15.196:3300/0 conn(0x7fcbe015eb80 0x7fcbe00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136162593 0 0) 0x7fcbe015ff70 con 0x7fcbe01626c0 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbcc003520 con 0x7fcbe01626c0 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966040008 0 0) 0x7fcbe00d1620 con 0x7fcbe00d5010 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe015ff70 con 0x7fcbe00d5010 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3351342985 0 0) 0x7fcbcc003520 con 0x7fcbe01626c0 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/3397686927 >> v1:172.21.15.196:6789/0 conn(0x7fcbe00d5010 legacy=0x7fcbe00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/3397686927 >> v1:172.21.15.99:6789/0 conn(0x7fcbe01626c0 legacy=0x7fcbe00d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.345+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/3397686927 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe0162ea0 con 0x7fcbe015eb80 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.346+0000 7fcbe5a8d640 1 --2- 172.21.15.99:0/3397686927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00d29b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0097d0 tx=0x7fcbdc02a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee7336efe276ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.346+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc037070 con 0x7fcbe015eb80 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.346+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc02abf0 con 0x7fcbe015eb80 2024-09-10T11:35:22.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.346+0000 7fcbe4a8b640 1 -- 172.21.15.99:0/3397686927 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4001670 con 0x7fcbe015eb80 2024-09-10T11:35:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/3397686927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 msgr2=0x7fcbe00d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/3397686927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00d29b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0097d0 tx=0x7fcbdc02a160 comp rx=0 tx=0).stop 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/3397686927 shutdown_connections 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/3397686927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00d29b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/3397686927 >> 172.21.15.99:0/3397686927 conn(0x7fcbe008e680 msgr2=0x7fcbe009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/3397686927 shutdown_connections 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.347+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/3397686927 wait complete. 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.348+0000 7fcbe7cf1640 1 Processor -- start 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.348+0000 7fcbe7cf1640 1 -- start start 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01626c0 0x7fcbe00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe00d2ef0 con 0x7fcbe00d5010 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0163460 con 0x7fcbe015eb80 2024-09-10T11:35:22.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe00d3070 con 0x7fcbe01626c0 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39468/0 (socket says 172.21.15.99:39468) 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/1942291769 learned_addr learned my addr 172.21.15.99:0/1942291769 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe528c640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe628e640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01626c0 0x7fcbe00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01626c0 msgr2=0x7fcbe00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01626c0 0x7fcbe00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 msgr2=0x7fcbe00cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 -- 172.21.15.99:0/1942291769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe015b8d0 con 0x7fcbe00d5010 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe5a8d640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc000c00 tx=0x7fcbdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc037040 con 0x7fcbe00d5010 2024-09-10T11:35:22.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbdc004230 con 0x7fcbe00d5010 2024-09-10T11:35:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc006e10 con 0x7fcbe00d5010 2024-09-10T11:35:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe015bbb0 con 0x7fcbe00d5010 2024-09-10T11:35:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.349+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe00ba9c0 con 0x7fcbe00d5010 2024-09-10T11:35:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.350+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe0069030 con 0x7fcbe00d5010 2024-09-10T11:35:22.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.351+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbdc003610 con 0x7fcbe00d5010 2024-09-10T11:35:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.351+0000 7fcbd27fc640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.351+0000 7fcbe528c640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.351+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fcbdc0ab980 con 0x7fcbe00d5010 2024-09-10T11:35:22.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.352+0000 7fcbe528c640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 0x7fcbc406e320 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbe01f8410 tx=0x7fcbd4001d00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.356+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcbdc07a300 con 0x7fcbe00d5010 2024-09-10T11:35:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.521+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbe00d29b0 con 0x7fcbe00d5010 2024-09-10T11:35:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.522+0000 7fcbd27fc640 1 -- 172.21.15.99:0/1942291769 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16784 (secure 0 0 0) 0x7fcbdc079c00 con 0x7fcbe00d5010 2024-09-10T11:35:22.523 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:22.523 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:18.629832+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.6","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 msgr2=0x7fcbc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 0x7fcbc406e320 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbe01f8410 tx=0x7fcbd4001d00 comp rx=0 tx=0).stop 2024-09-10T11:35:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 msgr2=0x7fcbe00ba480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc000c00 tx=0x7fcbdc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 shutdown_connections 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbc406be60 0x7fcbc406e320 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01626c0 0x7fcbe00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe015eb80 0x7fcbe00cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 --2- 172.21.15.99:0/1942291769 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbe00d5010 0x7fcbe00ba480 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 >> 172.21.15.99:0/1942291769 conn(0x7fcbe008e680 msgr2=0x7fcbe006d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 shutdown_connections 2024-09-10T11:35:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.524+0000 7fcbe7cf1640 1 -- 172.21.15.99:0/1942291769 wait complete. 2024-09-10T11:35:22.536 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:35:22.537 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:22.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.634+0000 7fe2a3e45640 1 Processor -- start 2024-09-10T11:35:22.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.634+0000 7fe2a3e45640 1 -- start start 2024-09-10T11:35:22.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a3e45640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe29c0d5010 0x7fe29c0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a3e45640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29c15ff70 con 0x7fe29c1626c0 2024-09-10T11:35:22.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a3e45640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29c0d1620 con 0x7fe29c15eb80 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a3e45640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe29c0d30c0 con 0x7fe29c0d5010 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a13e0640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe29c1626c0 0x7fe29c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39778/0 (socket says 172.21.15.99:39778) 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a13e0640 1 -- 172.21.15.99:0/22966322 learned_addr learned my addr 172.21.15.99:0/22966322 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a23e2640 1 --2- 172.21.15.99:0/22966322 >> v2:172.21.15.196:3300/0 conn(0x7fe29c0d5010 0x7fe29c0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1960645519 0 0) 0x7fe29c15ff70 con 0x7fe29c1626c0 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe288003520 con 0x7fe29c1626c0 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a23e2640 1 -- 172.21.15.99:0/22966322 >> v1:172.21.15.196:6789/0 conn(0x7fe29c15eb80 legacy=0x7fe29c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a23e2640 1 -- 172.21.15.99:0/22966322 >> v1:172.21.15.99:6789/0 conn(0x7fe29c1626c0 legacy=0x7fe29c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2273856658 0 0) 0x7fe288003520 con 0x7fe29c1626c0 2024-09-10T11:35:22.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.635+0000 7fe2a23e2640 1 -- 172.21.15.99:0/22966322 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c162bc0 con 0x7fe29c0d5010 2024-09-10T11:35:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.636+0000 7fe2a23e2640 1 --2- 172.21.15.99:0/22966322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c0d5010 0x7fe29c0d0770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe294009dc0 tx=0x7fe29402a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2bcd3ac4b1f55c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.636+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c001670 con 0x7fe29c0d5010 2024-09-10T11:35:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.636+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe294007540 con 0x7fe29c0d5010 2024-09-10T11:35:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.636+0000 7fe2a0bdf640 1 -- 172.21.15.99:0/22966322 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294007910 con 0x7fe29c0d5010 2024-09-10T11:35:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 -- 172.21.15.99:0/22966322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c0d5010 msgr2=0x7fe29c0d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/22966322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c0d5010 0x7fe29c0d0770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe294009dc0 tx=0x7fe29402a630 comp rx=0 tx=0).stop 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 -- 172.21.15.99:0/22966322 shutdown_connections 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/22966322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c0d5010 0x7fe29c0d0770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 -- 172.21.15.99:0/22966322 >> 172.21.15.99:0/22966322 conn(0x7fe29c08e680 msgr2=0x7fe29c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 -- 172.21.15.99:0/22966322 shutdown_connections 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.637+0000 7fe2a3e45640 1 -- 172.21.15.99:0/22966322 wait complete. 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.638+0000 7fe2a3e45640 1 Processor -- start 2024-09-10T11:35:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.638+0000 7fe2a3e45640 1 -- start start 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 0x7fe29c14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 0x7fe29c0ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c163460 con 0x7fe29c0d5010 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c163180 con 0x7fe29c1626c0 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a3e45640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c0d0cb0 con 0x7fe29c15eb80 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36880/0 (socket says 172.21.15.99:36880) 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 -- 172.21.15.99:0/1222187127 learned_addr learned my addr 172.21.15.99:0/1222187127 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a13e0640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 0x7fe29c14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a23e2640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 0x7fe29c0ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 -- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 msgr2=0x7fe29c14f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 0x7fe29c14f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 -- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 msgr2=0x7fe29c0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 0x7fe29c0ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a1be1640 1 -- 172.21.15.99:0/1222187127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c0cec40 con 0x7fe29c0d5010 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a13e0640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 0x7fe29c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.639+0000 7fe2a23e2640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 0x7fe29c0ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2a1be1640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c390 tx=0x7fe29800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298017070 con 0x7fe29c0d5010 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2980052d0 con 0x7fe29c0d5010 2024-09-10T11:35:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c15ace0 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298003830 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.640+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c0b1fd0 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.641+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c15aad0 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.641+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe29801a070 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.642+0000 7fe2927fc640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 0x7fe28006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.642+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fe29808a1f0 con 0x7fe29c0d5010 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.642+0000 7fe2a13e0640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 0x7fe28006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.642+0000 7fe2a13e0640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 0x7fe28006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0b5220 tx=0x7fe28c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.646+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe298058b60 con 0x7fe29c0d5010 2024-09-10T11:35:22.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.846+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe29c0d29b0 con 0x7fe29c0d5010 2024-09-10T11:35:22.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.847+0000 7fe2927fc640 1 -- 172.21.15.99:0/1222187127 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7fe298058460 con 0x7fe29c0d5010 2024-09-10T11:35:22.848 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:22.848 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:22.848 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 msgr2=0x7fe28006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 0x7fe28006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0b5220 tx=0x7fe28c009040 comp rx=0 tx=0).stop 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 msgr2=0x7fe29c15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c390 tx=0x7fe29800c850 comp rx=0 tx=0).stop 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 shutdown_connections 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe28006be30 0x7fe28006e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe29c1626c0 0x7fe29c0ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe29c15eb80 0x7fe29c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 --2- 172.21.15.99:0/1222187127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe29c0d5010 0x7fe29c15a2e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 >> 172.21.15.99:0/1222187127 conn(0x7fe29c08e680 msgr2=0x7fe29c0b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 shutdown_connections 2024-09-10T11:35:22.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.849+0000 7fe2a3e45640 1 -- 172.21.15.99:0/1222187127 wait complete. 2024-09-10T11:35:22.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:22.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.958+0000 7fa6e9065640 1 Processor -- start 2024-09-10T11:35:22.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.958+0000 7fa6e9065640 1 -- start start 2024-09-10T11:35:22.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e9065640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6e406bc10 0x7fa6e4163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e9065640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6e406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e9065640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6e40b5140 con 0x7fa6e4160030 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e9065640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa6e40b5420 con 0x7fa6e406bc10 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e9065640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e4162b40 con 0x7fa6e40b4930 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e259c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa6e4160030 0x7fa6e406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39790/0 (socket says 172.21.15.99:39790) 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.959+0000 7fa6e259c640 1 -- 172.21.15.99:0/2857239521 learned_addr learned my addr 172.21.15.99:0/2857239521 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e2d9d640 1 --2- 172.21.15.99:0/2857239521 >> v2:172.21.15.196:3300/0 conn(0x7fa6e406bc10 0x7fa6e4163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3333353077 0 0) 0x7fa6e40b5140 con 0x7fa6e4160030 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e359e640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6e406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d0003520 con 0x7fa6e4160030 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1590408750 0 0) 0x7fa6d0003520 con 0x7fa6e4160030 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 msgr2=0x7fa6e406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6e406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 >> v2:172.21.15.196:3300/0 conn(0x7fa6e406bc10 msgr2=0x7fa6e4163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 --2- 172.21.15.99:0/2857239521 >> v2:172.21.15.196:3300/0 conn(0x7fa6e406bc10 0x7fa6e4163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e4162e20 con 0x7fa6e4160030 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa6cc001670 con 0x7fa6e4160030 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 >> v1:172.21.15.99:6789/0 conn(0x7fa6e4160030 legacy=0x7fa6e406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00039b0 0x7fa6d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b5420 con 0x7fa6d0004800 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e4162b40 con 0x7fa6d00039b0 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d000b9c0 con 0x7fa6e40b4930 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e2d9d640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00039b0 0x7fa6d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e359e640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 -- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 msgr2=0x7fa6d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 -- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00039b0 msgr2=0x7fa6d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00039b0 0x7fa6d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e259c640 1 -- 172.21.15.99:0/2857239521 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d0003520 con 0x7fa6d0004800 2024-09-10T11:35:22.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.960+0000 7fa6e359e640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e40b4930 0x7fa6d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.961+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6d0004c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c210 tx=0x7fa6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44888aac2e6f669d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.961+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc007d70 con 0x7fa6d0004800 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.961+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6cc005af0 con 0x7fa6d0004800 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.961+0000 7fa6e1d9b640 1 -- 172.21.15.99:0/2857239521 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc007070 con 0x7fa6d0004800 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 -- 172.21.15.99:0/2857239521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 msgr2=0x7fa6d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2857239521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6d0004c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c210 tx=0x7fa6d0005830 comp rx=0 tx=0).stop 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e359e640 1 -- 172.21.15.99:0/2857239521 reap_dead start 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 -- 172.21.15.99:0/2857239521 shutdown_connections 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 -- 172.21.15.99:0/2857239521 >> 172.21.15.99:0/2857239521 conn(0x7fa6e40b3630 msgr2=0x7fa6e40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 -- 172.21.15.99:0/2857239521 shutdown_connections 2024-09-10T11:35:22.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.962+0000 7fa6e9065640 1 -- 172.21.15.99:0/2857239521 wait complete. 2024-09-10T11:35:22.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.963+0000 7fa6e9065640 1 Processor -- start 2024-09-10T11:35:22.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.963+0000 7fa6e9065640 1 -- start start 2024-09-10T11:35:22.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 0x7fa6e4144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e4144b50 0x7fa6e412bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b4d90 con 0x7fa6d0004800 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40d1560 con 0x7fa6e4144b50 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e9065640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e413e7c0 con 0x7fa6e415f290 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36896/0 (socket says 172.21.15.99:36896) 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 -- 172.21.15.99:0/2882732148 learned_addr learned my addr 172.21.15.99:0/2882732148 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e359e640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e4144b50 0x7fa6e412bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 0x7fa6e4144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 -- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 msgr2=0x7fa6e4144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 0x7fa6e4144610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 -- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e4144b50 msgr2=0x7fa6e412bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e4144b50 0x7fa6e412bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:22.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e2d9d640 1 -- 172.21.15.99:0/2882732148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e413e4e0 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.964+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 0x7fa6e4144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6e2d9d640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4011910 tx=0x7fa6d4011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d400faf0 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6d400fc90 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e4145c10 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4014440 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.965+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e413e120 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.966+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e40b5140 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.966+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6d4012040 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.967+0000 7fa6db7fe640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 0x7fa6c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.967+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fa6d408b490 con 0x7fa6d0004800 2024-09-10T11:35:22.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.967+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 0x7fa6c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:22.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.967+0000 7fa6e259c640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 0x7fa6c406e320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc033000 tx=0x7fa6cc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:22.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:22.971+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa6d4019030 con 0x7fa6d0004800 2024-09-10T11:35:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.170+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6e4073d40 con 0x7fa6d0004800 2024-09-10T11:35:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.170+0000 7fa6db7fe640 1 -- 172.21.15.99:0/2882732148 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5225 (secure 0 0 0) 0x7fa6d4059e00 con 0x7fa6d0004800 2024-09-10T11:35:23.171 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:23.171 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":48,"btime":"2024-09-10T11:35:04:580243+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:01.900031+0000","modified":"2024-09-10T11:35:03.905945+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6594},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6594":{"gid":6594,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6836/1212129697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1212129697},{"type":"v1","addr":"172.21.15.99:6836","nonce":1212129697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6594,"qdb_cluster":[6594]},"id":7}]} 2024-09-10T11:35:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 48 2024-09-10T11:35:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 msgr2=0x7fa6c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 0x7fa6c406e320 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc033000 tx=0x7fa6cc009030 comp rx=0 tx=0).stop 2024-09-10T11:35:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 msgr2=0x7fa6e413e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4011910 tx=0x7fa6d4011dd0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 shutdown_connections 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa6c406be60 0x7fa6c406e320 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e4144b50 0x7fa6e412bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e415f290 0x7fa6e4144610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 --2- 172.21.15.99:0/2882732148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6d0004800 0x7fa6e413e980 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 >> 172.21.15.99:0/2882732148 conn(0x7fa6e40b3630 msgr2=0x7fa6e412d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 shutdown_connections 2024-09-10T11:35:23.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.172+0000 7fa6e9065640 1 -- 172.21.15.99:0/2882732148 wait complete. 2024-09-10T11:35:23.184 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.282+0000 7fadcd427640 1 Processor -- start 2024-09-10T11:35:23.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.282+0000 7fadcd427640 1 -- start start 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadcd427640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fadc815eb80 0x7fadc80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadcd427640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc815ff70 con 0x7fadc81626c0 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadcd427640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc80d1620 con 0x7fadc80d5010 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadcd427640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fadc8162bc0 con 0x7fadc815eb80 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc6ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fadc815eb80 0x7fadc80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc67fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fadc81626c0 0x7fadc80d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39802/0 (socket says 172.21.15.99:39802) 2024-09-10T11:35:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc67fc640 1 -- 172.21.15.99:0/1752520444 learned_addr learned my addr 172.21.15.99:0/1752520444 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc6ffd640 1 -- 172.21.15.99:0/1752520444 >> v1:172.21.15.196:6789/0 conn(0x7fadc80d5010 legacy=0x7fadc80d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc6ffd640 1 -- 172.21.15.99:0/1752520444 >> v1:172.21.15.99:6789/0 conn(0x7fadc81626c0 legacy=0x7fadc80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc5ffb640 1 -- 172.21.15.99:0/1752520444 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503766165 0 0) 0x7fadc815ff70 con 0x7fadc81626c0 2024-09-10T11:35:23.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.283+0000 7fadc6ffd640 1 -- 172.21.15.99:0/1752520444 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc8162ea0 con 0x7fadc815eb80 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.284+0000 7fadc6ffd640 1 --2- 172.21.15.99:0/1752520444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc815eb80 0x7fadc80d29b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fadb80097d0 tx=0x7fadb802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425b4a4ae568e3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.284+0000 7fadc5ffb640 1 -- 172.21.15.99:0/1752520444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8037070 con 0x7fadc815eb80 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.284+0000 7fadc5ffb640 1 -- 172.21.15.99:0/1752520444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb8032440 con 0x7fadc815eb80 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.284+0000 7fadc5ffb640 1 -- 172.21.15.99:0/1752520444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8032810 con 0x7fadc815eb80 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 -- 172.21.15.99:0/1752520444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc815eb80 msgr2=0x7fadc80d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 --2- 172.21.15.99:0/1752520444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc815eb80 0x7fadc80d29b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fadb80097d0 tx=0x7fadb802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 -- 172.21.15.99:0/1752520444 shutdown_connections 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 --2- 172.21.15.99:0/1752520444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc815eb80 0x7fadc80d29b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 -- 172.21.15.99:0/1752520444 >> 172.21.15.99:0/1752520444 conn(0x7fadc808e680 msgr2=0x7fadc809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 -- 172.21.15.99:0/1752520444 shutdown_connections 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.285+0000 7fadcd427640 1 -- 172.21.15.99:0/1752520444 wait complete. 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.286+0000 7fadcd427640 1 Processor -- start 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.286+0000 7fadcd427640 1 -- start start 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc815eb80 0x7fadc814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 0x7fadc815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fadc80d2ef0 con 0x7fadc80d5010 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8163460 con 0x7fadc81626c0 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fadc80d3070 con 0x7fadc815eb80 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36908/0 (socket says 172.21.15.99:36908) 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 -- 172.21.15.99:0/2126763840 learned_addr learned my addr 172.21.15.99:0/2126763840 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc67fc640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc815eb80 0x7fadc814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc77fe640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 0x7fadc815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 -- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc815eb80 msgr2=0x7fadc814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc815eb80 0x7fadc814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 -- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 msgr2=0x7fadc815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 0x7fadc815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 -- 172.21.15.99:0/2126763840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc8160770 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc77fe640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 0x7fadc815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadc6ffd640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8037040 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb8042070 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc8160a50 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8030a70 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.287+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc8161010 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.289+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc80d29b0 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.289+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadb8030c10 con 0x7fadc80d5010 2024-09-10T11:35:23.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.289+0000 7fadaf7fe640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 0x7fadb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.289+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7fadb80ab960 con 0x7fadc80d5010 2024-09-10T11:35:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.289+0000 7fadc67fc640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 0x7fadb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.290+0000 7fadc67fc640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 0x7fadb406e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fadc815d060 tx=0x7fadbc007d00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.294+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fadb807a2d0 con 0x7fadc80d5010 2024-09-10T11:35:23.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.460+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadc80b5920 con 0x7fadc80d5010 2024-09-10T11:35:23.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.461+0000 7fadaf7fe640 1 -- 172.21.15.99:0/2126763840 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+16784 (secure 0 0 0) 0x7fadb8079bd0 con 0x7fadc80d5010 2024-09-10T11:35:23.462 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:23.462 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":127,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:18.629832+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:00.245532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:00.874362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.6","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 msgr2=0x7fadb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 0x7fadb406e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fadc815d060 tx=0x7fadbc007d00 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 msgr2=0x7fadc80cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb80046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 shutdown_connections 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fadb406be30 0x7fadb406e2f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc81626c0 0x7fadc815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fadc815eb80 0x7fadc814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 --2- 172.21.15.99:0/2126763840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadc80d5010 0x7fadc80cbd90 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 >> 172.21.15.99:0/2126763840 conn(0x7fadc808e680 msgr2=0x7fadc809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 shutdown_connections 2024-09-10T11:35:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.463+0000 7fadcd427640 1 -- 172.21.15.99:0/2126763840 wait complete. 2024-09-10T11:35:23.476 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:35:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.574+0000 7f5a3ebe4640 1 Processor -- start 2024-09-10T11:35:23.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 -- start start 2024-09-10T11:35:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a380d1760 0x7f5a380cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 0x7f5a380cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a380ce430 con 0x7f5a38160030 2024-09-10T11:35:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5a380ce710 con 0x7f5a380d1760 2024-09-10T11:35:23.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a3ebe4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5a380d1240 con 0x7f5a380d0130 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.575+0000 7f5a37fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5a380d1760 0x7f5a380cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a3c980640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5a38160030 0x7f5a380b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39818/0 (socket says 172.21.15.99:39818) 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a3c980640 1 -- 172.21.15.99:0/3270487396 learned_addr learned my addr 172.21.15.99:0/3270487396 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a37fff640 1 -- 172.21.15.99:0/3270487396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 msgr2=0x7f5a380cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a3d181640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 0x7f5a380cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a37fff640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 0x7f5a380cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a37fff640 1 -- 172.21.15.99:0/3270487396 >> v1:172.21.15.99:6789/0 conn(0x7f5a38160030 legacy=0x7f5a380b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a377fe640 1 -- 172.21.15.99:0/3270487396 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4245381939 0 0) 0x7f5a380ce430 con 0x7f5a38160030 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a37fff640 1 -- 172.21.15.99:0/3270487396 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a380ce030 con 0x7f5a380d1760 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a3d181640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 0x7f5a380cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.576+0000 7f5a37fff640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a380d1760 0x7f5a380cd5b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0097a0 tx=0x7f5a2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf38c5b0d3958980 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a377fe640 1 -- 172.21.15.99:0/3270487396 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c037070 con 0x7f5a380d1760 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a377fe640 1 -- 172.21.15.99:0/3270487396 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a2c032440 con 0x7f5a380d1760 2024-09-10T11:35:23.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a377fe640 1 -- 172.21.15.99:0/3270487396 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c032810 con 0x7f5a380d1760 2024-09-10T11:35:23.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/3270487396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a380d1760 msgr2=0x7f5a380cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a380d1760 0x7f5a380cd5b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0097a0 tx=0x7f5a2c02a180 comp rx=0 tx=0).stop 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/3270487396 shutdown_connections 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d0130 0x7f5a380cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/3270487396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a380d1760 0x7f5a380cd5b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/3270487396 >> 172.21.15.99:0/3270487396 conn(0x7f5a380b4650 msgr2=0x7f5a380b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/3270487396 shutdown_connections 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.577+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/3270487396 wait complete. 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.578+0000 7f5a3ebe4640 1 Processor -- start 2024-09-10T11:35:23.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.578+0000 7f5a3ebe4640 1 -- start start 2024-09-10T11:35:23.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a380d0130 0x7f5a3806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 0x7f5a3815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38160660 con 0x7f5a38160030 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a380b5950 con 0x7f5a380d0130 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3ebe4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3806eb20 con 0x7f5a380d1760 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3d181640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3d181640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36934/0 (socket says 172.21.15.99:36934) 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.579+0000 7f5a3d181640 1 -- 172.21.15.99:0/2695726330 learned_addr learned my addr 172.21.15.99:0/2695726330 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3c980640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a380d0130 0x7f5a3806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a37fff640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 0x7f5a3815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 -- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 msgr2=0x7f5a3815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 0x7f5a3815d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 -- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a380d0130 msgr2=0x7f5a3806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a380d0130 0x7f5a3806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 -- 172.21.15.99:0/2695726330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3806cec0 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a37fff640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 0x7f5a3815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3d181640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000cf20 tx=0x7f5a3000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3001a070 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3000d840 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3806d1a0 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000dc30 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.580+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3806d9a0 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.581+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a380b38f0 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.581+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5a3001d070 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.582+0000 7f5a357fa640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.582+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5767+0+0 (secure 0 0 0) 0x7f5a3008b610 con 0x7f5a38160030 2024-09-10T11:35:23.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.582+0000 7f5a3c980640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.582+0000 7f5a3c980640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 0x7f5a2406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000f60 tx=0x7f5a28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.586+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5a300599c0 con 0x7f5a38160030 2024-09-10T11:35:23.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.789+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a38070500 con 0x7f5a38160030 2024-09-10T11:35:23.790 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:35:23.790+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:35:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.793+0000 7f5a357fa640 1 -- 172.21.15.99:0/2695726330 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7f5a3005dcc0 con 0x7f5a38160030 2024-09-10T11:35:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:35:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.796+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 msgr2=0x7f5a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.796+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 0x7f5a2406e2f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000f60 tx=0x7f5a28009040 comp rx=0 tx=0).stop 2024-09-10T11:35:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.796+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 msgr2=0x7f5a380cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.796+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000cf20 tx=0x7f5a3000a7d0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 shutdown_connections 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5a2406be30 0x7f5a2406e2f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a38160030 0x7f5a380cc1c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5a380d1760 0x7f5a3815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 --2- 172.21.15.99:0/2695726330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a380d0130 0x7f5a3806e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 >> 172.21.15.99:0/2695726330 conn(0x7f5a380b4650 msgr2=0x7f5a38070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 shutdown_connections 2024-09-10T11:35:23.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.799+0000 7f5a3ebe4640 1 -- 172.21.15.99:0/2695726330 wait complete. 2024-09-10T11:35:23.801 INFO:tasks.ceph.mds.c.smithi099.stderr: -3306> 2024-09-10T11:35:08.557+0000 7f948d887640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:35:23.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:35:23.828 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:35:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.910+0000 7f243f65c640 1 Processor -- start 2024-09-10T11:35:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 -- start start 2024-09-10T11:35:23.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f243815eeb0 0x7f24380cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 0x7f24380d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f243815f290 con 0x7f24380d2160 2024-09-10T11:35:23.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24380d2800 con 0x7f243815eeb0 2024-09-10T11:35:23.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.911+0000 7f243f65c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24380d0e60 con 0x7f24380d3530 2024-09-10T11:35:23.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f243815eeb0 0x7f24380cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243dbf9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 0x7f24380d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f243815eeb0 0x7f24380cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:37530/0 (socket says 172.21.15.99:37530) 2024-09-10T11:35:23.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 -- 172.21.15.99:0/3491422853 learned_addr learned my addr 172.21.15.99:0/3491422853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:23.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 -- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 msgr2=0x7f24380d39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:23.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 0x7f24380d39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 -- 172.21.15.99:0/3491422853 >> v1:172.21.15.196:6789/0 conn(0x7f24380d2160 legacy=0x7f24380b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243d3f8640 1 -- 172.21.15.99:0/3491422853 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24380d2a50 con 0x7f243815eeb0 2024-09-10T11:35:23.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.912+0000 7f243dbf9640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 0x7f24380d39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.913+0000 7f243d3f8640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243815eeb0 0x7f24380cfa20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f243402a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c769b1b4ba8018b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.913+0000 7f242ffff640 1 -- 172.21.15.99:0/3491422853 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434037070 con 0x7f243815eeb0 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.913+0000 7f242ffff640 1 -- 172.21.15.99:0/3491422853 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2434032440 con 0x7f243815eeb0 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.913+0000 7f242ffff640 1 -- 172.21.15.99:0/3491422853 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434032810 con 0x7f243815eeb0 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 -- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243815eeb0 msgr2=0x7f24380cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243815eeb0 0x7f24380cfa20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f243402a160 comp rx=0 tx=0).stop 2024-09-10T11:35:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 -- 172.21.15.99:0/3491422853 shutdown_connections 2024-09-10T11:35:23.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d3530 0x7f24380d39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 --2- 172.21.15.99:0/3491422853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f243815eeb0 0x7f24380cfa20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 -- 172.21.15.99:0/3491422853 >> 172.21.15.99:0/3491422853 conn(0x7f24380d4c30 msgr2=0x7f24380d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:23.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 -- 172.21.15.99:0/3491422853 shutdown_connections 2024-09-10T11:35:23.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.914+0000 7f243f65c640 1 -- 172.21.15.99:0/3491422853 wait complete. 2024-09-10T11:35:23.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.915+0000 7f243f65c640 1 Processor -- start 2024-09-10T11:35:23.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.915+0000 7f243f65c640 1 -- start start 2024-09-10T11:35:23.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d2160 0x7f2438156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24380d3530 0x7f2438158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24380b4e00 con 0x7f243815eeb0 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24380b4c80 con 0x7f24380d3530 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24380d0a10 con 0x7f24380d2160 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36962/0 (socket says 172.21.15.99:36962) 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 -- 172.21.15.99:0/2682113645 learned_addr learned my addr 172.21.15.99:0/2682113645 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243cbf7640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24380d3530 0x7f2438158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243d3f8640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d2160 0x7f2438156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 -- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d2160 msgr2=0x7f2438156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d2160 0x7f2438156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 -- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24380d3530 msgr2=0x7f2438158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:23.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24380d3530 0x7f2438158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 -- 172.21.15.99:0/2682113645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2438159570 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243dbf9640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2430009e60 tx=0x7f243000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430018030 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243000cdf0 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2430003bb0 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.916+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2438159850 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.917+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2438159e10 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.918+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24380b40f0 con 0x7f243815eeb0 2024-09-10T11:35:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.918+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f243001b070 con 0x7f243815eeb0 2024-09-10T11:35:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.918+0000 7f242dffb640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 0x7f241c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.918+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5853+0+0 (secure 0 0 0) 0x7f243008b760 con 0x7f243815eeb0 2024-09-10T11:35:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.918+0000 7f243d3f8640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 0x7f241c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.919+0000 7f243d3f8640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 0x7f241c06e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2434034000 tx=0x7f243402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:23.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:23.923+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24380b40f0 con 0x7f243815eeb0 2024-09-10T11:35:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.120+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f24380babb0 con 0x7f243815eeb0 2024-09-10T11:35:24.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.796+0000 7f242dffb640 1 -- 172.21.15.99:0/2682113645 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f2430010040 con 0x7f243815eeb0 2024-09-10T11:35:24.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 msgr2=0x7f241c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 0x7f241c06e2f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2434034000 tx=0x7f243402e000 comp rx=0 tx=0).stop 2024-09-10T11:35:24.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 msgr2=0x7f24380ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2430009e60 tx=0x7f243000c490 comp rx=0 tx=0).stop 2024-09-10T11:35:24.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 shutdown_connections 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f241c06be30 0x7f241c06e2f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f243815eeb0 0x7f24380ca8f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24380d3530 0x7f2438158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 --2- 172.21.15.99:0/2682113645 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24380d2160 0x7f2438156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 >> 172.21.15.99:0/2682113645 conn(0x7f24380d4c30 msgr2=0x7f24380cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 shutdown_connections 2024-09-10T11:35:24.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.799+0000 7f243f65c640 1 -- 172.21.15.99:0/2682113645 wait complete. 2024-09-10T11:35:24.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:35:24.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 Processor -- start 2024-09-10T11:35:24.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 -- start start 2024-09-10T11:35:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 0x7f58440d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58440d2150 con 0x7f5844162d10 2024-09-10T11:35:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f58440d0d30 con 0x7f58440b59b0 2024-09-10T11:35:24.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.960+0000 7f584a998640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58440b4d30 con 0x7f58440d16f0 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:43234/0 (socket says 172.21.15.99:43234) 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 -- 172.21.15.99:0/1703972986 learned_addr learned my addr 172.21.15.99:0/1703972986 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f583bfff640 1 --2- 172.21.15.99:0/1703972986 >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 0x7f58440d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 -- 172.21.15.99:0/1703972986 >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 msgr2=0x7f58440d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 --2- 172.21.15.99:0/1703972986 >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 0x7f58440d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 -- 172.21.15.99:0/1703972986 >> v1:172.21.15.196:6789/0 conn(0x7f5844162d10 legacy=0x7f58441631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f5843fff640 1 -- 172.21.15.99:0/1703972986 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440b5780 con 0x7f58440d16f0 2024-09-10T11:35:24.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.961+0000 7f583bfff640 1 --2- 172.21.15.99:0/1703972986 >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 0x7f58440d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.962+0000 7f5843fff640 1 --2- 172.21.15.99:0/1703972986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5834009870 tx=0x7f583402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a86fffbc80401bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.962+0000 7f58437fe640 1 -- 172.21.15.99:0/1703972986 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834037070 con 0x7f58440d16f0 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.962+0000 7f58437fe640 1 -- 172.21.15.99:0/1703972986 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5834032440 con 0x7f58440d16f0 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.962+0000 7f58437fe640 1 -- 172.21.15.99:0/1703972986 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834032810 con 0x7f58440d16f0 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 -- 172.21.15.99:0/1703972986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 msgr2=0x7f58440d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 --2- 172.21.15.99:0/1703972986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5834009870 tx=0x7f583402a270 comp rx=0 tx=0).stop 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 -- 172.21.15.99:0/1703972986 shutdown_connections 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 --2- 172.21.15.99:0/1703972986 >> v2:172.21.15.196:3300/0 conn(0x7f58440b59b0 0x7f58440d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 --2- 172.21.15.99:0/1703972986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58440d16f0 0x7f58440d1ad0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 -- 172.21.15.99:0/1703972986 >> 172.21.15.99:0/1703972986 conn(0x7f58440d4c20 msgr2=0x7f58440d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 -- 172.21.15.99:0/1703972986 shutdown_connections 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.963+0000 7f584a998640 1 -- 172.21.15.99:0/1703972986 wait complete. 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.964+0000 7f584a998640 1 Processor -- start 2024-09-10T11:35:24.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.964+0000 7f584a998640 1 -- start start 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 0x7f584415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5844162d10 0x7f58441f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58440b4f80 con 0x7f58440d16f0 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58440b52f0 con 0x7f58440b59b0 2024-09-10T11:35:24.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f584a998640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58441638b0 con 0x7f5844162d10 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36976/0 (socket says 172.21.15.99:36976) 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 -- 172.21.15.99:0/1640855773 learned_addr learned my addr 172.21.15.99:0/1640855773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f5843fff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 0x7f584415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f5848f35640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5844162d10 0x7f58441f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 -- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5844162d10 msgr2=0x7f58441f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5844162d10 0x7f58441f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 -- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 msgr2=0x7f584415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 0x7f584415a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f583bfff640 1 -- 172.21.15.99:0/1640855773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440ca3b0 con 0x7f58440d16f0 2024-09-10T11:35:24.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.965+0000 7f5843fff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 0x7f584415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f583bfff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f582c00ebb0 tx=0x7f582c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c017070 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c00cc30 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c013470 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58440ca690 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.966+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58440b30a0 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.967+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58440d1ad0 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.967+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f582c01a070 con 0x7f58440d16f0 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.968+0000 7f58417fa640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 0x7f582406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:24.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.968+0000 7f5843fff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 0x7f582406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:24.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.968+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5853+0+0 (secure 0 0 0) 0x7f582c08b160 con 0x7f58440d16f0 2024-09-10T11:35:24.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.968+0000 7f5843fff640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 0x7f582406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5834034000 tx=0x7f583402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:24.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:24.972+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f582c059b10 con 0x7f58440d16f0 2024-09-10T11:35:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.138+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5844161cb0 con 0x7f58440d16f0 2024-09-10T11:35:25.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.805+0000 7f58417fa640 1 -- 172.21.15.99:0/1640855773 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) ==== 181+0+0 (secure 0 0 0) 0x7f582c059410 con 0x7f58440d16f0 2024-09-10T11:35:25.806 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.807+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 msgr2=0x7f582406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.807+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 0x7f582406e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5834034000 tx=0x7f583402e000 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.807+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 msgr2=0x7f58441f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.807+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f582c00ebb0 tx=0x7f582c00c520 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 shutdown_connections 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f582406be30 0x7f582406e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5844162d10 0x7f58441f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58440d16f0 0x7f58441f52d0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 --2- 172.21.15.99:0/1640855773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440b59b0 0x7f584415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 >> 172.21.15.99:0/1640855773 conn(0x7f58440d4c20 msgr2=0x7f5844160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 shutdown_connections 2024-09-10T11:35:25.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.808+0000 7f584a998640 1 -- 172.21.15.99:0/1640855773 wait complete. 2024-09-10T11:35:25.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:35:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.926+0000 7f58c63f6640 1 Processor -- start 2024-09-10T11:35:25.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.926+0000 7f58c63f6640 1 -- start start 2024-09-10T11:35:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c63f6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58c00b0090 0x7f58c00b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:25.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c63f6640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c00b40e0 con 0x7f58c015f1e0 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c63f6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c00b5ab0 con 0x7f58c015ee00 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c63f6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f58c0073c50 con 0x7f58c00b0090 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c4993640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58c00b0090 0x7f58c00b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58bf7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f58c015f1e0 0x7f58c00b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39824/0 (socket says 172.21.15.99:39824) 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58bf7fe640 1 -- 172.21.15.99:0/323799482 learned_addr learned my addr 172.21.15.99:0/323799482 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c4993640 1 -- 172.21.15.99:0/323799482 >> v1:172.21.15.196:6789/0 conn(0x7f58c015ee00 legacy=0x7f58c00b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c4993640 1 -- 172.21.15.99:0/323799482 >> v1:172.21.15.99:6789/0 conn(0x7f58c015f1e0 legacy=0x7f58c00b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.927+0000 7f58c4993640 1 -- 172.21.15.99:0/323799482 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c00b4380 con 0x7f58c00b0090 2024-09-10T11:35:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.928+0000 7f58c4993640 1 --2- 172.21.15.99:0/323799482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00b0090 0x7f58c00b5f90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009d90 tx=0x7f58b802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1ec33e1ece2c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.928+0000 7f58beffd640 1 -- 172.21.15.99:0/323799482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8007740 con 0x7f58c00b0090 2024-09-10T11:35:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.928+0000 7f58beffd640 1 -- 172.21.15.99:0/323799482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b80078e0 con 0x7f58c00b0090 2024-09-10T11:35:25.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.928+0000 7f58beffd640 1 -- 172.21.15.99:0/323799482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b8007cb0 con 0x7f58c00b0090 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 -- 172.21.15.99:0/323799482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00b0090 msgr2=0x7f58c00b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 --2- 172.21.15.99:0/323799482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00b0090 0x7f58c00b5f90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009d90 tx=0x7f58b802a6f0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 -- 172.21.15.99:0/323799482 shutdown_connections 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 --2- 172.21.15.99:0/323799482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00b0090 0x7f58c00b5f90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 -- 172.21.15.99:0/323799482 >> 172.21.15.99:0/323799482 conn(0x7f58c00730c0 msgr2=0x7f58c00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 -- 172.21.15.99:0/323799482 shutdown_connections 2024-09-10T11:35:25.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.929+0000 7f58c63f6640 1 -- 172.21.15.99:0/323799482 wait complete. 2024-09-10T11:35:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.930+0000 7f58c63f6640 1 Processor -- start 2024-09-10T11:35:25.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.930+0000 7f58c63f6640 1 -- start start 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 0x7f58c01e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c015f1e0 0x7f58c014c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00b0600 con 0x7f58c015ee00 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58c00afd60 con 0x7f58c015f1e0 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c63f6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58c00b0780 con 0x7f58c00b0090 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36980/0 (socket says 172.21.15.99:36980) 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 -- 172.21.15.99:0/4060232831 learned_addr learned my addr 172.21.15.99:0/4060232831 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58c4993640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c015f1e0 0x7f58c014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bffff640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 0x7f58c01e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:25.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 -- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 msgr2=0x7f58c01e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 0x7f58c01e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 -- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c015f1e0 msgr2=0x7f58c014c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c015f1e0 0x7f58c014c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bf7fe640 1 -- 172.21.15.99:0/4060232831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c01f5bc0 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.931+0000 7f58bffff640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 0x7f58c01e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58bf7fe640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f58b400c480 tx=0x7f58b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4017070 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b4005ce0 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b40072f0 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c01f5ea0 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.932+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c01e8670 con 0x7f58c015ee00 2024-09-10T11:35:25.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.933+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c0159a10 con 0x7f58c015ee00 2024-09-10T11:35:25.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.933+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b401a070 con 0x7f58c015ee00 2024-09-10T11:35:25.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.934+0000 7f58bcff9640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:25.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.934+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5357+0+0 (secure 0 0 0) 0x7f58b4089f60 con 0x7f58c015ee00 2024-09-10T11:35:25.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.934+0000 7f58bffff640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:25.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.934+0000 7f58bffff640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 0x7f58ac06e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000f60 tx=0x7f58b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:25.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:25.938+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58c0159a10 con 0x7f58c015ee00 2024-09-10T11:35:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.104+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58c00b5f90 con 0x7f58c015ee00 2024-09-10T11:35:26.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.806+0000 7f58bcff9640 1 -- 172.21.15.99:0/4060232831 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) ==== 169+0+0 (secure 0 0 0) 0x7f58b40584b0 con 0x7f58c015ee00 2024-09-10T11:35:26.808 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 msgr2=0x7f58ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 0x7f58ac06e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f58b0000f60 tx=0x7f58b0009040 comp rx=0 tx=0).stop 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 msgr2=0x7f58c01f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f58b400c480 tx=0x7f58b400c940 comp rx=0 tx=0).stop 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 shutdown_connections 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58ac06be30 0x7f58ac06e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c015f1e0 0x7f58c014c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c015ee00 0x7f58c01f8980 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 --2- 172.21.15.99:0/4060232831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c00b0090 0x7f58c01e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 >> 172.21.15.99:0/4060232831 conn(0x7f58c00730c0 msgr2=0x7f58c00b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 shutdown_connections 2024-09-10T11:35:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.808+0000 7f58c63f6640 1 -- 172.21.15.99:0/4060232831 wait complete. 2024-09-10T11:35:26.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:35:26.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.925+0000 7f966caac640 1 Processor -- start 2024-09-10T11:35:26.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.925+0000 7f966caac640 1 -- start start 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.926+0000 7f966caac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f966815c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.926+0000 7f966caac640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9668152e60 con 0x7f966806ece0 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.926+0000 7f966caac640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96680d15e0 con 0x7f96680d1200 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.926+0000 7f966caac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9668072680 con 0x7f9668073440 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9665d9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f966806ece0 0x7f966815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39840/0 (socket says 172.21.15.99:39840) 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9665d9b640 1 -- 172.21.15.99:0/3424443439 learned_addr learned my addr 172.21.15.99:0/3424443439 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830710074 0 0) 0x7f9668152e60 con 0x7f966806ece0 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f966815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9654003520 con 0x7f966806ece0 2024-09-10T11:35:26.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1257472721 0 0) 0x7f9654003520 con 0x7f966806ece0 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 msgr2=0x7f966815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f966815c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 >> v1:172.21.15.196:6789/0 conn(0x7f96680d1200 legacy=0x7f966806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9668072960 con 0x7f966806ece0 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9650001670 con 0x7f966806ece0 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 >> v1:172.21.15.99:6789/0 conn(0x7f966806ece0 legacy=0x7f966815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96540039e0 0x7f9654003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f9654004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9654004d40 0x7f96540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9668072680 con 0x7f9654004d40 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f965400baf0 con 0x7f96540039e0 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f965400bcf0 con 0x7f9668073440 2024-09-10T11:35:26.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9654004d40 0x7f96540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f966659c640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96540039e0 0x7f9654003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9665d9b640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f9654004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 -- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 msgr2=0x7f9654004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f9654004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 -- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96540039e0 msgr2=0x7f9654003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.927+0000 7f9666d9d640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96540039e0 0x7f9654003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f9666d9d640 1 -- 172.21.15.99:0/3424443439 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9654003520 con 0x7f9654004d40 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f9665d9b640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9668073440 0x7f9654004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f9666d9d640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9654004d40 0x7f96540051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9658009fc0 tx=0x7f9654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a75dba49583eab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658009c10 con 0x7f9654004d40 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9658036070 con 0x7f9654004d40 2024-09-10T11:35:26.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.928+0000 7f966559a640 1 -- 172.21.15.99:0/3424443439 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658031880 con 0x7f9654004d40 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 -- 172.21.15.99:0/3424443439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9654004d40 msgr2=0x7f96540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 --2- 172.21.15.99:0/3424443439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9654004d40 0x7f96540051c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9658009fc0 tx=0x7f9654005830 comp rx=0 tx=0).stop 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f9666d9d640 1 -- 172.21.15.99:0/3424443439 reap_dead start 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 -- 172.21.15.99:0/3424443439 shutdown_connections 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 -- 172.21.15.99:0/3424443439 >> 172.21.15.99:0/3424443439 conn(0x7f966806bc10 msgr2=0x7f966806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 -- 172.21.15.99:0/3424443439 shutdown_connections 2024-09-10T11:35:26.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.929+0000 7f966caac640 1 -- 172.21.15.99:0/3424443439 wait complete. 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.930+0000 7f966caac640 1 Processor -- start 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.930+0000 7f966caac640 1 -- start start 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9654004d40 0x7f96680ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815ac50 0x7f966816b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f966815cd90 con 0x7f966815b0d0 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9668072f20 con 0x7f9654004d40 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966caac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f966815c770 con 0x7f966815ac50 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f9666d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36998/0 (socket says 172.21.15.99:36998) 2024-09-10T11:35:26.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f9666d9d640 1 -- 172.21.15.99:0/2189187859 learned_addr learned my addr 172.21.15.99:0/2189187859 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f9665d9b640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815ac50 0x7f966816b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.931+0000 7f966659c640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9654004d40 0x7f96680ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 -- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815ac50 msgr2=0x7f966816b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815ac50 0x7f966816b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 -- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9654004d40 msgr2=0x7f96680ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9654004d40 0x7f96680ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 -- 172.21.15.99:0/2189187859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96680ac6b0 con 0x7f966815b0d0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f9666d9d640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f965800e230 tx=0x7f9658009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658009c10 con 0x7f966815b0d0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9658031dd0 con 0x7f966815b0d0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658003bd0 con 0x7f966815b0d0 2024-09-10T11:35:26.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96680ab0d0 con 0x7f966815b0d0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.932+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96680ab690 con 0x7f966815b0d0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.933+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966815c060 con 0x7f966815b0d0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.933+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9658005d10 con 0x7f966815b0d0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.934+0000 7f965effd640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.934+0000 7f966659c640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.934+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f96580ac110 con 0x7f966815b0d0 2024-09-10T11:35:26.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.934+0000 7f966659c640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 0x7f964806e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9660001760 tx=0x7f9660009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:26.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:26.938+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f965804a060 con 0x7f966815b0d0 2024-09-10T11:35:27.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.105+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f966806c020 con 0x7f966815b0d0 2024-09-10T11:35:27.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.108+0000 7f965effd640 1 -- 172.21.15.99:0/2189187859 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f965807a770 con 0x7f966815b0d0 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 msgr2=0x7f964806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 0x7f964806e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9660001760 tx=0x7f9660009040 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 msgr2=0x7f966816bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f965800e230 tx=0x7f9658009750 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 shutdown_connections 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f966815b0d0 0x7f966816bdb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f966815ac50 0x7f966816b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.110+0000 7f966caac640 1 --2- 172.21.15.99:0/2189187859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9654004d40 0x7f96680ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.111+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 >> 172.21.15.99:0/2189187859 conn(0x7f966806bc10 msgr2=0x7f96680ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.111+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 shutdown_connections 2024-09-10T11:35:27.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.111+0000 7f966caac640 1 -- 172.21.15.99:0/2189187859 wait complete. 2024-09-10T11:35:27.124 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-09-10T11:35:27.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.220+0000 7f8aea01e640 1 Processor -- start 2024-09-10T11:35:27.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.220+0000 7f8aea01e640 1 -- start start 2024-09-10T11:35:27.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8aea01e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae40d1200 0x7f8ae4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8aea01e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ae406e5d0 0x7f8ae406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8aea01e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae4152e60 con 0x7f8ae415c060 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8aea01e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae406ef80 con 0x7f8ae40d1200 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8aea01e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae40d15e0 con 0x7f8ae406e5d0 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae40d1200 0x7f8ae4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae3fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ae415c060 0x7f8ae415c4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39844/0 (socket says 172.21.15.99:39844) 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8ae40d1200 0x7f8ae4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37012/0 (socket says 172.21.15.99:37012) 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae3fff640 1 -- 172.21.15.99:0/947673843 learned_addr learned my addr 172.21.15.99:0/947673843 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae2ffd640 1 --2- 172.21.15.99:0/947673843 >> v2:172.21.15.196:3300/0 conn(0x7f8ae406e5d0 0x7f8ae406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae37fe640 1 -- 172.21.15.99:0/947673843 >> v2:172.21.15.196:3300/0 conn(0x7f8ae406e5d0 msgr2=0x7f8ae406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.221+0000 7f8ae37fe640 1 --2- 172.21.15.99:0/947673843 >> v2:172.21.15.196:3300/0 conn(0x7f8ae406e5d0 0x7f8ae406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae37fe640 1 -- 172.21.15.99:0/947673843 >> v1:172.21.15.99:6789/0 conn(0x7f8ae415c060 legacy=0x7f8ae415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae27fc640 1 -- 172.21.15.99:0/947673843 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962921239 0 0) 0x7f8ae4152e60 con 0x7f8ae415c060 2024-09-10T11:35:27.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae37fe640 1 -- 172.21.15.99:0/947673843 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4072680 con 0x7f8ae40d1200 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae37fe640 1 --2- 172.21.15.99:0/947673843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae40d1200 0x7f8ae4073440 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4071bb0 tx=0x7f8ad402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5612a7ff6d5dad3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae27fc640 1 -- 172.21.15.99:0/947673843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4037070 con 0x7f8ae40d1200 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae27fc640 1 -- 172.21.15.99:0/947673843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4032440 con 0x7f8ae40d1200 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.222+0000 7f8ae27fc640 1 -- 172.21.15.99:0/947673843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad402ae60 con 0x7f8ae40d1200 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 -- 172.21.15.99:0/947673843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae40d1200 msgr2=0x7f8ae4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 --2- 172.21.15.99:0/947673843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae40d1200 0x7f8ae4073440 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4071bb0 tx=0x7f8ad402a110 comp rx=0 tx=0).stop 2024-09-10T11:35:27.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 -- 172.21.15.99:0/947673843 shutdown_connections 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 --2- 172.21.15.99:0/947673843 >> v2:172.21.15.196:3300/0 conn(0x7f8ae406e5d0 0x7f8ae406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 --2- 172.21.15.99:0/947673843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae40d1200 0x7f8ae4073440 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 -- 172.21.15.99:0/947673843 >> 172.21.15.99:0/947673843 conn(0x7f8ae406bc10 msgr2=0x7f8ae406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 -- 172.21.15.99:0/947673843 shutdown_connections 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.223+0000 7f8aea01e640 1 -- 172.21.15.99:0/947673843 wait complete. 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.224+0000 7f8aea01e640 1 Processor -- start 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.224+0000 7f8aea01e640 1 -- start start 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ae40d1200 0x7f8ae414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ae415c060 0x7f8ae4169250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4072f20 con 0x7f8ae406e5d0 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4072c40 con 0x7f8ae40d1200 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8aea01e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae415cd90 con 0x7f8ae415c060 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37020/0 (socket says 172.21.15.99:37020) 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 -- 172.21.15.99:0/3549685570 learned_addr learned my addr 172.21.15.99:0/3549685570 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 -- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ae415c060 msgr2=0x7f8ae4169250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ae415c060 0x7f8ae4169250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 -- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ae40d1200 msgr2=0x7f8ae414c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ae40d1200 0x7f8ae414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 -- 172.21.15.99:0/3549685570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4169ab0 con 0x7f8ae406e5d0 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.225+0000 7f8ae37fe640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4000c00 tx=0x7f8ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.226+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4037040 con 0x7f8ae406e5d0 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.226+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4042070 con 0x7f8ae406e5d0 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.226+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae4169d90 con 0x7f8ae406e5d0 2024-09-10T11:35:27.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.226+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4030bd0 con 0x7f8ae406e5d0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.226+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae416a350 con 0x7f8ae406e5d0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.227+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae406ea40 con 0x7f8ae406e5d0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.227+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ad4047460 con 0x7f8ae406e5d0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.227+0000 7f8acbfff640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 0x7f8ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.228+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f8ad40ab750 con 0x7f8ae406e5d0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.228+0000 7f8ae2ffd640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 0x7f8ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.228+0000 7f8ae2ffd640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 0x7f8ad006e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.232+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ad4032e20 con 0x7f8ae406e5d0 2024-09-10T11:35:27.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.459+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7f8ae406ece0 con 0x7f8ae406e5d0 2024-09-10T11:35:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.820+0000 7f8acbfff640 1 -- 172.21.15.99:0/3549685570 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v272) ==== 128+0+0 (secure 0 0 0) 0x7f8ad407a400 con 0x7f8ae406e5d0 2024-09-10T11:35:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 msgr2=0x7f8ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 0x7f8ad006e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006fd0 tx=0x7f8ad8008040 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 msgr2=0x7f8ae4157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4000c00 tx=0x7f8ad40046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 shutdown_connections 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8ad006be30 0x7f8ad006e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ae415c060 0x7f8ae4169250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ae40d1200 0x7f8ae414c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 --2- 172.21.15.99:0/3549685570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ae406e5d0 0x7f8ae4157720 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.822+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 >> 172.21.15.99:0/3549685570 conn(0x7f8ae406bc10 msgr2=0x7f8ae415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.823+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 shutdown_connections 2024-09-10T11:35:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.823+0000 7f8aea01e640 1 -- 172.21.15.99:0/3549685570 wait complete. 2024-09-10T11:35:27.836 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-09-10T11:35:27.836 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-09-10T11:35:27.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.936+0000 7fe04e2d0640 1 Processor -- start 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.936+0000 7fe04e2d0640 1 -- start start 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04e2d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe04815ee00 0x7fe0480b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04e2d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe0480b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04e2d0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0480b40e0 con 0x7fe04815f1e0 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04e2d0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe0480b5ab0 con 0x7fe04815ee00 2024-09-10T11:35:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04e2d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe048073c50 con 0x7fe0480b0090 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe04815ee00 0x7fe0480b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe04815ee00 0x7fe0480b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37028/0 (socket says 172.21.15.99:37028) 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 -- 172.21.15.99:0/3209512549 learned_addr learned my addr 172.21.15.99:0/3209512549 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04c86d640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe0480b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 -- 172.21.15.99:0/3209512549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 msgr2=0x7fe0480b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe0480b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 -- 172.21.15.99:0/3209512549 >> v1:172.21.15.196:6789/0 conn(0x7fe04815f1e0 legacy=0x7fe0480b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe047fff640 1 -- 172.21.15.99:0/3209512549 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0480b4380 con 0x7fe04815ee00 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.937+0000 7fe04c86d640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe0480b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.938+0000 7fe047fff640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe0480b5f90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe038009870 tx=0x7fe03802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed3e65b182439f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.938+0000 7fe046ffd640 1 -- 172.21.15.99:0/3209512549 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038037070 con 0x7fe04815ee00 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.938+0000 7fe046ffd640 1 -- 172.21.15.99:0/3209512549 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe038032440 con 0x7fe04815ee00 2024-09-10T11:35:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.938+0000 7fe046ffd640 1 -- 172.21.15.99:0/3209512549 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038032810 con 0x7fe04815ee00 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 -- 172.21.15.99:0/3209512549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 msgr2=0x7fe0480b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe0480b5f90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe038009870 tx=0x7fe03802a270 comp rx=0 tx=0).stop 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 -- 172.21.15.99:0/3209512549 shutdown_connections 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe0480b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/3209512549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe0480b5f90 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 -- 172.21.15.99:0/3209512549 >> 172.21.15.99:0/3209512549 conn(0x7fe0480730c0 msgr2=0x7fe0480734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 -- 172.21.15.99:0/3209512549 shutdown_connections 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.939+0000 7fe04e2d0640 1 -- 172.21.15.99:0/3209512549 wait complete. 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.940+0000 7fe04e2d0640 1 Processor -- start 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.940+0000 7fe04e2d0640 1 -- start start 2024-09-10T11:35:27.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe04814b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe04815f1e0 0x7fe048160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0480b21b0 con 0x7fe04815ee00 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0480afd60 con 0x7fe04815f1e0 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04e2d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe0480b0600 con 0x7fe0480b0090 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37040/0 (socket says 172.21.15.99:37040) 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 -- 172.21.15.99:0/2930217438 learned_addr learned my addr 172.21.15.99:0/2930217438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe047fff640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe04814b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe04c86d640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe04815f1e0 0x7fe048160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 -- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 msgr2=0x7fe04814b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe04814b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 -- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe04815f1e0 msgr2=0x7fe048160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe04815f1e0 0x7fe048160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 -- 172.21.15.99:0/2930217438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04809d360 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe0477fe640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c3c0 tx=0x7fe03c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.941+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0052d0 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.942+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe03c005490 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.942+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe048160b40 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.942+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c007260 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.942+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04809b1a0 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.943+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0480b5f90 con 0x7fe04815ee00 2024-09-10T11:35:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.943+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe03c007400 con 0x7fe04815ee00 2024-09-10T11:35:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.944+0000 7fe044ff9640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.944+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fe03c089f60 con 0x7fe04815ee00 2024-09-10T11:35:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.944+0000 7fe047fff640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.944+0000 7fe047fff640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 0x7fe03406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe038034000 tx=0x7fe03802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:27.948+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe03c0585c0 con 0x7fe04815ee00 2024-09-10T11:35:28.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.113+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0480b4bf0 con 0x7fe04815ee00 2024-09-10T11:35:28.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.113+0000 7fe044ff9640 1 -- 172.21.15.99:0/2930217438 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+13043 (secure 0 0 0) 0x7fe03c05c8c0 con 0x7fe04815ee00 2024-09-10T11:35:28.115 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:28.116 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":130,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:26.806032+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6836/1212129697":"2024-09-11T11:35:23.790494+0000","172.21.15.99:6834/1212129697":"2024-09-11T11:35:23.790494+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 msgr2=0x7fe03406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 0x7fe03406e2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe038034000 tx=0x7fe03802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 msgr2=0x7fe04815ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c3c0 tx=0x7fe03c00c880 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 shutdown_connections 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe03406be30 0x7fe03406e2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe04815f1e0 0x7fe048160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe04815ee00 0x7fe04815ffd0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 --2- 172.21.15.99:0/2930217438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0480b0090 0x7fe04814b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.115+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 >> 172.21.15.99:0/2930217438 conn(0x7fe0480730c0 msgr2=0x7fe0480b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.116+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 shutdown_connections 2024-09-10T11:35:28.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.116+0000 7fe04e2d0640 1 -- 172.21.15.99:0/2930217438 wait complete. 2024-09-10T11:35:28.129 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:35:28.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:35:28.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.229+0000 7f048585c640 1 Processor -- start 2024-09-10T11:35:28.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.229+0000 7f048585c640 1 -- start start 2024-09-10T11:35:28.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f048585c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f04800d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f048585c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f048006f000 0x7f048006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f048585c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04800ceb10 con 0x7f04800d0fb0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f048585c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f04800b3840 con 0x7f048006f000 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f048585c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04800d13f0 con 0x7f048015c7b0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047f7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f048006f000 0x7f048006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f04767fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f04800d0fb0 0x7f048006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39852/0 (socket says 172.21.15.99:39852) 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f04767fc640 1 -- 172.21.15.99:0/563834829 learned_addr learned my addr 172.21.15.99:0/563834829 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047e7fc640 1 -- 172.21.15.99:0/563834829 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2095459006 0 0) 0x7f0468001110 con 0x7f04800d0fb0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047effd640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f04800d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047e7fc640 1 -- 172.21.15.99:0/563834829 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f046c003520 con 0x7f04800d0fb0 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047f7fe640 1 -- 172.21.15.99:0/563834829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 msgr2=0x7f04800d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047f7fe640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f04800d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047f7fe640 1 -- 172.21.15.99:0/563834829 >> v1:172.21.15.99:6789/0 conn(0x7f04800d0fb0 legacy=0x7f048006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.230+0000 7f047f7fe640 1 -- 172.21.15.99:0/563834829 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04800b3a90 con 0x7f048006f000 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.231+0000 7f047f7fe640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006f000 0x7f048006d4c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79871d6a451dc530 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.231+0000 7f047e7fc640 1 -- 172.21.15.99:0/563834829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478007740 con 0x7f048006f000 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.231+0000 7f047e7fc640 1 -- 172.21.15.99:0/563834829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04780078e0 con 0x7f048006f000 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.231+0000 7f047e7fc640 1 -- 172.21.15.99:0/563834829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478007cb0 con 0x7f048006f000 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 -- 172.21.15.99:0/563834829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006f000 msgr2=0x7f048006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006f000 0x7f048006d4c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0478009d90 tx=0x7f047802a660 comp rx=0 tx=0).stop 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 -- 172.21.15.99:0/563834829 shutdown_connections 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f048006f000 0x7f048006d4c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 --2- 172.21.15.99:0/563834829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f04800d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 -- 172.21.15.99:0/563834829 >> 172.21.15.99:0/563834829 conn(0x7f048006bc10 msgr2=0x7f048006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 -- 172.21.15.99:0/563834829 shutdown_connections 2024-09-10T11:35:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.232+0000 7f048585c640 1 -- 172.21.15.99:0/563834829 wait complete. 2024-09-10T11:35:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.233+0000 7f048585c640 1 Processor -- start 2024-09-10T11:35:28.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.233+0000 7f048585c640 1 -- start start 2024-09-10T11:35:28.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f048006f000 0x7f0480152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f0480151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04800b4330 con 0x7f04800d0fb0 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04800b4050 con 0x7f048006f000 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f048585c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04800b4610 con 0x7f048015c7b0 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37072/0 (socket says 172.21.15.99:37072) 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 -- 172.21.15.99:0/3931781787 learned_addr learned my addr 172.21.15.99:0/3931781787 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f047f7fe640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f0480151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f047effd640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f048006f000 0x7f0480152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 -- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 msgr2=0x7f0480151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f0480151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 -- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f048006f000 msgr2=0x7f0480152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f048006f000 0x7f0480152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.234+0000 7f04767fc640 1 -- 172.21.15.99:0/3931781787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480152af0 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f04767fc640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f046800c760 tx=0x7f046800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468017070 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0468005ad0 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0480152dd0 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468007e20 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.235+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0480150020 con 0x7f04800d0fb0 2024-09-10T11:35:28.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.236+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048015ccb0 con 0x7f04800d0fb0 2024-09-10T11:35:28.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.236+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f046801a070 con 0x7f04800d0fb0 2024-09-10T11:35:28.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.237+0000 7f047cff9640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 0x7f046006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.237+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f0468089f20 con 0x7f04800d0fb0 2024-09-10T11:35:28.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.237+0000 7f047effd640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 0x7f046006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.237+0000 7f047effd640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 0x7f046006e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0470006fd0 tx=0x7f0470008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.241+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f04800ceb10 con 0x7f04800d0fb0 2024-09-10T11:35:28.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.406+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0480159640 con 0x7f04800d0fb0 2024-09-10T11:35:28.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.407+0000 7f047cff9640 1 -- 172.21.15.99:0/3931781787 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7f0468058bd0 con 0x7f04800d0fb0 2024-09-10T11:35:28.409 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:35:28.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 msgr2=0x7f046006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 0x7f046006e2f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0470006fd0 tx=0x7f0470008040 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 msgr2=0x7f048014dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f046800c760 tx=0x7f046800cc20 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 shutdown_connections 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f046006be30 0x7f046006e2f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f048015c7b0 0x7f0480151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04800d0fb0 0x7f048014dc10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 --2- 172.21.15.99:0/3931781787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f048006f000 0x7f0480152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 >> 172.21.15.99:0/3931781787 conn(0x7f048006bc10 msgr2=0x7f04800b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 shutdown_connections 2024-09-10T11:35:28.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.409+0000 7f048585c640 1 -- 172.21.15.99:0/3931781787 wait complete. 2024-09-10T11:35:28.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-09-10T11:35:28.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.518+0000 7fef24913640 1 Processor -- start 2024-09-10T11:35:28.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.518+0000 7fef24913640 1 -- start start 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef24913640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef2015ba90 0x7fef200ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef24913640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fef200af3d0 0x7fef200b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef24913640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef20073d70 con 0x7fef200b02a0 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef24913640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fef200b5360 con 0x7fef200af3d0 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef24913640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef200b55f0 con 0x7fef2015ba90 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fef200af3d0 0x7fef200b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fef200af3d0 0x7fef200b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37076/0 (socket says 172.21.15.99:37076) 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2234751125 learned_addr learned my addr 172.21.15.99:0/2234751125 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1e59c640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef2015ba90 0x7fef200ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2234751125 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef2015ba90 msgr2=0x7fef200ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef2015ba90 0x7fef200ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2234751125 >> v1:172.21.15.196:6789/0 conn(0x7fef200b02a0 legacy=0x7fef200aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.519+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2234751125 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef200b0970 con 0x7fef200af3d0 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.520+0000 7fef1ed9d640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef200af3d0 0x7fef200b1450 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0aa887db53a1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.520+0000 7fef1d59a640 1 -- 172.21.15.99:0/2234751125 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18036070 con 0x7fef200af3d0 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.520+0000 7fef1d59a640 1 -- 172.21.15.99:0/2234751125 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18031600 con 0x7fef200af3d0 2024-09-10T11:35:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.520+0000 7fef1d59a640 1 -- 172.21.15.99:0/2234751125 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180319d0 con 0x7fef200af3d0 2024-09-10T11:35:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 -- 172.21.15.99:0/2234751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef200af3d0 msgr2=0x7fef200b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef200af3d0 0x7fef200b1450 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802a6c0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 -- 172.21.15.99:0/2234751125 shutdown_connections 2024-09-10T11:35:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef200af3d0 0x7fef200b1450 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 --2- 172.21.15.99:0/2234751125 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef2015ba90 0x7fef200ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 -- 172.21.15.99:0/2234751125 >> 172.21.15.99:0/2234751125 conn(0x7fef200730c0 msgr2=0x7fef200734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 -- 172.21.15.99:0/2234751125 shutdown_connections 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.521+0000 7fef24913640 1 -- 172.21.15.99:0/2234751125 wait complete. 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.522+0000 7fef24913640 1 Processor -- start 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.522+0000 7fef24913640 1 -- start start 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 0x7fef201f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 0x7fef2014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fef2015be70 con 0x7fef2015ba90 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef200b0f30 con 0x7fef200af3d0 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef24913640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fef2015bff0 con 0x7fef200b02a0 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37082/0 (socket says 172.21.15.99:37082) 2024-09-10T11:35:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2786802509 learned_addr learned my addr 172.21.15.99:0/2786802509 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1e59c640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 0x7fef201f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1dd9b640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 0x7fef2014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 msgr2=0x7fef2014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 0x7fef2014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 msgr2=0x7fef201f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 0x7fef201f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1ed9d640 1 -- 172.21.15.99:0/2786802509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef201f7e50 con 0x7fef2015ba90 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1e59c640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 0x7fef201f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.523+0000 7fef1dd9b640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 0x7fef2014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef1ed9d640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fef18034040 tx=0x7fef18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18036050 con 0x7fef2015ba90 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef18042070 con 0x7fef2015ba90 2024-09-10T11:35:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef201552c0 con 0x7fef2015ba90 2024-09-10T11:35:28.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18009510 con 0x7fef2015ba90 2024-09-10T11:35:28.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.524+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2014dc50 con 0x7fef2015ba90 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.525+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef200ae780 con 0x7fef2015ba90 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.525+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef18007bf0 con 0x7fef2015ba90 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.526+0000 7fef0effd640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 0x7fef0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.526+0000 7fef1e59c640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 0x7fef0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.526+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7fef180aba40 con 0x7fef2015ba90 2024-09-10T11:35:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.526+0000 7fef1e59c640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 0x7fef0006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b820 tx=0x7fef10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.530+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef1807a780 con 0x7fef2015ba90 2024-09-10T11:35:28.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.758+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7fef201555a0 con 0x7fef2015ba90 2024-09-10T11:35:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.821+0000 7fef0effd640 1 -- 172.21.15.99:0/2786802509 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v273) ==== 143+0+0 (secure 0 0 0) 0x7fef1807a080 con 0x7fef2015ba90 2024-09-10T11:35:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 msgr2=0x7fef0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 0x7fef0006e2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef1000b820 tx=0x7fef10008040 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 msgr2=0x7fef200d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fef18034040 tx=0x7fef18004a40 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 shutdown_connections 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fef0006be30 0x7fef0006e2f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fef2015ba90 0x7fef200d1090 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef200b02a0 0x7fef2014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 --2- 172.21.15.99:0/2786802509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef200af3d0 0x7fef201f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.823+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 >> 172.21.15.99:0/2786802509 conn(0x7fef200730c0 msgr2=0x7fef2015a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.824+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 shutdown_connections 2024-09-10T11:35:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.824+0000 7fef24913640 1 -- 172.21.15.99:0/2786802509 wait complete. 2024-09-10T11:35:28.836 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:35:28.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:35:28.882 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:35:28.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.937+0000 7f4b26aaf640 1 Processor -- start 2024-09-10T11:35:28.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.937+0000 7f4b26aaf640 1 -- start start 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b26aaf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b2015c7b0 0x7f4b200d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b26aaf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b2006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b26aaf640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b200ceb10 con 0x7f4b200d0fb0 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b26aaf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b200d1420 con 0x7f4b2015c7b0 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b26aaf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200b3840 con 0x7f4b2006f000 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b1ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4b200d0fb0 0x7f4b2006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39866/0 (socket says 172.21.15.99:39866) 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.938+0000 7f4b1ffff640 1 -- 172.21.15.99:0/1257219685 learned_addr learned my addr 172.21.15.99:0/1257219685 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b2504c640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b2006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b2484b640 1 --2- 172.21.15.99:0/1257219685 >> v2:172.21.15.196:3300/0 conn(0x7f4b2015c7b0 0x7f4b200d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 952594166 0 0) 0x7f4b14001110 con 0x7f4b200d0fb0 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b08003520 con 0x7f4b200d0fb0 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 291000840 0 0) 0x7f4b08003520 con 0x7f4b200d0fb0 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 msgr2=0x7f4b2006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b2006d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 >> v2:172.21.15.196:3300/0 conn(0x7f4b2015c7b0 msgr2=0x7f4b200d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 --2- 172.21.15.99:0/1257219685 >> v2:172.21.15.196:3300/0 conn(0x7f4b2015c7b0 0x7f4b200d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b200b3a90 con 0x7f4b200d0fb0 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4b14001840 con 0x7f4b200d0fb0 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 >> v1:172.21.15.99:6789/0 conn(0x7f4b200d0fb0 legacy=0x7f4b2006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b08003f30 0x7f4b08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08004d40 0x7f4b080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200d1420 con 0x7f4b08003f30 2024-09-10T11:35:28.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b3840 con 0x7f4b08004d40 2024-09-10T11:35:28.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0800b9c0 con 0x7f4b2006f000 2024-09-10T11:35:28.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b08003f30 0x7f4b08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b2504c640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08004d40 0x7f4b080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b2484b640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 -- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 msgr2=0x7f4b080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 -- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08004d40 msgr2=0x7f4b080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08004d40 0x7f4b080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b1ffff640 1 -- 172.21.15.99:0/1257219685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b08003520 con 0x7f4b08003f30 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.939+0000 7f4b2484b640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b2006f000 0x7f4b080039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.940+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b08003f30 0x7f4b08004800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c3b0 tx=0x7f4b08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e35267c8e537e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.940+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400c0e0 con 0x7f4b08003f30 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.940+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14034070 con 0x7f4b08003f30 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.940+0000 7f4b1f7fe640 1 -- 172.21.15.99:0/1257219685 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14005d10 con 0x7f4b08003f30 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 -- 172.21.15.99:0/1257219685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b08003f30 msgr2=0x7f4b08004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/1257219685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b08003f30 0x7f4b08004800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c3b0 tx=0x7f4b08005830 comp rx=0 tx=0).stop 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b2504c640 1 -- 172.21.15.99:0/1257219685 reap_dead start 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 -- 172.21.15.99:0/1257219685 shutdown_connections 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 -- 172.21.15.99:0/1257219685 >> 172.21.15.99:0/1257219685 conn(0x7f4b2006bc10 msgr2=0x7f4b2006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 -- 172.21.15.99:0/1257219685 shutdown_connections 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.941+0000 7f4b26aaf640 1 -- 172.21.15.99:0/1257219685 wait complete. 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.942+0000 7f4b26aaf640 1 Processor -- start 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.942+0000 7f4b26aaf640 1 -- start start 2024-09-10T11:35:28.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 0x7f4b20154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 0x7f4b2015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b4330 con 0x7f4b2014ded0 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b200b4050 con 0x7f4b201512b0 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b26aaf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b200b4610 con 0x7f4b08003f30 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37100/0 (socket says 172.21.15.99:37100) 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 -- 172.21.15.99:0/2883853783 learned_addr learned my addr 172.21.15.99:0/2883853783 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b2504c640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 0x7f4b2015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b2484b640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 0x7f4b20154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 -- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 msgr2=0x7f4b20154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 0x7f4b20154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 -- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 msgr2=0x7f4b2015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 0x7f4b2015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 -- 172.21.15.99:0/2883853783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b20150210 con 0x7f4b2014ded0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b2484b640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 0x7f4b20154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b2504c640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 0x7f4b2015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1ffff640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c4a0 tx=0x7f4b200b3c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.943+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14005e60 con 0x7f4b2014ded0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.944+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b14007270 con 0x7f4b2014ded0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.944+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14007640 con 0x7f4b2014ded0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.944+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b201504f0 con 0x7f4b2014ded0 2024-09-10T11:35:28.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.944+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b20150ab0 con 0x7f4b2014ded0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.945+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2015cb90 con 0x7f4b2014ded0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.945+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b1402f460 con 0x7f4b2014ded0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.945+0000 7f4b1d7fa640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 0x7f4b0c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.946+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f4b140ab160 con 0x7f4b2014ded0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.946+0000 7f4b2484b640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 0x7f4b0c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.946+0000 7f4b2484b640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 0x7f4b0c06e320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009a00 tx=0x7f4b10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:28.950+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b14040070 con 0x7f4b2014ded0 2024-09-10T11:35:29.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.118+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4b20157cc0 con 0x7f4b2014ded0 2024-09-10T11:35:29.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.121+0000 7f4b1d7fa640 1 -- 172.21.15.99:0/2883853783 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f4b200ceb10 con 0x7f4b2014ded0 2024-09-10T11:35:29.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 msgr2=0x7f4b0c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 0x7f4b0c06e320 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009a00 tx=0x7f4b10008040 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 msgr2=0x7f4b2014d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400c4a0 tx=0x7f4b200b3c60 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 shutdown_connections 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b0c06be60 0x7f4b0c06e320 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b201512b0 0x7f4b2015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b2014ded0 0x7f4b2014d6c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 --2- 172.21.15.99:0/2883853783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b08003f30 0x7f4b20154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 >> 172.21.15.99:0/2883853783 conn(0x7f4b2006bc10 msgr2=0x7f4b200b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 shutdown_connections 2024-09-10T11:35:29.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.123+0000 7f4b26aaf640 1 -- 172.21.15.99:0/2883853783 wait complete. 2024-09-10T11:35:29.136 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:35:29.136 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:35:29.210 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:35:29.275 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:35:29.276 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:29.276 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:29.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:35:29.293 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:35:29.293 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:29.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:35:29.312 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:29.312 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:29.347 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:35:29.347 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:35:29.347 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:29.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:35:29.370 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:35:29.370 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:29.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.483+0000 7f6321168640 1 Processor -- start 2024-09-10T11:35:29.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.483+0000 7f6321168640 1 -- start start 2024-09-10T11:35:29.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6321168640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f631c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6321168640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0b40e0 con 0x7f631c15ee00 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6321168640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0b5ab0 con 0x7f631c0b0090 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6321168640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f631c073c50 con 0x7f631c15f1e0 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f631ad9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f631c15ee00 0x7f631c0b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39880/0 (socket says 172.21.15.99:39880) 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f631ad9d640 1 -- 172.21.15.99:0/1478341337 learned_addr learned my addr 172.21.15.99:0/1478341337 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806977999 0 0) 0x7f631c0b40e0 con 0x7f631c15ee00 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6304003520 con 0x7f631c15ee00 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.484+0000 7f631a59c640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f631c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3683423902 0 0) 0x7f6304003520 con 0x7f631c15ee00 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 msgr2=0x7f631c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f631c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 >> v1:172.21.15.196:6789/0 conn(0x7f631c0b0090 legacy=0x7f631c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c0b4380 con 0x7f631c15ee00 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f630c001670 con 0x7f631c15ee00 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 >> v1:172.21.15.99:6789/0 conn(0x7f631c15ee00 legacy=0x7f631c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63040039e0 0x7f6304003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f6304004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6304004d40 0x7f63040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f631c073c50 con 0x7f63040039e0 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f630400baa0 con 0x7f6304004d40 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f630400bca0 con 0x7f631c15f1e0 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63040039e0 0x7f6304003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631a59c640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f6304004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631b59e640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6304004d40 0x7f63040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 -- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 msgr2=0x7f6304004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631c15f1e0 0x7f6304004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 -- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6304004d40 msgr2=0x7f63040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6304004d40 0x7f63040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.485+0000 7f631ad9d640 1 -- 172.21.15.99:0/1478341337 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6304003520 con 0x7f63040039e0 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f631ad9d640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63040039e0 0x7f6304003e00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f630c001eb0 tx=0x7f6304005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6afd67a29b3de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c007d30 con 0x7f63040039e0 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0058d0 con 0x7f63040039e0 2024-09-10T11:35:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6319d9b640 1 -- 172.21.15.99:0/1478341337 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c005ce0 con 0x7f63040039e0 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6321168640 1 -- 172.21.15.99:0/1478341337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63040039e0 msgr2=0x7f6304003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6321168640 1 --2- 172.21.15.99:0/1478341337 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63040039e0 0x7f6304003e00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f630c001eb0 tx=0x7f6304005830 comp rx=0 tx=0).stop 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f631b59e640 1 -- 172.21.15.99:0/1478341337 reap_dead start 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6321168640 1 -- 172.21.15.99:0/1478341337 shutdown_connections 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.486+0000 7f6321168640 1 -- 172.21.15.99:0/1478341337 >> 172.21.15.99:0/1478341337 conn(0x7f631c0730c0 msgr2=0x7f631c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.487+0000 7f6321168640 1 -- 172.21.15.99:0/1478341337 shutdown_connections 2024-09-10T11:35:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.487+0000 7f6321168640 1 -- 172.21.15.99:0/1478341337 wait complete. 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.488+0000 7f6321168640 1 Processor -- start 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.488+0000 7f6321168640 1 -- start start 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63040039e0 0x7f631c1ea140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 0x7f631c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0b0600 con 0x7f631c0ca3c0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0afd60 con 0x7f631c15b2e0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0b0780 con 0x7f63040039e0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37108/0 (socket says 172.21.15.99:37108) 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 -- 172.21.15.99:0/2032513082 learned_addr learned my addr 172.21.15.99:0/2032513082 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631a59c640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 0x7f631c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631ad9d640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63040039e0 0x7f631c1ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 -- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63040039e0 msgr2=0x7f631c1ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63040039e0 0x7f631c1ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 -- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 msgr2=0x7f631c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 0x7f631c161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 -- 172.21.15.99:0/2032513082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c1ea800 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631a59c640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 0x7f631c161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f631b59e640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f631c0b3e00 tx=0x7f631400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314003970 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6314018070 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c1eaae0 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6314003d70 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.489+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c1eb0a0 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.491+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c0b5f90 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.491+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63140050d0 con 0x7f631c0ca3c0 2024-09-10T11:35:29.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.491+0000 7f63037fe640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 0x7f630806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.491+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f631408a9a0 con 0x7f631c0ca3c0 2024-09-10T11:35:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.491+0000 7f631ad9d640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 0x7f630806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.492+0000 7f631ad9d640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 0x7f630806e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f630c033000 tx=0x7f630c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.496+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6314059650 con 0x7f631c0ca3c0 2024-09-10T11:35:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.695+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f631c0b3840 con 0x7f631c0ca3c0 2024-09-10T11:35:29.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.695+0000 7f63037fe640 1 -- 172.21.15.99:0/2032513082 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+3911 (secure 0 0 0) 0x7f6314058f50 con 0x7f631c0ca3c0 2024-09-10T11:35:29.698 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:29.698 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":50,"btime":"2024-09-10T11:35:24:793977+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/504861533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":504861533},{"type":"v1","addr":"172.21.15.196:6834","nonce":504861533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/515677503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":515677503},{"type":"v1","addr":"172.21.15.196:6835","nonce":515677503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1787413609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1787413609},{"type":"v1","addr":"172.21.15.99:6837","nonce":1787413609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":6900,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3211176391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3211176391},{"type":"v1","addr":"172.21.15.99:6836","nonce":3211176391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[]} 2024-09-10T11:35:29.698 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 50 2024-09-10T11:35:29.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 msgr2=0x7f630806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 0x7f630806e2f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f630c033000 tx=0x7f630c02e000 comp rx=0 tx=0).stop 2024-09-10T11:35:29.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 msgr2=0x7f631c0ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f631c0b3e00 tx=0x7f631400ec10 comp rx=0 tx=0).stop 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 shutdown_connections 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f630806be30 0x7f630806e2f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f631c0ca3c0 0x7f631c0ca830 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f631c15b2e0 0x7f631c161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 --2- 172.21.15.99:0/2032513082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63040039e0 0x7f631c1ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 >> 172.21.15.99:0/2032513082 conn(0x7f631c0730c0 msgr2=0x7f631c15c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 shutdown_connections 2024-09-10T11:35:29.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.697+0000 7f6321168640 1 -- 172.21.15.99:0/2032513082 wait complete. 2024-09-10T11:35:29.710 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:35:29.711 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:35:29.711 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:35:29.711 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:35:29.711 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:35:29.711 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:35:29.711 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:35:29.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:35:29.712 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:35:29.712 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:35:29.712 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:35:29.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:35:29.712 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:35:29.712 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:35:29.713 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:35:29.713 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:35:29.723 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:35:29.723 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:35:29.724 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:35:29.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:35:29.727 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:35:29.727 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:35:29.728 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:35:29.728 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:35:29.765 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:35:29.765 DEBUG:teuthology.parallel:result is None 2024-09-10T11:35:29.766 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:35:29.766 DEBUG:teuthology.parallel:result is None 2024-09-10T11:35:29.770 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:35:29.771 DEBUG:teuthology.parallel:result is None 2024-09-10T11:35:29.772 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:35:29.772 DEBUG:teuthology.parallel:result is None 2024-09-10T11:35:29.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:35:29.840 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:35:29.841 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:35:29.855 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:35:29.855 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:35:29.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.875+0000 7f716fb6d640 1 Processor -- start 2024-09-10T11:35:29.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.875+0000 7f716fb6d640 1 -- start start 2024-09-10T11:35:29.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716fb6d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71681626c0 0x7f71680d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716fb6d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f716815ff70 con 0x7f716815eb80 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716fb6d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71680d1620 con 0x7f71680d4e40 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716fb6d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f71680d30c0 con 0x7f71681626c0 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716d909640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f716815eb80 0x7f71680d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39892/0 (socket says 172.21.15.99:39892) 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716d909640 1 -- 172.21.15.99:0/3467907202 learned_addr learned my addr 172.21.15.99:0/3467907202 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2868430234 0 0) 0x7f716815ff70 con 0x7f716815eb80 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7154003520 con 0x7f716815eb80 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716d108640 1 --2- 172.21.15.99:0/3467907202 >> v2:172.21.15.196:3300/0 conn(0x7f71681626c0 0x7f71680d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3637978585 0 0) 0x7f7154003520 con 0x7f716815eb80 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 >> v2:172.21.15.196:3300/0 conn(0x7f71681626c0 msgr2=0x7f71680d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 --2- 172.21.15.99:0/3467907202 >> v2:172.21.15.196:3300/0 conn(0x7f71681626c0 0x7f71680d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.876+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 >> v1:172.21.15.196:6789/0 conn(0x7f71680d4e40 legacy=0x7f71680d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7168162bc0 con 0x7f716815eb80 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7158001670 con 0x7f716815eb80 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 >> v1:172.21.15.99:6789/0 conn(0x7f716815eb80 legacy=0x7f71680d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7154003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7154005210 0x7f715400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71680d30c0 con 0x7f71540039e0 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f715400c140 con 0x7f7154005210 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f715400c340 con 0x7f7154004800 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7154003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716e10a640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7154005210 0x7f715400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d108640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 -- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154004800 msgr2=0x7f7154004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7154004800 0x7f7154004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 -- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7154005210 msgr2=0x7f715400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7154005210 0x7f715400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716d909640 1 -- 172.21.15.99:0/3467907202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7154003520 con 0x7f71540039e0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716e10a640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7154005210 0x7f715400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.877+0000 7f716e10a640 1 -- 172.21.15.99:0/3467907202 reap_dead start 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716d909640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7154003e00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f715800c210 tx=0x7f7154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30107f2e55da854 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158007d30 con 0x7f71540039e0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158005ad0 con 0x7f71540039e0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716c907640 1 -- 172.21.15.99:0/3467907202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158007070 con 0x7f71540039e0 2024-09-10T11:35:29.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 -- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 msgr2=0x7f7154003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7154003e00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f715800c210 tx=0x7f7154005830 comp rx=0 tx=0).stop 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 -- 172.21.15.99:0/3467907202 shutdown_connections 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 --2- 172.21.15.99:0/3467907202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7154003e00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 -- 172.21.15.99:0/3467907202 >> 172.21.15.99:0/3467907202 conn(0x7f716808e680 msgr2=0x7f716809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.878+0000 7f716fb6d640 1 -- 172.21.15.99:0/3467907202 shutdown_connections 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.879+0000 7f716fb6d640 1 -- 172.21.15.99:0/3467907202 wait complete. 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 Processor -- start 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 -- start start 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 0x7f71681f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 0x7f71680cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7168163460 con 0x7f71540039e0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7168163180 con 0x7f71680ce5e0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716fb6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71680d5290 con 0x7f716815c9e0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716d909640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716d909640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37148/0 (socket says 172.21.15.99:37148) 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.880+0000 7f716d909640 1 -- 172.21.15.99:0/1379591417 learned_addr learned my addr 172.21.15.99:0/1379591417 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716e10a640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 0x7f71680cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d108640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 0x7f71681f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 -- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 msgr2=0x7f71681f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 0x7f71681f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 -- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 msgr2=0x7f71680cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:29.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 0x7f71680cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 -- 172.21.15.99:0/1379591417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71681f8540 con 0x7f71540039e0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716e10a640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 0x7f71680cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d108640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 0x7f71681f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716d909640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f715800c330 tx=0x7f7158001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158041070 con 0x7f71540039e0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7158000ed0 con 0x7f71540039e0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71681f8820 con 0x7f71540039e0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158007290 con 0x7f71540039e0 2024-09-10T11:35:29.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.881+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71681f8230 con 0x7f71540039e0 2024-09-10T11:35:29.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.882+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71680d4900 con 0x7f71540039e0 2024-09-10T11:35:29.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.883+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7158044070 con 0x7f71540039e0 2024-09-10T11:35:29.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.883+0000 7f715e7fc640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 0x7f714c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:29.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.884+0000 7f716d108640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 0x7f714c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:29.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.884+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 4943+0+0 (secure 0 0 0) 0x7f71580aaee0 con 0x7f71540039e0 2024-09-10T11:35:29.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.884+0000 7f716d108640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 0x7f714c06e320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71681f7c50 tx=0x7f7164008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:29.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:29.888+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f715804b070 con 0x7f71540039e0 2024-09-10T11:35:30.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.064+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f71680b5260 con 0x7f71540039e0 2024-09-10T11:35:30.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.839+0000 7f715e7fc640 1 -- 172.21.15.99:0/1379591417 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) ==== 97+0+0 (secure 0 0 0) 0x7f7158079b90 con 0x7f71540039e0 2024-09-10T11:35:30.841 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:35:30.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 msgr2=0x7f714c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 0x7f714c06e320 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f71681f7c50 tx=0x7f7164008040 comp rx=0 tx=0).stop 2024-09-10T11:35:30.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 msgr2=0x7f7168153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f715800c330 tx=0x7f7158001a40 comp rx=0 tx=0).stop 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 shutdown_connections 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f714c06be60 0x7f714c06e320 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71680ce5e0 0x7f71680cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f716815c9e0 0x7f71681f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 --2- 172.21.15.99:0/1379591417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71540039e0 0x7f7168153360 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 >> 172.21.15.99:0/1379591417 conn(0x7f716808e680 msgr2=0x7f71680b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 shutdown_connections 2024-09-10T11:35:30.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.843+0000 7f716fb6d640 1 -- 172.21.15.99:0/1379591417 wait complete. 2024-09-10T11:35:30.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:35:30.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.961+0000 7f53dd79f640 1 Processor -- start 2024-09-10T11:35:30.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.961+0000 7f53dd79f640 1 -- start start 2024-09-10T11:35:30.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53dd79f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53dd79f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53dd79f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d8152e60 con 0x7f53d815c060 2024-09-10T11:35:30.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53dd79f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53d806ef50 con 0x7f53d806e5d0 2024-09-10T11:35:30.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53dd79f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53d80d15e0 con 0x7f53d80d1200 2024-09-10T11:35:30.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d67fc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d6ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d6ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:43476/0 (socket says 172.21.15.99:43476) 2024-09-10T11:35:30.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d67fc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:37748/0 (socket says 172.21.15.99:37748) 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d6ffd640 1 -- 172.21.15.99:0/2344638116 learned_addr learned my addr 172.21.15.99:0/2344638116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.962+0000 7f53d77fe640 1 --1- 172.21.15.99:0/2344638116 >> v1:172.21.15.196:6789/0 conn(0x7f53d815c060 0x7f53d815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:46478/0 (socket says 172.21.15.99:46478) 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.963+0000 7f53d6ffd640 1 -- 172.21.15.99:0/2344638116 >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 msgr2=0x7f53d806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.963+0000 7f53d6ffd640 1 --2- 172.21.15.99:0/2344638116 >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.963+0000 7f53d6ffd640 1 -- 172.21.15.99:0/2344638116 >> v1:172.21.15.196:6789/0 conn(0x7f53d815c060 legacy=0x7f53d815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.963+0000 7f53d6ffd640 1 -- 172.21.15.99:0/2344638116 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d8072680 con 0x7f53d80d1200 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.963+0000 7f53d67fc640 1 --2- 172.21.15.99:0/2344638116 >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53d6ffd640 1 --2- 172.21.15.99:0/2344638116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009870 tx=0x7f53c802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f27b4bfb2fa9c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53d5ffb640 1 -- 172.21.15.99:0/2344638116 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037070 con 0x7f53d80d1200 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53d5ffb640 1 -- 172.21.15.99:0/2344638116 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c8032440 con 0x7f53d80d1200 2024-09-10T11:35:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53d5ffb640 1 -- 172.21.15.99:0/2344638116 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8032810 con 0x7f53d80d1200 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 -- 172.21.15.99:0/2344638116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 msgr2=0x7f53d8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 --2- 172.21.15.99:0/2344638116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009870 tx=0x7f53c802a1c0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 -- 172.21.15.99:0/2344638116 shutdown_connections 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 --2- 172.21.15.99:0/2344638116 >> v2:172.21.15.196:3300/0 conn(0x7f53d806e5d0 0x7f53d806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 --2- 172.21.15.99:0/2344638116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d80d1200 0x7f53d8073440 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 -- 172.21.15.99:0/2344638116 >> 172.21.15.99:0/2344638116 conn(0x7f53d806bc10 msgr2=0x7f53d806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 -- 172.21.15.99:0/2344638116 shutdown_connections 2024-09-10T11:35:30.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.964+0000 7f53dd79f640 1 -- 172.21.15.99:0/2344638116 wait complete. 2024-09-10T11:35:30.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 Processor -- start 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 -- start start 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d806e5d0 0x7f53d813a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d80d1200 0x7f53d813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8072f20 con 0x7f53d815c060 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53d8072c40 con 0x7f53d80d1200 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.966+0000 7f53dd79f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53d815cd90 con 0x7f53d806e5d0 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37162/0 (socket says 172.21.15.99:37162) 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 -- 172.21.15.99:0/1561346635 learned_addr learned my addr 172.21.15.99:0/1561346635 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d67fc640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d80d1200 0x7f53d813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d6ffd640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d806e5d0 0x7f53d813a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 -- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d806e5d0 msgr2=0x7f53d813a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d806e5d0 0x7f53d813a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 -- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d80d1200 msgr2=0x7f53d813ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d80d1200 0x7f53d813ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 -- 172.21.15.99:0/1561346635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d813b8f0 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53d77fe640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c780 tx=0x7f53d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d0017070 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d000a640 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d813bbd0 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d000a9f0 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.967+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d813c190 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.968+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d806ea10 con 0x7f53d815c060 2024-09-10T11:35:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.969+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53d001a070 con 0x7f53d815c060 2024-09-10T11:35:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.969+0000 7f53bf7fe640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 0x7f53c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.969+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7f53d008a0b0 con 0x7f53d815c060 2024-09-10T11:35:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.969+0000 7f53d6ffd640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 0x7f53c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.970+0000 7f53d6ffd640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 0x7f53c406e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f53c8034000 tx=0x7f53c802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:30.973+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53d00586e0 con 0x7f53d815c060 2024-09-10T11:35:31.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.200+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f53d80b4ce0 con 0x7f53d815c060 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.200+0000 7f53bf7fe640 1 -- 172.21.15.99:0/1561346635 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f53d005c9e0 con 0x7f53d815c060 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:35:31.202 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:35:31.203 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:35:31.205 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:35:31.208 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.209 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.210 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.211 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:35:31.213 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:35:31.214 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:35:31.215 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:31.216 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 msgr2=0x7f53c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 0x7f53c406e2f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f53c8034000 tx=0x7f53c802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 msgr2=0x7f53d8155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f53d000c780 tx=0x7f53d000cc40 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 shutdown_connections 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c406be30 0x7f53c406e2f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53d815c060 0x7f53d8155200 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d80d1200 0x7f53d813ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 --2- 172.21.15.99:0/1561346635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53d806e5d0 0x7f53d813a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 >> 172.21.15.99:0/1561346635 conn(0x7f53d806bc10 msgr2=0x7f53d80b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 shutdown_connections 2024-09-10T11:35:31.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.202+0000 7f53dd79f640 1 -- 172.21.15.99:0/1561346635 wait complete. 2024-09-10T11:35:31.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:35:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.315+0000 7fe52fd31640 1 Processor -- start 2024-09-10T11:35:31.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.315+0000 7fe52fd31640 1 -- start start 2024-09-10T11:35:31.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52fd31640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5280714a0 0x7fe528163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52fd31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5280d0740 0x7fe5280d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52fd31640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5280b5d30 con 0x7fe528163140 2024-09-10T11:35:31.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52fd31640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe5280d10d0 con 0x7fe5280714a0 2024-09-10T11:35:31.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52fd31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe528071980 con 0x7fe5280d0740 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5280714a0 0x7fe528163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe5280714a0 0x7fe528163520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37178/0 (socket says 172.21.15.99:37178) 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 -- 172.21.15.99:0/4099906914 learned_addr learned my addr 172.21.15.99:0/4099906914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52dacd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe528163140 0x7fe5280b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39900/0 (socket says 172.21.15.99:39900) 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 -- 172.21.15.99:0/4099906914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5280d0740 msgr2=0x7fe5280d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 --2- 172.21.15.99:0/4099906914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5280d0740 0x7fe5280d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 -- 172.21.15.99:0/4099906914 >> v1:172.21.15.99:6789/0 conn(0x7fe528163140 legacy=0x7fe5280b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52cacb640 1 -- 172.21.15.99:0/4099906914 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 941113669 0 0) 0x7fe5280b5d30 con 0x7fe528163140 2024-09-10T11:35:31.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.316+0000 7fe52d2cc640 1 -- 172.21.15.99:0/4099906914 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe528071c60 con 0x7fe5280714a0 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.317+0000 7fe52d2cc640 1 --2- 172.21.15.99:0/4099906914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280714a0 0x7fe528163520 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe518009870 tx=0x7fe51802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1ce2e31c7701c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.317+0000 7fe52cacb640 1 -- 172.21.15.99:0/4099906914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518037070 con 0x7fe5280714a0 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.317+0000 7fe52cacb640 1 -- 172.21.15.99:0/4099906914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe518032440 con 0x7fe5280714a0 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.317+0000 7fe52cacb640 1 -- 172.21.15.99:0/4099906914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518032810 con 0x7fe5280714a0 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 -- 172.21.15.99:0/4099906914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280714a0 msgr2=0x7fe528163520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 --2- 172.21.15.99:0/4099906914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280714a0 0x7fe528163520 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe518009870 tx=0x7fe51802a270 comp rx=0 tx=0).stop 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 -- 172.21.15.99:0/4099906914 shutdown_connections 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 --2- 172.21.15.99:0/4099906914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5280d0740 0x7fe5280d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 --2- 172.21.15.99:0/4099906914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280714a0 0x7fe528163520 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 -- 172.21.15.99:0/4099906914 >> 172.21.15.99:0/4099906914 conn(0x7fe52806bc10 msgr2=0x7fe52806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 -- 172.21.15.99:0/4099906914 shutdown_connections 2024-09-10T11:35:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.318+0000 7fe52fd31640 1 -- 172.21.15.99:0/4099906914 wait complete. 2024-09-10T11:35:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.319+0000 7fe52fd31640 1 Processor -- start 2024-09-10T11:35:31.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.319+0000 7fe52fd31640 1 -- start start 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 0x7fe528151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 0x7fe5280b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe528072500 con 0x7fe5280d0740 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe528072220 con 0x7fe5280714a0 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe528072a20 con 0x7fe528163140 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37186/0 (socket says 172.21.15.99:37186) 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 -- 172.21.15.99:0/930670309 learned_addr learned my addr 172.21.15.99:0/930670309 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52e2ce640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 0x7fe5280b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52dacd640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 0x7fe528151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 -- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 msgr2=0x7fe5280b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 0x7fe5280b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 -- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 msgr2=0x7fe528151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 0x7fe528151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 -- 172.21.15.99:0/930670309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52815c400 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52e2ce640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 0x7fe5280b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52dacd640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 0x7fe528151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52d2cc640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe5180090b0 tx=0x7fe5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518037040 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe518004030 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280aa800 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518004440 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.320+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5280aadc0 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.321+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280d0b90 con 0x7fe5280d0740 2024-09-10T11:35:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.322+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe51803a040 con 0x7fe5280d0740 2024-09-10T11:35:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.322+0000 7fe51e7fc640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 0x7fe50c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.323+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7fe5180ab1c0 con 0x7fe5280d0740 2024-09-10T11:35:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.323+0000 7fe52dacd640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 0x7fe50c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.323+0000 7fe52dacd640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 0x7fe50c06e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe524007fd0 tx=0x7fe524009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.326+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe518042020 con 0x7fe5280d0740 2024-09-10T11:35:31.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.496+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe52806bff0 con 0x7fe5280d0740 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.497+0000 7fe51e7fc640 1 -- 172.21.15.99:0/930670309 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) ==== 84+0+65 (secure 0 0 0) 0x7fe51807a070 con 0x7fe5280d0740 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:31.499 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:35:31.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 msgr2=0x7fe50c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 0x7fe50c06e320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe524007fd0 tx=0x7fe524009040 comp rx=0 tx=0).stop 2024-09-10T11:35:31.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 msgr2=0x7fe52815bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe5180090b0 tx=0x7fe5180046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 shutdown_connections 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe50c06be60 0x7fe50c06e320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe528163140 0x7fe5280b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5280d0740 0x7fe52815bec0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 --2- 172.21.15.99:0/930670309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5280714a0 0x7fe528151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 >> 172.21.15.99:0/930670309 conn(0x7fe52806bc10 msgr2=0x7fe5281600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 shutdown_connections 2024-09-10T11:35:31.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.499+0000 7fe52fd31640 1 -- 172.21.15.99:0/930670309 wait complete. 2024-09-10T11:35:31.512 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:35:31.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:35:31.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.611+0000 7fb8a482c640 1 Processor -- start 2024-09-10T11:35:31.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.611+0000 7fb8a482c640 1 -- start start 2024-09-10T11:35:31.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.612+0000 7fb8a482c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.612+0000 7fb8a482c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb8a00b59b0 0x7fb8a00d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.612+0000 7fb8a482c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a00d2150 con 0x7fb8a0162d10 2024-09-10T11:35:31.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.612+0000 7fb8a482c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a00d0d30 con 0x7fb8a00d16f0 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.612+0000 7fb8a482c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a00b4d30 con 0x7fb8a00b59b0 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37200/0 (socket says 172.21.15.99:37200) 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 -- 172.21.15.99:0/2701314720 learned_addr learned my addr 172.21.15.99:0/2701314720 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89ed9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb8a0162d10 0x7fb8a0163140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39914/0 (socket says 172.21.15.99:39914) 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89dd9b640 1 --2- 172.21.15.99:0/2701314720 >> v2:172.21.15.196:3300/0 conn(0x7fb8a00b59b0 0x7fb8a00d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 -- 172.21.15.99:0/2701314720 >> v2:172.21.15.196:3300/0 conn(0x7fb8a00b59b0 msgr2=0x7fb8a00d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 --2- 172.21.15.99:0/2701314720 >> v2:172.21.15.196:3300/0 conn(0x7fb8a00b59b0 0x7fb8a00d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 -- 172.21.15.99:0/2701314720 >> v1:172.21.15.99:6789/0 conn(0x7fb8a0162d10 legacy=0x7fb8a0163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89d59a640 1 -- 172.21.15.99:0/2701314720 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 472747012 0 0) 0x7fb8a00d2150 con 0x7fb8a0162d10 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.613+0000 7fb89e59c640 1 -- 172.21.15.99:0/2701314720 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00b5780 con 0x7fb8a00d16f0 2024-09-10T11:35:31.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb89e59c640 1 --2- 172.21.15.99:0/2701314720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb8a008d7f0 tx=0x7fb89002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79baa7d2b3d7411a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb89d59a640 1 -- 172.21.15.99:0/2701314720 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890037070 con 0x7fb8a00d16f0 2024-09-10T11:35:31.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb89d59a640 1 -- 172.21.15.99:0/2701314720 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb890032440 con 0x7fb8a00d16f0 2024-09-10T11:35:31.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb89d59a640 1 -- 172.21.15.99:0/2701314720 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890032810 con 0x7fb8a00d16f0 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 -- 172.21.15.99:0/2701314720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a00d16f0 msgr2=0x7fb8a00d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 --2- 172.21.15.99:0/2701314720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb8a008d7f0 tx=0x7fb89002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 -- 172.21.15.99:0/2701314720 shutdown_connections 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 --2- 172.21.15.99:0/2701314720 >> v2:172.21.15.196:3300/0 conn(0x7fb8a00b59b0 0x7fb8a00d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 --2- 172.21.15.99:0/2701314720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00d1ad0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 -- 172.21.15.99:0/2701314720 >> 172.21.15.99:0/2701314720 conn(0x7fb8a00d4c20 msgr2=0x7fb8a00d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 -- 172.21.15.99:0/2701314720 shutdown_connections 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.614+0000 7fb8a482c640 1 -- 172.21.15.99:0/2701314720 wait complete. 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.616+0000 7fb8a482c640 1 Processor -- start 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.616+0000 7fb8a482c640 1 -- start start 2024-09-10T11:35:31.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.616+0000 7fb8a482c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8a00b59b0 0x7fb8a0157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.616+0000 7fb8a482c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.616+0000 7fb8a482c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb8a482c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00b4f80 con 0x7fb8a0162d10 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb8a482c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a00b52f0 con 0x7fb8a00d16f0 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb8a482c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a0163850 con 0x7fb8a00b59b0 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37210/0 (socket says 172.21.15.99:37210) 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 -- 172.21.15.99:0/3913045995 learned_addr learned my addr 172.21.15.99:0/3913045995 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89dd9b640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89e59c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8a00b59b0 0x7fb8a0157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 -- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8a00b59b0 msgr2=0x7fb8a0157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8a00b59b0 0x7fb8a0157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 -- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00d16f0 msgr2=0x7fb8a00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 -- 172.21.15.99:0/3913045995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a00c87f0 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb89ed9d640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb89800c700 tx=0x7fb89800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898018070 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89800ce10 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89800a860 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a00c8a60 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.617+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a00c9020 con 0x7fb8a0162d10 2024-09-10T11:35:31.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.618+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a00d1ad0 con 0x7fb8a0162d10 2024-09-10T11:35:31.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.619+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb89801b070 con 0x7fb8a0162d10 2024-09-10T11:35:31.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.619+0000 7fb88effd640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 0x7fb88006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.619+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 4857+0+0 (secure 0 0 0) 0x7fb898089f90 con 0x7fb8a0162d10 2024-09-10T11:35:31.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.619+0000 7fb89e59c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 0x7fb88006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.620+0000 7fb89e59c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 0x7fb88006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb890034000 tx=0x7fb89002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.623+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb898058640 con 0x7fb8a0162d10 2024-09-10T11:35:31.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.790+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb8a00b4a50 con 0x7fb8a0162d10 2024-09-10T11:35:31.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.860+0000 7fb88effd640 1 -- 172.21.15.99:0/3913045995 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) ==== 138+0+0 (secure 0 0 0) 0x7fb89805c940 con 0x7fb8a0162d10 2024-09-10T11:35:31.862 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:35:31.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 msgr2=0x7fb88006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 0x7fb88006e320 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb890034000 tx=0x7fb89002e000 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 msgr2=0x7fb8a00b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb89800c700 tx=0x7fb89800cbc0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 shutdown_connections 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.862+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb88006be60 0x7fb88006e320 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8a0162d10 0x7fb8a00b35d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8a00d16f0 0x7fb8a00b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 --2- 172.21.15.99:0/3913045995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8a00b59b0 0x7fb8a0157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 >> 172.21.15.99:0/3913045995 conn(0x7fb8a00d4c20 msgr2=0x7fb8a01613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 shutdown_connections 2024-09-10T11:35:31.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.863+0000 7fb8a482c640 1 -- 172.21.15.99:0/3913045995 wait complete. 2024-09-10T11:35:31.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:35:31.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 Processor -- start 2024-09-10T11:35:31.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 -- start start 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f065c1626c0 0x7f065c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065c15ff70 con 0x7f065c0d4e40 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f065c0d1620 con 0x7f065c15eb80 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.981+0000 7f066300a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f065c0d30c0 con 0x7f065c1626c0 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f065c1626c0 0x7f065c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f065c1626c0 0x7f065c0d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37212/0 (socket says 172.21.15.99:37212) 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f06615a7640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f065c0d4e40 0x7f065c0d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39916/0 (socket says 172.21.15.99:39916) 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 -- 172.21.15.99:0/2134730632 learned_addr learned my addr 172.21.15.99:0/2134730632 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 -- 172.21.15.99:0/2134730632 >> v1:172.21.15.196:6789/0 conn(0x7f065c15eb80 legacy=0x7f065c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:31.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065b7fe640 1 -- 172.21.15.99:0/2134730632 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1708630220 0 0) 0x7f065c15ff70 con 0x7f065c0d4e40 2024-09-10T11:35:31.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 -- 172.21.15.99:0/2134730632 >> v1:172.21.15.99:6789/0 conn(0x7f065c0d4e40 legacy=0x7f065c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.982+0000 7f065bfff640 1 -- 172.21.15.99:0/2134730632 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c162bc0 con 0x7f065c1626c0 2024-09-10T11:35:31.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f065bfff640 1 --2- 172.21.15.99:0/2134730632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c1626c0 0x7f065c0d4900 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f064802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce09fecd2c49f217 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f065b7fe640 1 -- 172.21.15.99:0/2134730632 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648037070 con 0x7f065c1626c0 2024-09-10T11:35:31.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f065b7fe640 1 -- 172.21.15.99:0/2134730632 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0648032440 con 0x7f065c1626c0 2024-09-10T11:35:31.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f065b7fe640 1 -- 172.21.15.99:0/2134730632 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648032810 con 0x7f065c1626c0 2024-09-10T11:35:31.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 -- 172.21.15.99:0/2134730632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c1626c0 msgr2=0x7f065c0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 --2- 172.21.15.99:0/2134730632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c1626c0 0x7f065c0d4900 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f064802a290 comp rx=0 tx=0).stop 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 -- 172.21.15.99:0/2134730632 shutdown_connections 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 --2- 172.21.15.99:0/2134730632 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c1626c0 0x7f065c0d4900 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 -- 172.21.15.99:0/2134730632 >> 172.21.15.99:0/2134730632 conn(0x7f065c08e680 msgr2=0x7f065c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 -- 172.21.15.99:0/2134730632 shutdown_connections 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.983+0000 7f066300a640 1 -- 172.21.15.99:0/2134730632 wait complete. 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.984+0000 7f066300a640 1 Processor -- start 2024-09-10T11:35:31.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 -- start start 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 0x7f065c161d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 0x7f065c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f065c163460 con 0x7f065c0d4e40 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f065c163180 con 0x7f065c1626c0 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.985+0000 7f066300a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0d5290 con 0x7f065c15eb80 2024-09-10T11:35:31.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37226/0 (socket says 172.21.15.99:37226) 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 -- 172.21.15.99:0/1509100444 learned_addr learned my addr 172.21.15.99:0/1509100444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f06615a7640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 0x7f065c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f065bfff640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 0x7f065c161d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 -- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 msgr2=0x7f065c161d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 0x7f065c161d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 -- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 msgr2=0x7f065c159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 0x7f065c159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 -- 172.21.15.99:0/1509100444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c160cd0 con 0x7f065c0d4e40 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f065bfff640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 0x7f065c161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f06615a7640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 0x7f065c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f0660da6640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f065000c370 tx=0x7f065000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650017070 con 0x7f065c0d4e40 2024-09-10T11:35:31.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06500052d0 con 0x7f065c0d4e40 2024-09-10T11:35:31.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c160fb0 con 0x7f065c0d4e40 2024-09-10T11:35:31.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650003850 con 0x7f065c0d4e40 2024-09-10T11:35:31.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.986+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c161570 con 0x7f065c0d4e40 2024-09-10T11:35:31.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.987+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c0d4900 con 0x7f065c0d4e40 2024-09-10T11:35:31.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.988+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f065001a070 con 0x7f065c0d4e40 2024-09-10T11:35:31.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.988+0000 7f06597fa640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 0x7f064006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:31.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.988+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5227+0+0 (secure 0 0 0) 0x7f0650089f60 con 0x7f065c0d4e40 2024-09-10T11:35:31.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.988+0000 7f065bfff640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 0x7f064006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:31.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.989+0000 7f065bfff640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 0x7f064006e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0648034000 tx=0x7f064802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:31.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:31.993+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0650058b80 con 0x7f065c0d4e40 2024-09-10T11:35:32.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.161+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f065c09e280 con 0x7f065c0d4e40 2024-09-10T11:35:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.862+0000 7f06597fa640 1 -- 172.21.15.99:0/1509100444 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) ==== 170+0+0 (secure 0 0 0) 0x7f0650058480 con 0x7f065c0d4e40 2024-09-10T11:35:32.864 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:35:32.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.866+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 msgr2=0x7f064006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:32.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.866+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 0x7f064006e2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0648034000 tx=0x7f064802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:32.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.866+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 msgr2=0x7f065c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:32.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.866+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f065000c370 tx=0x7f065000c830 comp rx=0 tx=0).stop 2024-09-10T11:35:32.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 shutdown_connections 2024-09-10T11:35:32.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f064006be30 0x7f064006e2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f065c1626c0 0x7f065c159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f065c15eb80 0x7f065c161d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 --2- 172.21.15.99:0/1509100444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f065c0d4e40 0x7f065c1596f0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.867+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 >> 172.21.15.99:0/1509100444 conn(0x7f065c08e680 msgr2=0x7f065c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:32.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.871+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 shutdown_connections 2024-09-10T11:35:32.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.872+0000 7f066300a640 1 -- 172.21.15.99:0/1509100444 wait complete. 2024-09-10T11:35:32.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:35:32.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.989+0000 7f7586d23640 1 Processor -- start 2024-09-10T11:35:32.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 -- start start 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75801626c0 0x7f75800d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f75800d4e40 0x7f75800d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758015ff70 con 0x7f758015eb80 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f75800d30c0 con 0x7f75801626c0 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f7586d23640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f75800d1620 con 0x7f75800d4e40 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f757ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75801626c0 0x7f75800d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f757ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f75801626c0 0x7f75800d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37448/0 (socket says 172.21.15.99:37448) 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.990+0000 7f757ffff640 1 -- 172.21.15.99:0/3009639601 learned_addr learned my addr 172.21.15.99:0/3009639601 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:32.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f75852c0640 1 --2- 172.21.15.99:0/3009639601 >> v2:172.21.15.196:3300/0 conn(0x7f75800d4e40 0x7f75800d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f757ffff640 1 -- 172.21.15.99:0/3009639601 >> v2:172.21.15.196:3300/0 conn(0x7f75800d4e40 msgr2=0x7f75800d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f757ffff640 1 --2- 172.21.15.99:0/3009639601 >> v2:172.21.15.196:3300/0 conn(0x7f75800d4e40 0x7f75800d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f757ffff640 1 -- 172.21.15.99:0/3009639601 >> v1:172.21.15.196:6789/0 conn(0x7f758015eb80 legacy=0x7f75800d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f757ffff640 1 -- 172.21.15.99:0/3009639601 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7580162bc0 con 0x7f75801626c0 2024-09-10T11:35:32.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.991+0000 7f757ffff640 1 --2- 172.21.15.99:0/3009639601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75801626c0 0x7f75800d4900 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7568009870 tx=0x7f756802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208e99e661f9d32d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f757f7fe640 1 -- 172.21.15.99:0/3009639601 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568037070 con 0x7f75801626c0 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f757f7fe640 1 -- 172.21.15.99:0/3009639601 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7568032440 con 0x7f75801626c0 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f757f7fe640 1 -- 172.21.15.99:0/3009639601 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568032810 con 0x7f75801626c0 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 -- 172.21.15.99:0/3009639601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75801626c0 msgr2=0x7f75800d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 --2- 172.21.15.99:0/3009639601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75801626c0 0x7f75800d4900 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7568009870 tx=0x7f756802a210 comp rx=0 tx=0).stop 2024-09-10T11:35:32.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 -- 172.21.15.99:0/3009639601 shutdown_connections 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 --2- 172.21.15.99:0/3009639601 >> v2:172.21.15.196:3300/0 conn(0x7f75800d4e40 0x7f75800d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 --2- 172.21.15.99:0/3009639601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75801626c0 0x7f75800d4900 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 -- 172.21.15.99:0/3009639601 >> 172.21.15.99:0/3009639601 conn(0x7f758008e680 msgr2=0x7f758009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 -- 172.21.15.99:0/3009639601 shutdown_connections 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.992+0000 7f7586d23640 1 -- 172.21.15.99:0/3009639601 wait complete. 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.993+0000 7f7586d23640 1 Processor -- start 2024-09-10T11:35:32.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.993+0000 7f7586d23640 1 -- start start 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 0x7f758006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75801626c0 0x7f7580156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7580163460 con 0x7f758015eb80 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7580163180 con 0x7f75801626c0 2024-09-10T11:35:32.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7586d23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f75800d5290 con 0x7f75800d4e40 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f757ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f757ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37462/0 (socket says 172.21.15.99:37462) 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f757ffff640 1 -- 172.21.15.99:0/2871770583 learned_addr learned my addr 172.21.15.99:0/2871770583 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f75852c0640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75801626c0 0x7f7580156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.994+0000 7f7584abf640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 0x7f758006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 -- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 msgr2=0x7f758006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 0x7f758006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 -- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75801626c0 msgr2=0x7f7580156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75801626c0 0x7f7580156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 -- 172.21.15.99:0/2871770583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758015c3e0 con 0x7f758015eb80 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f7584abf640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 0x7f758006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757ffff640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f756802a250 tx=0x7f75680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:32.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568041020 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75680306e0 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568037040 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758015c6c0 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.995+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7580157500 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.996+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75800d4900 con 0x7f758015eb80 2024-09-10T11:35:33.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.997+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7568044020 con 0x7f758015eb80 2024-09-10T11:35:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.997+0000 7f757d7fa640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.997+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5609+0+0 (secure 0 0 0) 0x7f75680ab5a0 con 0x7f758015eb80 2024-09-10T11:35:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.997+0000 7f7584abf640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:33.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:32.997+0000 7f7584abf640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 0x7f756006e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7570001d60 tx=0x7f7570009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:33.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.001+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f756804b030 con 0x7f758015eb80 2024-09-10T11:35:33.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.200+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f758009e120 con 0x7f758015eb80 2024-09-10T11:35:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.874+0000 7f757d7fa640 1 -- 172.21.15.99:0/2871770583 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v53) ==== 174+0+0 (secure 0 0 0) 0x7f7568079fb0 con 0x7f758015eb80 2024-09-10T11:35:33.877 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T11:35:33.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 msgr2=0x7f756006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:33.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 0x7f756006e2f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7570001d60 tx=0x7f7570009040 comp rx=0 tx=0).stop 2024-09-10T11:35:33.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 msgr2=0x7f75801566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:33.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f756802a250 tx=0x7f75680046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:33.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 shutdown_connections 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f756006be30 0x7f756006e2f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75801626c0 0x7f7580156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f758015eb80 0x7f75801566e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 --2- 172.21.15.99:0/2871770583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75800d4e40 0x7f758006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 >> 172.21.15.99:0/2871770583 conn(0x7f758008e680 msgr2=0x7f75800cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 shutdown_connections 2024-09-10T11:35:33.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.881+0000 7f7586d23640 1 -- 172.21.15.99:0/2871770583 wait complete. 2024-09-10T11:35:33.899 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:33.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.997+0000 7fd90fd6d640 1 Processor -- start 2024-09-10T11:35:34.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.998+0000 7fd90fd6d640 1 -- start start 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.998+0000 7fd90fd6d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9080d5010 0x7fd9080d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.998+0000 7fd90fd6d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90815ff70 con 0x7fd9081626c0 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.998+0000 7fd90fd6d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9080d1620 con 0x7fd90815eb80 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.998+0000 7fd90fd6d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd9080d30c0 con 0x7fd9080d5010 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90d308640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd9081626c0 0x7fd9080d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46706/0 (socket says 172.21.15.99:46706) 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90d308640 1 -- 172.21.15.99:0/2695542846 learned_addr learned my addr 172.21.15.99:0/2695542846 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90e30a640 1 --2- 172.21.15.99:0/2695542846 >> v2:172.21.15.99:3300/0 conn(0x7fd9080d5010 0x7fd9080d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90e30a640 1 -- 172.21.15.99:0/2695542846 >> v1:172.21.15.196:6789/0 conn(0x7fd90815eb80 legacy=0x7fd9080d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90e30a640 1 -- 172.21.15.99:0/2695542846 >> v1:172.21.15.99:6789/0 conn(0x7fd9081626c0 legacy=0x7fd9080d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90e30a640 1 -- 172.21.15.99:0/2695542846 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908162bc0 con 0x7fd9080d5010 2024-09-10T11:35:34.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:33.999+0000 7fd90cb07640 1 -- 172.21.15.99:0/2695542846 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3733478748 0 0) 0x7fd90815ff70 con 0x7fd9081626c0 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90e30a640 1 --2- 172.21.15.99:0/2695542846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9080d5010 0x7fd9080d0770 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd9000097d0 tx=0x7fd90002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ada164d3cac0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90cb07640 1 -- 172.21.15.99:0/2695542846 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037070 con 0x7fd9080d5010 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90cb07640 1 -- 172.21.15.99:0/2695542846 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900032440 con 0x7fd9080d5010 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90cb07640 1 -- 172.21.15.99:0/2695542846 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900032810 con 0x7fd9080d5010 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 -- 172.21.15.99:0/2695542846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9080d5010 msgr2=0x7fd9080d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/2695542846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9080d5010 0x7fd9080d0770 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd9000097d0 tx=0x7fd90002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 -- 172.21.15.99:0/2695542846 shutdown_connections 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/2695542846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9080d5010 0x7fd9080d0770 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 -- 172.21.15.99:0/2695542846 >> 172.21.15.99:0/2695542846 conn(0x7fd90808e680 msgr2=0x7fd90809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 -- 172.21.15.99:0/2695542846 shutdown_connections 2024-09-10T11:35:34.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.000+0000 7fd90fd6d640 1 -- 172.21.15.99:0/2695542846 wait complete. 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.001+0000 7fd90fd6d640 1 Processor -- start 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 -- start start 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9080d5010 0x7fd9080b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd90815eb80 0x7fd9080ba540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd908163460 con 0x7fd9081626c0 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd908163180 con 0x7fd9080d5010 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90fd6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9080d0cb0 con 0x7fd90815eb80 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90e30a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90e30a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37482/0 (socket says 172.21.15.99:37482) 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90e30a640 1 -- 172.21.15.99:0/1998909937 learned_addr learned my addr 172.21.15.99:0/1998909937 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90db09640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9080d5010 0x7fd9080b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.002+0000 7fd90d308640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd90815eb80 0x7fd9080ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 -- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd90815eb80 msgr2=0x7fd9080ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd90815eb80 0x7fd9080ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 -- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9080d5010 msgr2=0x7fd9080b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9080d5010 0x7fd9080b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 -- 172.21.15.99:0/1998909937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd908152a20 con 0x7fd9081626c0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90e30a640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd900035040 tx=0x7fd9000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900037050 con 0x7fd9081626c0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900038040 con 0x7fd9081626c0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9080b3660 con 0x7fd9081626c0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900004490 con 0x7fd9081626c0 2024-09-10T11:35:34.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.003+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9080b2360 con 0x7fd9081626c0 2024-09-10T11:35:34.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.004+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9080d5bd0 con 0x7fd9081626c0 2024-09-10T11:35:34.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.004+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd900030980 con 0x7fd9081626c0 2024-09-10T11:35:34.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.005+0000 7fd9067fc640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 0x7fd8f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.005+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7fd9000abb70 con 0x7fd9081626c0 2024-09-10T11:35:34.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.005+0000 7fd90db09640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 0x7fd8f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.005+0000 7fd90db09640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 0x7fd8f406e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8007fd0 tx=0x7fd8f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.009+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9080d5bd0 con 0x7fd9081626c0 2024-09-10T11:35:34.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.175+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9080d29b0 con 0x7fd9081626c0 2024-09-10T11:35:34.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.176+0000 7fd9067fc640 1 -- 172.21.15.99:0/1998909937 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16757 (secure 0 0 0) 0x7fd90007a5b0 con 0x7fd9081626c0 2024-09-10T11:35:34.178 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:34.178 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":137,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:33.202044+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:31.809882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:32.181001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 msgr2=0x7fd8f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 0x7fd8f406e2f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8007fd0 tx=0x7fd8f8009040 comp rx=0 tx=0).stop 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 msgr2=0x7fd9080b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd900035040 tx=0x7fd9000046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 shutdown_connections 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8f406be30 0x7fd8f406e2f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9081626c0 0x7fd9080b7d60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd90815eb80 0x7fd9080ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 --2- 172.21.15.99:0/1998909937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9080d5010 0x7fd9080b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 >> 172.21.15.99:0/1998909937 conn(0x7fd90808e680 msgr2=0x7fd9080cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 shutdown_connections 2024-09-10T11:35:34.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.178+0000 7fd90fd6d640 1 -- 172.21.15.99:0/1998909937 wait complete. 2024-09-10T11:35:34.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:34.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.288+0000 7ff1d126e640 1 Processor -- start 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.288+0000 7ff1d126e640 1 -- start start 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.288+0000 7ff1d126e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1d126e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 0x7ff1cc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1d126e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc0d52f0 con 0x7ff1cc0d4e40 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1d126e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc15ff70 con 0x7ff1cc15eb80 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1d126e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0d1620 con 0x7ff1cc1626c0 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37494/0 (socket says 172.21.15.99:37494) 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 -- 172.21.15.99:0/1537707774 learned_addr learned my addr 172.21.15.99:0/1537707774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:34.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1ca7fc640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 0x7ff1cc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 -- 172.21.15.99:0/1537707774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 msgr2=0x7ff1cc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 0x7ff1cc0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 -- 172.21.15.99:0/1537707774 >> v1:172.21.15.196:6789/0 conn(0x7ff1cc0d4e40 legacy=0x7ff1cc0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1caffd640 1 -- 172.21.15.99:0/1537707774 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc162bc0 con 0x7ff1cc15eb80 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.289+0000 7ff1ca7fc640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 0x7ff1cc0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1caffd640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7691e29ced6060f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1c9ffb640 1 -- 172.21.15.99:0/1537707774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc037070 con 0x7ff1cc15eb80 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1c9ffb640 1 -- 172.21.15.99:0/1537707774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc032440 con 0x7ff1cc15eb80 2024-09-10T11:35:34.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1c9ffb640 1 -- 172.21.15.99:0/1537707774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc032810 con 0x7ff1cc15eb80 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 -- 172.21.15.99:0/1537707774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc15eb80 msgr2=0x7ff1cc0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009870 tx=0x7ff1bc02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 -- 172.21.15.99:0/1537707774 shutdown_connections 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc1626c0 0x7ff1cc0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 --2- 172.21.15.99:0/1537707774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc15eb80 0x7ff1cc0d29b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.290+0000 7ff1d126e640 1 -- 172.21.15.99:0/1537707774 >> 172.21.15.99:0/1537707774 conn(0x7ff1cc08e680 msgr2=0x7ff1cc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.291+0000 7ff1d126e640 1 -- 172.21.15.99:0/1537707774 shutdown_connections 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.291+0000 7ff1d126e640 1 -- 172.21.15.99:0/1537707774 wait complete. 2024-09-10T11:35:34.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.292+0000 7ff1d126e640 1 Processor -- start 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.292+0000 7ff1d126e640 1 -- start start 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.292+0000 7ff1d126e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1cc0d4e40 0x7ff1cc0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc15eb80 0x7ff1cc0ba220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc163460 con 0x7ff1cc1626c0 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc163180 con 0x7ff1cc0d4e40 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0d2ef0 con 0x7ff1cc15eb80 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37502/0 (socket says 172.21.15.99:37502) 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1caffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1cc0d4e40 0x7ff1cc0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 -- 172.21.15.99:0/3706213357 learned_addr learned my addr 172.21.15.99:0/3706213357 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:34.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1ca7fc640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc15eb80 0x7ff1cc0ba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 -- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc15eb80 msgr2=0x7ff1cc0ba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc15eb80 0x7ff1cc0ba220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 -- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1cc0d4e40 msgr2=0x7ff1cc0b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1cc0d4e40 0x7ff1cc0b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 -- 172.21.15.99:0/3706213357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc0b8030 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1cb7fe640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c6e0 tx=0x7ff1c400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4003b70 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4003d30 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc15b540 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4005310 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.293+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc15bb00 con 0x7ff1cc1626c0 2024-09-10T11:35:34.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.294+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc0d29b0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.295+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1c40054b0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.295+0000 7ff1b37fe640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 0x7ff1b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.295+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7ff1c408aad0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.295+0000 7ff1caffd640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 0x7ff1b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.296+0000 7ff1caffd640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 0x7ff1b806e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc034000 tx=0x7ff1bc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.300+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1cc0d52f0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.465+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1cc09e3b0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.466+0000 7ff1b37fe640 1 -- 172.21.15.99:0/3706213357 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+16757 (secure 0 0 0) 0x7ff1c4058ec0 con 0x7ff1cc1626c0 2024-09-10T11:35:34.468 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:34.468 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":137,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:33.202044+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:31.809882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:32.181001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 msgr2=0x7ff1b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 0x7ff1b806e2f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc034000 tx=0x7ff1bc02e000 comp rx=0 tx=0).stop 2024-09-10T11:35:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 msgr2=0x7ff1cc159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400c6e0 tx=0x7ff1c400cba0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 shutdown_connections 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff1b806be30 0x7ff1b806e2f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1cc1626c0 0x7ff1cc159d10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff1cc15eb80 0x7ff1cc0ba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 --2- 172.21.15.99:0/3706213357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1cc0d4e40 0x7ff1cc0b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 >> 172.21.15.99:0/3706213357 conn(0x7ff1cc08e680 msgr2=0x7ff1cc06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 shutdown_connections 2024-09-10T11:35:34.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.468+0000 7ff1d126e640 1 -- 172.21.15.99:0/3706213357 wait complete. 2024-09-10T11:35:34.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:35:34.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.580+0000 7faf70303640 1 Processor -- start 2024-09-10T11:35:34.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.580+0000 7faf70303640 1 -- start start 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf68145360 0x7faf680d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680ce270 0x7faf680b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf680b60d0 0x7faf680cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7faf680b6540 con 0x7faf68145360 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7faf68145740 con 0x7faf680b60d0 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf70303640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faf6806e5a0 con 0x7faf680ce270 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf6e09f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf68145360 0x7faf680d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf6e09f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf68145360 0x7faf680d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37516/0 (socket says 172.21.15.99:37516) 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf6e09f640 1 -- 172.21.15.99:0/1567724649 learned_addr learned my addr 172.21.15.99:0/1567724649 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:34.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf6d89e640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680ce270 0x7faf680b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.581+0000 7faf6e8a0640 1 --2- 172.21.15.99:0/1567724649 >> v2:172.21.15.196:3300/0 conn(0x7faf680b60d0 0x7faf680cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 -- 172.21.15.99:0/1567724649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680ce270 msgr2=0x7faf680b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680ce270 0x7faf680b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 -- 172.21.15.99:0/1567724649 >> v2:172.21.15.196:3300/0 conn(0x7faf680b60d0 msgr2=0x7faf680cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 --2- 172.21.15.99:0/1567724649 >> v2:172.21.15.196:3300/0 conn(0x7faf680b60d0 0x7faf680cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 -- 172.21.15.99:0/1567724649 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6806e880 con 0x7faf68145360 2024-09-10T11:35:34.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6e09f640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf68145360 0x7faf680d10f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faf680b4b30 tx=0x7faf6402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e75f4b7aa555346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6d09d640 1 -- 172.21.15.99:0/1567724649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64036020 con 0x7faf68145360 2024-09-10T11:35:34.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6d09d640 1 -- 172.21.15.99:0/1567724649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf64032820 con 0x7faf68145360 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.582+0000 7faf6d09d640 1 -- 172.21.15.99:0/1567724649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64032bf0 con 0x7faf68145360 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 -- 172.21.15.99:0/1567724649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf68145360 msgr2=0x7faf680d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf68145360 0x7faf680d10f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7faf680b4b30 tx=0x7faf6402c920 comp rx=0 tx=0).stop 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 -- 172.21.15.99:0/1567724649 shutdown_connections 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 --2- 172.21.15.99:0/1567724649 >> v2:172.21.15.196:3300/0 conn(0x7faf680b60d0 0x7faf680cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680ce270 0x7faf680b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 --2- 172.21.15.99:0/1567724649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf68145360 0x7faf680d10f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 -- 172.21.15.99:0/1567724649 >> 172.21.15.99:0/1567724649 conn(0x7faf68072680 msgr2=0x7faf68072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 -- 172.21.15.99:0/1567724649 shutdown_connections 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.583+0000 7faf70303640 1 -- 172.21.15.99:0/1567724649 wait complete. 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.584+0000 7faf70303640 1 Processor -- start 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.584+0000 7faf70303640 1 -- start start 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 0x7faf68156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 0x7faf68158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf680ce6e0 con 0x7faf680ce270 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf6806ee40 con 0x7faf68145360 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf70303640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faf680ce860 con 0x7faf680b60d0 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37528/0 (socket says 172.21.15.99:37528) 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 -- 172.21.15.99:0/3028495371 learned_addr learned my addr 172.21.15.99:0/3028495371 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:34.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6e09f640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 0x7faf68156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6e8a0640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 0x7faf68158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 -- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 msgr2=0x7faf68156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 0x7faf68156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 -- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 msgr2=0x7faf68158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 0x7faf68158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6d89e640 1 -- 172.21.15.99:0/3028495371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf681589e0 con 0x7faf680ce270 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6e09f640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 0x7faf68156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.585+0000 7faf6e8a0640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 0x7faf68158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf6d89e640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9b0 tx=0x7faf5800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58017070 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf5800c9a0 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf68155d60 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5800cd90 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.586+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf68156320 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.587+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf680b22d0 con 0x7faf680ce270 2024-09-10T11:35:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.587+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf5801a070 con 0x7faf680ce270 2024-09-10T11:35:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.588+0000 7faf5effd640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 0x7faf4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.588+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5709+0+0 (secure 0 0 0) 0x7faf5808ae00 con 0x7faf680ce270 2024-09-10T11:35:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.588+0000 7faf6e09f640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 0x7faf4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.588+0000 7faf6e09f640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 0x7faf4c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faf64037000 tx=0x7faf6403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.592+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf58059840 con 0x7faf680ce270 2024-09-10T11:35:34.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.791+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf680b3410 con 0x7faf680ce270 2024-09-10T11:35:34.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.877+0000 7faf5effd640 1 -- 172.21.15.99:0/3028495371 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7faf58059140 con 0x7faf680ce270 2024-09-10T11:35:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 msgr2=0x7faf4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 0x7faf4c06e2f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7faf64037000 tx=0x7faf6403d040 comp rx=0 tx=0).stop 2024-09-10T11:35:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 msgr2=0x7faf6815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e9b0 tx=0x7faf5800ee70 comp rx=0 tx=0).stop 2024-09-10T11:35:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 shutdown_connections 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf4c06be30 0x7faf4c06e2f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf68145360 0x7faf68158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf680ce270 0x7faf6815ed60 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 --2- 172.21.15.99:0/3028495371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf680b60d0 0x7faf68156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 >> 172.21.15.99:0/3028495371 conn(0x7faf68072680 msgr2=0x7faf680b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 shutdown_connections 2024-09-10T11:35:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.880+0000 7faf70303640 1 -- 172.21.15.99:0/3028495371 wait complete. 2024-09-10T11:35:34.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:35:34.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:35:34.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 Processor -- start 2024-09-10T11:35:34.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 -- start start 2024-09-10T11:35:34.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b681626c0 0x7f5b680d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:34.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b6815ff70 con 0x7f5b680d4e40 2024-09-10T11:35:34.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b680d1620 con 0x7f5b6815eb80 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.994+0000 7f5b6e32c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5b680d30c0 con 0x7f5b681626c0 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b6c8c9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5b680d4e40 0x7f5b680d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46708/0 (socket says 172.21.15.99:46708) 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/763000623 learned_addr learned my addr 172.21.15.99:0/763000623 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b677fe640 1 --2- 172.21.15.99:0/763000623 >> v2:172.21.15.196:3300/0 conn(0x7f5b681626c0 0x7f5b680d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892116876 0 0) 0x7f5b6815ff70 con 0x7f5b680d4e40 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b50003520 con 0x7f5b680d4e40 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143698416 0 0) 0x7f5b50003520 con 0x7f5b680d4e40 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 >> v2:172.21.15.196:3300/0 conn(0x7f5b681626c0 msgr2=0x7f5b680d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 --2- 172.21.15.99:0/763000623 >> v2:172.21.15.196:3300/0 conn(0x7f5b681626c0 0x7f5b680d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 >> v1:172.21.15.196:6789/0 conn(0x7f5b6815eb80 legacy=0x7f5b680d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b68162bc0 con 0x7f5b680d4e40 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5b600019f0 con 0x7f5b680d4e40 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 >> v1:172.21.15.99:6789/0 conn(0x7f5b680d4e40 legacy=0x7f5b680d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b500039e0 0x7f5b50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 0x7f5b50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50005210 0x7f5b5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5b680d30c0 con 0x7f5b50004800 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5000c140 con 0x7f5b500039e0 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5000c340 con 0x7f5b50005210 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b677fe640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 0x7f5b50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b67fff640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b500039e0 0x7f5b50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.995+0000 7f5b6c8c9640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50005210 0x7f5b5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 -- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50005210 msgr2=0x7f5b5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50005210 0x7f5b5000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 -- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b500039e0 msgr2=0x7f5b50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b500039e0 0x7f5b50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 -- 172.21.15.99:0/763000623 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b50003520 con 0x7f5b50004800 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/763000623 reap_dead start 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b677fe640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 0x7f5b50004c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009a90 tx=0x7f5b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f20ca1046e13bda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c036070 con 0x7f5b50004800 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5c031400 con 0x7f5b50004800 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.996+0000 7f5b66ffd640 1 -- 172.21.15.99:0/763000623 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c009690 con 0x7f5b50004800 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 -- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 msgr2=0x7f5b50004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 0x7f5b50004c80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009a90 tx=0x7f5b50005830 comp rx=0 tx=0).stop 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 -- 172.21.15.99:0/763000623 shutdown_connections 2024-09-10T11:35:35.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/763000623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b50004800 0x7f5b50004c80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 -- 172.21.15.99:0/763000623 >> 172.21.15.99:0/763000623 conn(0x7f5b6808e680 msgr2=0x7f5b6809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:35.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 -- 172.21.15.99:0/763000623 shutdown_connections 2024-09-10T11:35:35.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.997+0000 7f5b6e32c640 1 -- 172.21.15.99:0/763000623 wait complete. 2024-09-10T11:35:35.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.999+0000 7f5b6e32c640 1 Processor -- start 2024-09-10T11:35:35.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:34.999+0000 7f5b6e32c640 1 -- start start 2024-09-10T11:35:35.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50004800 0x7f5b68161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b680ce170 0x7f5b680ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68163460 con 0x7f5b680cc1f0 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68163180 con 0x7f5b680ce170 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6e32c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5b680d5290 con 0x7f5b50004800 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37554/0 (socket says 172.21.15.99:37554) 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/3766808497 learned_addr learned my addr 172.21.15.99:0/3766808497 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b67fff640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50004800 0x7f5b68161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b677fe640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b680ce170 0x7f5b680ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50004800 msgr2=0x7f5b68161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50004800 0x7f5b68161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b680ce170 msgr2=0x7f5b680ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b680ce170 0x7f5b680ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 -- 172.21.15.99:0/3766808497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6815b7d0 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b6c8c9640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5b680b44a0 tx=0x7f5b6000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.000+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000eb30 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.001+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b60018070 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.001+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6815bab0 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.001+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60013420 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.001+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6815c070 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.001+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b680d4900 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.002+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b600135c0 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.002+0000 7f5b64ff9640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 0x7f5b5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.003+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f5b6008af20 con 0x7f5b680cc1f0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.003+0000 7f5b67fff640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 0x7f5b5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:35.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.003+0000 7f5b67fff640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 0x7f5b5406e320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:35.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.007+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b60059310 con 0x7f5b680cc1f0 2024-09-10T11:35:35.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.205+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5b68073120 con 0x7f5b680cc1f0 2024-09-10T11:35:35.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.882+0000 7f5b64ff9640 1 -- 172.21.15.99:0/3766808497 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7f5b6005d610 con 0x7f5b680cc1f0 2024-09-10T11:35:35.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.884+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 msgr2=0x7f5b5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.884+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 0x7f5b5406e320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5b58000c00 tx=0x7f5b58009040 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.884+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 msgr2=0x7f5b680cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.884+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5b680b44a0 tx=0x7f5b6000e530 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 shutdown_connections 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5b5406be60 0x7f5b5406e320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b680cc1f0 0x7f5b680cc670 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b680ce170 0x7f5b680ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 --2- 172.21.15.99:0/3766808497 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b50004800 0x7f5b68161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 >> 172.21.15.99:0/3766808497 conn(0x7f5b6808e680 msgr2=0x7f5b6806d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 shutdown_connections 2024-09-10T11:35:35.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:35.885+0000 7f5b6e32c640 1 -- 172.21.15.99:0/3766808497 wait complete. 2024-09-10T11:35:35.900 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:35:35.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:36.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.001+0000 7f9ef2ce4640 1 Processor -- start 2024-09-10T11:35:36.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.001+0000 7f9ef2ce4640 1 -- start start 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eec15c7b0 0x7f9eec0d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9eec0d0fb0 0x7f9eec06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9eec0d1420 con 0x7f9eec15c7b0 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9eec06f280 con 0x7f9eec0d0fb0 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef2ce4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec0b3840 con 0x7f9eec06ee30 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eec15c7b0 0x7f9eec0d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eec15c7b0 0x7f9eec0d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37556/0 (socket says 172.21.15.99:37556) 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 -- 172.21.15.99:0/1565459773 learned_addr learned my addr 172.21.15.99:0/1565459773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 -- 172.21.15.99:0/1565459773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 msgr2=0x7f9eec1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/1565459773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 -- 172.21.15.99:0/1565459773 >> v2:172.21.15.196:3300/0 conn(0x7f9eec0d0fb0 msgr2=0x7f9eec06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/1565459773 >> v2:172.21.15.196:3300/0 conn(0x7f9eec0d0fb0 0x7f9eec06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.002+0000 7f9ef0a80640 1 -- 172.21.15.99:0/1565459773 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec0ceb10 con 0x7f9eec15c7b0 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.003+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/1565459773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec0d0a70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0d0250 tx=0x7f9edc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bb6500f4dd7095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.003+0000 7f9eeb7fe640 1 -- 172.21.15.99:0/1565459773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc037070 con 0x7f9eec15c7b0 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.003+0000 7f9eeb7fe640 1 -- 172.21.15.99:0/1565459773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc032440 con 0x7f9eec15c7b0 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.003+0000 7f9eeb7fe640 1 -- 172.21.15.99:0/1565459773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc032810 con 0x7f9eec15c7b0 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/1565459773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 msgr2=0x7f9eec0d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/1565459773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec0d0a70 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0d0250 tx=0x7f9edc02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/1565459773 shutdown_connections 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/1565459773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/1565459773 >> v2:172.21.15.196:3300/0 conn(0x7f9eec0d0fb0 0x7f9eec06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/1565459773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec0d0a70 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/1565459773 >> 172.21.15.99:0/1565459773 conn(0x7f9eec06bc10 msgr2=0x7f9eec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/1565459773 shutdown_connections 2024-09-10T11:35:36.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.004+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/1565459773 wait complete. 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.005+0000 7f9ef2ce4640 1 Processor -- start 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.005+0000 7f9ef2ce4640 1 -- start start 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec13cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 0x7f9eec150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec0b4050 con 0x7f9eec15c7b0 2024-09-10T11:35:36.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec0b3d70 con 0x7f9eec0d0fb0 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef2ce4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec0b4330 con 0x7f9eec06ee30 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37560/0 (socket says 172.21.15.99:37560) 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 -- 172.21.15.99:0/3923342568 learned_addr learned my addr 172.21.15.99:0/3923342568 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec13cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9eebfff640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 0x7f9eec150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 -- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 msgr2=0x7f9eec13cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec13cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 -- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 msgr2=0x7f9eec150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 0x7f9eec150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9ef1281640 1 -- 172.21.15.99:0/3923342568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec13d060 con 0x7f9eec15c7b0 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.006+0000 7f9eebfff640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 0x7f9eec150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:36.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ef1281640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b970 tx=0x7f9ee400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4019070 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee400cb60 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4014440 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec14f9d0 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.007+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec14ff90 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.008+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec15cb90 con 0x7f9eec15c7b0 2024-09-10T11:35:36.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.008+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ee401c070 con 0x7f9eec15c7b0 2024-09-10T11:35:36.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.009+0000 7f9ee97fa640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 0x7f9ed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.009+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 0x7f9ed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.009+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f9ee408aac0 con 0x7f9eec15c7b0 2024-09-10T11:35:36.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.009+0000 7f9ef0a80640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 0x7f9ed806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9edc034000 tx=0x7f9edc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.013+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ee4058e20 con 0x7f9eec15c7b0 2024-09-10T11:35:36.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.213+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eec0affb0 con 0x7f9eec15c7b0 2024-09-10T11:35:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.214+0000 7f9ee97fa640 1 -- 172.21.15.99:0/3923342568 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f9ee405d120 con 0x7f9eec15c7b0 2024-09-10T11:35:36.216 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:36.216 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:35:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:35:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 msgr2=0x7f9ed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 0x7f9ed806e2f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9edc034000 tx=0x7f9edc02e000 comp rx=0 tx=0).stop 2024-09-10T11:35:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 msgr2=0x7f9eec158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9ee400b970 tx=0x7f9ee400be30 comp rx=0 tx=0).stop 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 shutdown_connections 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ed806be30 0x7f9ed806e2f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eec15c7b0 0x7f9eec158ac0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eec0d0fb0 0x7f9eec150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 --2- 172.21.15.99:0/3923342568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eec06ee30 0x7f9eec13cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 >> 172.21.15.99:0/3923342568 conn(0x7f9eec06bc10 msgr2=0x7f9eec0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 shutdown_connections 2024-09-10T11:35:36.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.216+0000 7f9ef2ce4640 1 -- 172.21.15.99:0/3923342568 wait complete. 2024-09-10T11:35:36.229 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:35:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.326+0000 7fb83f7bd640 1 Processor -- start 2024-09-10T11:35:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.326+0000 7fb83f7bd640 1 -- start start 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83f7bd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb83806e4d0 0x7fb838167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83f7bd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8380ceb10 con 0x7fb83815c7b0 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83f7bd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8380b3a90 con 0x7fb8380d1180 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83f7bd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb8380b3840 con 0x7fb83806e4d0 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb83806e4d0 0x7fb838167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb83806e4d0 0x7fb838167610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37574/0 (socket says 172.21.15.99:37574) 2024-09-10T11:35:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83d559640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb83815c7b0 0x7fb8380d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46720/0 (socket says 172.21.15.99:46720) 2024-09-10T11:35:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 -- 172.21.15.99:0/1450688239 learned_addr learned my addr 172.21.15.99:0/1450688239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 -- 172.21.15.99:0/1450688239 >> v1:172.21.15.196:6789/0 conn(0x7fb8380d1180 legacy=0x7fb83806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 -- 172.21.15.99:0/1450688239 >> v1:172.21.15.99:6789/0 conn(0x7fb83815c7b0 legacy=0x7fb8380d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb82ffff640 1 -- 172.21.15.99:0/1450688239 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604264557 0 0) 0x7fb834001110 con 0x7fb83815c7b0 2024-09-10T11:35:36.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.327+0000 7fb83dd5a640 1 -- 172.21.15.99:0/1450688239 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8380b3d70 con 0x7fb83806e4d0 2024-09-10T11:35:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83dd5a640 1 --2- 172.21.15.99:0/1450688239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83806e4d0 0x7fb838167610 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb830009e50 tx=0x7fb83002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9b59a20dc87daf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb82ffff640 1 -- 172.21.15.99:0/1450688239 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830036070 con 0x7fb83806e4d0 2024-09-10T11:35:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb82ffff640 1 -- 172.21.15.99:0/1450688239 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb830031600 con 0x7fb83806e4d0 2024-09-10T11:35:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb82ffff640 1 -- 172.21.15.99:0/1450688239 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300319d0 con 0x7fb83806e4d0 2024-09-10T11:35:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83f7bd640 1 -- 172.21.15.99:0/1450688239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83806e4d0 msgr2=0x7fb838167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/1450688239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83806e4d0 0x7fb838167610 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb830009e50 tx=0x7fb83002a760 comp rx=0 tx=0).stop 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83f7bd640 1 -- 172.21.15.99:0/1450688239 shutdown_connections 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/1450688239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83806e4d0 0x7fb838167610 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.328+0000 7fb83f7bd640 1 -- 172.21.15.99:0/1450688239 >> 172.21.15.99:0/1450688239 conn(0x7fb83806bc10 msgr2=0x7fb83806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.329+0000 7fb83f7bd640 1 -- 172.21.15.99:0/1450688239 shutdown_connections 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.329+0000 7fb83f7bd640 1 -- 172.21.15.99:0/1450688239 wait complete. 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 Processor -- start 2024-09-10T11:35:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 -- start start 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 0x7fb838150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8380d1180 0x7fb8381509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb8380b4610 con 0x7fb83815c7b0 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8380b4370 con 0x7fb83806e4d0 2024-09-10T11:35:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83f7bd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb83815cb90 con 0x7fb8380d1180 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.330+0000 7fb83dd5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37582/0 (socket says 172.21.15.99:37582) 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 -- 172.21.15.99:0/3175129832 learned_addr learned my addr 172.21.15.99:0/3175129832 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83d559640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 0x7fb838150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83cd58640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8380d1180 0x7fb8381509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 -- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8380d1180 msgr2=0x7fb8381509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8380d1180 0x7fb8381509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 -- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 msgr2=0x7fb838150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 0x7fb838150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 -- 172.21.15.99:0/3175129832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381e62f0 con 0x7fb83815c7b0 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83d559640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 0x7fb838150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83dd5a640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb830034040 tx=0x7fb830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830041020 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb830007780 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830036040 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381e65d0 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.331+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381e6b90 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.332+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8380d0a70 con 0x7fb83815c7b0 2024-09-10T11:35:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.333+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb830044020 con 0x7fb83815c7b0 2024-09-10T11:35:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.333+0000 7fb82dffb640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 0x7fb81c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.333+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fb8300abb80 con 0x7fb83815c7b0 2024-09-10T11:35:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.333+0000 7fb83d559640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 0x7fb81c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.334+0000 7fb83d559640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 0x7fb81c06e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb834000f60 tx=0x7fb834007cc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.337+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb83004b030 con 0x7fb83815c7b0 2024-09-10T11:35:36.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.502+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8380b5a70 con 0x7fb83815c7b0 2024-09-10T11:35:36.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.504+0000 7fb82dffb640 1 -- 172.21.15.99:0/3175129832 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+16744 (secure 0 0 0) 0x7fb83007a5f0 con 0x7fb83815c7b0 2024-09-10T11:35:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":138,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:35:34.865583+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:31.809882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:32.181001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 msgr2=0x7fb81c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 0x7fb81c06e2f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb834000f60 tx=0x7fb834007cc0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 msgr2=0x7fb8381547a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb830034040 tx=0x7fb830004a40 comp rx=0 tx=0).stop 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 shutdown_connections 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb81c06be30 0x7fb81c06e2f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb83815c7b0 0x7fb8381547a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8380d1180 0x7fb8381509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 --2- 172.21.15.99:0/3175129832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb83806e4d0 0x7fb838150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 >> 172.21.15.99:0/3175129832 conn(0x7fb83806bc10 msgr2=0x7fb8381591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 shutdown_connections 2024-09-10T11:35:36.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.506+0000 7fb83f7bd640 1 -- 172.21.15.99:0/3175129832 wait complete. 2024-09-10T11:35:36.519 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:35:36.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:36.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.616+0000 7f07c2178640 1 Processor -- start 2024-09-10T11:35:36.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.616+0000 7f07c2178640 1 -- start start 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07c2178640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 0x7f07bc167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07c2178640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc0ceb10 con 0x7f07bc0d1180 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07c2178640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc0b3a90 con 0x7f07bc15c7b0 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07c2178640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc0b3840 con 0x7f07bc06e4d0 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07baffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f07bc0d1180 0x7f07bc06e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46730/0 (socket says 172.21.15.99:46730) 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07baffd640 1 -- 172.21.15.99:0/513161166 learned_addr learned my addr 172.21.15.99:0/513161166 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:36.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07bbfff640 1 --2- 172.21.15.99:0/513161166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 0x7f07bc167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3163253687 0 0) 0x7f07b0001110 con 0x7f07bc0d1180 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07a4003520 con 0x7f07bc0d1180 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07bbfff640 1 -- 172.21.15.99:0/513161166 >> v1:172.21.15.196:6789/0 conn(0x7f07bc15c7b0 legacy=0x7f07bc0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2704033066 0 0) 0x7f07bc0b3a90 con 0x7f07bc15c7b0 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07bbfff640 1 -- 172.21.15.99:0/513161166 >> v1:172.21.15.99:6789/0 conn(0x7f07bc0d1180 legacy=0x7f07bc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.617+0000 7f07bbfff640 1 -- 172.21.15.99:0/513161166 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc0b3d70 con 0x7f07bc06e4d0 2024-09-10T11:35:36.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.618+0000 7f07bbfff640 1 --2- 172.21.15.99:0/513161166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 0x7f07bc167610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b39c0 tx=0x7f07b402a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631908cf74c06860 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.618+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4035020 con 0x7f07bc06e4d0 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.618+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b4007540 con 0x7f07bc06e4d0 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.618+0000 7f07ba7fc640 1 -- 172.21.15.99:0/513161166 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0001840 con 0x7f07bc06e4d0 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 -- 172.21.15.99:0/513161166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 msgr2=0x7f07bc167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 --2- 172.21.15.99:0/513161166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 0x7f07bc167610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b39c0 tx=0x7f07b402a630 comp rx=0 tx=0).stop 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 -- 172.21.15.99:0/513161166 shutdown_connections 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 --2- 172.21.15.99:0/513161166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc06e4d0 0x7f07bc167610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 -- 172.21.15.99:0/513161166 >> 172.21.15.99:0/513161166 conn(0x7f07bc06bc10 msgr2=0x7f07bc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 -- 172.21.15.99:0/513161166 shutdown_connections 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.619+0000 7f07c2178640 1 -- 172.21.15.99:0/513161166 wait complete. 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.620+0000 7f07c2178640 1 Processor -- start 2024-09-10T11:35:36.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.620+0000 7f07c2178640 1 -- start start 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc06e4d0 0x7f07bc14daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc15c7b0 0x7f07bc0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc0b4610 con 0x7f07bc0d1180 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc0b4330 con 0x7f07bc06e4d0 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07c2178640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc15cb90 con 0x7f07bc15c7b0 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37592/0 (socket says 172.21.15.99:37592) 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 -- 172.21.15.99:0/3044710069 learned_addr learned my addr 172.21.15.99:0/3044710069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07bb7fe640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc06e4d0 0x7f07bc14daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07bbfff640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc15c7b0 0x7f07bc0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 -- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc15c7b0 msgr2=0x7f07bc0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc15c7b0 0x7f07bc0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 -- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc06e4d0 msgr2=0x7f07bc14daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc06e4d0 0x7f07bc14daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 -- 172.21.15.99:0/3044710069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc0718c0 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07baffd640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c780 tx=0x7f07b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0018070 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.621+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b00074e0 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.622+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00049e0 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.622+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc071ba0 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.622+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc072160 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.623+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc0d0a70 con 0x7f07bc0d1180 2024-09-10T11:35:36.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.623+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07b001b070 con 0x7f07bc0d1180 2024-09-10T11:35:36.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.623+0000 7f07a3fff640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 0x7f07a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.624+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f07b008a460 con 0x7f07bc0d1180 2024-09-10T11:35:36.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.624+0000 7f07bb7fe640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 0x7f07a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.624+0000 7f07bb7fe640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 0x7f07a806e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f07ac000f60 tx=0x7f07ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.628+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07bc0ceb10 con 0x7f07bc0d1180 2024-09-10T11:35:36.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.788+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07bc158140 con 0x7f07a806be60 2024-09-10T11:35:36.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.793+0000 7f07a3fff640 1 -- 172.21.15.99:0/3044710069 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237341 (secure 0 0 0) 0x7f07bc158140 con 0x7f07a806be60 2024-09-10T11:35:36.798 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:36.799 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:36.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 msgr2=0x7f07a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 0x7f07a806e320 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f07ac000f60 tx=0x7f07ac009040 comp rx=0 tx=0).stop 2024-09-10T11:35:36.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 msgr2=0x7f07bc150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f07b000c780 tx=0x7f07b000cc40 comp rx=0 tx=0).stop 2024-09-10T11:35:36.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 shutdown_connections 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f07a806be60 0x7f07a806e320 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07bc15c7b0 0x7f07bc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07bc0d1180 0x7f07bc150e80 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 --2- 172.21.15.99:0/3044710069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07bc06e4d0 0x7f07bc14daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 >> 172.21.15.99:0/3044710069 conn(0x7f07bc06bc10 msgr2=0x7f07bc15a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.796+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 shutdown_connections 2024-09-10T11:35:36.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.797+0000 7f07c2178640 1 -- 172.21.15.99:0/3044710069 wait complete. 2024-09-10T11:35:36.809 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-10T11:35:34.972859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":52,"acting":52,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":2,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555576,"statfs":{"total":1600000000,"available":1592910609,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000593"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865050+0000","last_change":"2024-09-10T11:35:31.865050+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865050+0000","last_undegraded":"2024-09-10T11:35:31.865050+0000","last_fullsized":"2024-09-10T11:35:31.865050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865033+0000","last_change":"2024-09-10T11:35:31.865033+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865033+0000","last_undegraded":"2024-09-10T11:35:31.865033+0000","last_fullsized":"2024-09-10T11:35:31.865033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865022+0000","last_change":"2024-09-10T11:35:31.865022+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865022+0000","last_undegraded":"2024-09-10T11:35:31.865022+0000","last_fullsized":"2024-09-10T11:35:31.865022+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863057+0000","last_change":"2024-09-10T11:35:31.863057+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863057+0000","last_undegraded":"2024-09-10T11:35:31.863057+0000","last_fullsized":"2024-09-10T11:35:31.863057+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.867052+0000","last_change":"2024-09-10T11:35:31.867052+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.867052+0000","last_undegraded":"2024-09-10T11:35:31.867052+0000","last_fullsized":"2024-09-10T11:35:31.867052+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865825+0000","last_change":"2024-09-10T11:35:31.865825+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865825+0000","last_undegraded":"2024-09-10T11:35:31.865825+0000","last_fullsized":"2024-09-10T11:35:31.865825+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.864628+0000","last_change":"2024-09-10T11:35:31.864628+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.864628+0000","last_undegraded":"2024-09-10T11:35:31.864628+0000","last_fullsized":"2024-09-10T11:35:31.864628+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.864628+0000","last_change":"2024-09-10T11:35:31.864628+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.864628+0000","last_undegraded":"2024-09-10T11:35:31.864628+0000","last_fullsized":"2024-09-10T11:35:31.864628+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865688+0000","last_change":"2024-09-10T11:35:31.865688+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865688+0000","last_undegraded":"2024-09-10T11:35:31.865688+0000","last_fullsized":"2024-09-10T11:35:31.865688+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863841+0000","last_change":"2024-09-10T11:35:31.863841+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863841+0000","last_undegraded":"2024-09-10T11:35:31.863841+0000","last_fullsized":"2024-09-10T11:35:31.863841+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865294+0000","last_change":"2024-09-10T11:35:31.865294+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865294+0000","last_undegraded":"2024-09-10T11:35:31.865294+0000","last_fullsized":"2024-09-10T11:35:31.865294+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863061+0000","last_change":"2024-09-10T11:35:31.863061+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863061+0000","last_undegraded":"2024-09-10T11:35:31.863061+0000","last_fullsized":"2024-09-10T11:35:31.863061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.866805+0000","last_change":"2024-09-10T11:35:31.866805+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.866805+0000","last_undegraded":"2024-09-10T11:35:31.866805+0000","last_fullsized":"2024-09-10T11:35:31.866805+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865033+0000","last_change":"2024-09-10T11:35:31.865033+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865033+0000","last_undegraded":"2024-09-10T11:35:31.865033+0000","last_fullsized":"2024-09-10T11:35:31.865033+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.864955+0000","last_change":"2024-09-10T11:35:31.864955+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.864955+0000","last_undegraded":"2024-09-10T11:35:31.864955+0000","last_fullsized":"2024-09-10T11:35:31.864955+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.864629+0000","last_change":"2024-09-10T11:35:31.864629+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.864629+0000","last_undegraded":"2024-09-10T11:35:31.864629+0000","last_fullsized":"2024-09-10T11:35:31.864629+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.866291+0000","last_change":"2024-09-10T11:35:31.866291+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.866291+0000","last_undegraded":"2024-09-10T11:35:31.866291+0000","last_fullsized":"2024-09-10T11:35:31.866291+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863870+0000","last_change":"2024-09-10T11:35:31.863870+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863870+0000","last_undegraded":"2024-09-10T11:35:31.863870+0000","last_fullsized":"2024-09-10T11:35:31.863870+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863295+0000","last_change":"2024-09-10T11:35:31.863295+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863295+0000","last_undegraded":"2024-09-10T11:35:31.863295+0000","last_fullsized":"2024-09-10T11:35:31.863295+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863292+0000","last_change":"2024-09-10T11:35:31.863292+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863292+0000","last_undegraded":"2024-09-10T11:35:31.863292+0000","last_fullsized":"2024-09-10T11:35:31.863292+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863290+0000","last_change":"2024-09-10T11:35:31.863290+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863290+0000","last_undegraded":"2024-09-10T11:35:31.863290+0000","last_fullsized":"2024-09-10T11:35:31.863290+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.863472+0000","last_change":"2024-09-10T11:35:31.863472+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.863472+0000","last_undegraded":"2024-09-10T11:35:31.863472+0000","last_fullsized":"2024-09-10T11:35:31.863472+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:31.857526+0000","last_change":"2024-09-10T11:35:31.857526+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.857526+0000","last_undegraded":"2024-09-10T11:35:31.857526+0000","last_fullsized":"2024-09-10T11:35:31.857526+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.866192+0000","last_change":"2024-09-10T11:35:31.866192+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.866192+0000","last_undegraded":"2024-09-10T11:35:31.866192+0000","last_fullsized":"2024-09-10T11:35:31.866192+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.864286+0000","last_change":"2024-09-10T11:35:31.864286+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.864286+0000","last_undegraded":"2024-09-10T11:35:31.864286+0000","last_fullsized":"2024-09-10T11:35:31.864286+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T11:35:32.861773+0000","last_change":"2024-09-10T11:35:32.861773+0000","last_active":"2024-09-10T11:35:32.861773+0000","last_peered":"2024-09-10T11:35:32.861773+0000","last_clean":"2024-09-10T11:35:32.861773+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:32.861773+0000","last_undegraded":"2024-09-10T11:35:32.861773+0000","last_fullsized":"2024-09-10T11:35:32.861773+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":135,"state":"creating+peering","last_fresh":"2024-09-10T11:35:31.865050+0000","last_change":"2024-09-10T11:35:31.865050+0000","last_active":"2024-09-10T11:35:31.857526+0000","last_peered":"2024-09-10T11:35:31.857526+0000","last_clean":"2024-09-10T11:35:31.857526+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T11:35:31.865050+0000","last_undegraded":"2024-09-10T11:35:31.865050+0000","last_fullsized":"2024-09-10T11:35:31.865050+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":403,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-10T11:35:31.862723+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:31.862723+0000","last_peered":"2024-09-10T11:35:31.862723+0000","last_clean":"2024-09-10T11:35:31.862723+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:31.862723+0000","last_undegraded":"2024-09-10T11:35:31.862723+0000","last_fullsized":"2024-09-10T11:35:31.862723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":50,"acting":50,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265843,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194014,"statfs":{"total":200000000,"available":198670418,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1298,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194014,"statfs":{"total":200000000,"available":198670418,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":721,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194590,"statfs":{"total":200000000,"available":199260786,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:36.813 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:35:36.845 INFO:tasks.ceph.mds.b.smithi196.stderr:2024-09-10T11:35:36.844+0000 7f6e81d90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:35:36.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.947+0000 7f229bc5e640 1 Processor -- start 2024-09-10T11:35:36.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.947+0000 7f229bc5e640 1 -- start start 2024-09-10T11:35:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229bc5e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f229415cb70 0x7f2294072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229bc5e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2294072650 con 0x7f22940b0fc0 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229bc5e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229415db70 con 0x7f22940b4150 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229bc5e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f229415de50 con 0x7f229415cb70 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229a1fb640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f22940b0fc0 0x7f229415cf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46734/0 (socket says 172.21.15.99:46734) 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f229a1fb640 1 -- 172.21.15.99:0/1265744379 learned_addr learned my addr 172.21.15.99:0/1265744379 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2796205216 0 0) 0x7f2294072650 con 0x7f22940b0fc0 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22999fa640 1 --2- 172.21.15.99:0/1265744379 >> v2:172.21.15.196:3300/0 conn(0x7f229415cb70 0x7f2294072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f227c003520 con 0x7f22940b0fc0 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2273907568 0 0) 0x7f227c003520 con 0x7f22940b0fc0 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 >> v2:172.21.15.196:3300/0 conn(0x7f229415cb70 msgr2=0x7f2294072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 --2- 172.21.15.99:0/1265744379 >> v2:172.21.15.196:3300/0 conn(0x7f229415cb70 0x7f2294072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 >> v1:172.21.15.196:6789/0 conn(0x7f22940b4150 legacy=0x7f22940b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22940728c0 con 0x7f22940b0fc0 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f228c001670 con 0x7f22940b0fc0 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 >> v1:172.21.15.99:6789/0 conn(0x7f22940b0fc0 legacy=0x7f229415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f229415de50 con 0x7f227c004800 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f227c00c0c0 con 0x7f227c003a00 2024-09-10T11:35:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f227c00c2c0 con 0x7f227c0051c0 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.948+0000 7f22991f9640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22999fa640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f229a1fb640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 -- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227c0051c0 msgr2=0x7f227c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 -- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f227c003a00 msgr2=0x7f227c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f227c003a00 0x7f227c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 -- 172.21.15.99:0/1265744379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227c003520 con 0x7f227c004800 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f229a1fb640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227c0051c0 0x7f227c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:36.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f229a1fb640 1 -- 172.21.15.99:0/1265744379 reap_dead start 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.949+0000 7f22991f9640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f227c004c80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2288001750 tx=0x7f227c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d7e8a48c762794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2288001370 con 0x7f227c004800 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22880052d0 con 0x7f227c004800 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f22989f8640 1 -- 172.21.15.99:0/1265744379 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228802abc0 con 0x7f227c004800 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 -- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 msgr2=0x7f227c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f227c004c80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f2288001750 tx=0x7f227c005830 comp rx=0 tx=0).stop 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 -- 172.21.15.99:0/1265744379 shutdown_connections 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 --2- 172.21.15.99:0/1265744379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f227c004c80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 -- 172.21.15.99:0/1265744379 >> 172.21.15.99:0/1265744379 conn(0x7f22940b3630 msgr2=0x7f22940b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 -- 172.21.15.99:0/1265744379 shutdown_connections 2024-09-10T11:35:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.950+0000 7f229bc5e640 1 -- 172.21.15.99:0/1265744379 wait complete. 2024-09-10T11:35:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.951+0000 7f229bc5e640 1 Processor -- start 2024-09-10T11:35:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.951+0000 7f229bc5e640 1 -- start start 2024-09-10T11:35:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22940ae8a0 0x7f229414cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 0x7f229414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f22940b0a30 con 0x7f227c004800 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f22940b08b0 con 0x7f229414dd70 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229bc5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f22940b0bb0 con 0x7f22940ae8a0 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f22999fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f22999fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37620/0 (socket says 172.21.15.99:37620) 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f22999fa640 1 -- 172.21.15.99:0/3929956996 learned_addr learned my addr 172.21.15.99:0/3929956996 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f22991f9640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22940ae8a0 0x7f229414cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.952+0000 7f229a1fb640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 0x7f229414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 -- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22940ae8a0 msgr2=0x7f229414cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22940ae8a0 0x7f229414cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 -- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 msgr2=0x7f229414e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 0x7f229414e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 -- 172.21.15.99:0/3929956996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22940afeb0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f229a1fb640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 0x7f229414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22999fa640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2284011e30 tx=0x7f228400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284015630 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22840157d0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22940aee10 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284015ba0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.953+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22940af3d0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.954+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2294072de0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.954+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f228401d400 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.955+0000 7f22927fc640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.955+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f228408bfd0 con 0x7f227c004800 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.955+0000 7f22991f9640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:36.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.955+0000 7f22991f9640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 0x7f228006e320 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2288032000 tx=0x7f228802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:36.959+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2284019030 con 0x7f227c004800 2024-09-10T11:35:37.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.121+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2294160180 con 0x7f228006be60 2024-09-10T11:35:37.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.127+0000 7f22927fc640 1 -- 172.21.15.99:0/3929956996 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247993 (secure 0 0 0) 0x7f228807e8d0 con 0x7f228006be60 2024-09-10T11:35:37.131 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:37.133 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:35:37.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 msgr2=0x7f228006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 0x7f228006e320 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2288032000 tx=0x7f228802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:37.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 msgr2=0x7f22940d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2284011e30 tx=0x7f228400f580 comp rx=0 tx=0).stop 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 shutdown_connections 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f228006be60 0x7f228006e320 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f229414dd70 0x7f229414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22940ae8a0 0x7f229414cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 --2- 172.21.15.99:0/3929956996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f227c004800 0x7f22940d0ef0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.129+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 >> 172.21.15.99:0/3929956996 conn(0x7f22940b3630 msgr2=0x7f229406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.130+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 shutdown_connections 2024-09-10T11:35:37.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.130+0000 7f229bc5e640 1 -- 172.21.15.99:0/3929956996 wait complete. 2024-09-10T11:35:37.143 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-10T11:35:36.973136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7129,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555370,"statfs":{"total":1600000000,"available":1592699396,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.111032"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869150+0000","last_change":"2024-09-10T11:35:33.880124+0000","last_active":"2024-09-10T11:35:34.869150+0000","last_peered":"2024-09-10T11:35:34.869150+0000","last_clean":"2024-09-10T11:35:34.869150+0000","last_became_active":"2024-09-10T11:35:33.879590+0000","last_became_peered":"2024-09-10T11:35:33.879590+0000","last_unstale":"2024-09-10T11:35:34.869150+0000","last_undegraded":"2024-09-10T11:35:34.869150+0000","last_fullsized":"2024-09-10T11:35:34.869150+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:41:29.787808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000349+0000","last_change":"2024-09-10T11:35:33.880761+0000","last_active":"2024-09-10T11:35:35.000349+0000","last_peered":"2024-09-10T11:35:35.000349+0000","last_clean":"2024-09-10T11:35:35.000349+0000","last_became_active":"2024-09-10T11:35:33.880279+0000","last_became_peered":"2024-09-10T11:35:33.880279+0000","last_unstale":"2024-09-10T11:35:35.000349+0000","last_undegraded":"2024-09-10T11:35:35.000349+0000","last_fullsized":"2024-09-10T11:35:35.000349+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:22.147954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868868+0000","last_change":"2024-09-10T11:35:33.884586+0000","last_active":"2024-09-10T11:35:34.868868+0000","last_peered":"2024-09-10T11:35:34.868868+0000","last_clean":"2024-09-10T11:35:34.868868+0000","last_became_active":"2024-09-10T11:35:33.884338+0000","last_became_peered":"2024-09-10T11:35:33.884338+0000","last_unstale":"2024-09-10T11:35:34.868868+0000","last_undegraded":"2024-09-10T11:35:34.868868+0000","last_fullsized":"2024-09-10T11:35:34.868868+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:30.774749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869957+0000","last_change":"2024-09-10T11:35:33.879990+0000","last_active":"2024-09-10T11:35:34.869957+0000","last_peered":"2024-09-10T11:35:34.869957+0000","last_clean":"2024-09-10T11:35:34.869957+0000","last_became_active":"2024-09-10T11:35:33.879755+0000","last_became_peered":"2024-09-10T11:35:33.879755+0000","last_unstale":"2024-09-10T11:35:34.869957+0000","last_undegraded":"2024-09-10T11:35:34.869957+0000","last_fullsized":"2024-09-10T11:35:34.869957+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:41.678586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.001179+0000","last_change":"2024-09-10T11:35:33.876777+0000","last_active":"2024-09-10T11:35:35.001179+0000","last_peered":"2024-09-10T11:35:35.001179+0000","last_clean":"2024-09-10T11:35:35.001179+0000","last_became_active":"2024-09-10T11:35:33.876499+0000","last_became_peered":"2024-09-10T11:35:33.876499+0000","last_unstale":"2024-09-10T11:35:35.001179+0000","last_undegraded":"2024-09-10T11:35:35.001179+0000","last_fullsized":"2024-09-10T11:35:35.001179+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:38.185556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.889948+0000","last_change":"2024-09-10T11:35:33.881197+0000","last_active":"2024-09-10T11:35:34.889948+0000","last_peered":"2024-09-10T11:35:34.889948+0000","last_clean":"2024-09-10T11:35:34.889948+0000","last_became_active":"2024-09-10T11:35:33.880906+0000","last_became_peered":"2024-09-10T11:35:33.880906+0000","last_unstale":"2024-09-10T11:35:34.889948+0000","last_undegraded":"2024-09-10T11:35:34.889948+0000","last_fullsized":"2024-09-10T11:35:34.889948+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:34.901641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888389+0000","last_change":"2024-09-10T11:35:33.876315+0000","last_active":"2024-09-10T11:35:34.888389+0000","last_peered":"2024-09-10T11:35:34.888389+0000","last_clean":"2024-09-10T11:35:34.888389+0000","last_became_active":"2024-09-10T11:35:33.876071+0000","last_became_peered":"2024-09-10T11:35:33.876071+0000","last_unstale":"2024-09-10T11:35:34.888389+0000","last_undegraded":"2024-09-10T11:35:34.888389+0000","last_fullsized":"2024-09-10T11:35:34.888389+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:22.943668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888391+0000","last_change":"2024-09-10T11:35:33.884015+0000","last_active":"2024-09-10T11:35:34.888391+0000","last_peered":"2024-09-10T11:35:34.888391+0000","last_clean":"2024-09-10T11:35:34.888391+0000","last_became_active":"2024-09-10T11:35:33.883828+0000","last_became_peered":"2024-09-10T11:35:33.883828+0000","last_unstale":"2024-09-10T11:35:34.888391+0000","last_undegraded":"2024-09-10T11:35:34.888391+0000","last_fullsized":"2024-09-10T11:35:34.888391+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:05.689676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869037+0000","last_change":"2024-09-10T11:35:33.880906+0000","last_active":"2024-09-10T11:35:34.869037+0000","last_peered":"2024-09-10T11:35:34.869037+0000","last_clean":"2024-09-10T11:35:34.869037+0000","last_became_active":"2024-09-10T11:35:33.880423+0000","last_became_peered":"2024-09-10T11:35:33.880423+0000","last_unstale":"2024-09-10T11:35:34.869037+0000","last_undegraded":"2024-09-10T11:35:34.869037+0000","last_fullsized":"2024-09-10T11:35:34.869037+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:52.502242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000589+0000","last_change":"2024-09-10T11:35:33.874937+0000","last_active":"2024-09-10T11:35:35.000589+0000","last_peered":"2024-09-10T11:35:35.000589+0000","last_clean":"2024-09-10T11:35:35.000589+0000","last_became_active":"2024-09-10T11:35:33.873848+0000","last_became_peered":"2024-09-10T11:35:33.873848+0000","last_unstale":"2024-09-10T11:35:35.000589+0000","last_undegraded":"2024-09-10T11:35:35.000589+0000","last_fullsized":"2024-09-10T11:35:35.000589+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:57.204182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888415+0000","last_change":"2024-09-10T11:35:33.881687+0000","last_active":"2024-09-10T11:35:34.888415+0000","last_peered":"2024-09-10T11:35:34.888415+0000","last_clean":"2024-09-10T11:35:34.888415+0000","last_became_active":"2024-09-10T11:35:33.881220+0000","last_became_peered":"2024-09-10T11:35:33.881220+0000","last_unstale":"2024-09-10T11:35:34.888415+0000","last_undegraded":"2024-09-10T11:35:34.888415+0000","last_fullsized":"2024-09-10T11:35:34.888415+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:35.724998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000139+0000","last_change":"2024-09-10T11:35:33.876449+0000","last_active":"2024-09-10T11:35:35.000139+0000","last_peered":"2024-09-10T11:35:35.000139+0000","last_clean":"2024-09-10T11:35:35.000139+0000","last_became_active":"2024-09-10T11:35:33.875770+0000","last_became_peered":"2024-09-10T11:35:33.875770+0000","last_unstale":"2024-09-10T11:35:35.000139+0000","last_undegraded":"2024-09-10T11:35:35.000139+0000","last_fullsized":"2024-09-10T11:35:35.000139+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:11.098273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000682+0000","last_change":"2024-09-10T11:35:33.883731+0000","last_active":"2024-09-10T11:35:35.000682+0000","last_peered":"2024-09-10T11:35:35.000682+0000","last_clean":"2024-09-10T11:35:35.000682+0000","last_became_active":"2024-09-10T11:35:33.883387+0000","last_became_peered":"2024-09-10T11:35:33.883387+0000","last_unstale":"2024-09-10T11:35:35.000682+0000","last_undegraded":"2024-09-10T11:35:35.000682+0000","last_fullsized":"2024-09-10T11:35:35.000682+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:49.796401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.999865+0000","last_change":"2024-09-10T11:35:33.879587+0000","last_active":"2024-09-10T11:35:34.999865+0000","last_peered":"2024-09-10T11:35:34.999865+0000","last_clean":"2024-09-10T11:35:34.999865+0000","last_became_active":"2024-09-10T11:35:33.879320+0000","last_became_peered":"2024-09-10T11:35:33.879320+0000","last_unstale":"2024-09-10T11:35:34.999865+0000","last_undegraded":"2024-09-10T11:35:34.999865+0000","last_fullsized":"2024-09-10T11:35:34.999865+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:19.684284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870212+0000","last_change":"2024-09-10T11:35:33.875387+0000","last_active":"2024-09-10T11:35:34.870212+0000","last_peered":"2024-09-10T11:35:34.870212+0000","last_clean":"2024-09-10T11:35:34.870212+0000","last_became_active":"2024-09-10T11:35:33.874529+0000","last_became_peered":"2024-09-10T11:35:33.874529+0000","last_unstale":"2024-09-10T11:35:34.870212+0000","last_undegraded":"2024-09-10T11:35:34.870212+0000","last_fullsized":"2024-09-10T11:35:34.870212+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:36:37.194046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000571+0000","last_change":"2024-09-10T11:35:33.876987+0000","last_active":"2024-09-10T11:35:35.000571+0000","last_peered":"2024-09-10T11:35:35.000571+0000","last_clean":"2024-09-10T11:35:35.000571+0000","last_became_active":"2024-09-10T11:35:33.875866+0000","last_became_peered":"2024-09-10T11:35:33.875866+0000","last_unstale":"2024-09-10T11:35:35.000571+0000","last_undegraded":"2024-09-10T11:35:35.000571+0000","last_fullsized":"2024-09-10T11:35:35.000571+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:38.559949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868836+0000","last_change":"2024-09-10T11:35:33.878297+0000","last_active":"2024-09-10T11:35:34.868836+0000","last_peered":"2024-09-10T11:35:34.868836+0000","last_clean":"2024-09-10T11:35:34.868836+0000","last_became_active":"2024-09-10T11:35:33.878072+0000","last_became_peered":"2024-09-10T11:35:33.878072+0000","last_unstale":"2024-09-10T11:35:34.868836+0000","last_undegraded":"2024-09-10T11:35:34.868836+0000","last_fullsized":"2024-09-10T11:35:34.868836+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:14.230895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888548+0000","last_change":"2024-09-10T11:35:33.871152+0000","last_active":"2024-09-10T11:35:34.888548+0000","last_peered":"2024-09-10T11:35:34.888548+0000","last_clean":"2024-09-10T11:35:34.888548+0000","last_became_active":"2024-09-10T11:35:33.870992+0000","last_became_peered":"2024-09-10T11:35:33.870992+0000","last_unstale":"2024-09-10T11:35:34.888548+0000","last_undegraded":"2024-09-10T11:35:34.888548+0000","last_fullsized":"2024-09-10T11:35:34.888548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:14:58.273818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868836+0000","last_change":"2024-09-10T11:35:33.879734+0000","last_active":"2024-09-10T11:35:34.868836+0000","last_peered":"2024-09-10T11:35:34.868836+0000","last_clean":"2024-09-10T11:35:34.868836+0000","last_became_active":"2024-09-10T11:35:33.879512+0000","last_became_peered":"2024-09-10T11:35:33.879512+0000","last_unstale":"2024-09-10T11:35:34.868836+0000","last_undegraded":"2024-09-10T11:35:34.868836+0000","last_fullsized":"2024-09-10T11:35:34.868836+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:18.802037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868835+0000","last_change":"2024-09-10T11:35:33.884257+0000","last_active":"2024-09-10T11:35:34.868835+0000","last_peered":"2024-09-10T11:35:34.868835+0000","last_clean":"2024-09-10T11:35:34.868835+0000","last_became_active":"2024-09-10T11:35:33.884045+0000","last_became_peered":"2024-09-10T11:35:33.884045+0000","last_unstale":"2024-09-10T11:35:34.868835+0000","last_undegraded":"2024-09-10T11:35:34.868835+0000","last_fullsized":"2024-09-10T11:35:34.868835+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:01.239338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888510+0000","last_change":"2024-09-10T11:35:33.881480+0000","last_active":"2024-09-10T11:35:34.888510+0000","last_peered":"2024-09-10T11:35:34.888510+0000","last_clean":"2024-09-10T11:35:34.888510+0000","last_became_active":"2024-09-10T11:35:33.880852+0000","last_became_peered":"2024-09-10T11:35:33.880852+0000","last_unstale":"2024-09-10T11:35:34.888510+0000","last_undegraded":"2024-09-10T11:35:34.888510+0000","last_fullsized":"2024-09-10T11:35:34.888510+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:02.245264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.889869+0000","last_change":"2024-09-10T11:35:33.873169+0000","last_active":"2024-09-10T11:35:34.889869+0000","last_peered":"2024-09-10T11:35:34.889869+0000","last_clean":"2024-09-10T11:35:34.889869+0000","last_became_active":"2024-09-10T11:35:33.872918+0000","last_became_peered":"2024-09-10T11:35:33.872918+0000","last_unstale":"2024-09-10T11:35:34.889869+0000","last_undegraded":"2024-09-10T11:35:34.889869+0000","last_fullsized":"2024-09-10T11:35:34.889869+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:02.977195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.999867+0000","last_change":"2024-09-10T11:35:33.884689+0000","last_active":"2024-09-10T11:35:34.999867+0000","last_peered":"2024-09-10T11:35:34.999867+0000","last_clean":"2024-09-10T11:35:34.999867+0000","last_became_active":"2024-09-10T11:35:33.884224+0000","last_became_peered":"2024-09-10T11:35:33.884224+0000","last_unstale":"2024-09-10T11:35:34.999867+0000","last_undegraded":"2024-09-10T11:35:34.999867+0000","last_fullsized":"2024-09-10T11:35:34.999867+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:11.315214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870093+0000","last_change":"2024-09-10T11:35:33.881099+0000","last_active":"2024-09-10T11:35:34.870093+0000","last_peered":"2024-09-10T11:35:34.870093+0000","last_clean":"2024-09-10T11:35:34.870093+0000","last_became_active":"2024-09-10T11:35:33.880662+0000","last_became_peered":"2024-09-10T11:35:33.880662+0000","last_unstale":"2024-09-10T11:35:34.870093+0000","last_undegraded":"2024-09-10T11:35:34.870093+0000","last_fullsized":"2024-09-10T11:35:34.870093+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:23:30.154013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888419+0000","last_change":"2024-09-10T11:35:33.874907+0000","last_active":"2024-09-10T11:35:34.888419+0000","last_peered":"2024-09-10T11:35:34.888419+0000","last_clean":"2024-09-10T11:35:34.888419+0000","last_became_active":"2024-09-10T11:35:33.874648+0000","last_became_peered":"2024-09-10T11:35:33.874648+0000","last_unstale":"2024-09-10T11:35:34.888419+0000","last_undegraded":"2024-09-10T11:35:34.888419+0000","last_fullsized":"2024-09-10T11:35:34.888419+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:30.708014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890268+0000","last_change":"2024-09-10T11:35:33.876641+0000","last_active":"2024-09-10T11:35:34.890268+0000","last_peered":"2024-09-10T11:35:34.890268+0000","last_clean":"2024-09-10T11:35:34.890268+0000","last_became_active":"2024-09-10T11:35:33.876391+0000","last_became_peered":"2024-09-10T11:35:33.876391+0000","last_unstale":"2024-09-10T11:35:34.890268+0000","last_undegraded":"2024-09-10T11:35:34.890268+0000","last_fullsized":"2024-09-10T11:35:34.890268+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:41.381704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870727+0000","last_change":"2024-09-10T11:35:33.875746+0000","last_active":"2024-09-10T11:35:34.870727+0000","last_peered":"2024-09-10T11:35:34.870727+0000","last_clean":"2024-09-10T11:35:34.870727+0000","last_became_active":"2024-09-10T11:35:33.875482+0000","last_became_peered":"2024-09-10T11:35:33.875482+0000","last_unstale":"2024-09-10T11:35:34.870727+0000","last_undegraded":"2024-09-10T11:35:34.870727+0000","last_fullsized":"2024-09-10T11:35:34.870727+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:46:34.772784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.889915+0000","last_change":"2024-09-10T11:35:33.881525+0000","last_active":"2024-09-10T11:35:34.889915+0000","last_peered":"2024-09-10T11:35:34.889915+0000","last_clean":"2024-09-10T11:35:34.889915+0000","last_became_active":"2024-09-10T11:35:33.880924+0000","last_became_peered":"2024-09-10T11:35:33.880924+0000","last_unstale":"2024-09-10T11:35:34.889915+0000","last_undegraded":"2024-09-10T11:35:34.889915+0000","last_fullsized":"2024-09-10T11:35:34.889915+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:00.845269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000006+0000","last_change":"2024-09-10T11:35:33.876407+0000","last_active":"2024-09-10T11:35:35.000006+0000","last_peered":"2024-09-10T11:35:35.000006+0000","last_clean":"2024-09-10T11:35:35.000006+0000","last_became_active":"2024-09-10T11:35:33.876186+0000","last_became_peered":"2024-09-10T11:35:33.876186+0000","last_unstale":"2024-09-10T11:35:35.000006+0000","last_undegraded":"2024-09-10T11:35:35.000006+0000","last_fullsized":"2024-09-10T11:35:35.000006+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:15.137485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888574+0000","last_change":"2024-09-10T11:35:33.873019+0000","last_active":"2024-09-10T11:35:34.888574+0000","last_peered":"2024-09-10T11:35:34.888574+0000","last_clean":"2024-09-10T11:35:34.888574+0000","last_became_active":"2024-09-10T11:35:33.872760+0000","last_became_peered":"2024-09-10T11:35:33.872760+0000","last_unstale":"2024-09-10T11:35:34.888574+0000","last_undegraded":"2024-09-10T11:35:34.888574+0000","last_fullsized":"2024-09-10T11:35:34.888574+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:25.575001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869304+0000","last_change":"2024-09-10T11:35:33.879938+0000","last_active":"2024-09-10T11:35:34.869304+0000","last_peered":"2024-09-10T11:35:34.869304+0000","last_clean":"2024-09-10T11:35:34.869304+0000","last_became_active":"2024-09-10T11:35:33.879229+0000","last_became_peered":"2024-09-10T11:35:33.879229+0000","last_unstale":"2024-09-10T11:35:34.869304+0000","last_undegraded":"2024-09-10T11:35:34.869304+0000","last_fullsized":"2024-09-10T11:35:34.869304+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:20:22.638881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000433+0000","last_change":"2024-09-10T11:35:33.876693+0000","last_active":"2024-09-10T11:35:35.000433+0000","last_peered":"2024-09-10T11:35:35.000433+0000","last_clean":"2024-09-10T11:35:35.000433+0000","last_became_active":"2024-09-10T11:35:33.876205+0000","last_became_peered":"2024-09-10T11:35:33.876205+0000","last_unstale":"2024-09-10T11:35:35.000433+0000","last_undegraded":"2024-09-10T11:35:35.000433+0000","last_fullsized":"2024-09-10T11:35:35.000433+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:17:58.548229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869176+0000","last_change":"2024-09-10T11:35:33.880703+0000","last_active":"2024-09-10T11:35:34.869176+0000","last_peered":"2024-09-10T11:35:34.869176+0000","last_clean":"2024-09-10T11:35:34.869176+0000","last_became_active":"2024-09-10T11:35:33.880017+0000","last_became_peered":"2024-09-10T11:35:33.880017+0000","last_unstale":"2024-09-10T11:35:34.869176+0000","last_undegraded":"2024-09-10T11:35:34.869176+0000","last_fullsized":"2024-09-10T11:35:34.869176+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:18.540702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000248+0000","last_change":"2024-09-10T11:35:32.873514+0000","last_active":"2024-09-10T11:35:35.000248+0000","last_peered":"2024-09-10T11:35:35.000248+0000","last_clean":"2024-09-10T11:35:35.000248+0000","last_became_active":"2024-09-10T11:35:32.872986+0000","last_became_peered":"2024-09-10T11:35:32.872986+0000","last_unstale":"2024-09-10T11:35:35.000248+0000","last_undegraded":"2024-09-10T11:35:35.000248+0000","last_fullsized":"2024-09-10T11:35:35.000248+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:28.443413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000008+0000","last_change":"2024-09-10T11:35:33.884457+0000","last_active":"2024-09-10T11:35:35.000008+0000","last_peered":"2024-09-10T11:35:35.000008+0000","last_clean":"2024-09-10T11:35:35.000008+0000","last_became_active":"2024-09-10T11:35:33.883914+0000","last_became_peered":"2024-09-10T11:35:33.883914+0000","last_unstale":"2024-09-10T11:35:35.000008+0000","last_undegraded":"2024-09-10T11:35:35.000008+0000","last_fullsized":"2024-09-10T11:35:35.000008+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:39.344253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000195+0000","last_change":"2024-09-10T11:35:32.871442+0000","last_active":"2024-09-10T11:35:35.000195+0000","last_peered":"2024-09-10T11:35:35.000195+0000","last_clean":"2024-09-10T11:35:35.000195+0000","last_became_active":"2024-09-10T11:35:32.871230+0000","last_became_peered":"2024-09-10T11:35:32.871230+0000","last_unstale":"2024-09-10T11:35:35.000195+0000","last_undegraded":"2024-09-10T11:35:35.000195+0000","last_fullsized":"2024-09-10T11:35:35.000195+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:42.863601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890119+0000","last_change":"2024-09-10T11:35:33.873780+0000","last_active":"2024-09-10T11:35:34.890119+0000","last_peered":"2024-09-10T11:35:34.890119+0000","last_clean":"2024-09-10T11:35:34.890119+0000","last_became_active":"2024-09-10T11:35:33.873526+0000","last_became_peered":"2024-09-10T11:35:33.873526+0000","last_unstale":"2024-09-10T11:35:34.890119+0000","last_undegraded":"2024-09-10T11:35:34.890119+0000","last_fullsized":"2024-09-10T11:35:34.890119+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:18.507607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870952+0000","last_change":"2024-09-10T11:35:32.881290+0000","last_active":"2024-09-10T11:35:34.870952+0000","last_peered":"2024-09-10T11:35:34.870952+0000","last_clean":"2024-09-10T11:35:34.870952+0000","last_became_active":"2024-09-10T11:35:32.881093+0000","last_became_peered":"2024-09-10T11:35:32.881093+0000","last_unstale":"2024-09-10T11:35:34.870952+0000","last_undegraded":"2024-09-10T11:35:34.870952+0000","last_fullsized":"2024-09-10T11:35:34.870952+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:25.516283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868923+0000","last_change":"2024-09-10T11:35:33.879155+0000","last_active":"2024-09-10T11:35:34.868923+0000","last_peered":"2024-09-10T11:35:34.868923+0000","last_clean":"2024-09-10T11:35:34.868923+0000","last_became_active":"2024-09-10T11:35:33.878928+0000","last_became_peered":"2024-09-10T11:35:33.878928+0000","last_unstale":"2024-09-10T11:35:34.868923+0000","last_undegraded":"2024-09-10T11:35:34.868923+0000","last_fullsized":"2024-09-10T11:35:34.868923+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:42.581485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870721+0000","last_change":"2024-09-10T11:35:32.879625+0000","last_active":"2024-09-10T11:35:34.870721+0000","last_peered":"2024-09-10T11:35:34.870721+0000","last_clean":"2024-09-10T11:35:34.870721+0000","last_became_active":"2024-09-10T11:35:32.879202+0000","last_became_peered":"2024-09-10T11:35:32.879202+0000","last_unstale":"2024-09-10T11:35:34.870721+0000","last_undegraded":"2024-09-10T11:35:34.870721+0000","last_fullsized":"2024-09-10T11:35:34.870721+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:57.077922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869379+0000","last_change":"2024-09-10T11:35:33.886741+0000","last_active":"2024-09-10T11:35:34.869379+0000","last_peered":"2024-09-10T11:35:34.869379+0000","last_clean":"2024-09-10T11:35:34.869379+0000","last_became_active":"2024-09-10T11:35:33.885419+0000","last_became_peered":"2024-09-10T11:35:33.885419+0000","last_unstale":"2024-09-10T11:35:34.869379+0000","last_undegraded":"2024-09-10T11:35:34.869379+0000","last_fullsized":"2024-09-10T11:35:34.869379+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:10.073748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869078+0000","last_change":"2024-09-10T11:35:32.876569+0000","last_active":"2024-09-10T11:35:34.869078+0000","last_peered":"2024-09-10T11:35:34.869078+0000","last_clean":"2024-09-10T11:35:34.869078+0000","last_became_active":"2024-09-10T11:35:32.876318+0000","last_became_peered":"2024-09-10T11:35:32.876318+0000","last_unstale":"2024-09-10T11:35:34.869078+0000","last_undegraded":"2024-09-10T11:35:34.869078+0000","last_fullsized":"2024-09-10T11:35:34.869078+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:40.045819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888533+0000","last_change":"2024-09-10T11:35:33.878891+0000","last_active":"2024-09-10T11:35:34.888533+0000","last_peered":"2024-09-10T11:35:34.888533+0000","last_clean":"2024-09-10T11:35:34.888533+0000","last_became_active":"2024-09-10T11:35:33.878640+0000","last_became_peered":"2024-09-10T11:35:33.878640+0000","last_unstale":"2024-09-10T11:35:34.888533+0000","last_undegraded":"2024-09-10T11:35:34.888533+0000","last_fullsized":"2024-09-10T11:35:34.888533+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:54.669341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869122+0000","last_change":"2024-09-10T11:35:32.873937+0000","last_active":"2024-09-10T11:35:34.869122+0000","last_peered":"2024-09-10T11:35:34.869122+0000","last_clean":"2024-09-10T11:35:34.869122+0000","last_became_active":"2024-09-10T11:35:32.872794+0000","last_became_peered":"2024-09-10T11:35:32.872794+0000","last_unstale":"2024-09-10T11:35:34.869122+0000","last_undegraded":"2024-09-10T11:35:34.869122+0000","last_fullsized":"2024-09-10T11:35:34.869122+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:56.724660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000730+0000","last_change":"2024-09-10T11:35:33.884700+0000","last_active":"2024-09-10T11:35:35.000730+0000","last_peered":"2024-09-10T11:35:35.000730+0000","last_clean":"2024-09-10T11:35:35.000730+0000","last_became_active":"2024-09-10T11:35:33.884047+0000","last_became_peered":"2024-09-10T11:35:33.884047+0000","last_unstale":"2024-09-10T11:35:35.000730+0000","last_undegraded":"2024-09-10T11:35:35.000730+0000","last_fullsized":"2024-09-10T11:35:35.000730+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:38.642681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888572+0000","last_change":"2024-09-10T11:35:32.878360+0000","last_active":"2024-09-10T11:35:34.888572+0000","last_peered":"2024-09-10T11:35:34.888572+0000","last_clean":"2024-09-10T11:35:34.888572+0000","last_became_active":"2024-09-10T11:35:32.878121+0000","last_became_peered":"2024-09-10T11:35:32.878121+0000","last_unstale":"2024-09-10T11:35:34.888572+0000","last_undegraded":"2024-09-10T11:35:34.888572+0000","last_fullsized":"2024-09-10T11:35:34.888572+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:51.267662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888402+0000","last_change":"2024-09-10T11:35:33.880662+0000","last_active":"2024-09-10T11:35:34.888402+0000","last_peered":"2024-09-10T11:35:34.888402+0000","last_clean":"2024-09-10T11:35:34.888402+0000","last_became_active":"2024-09-10T11:35:33.880261+0000","last_became_peered":"2024-09-10T11:35:33.880261+0000","last_unstale":"2024-09-10T11:35:34.888402+0000","last_undegraded":"2024-09-10T11:35:34.888402+0000","last_fullsized":"2024-09-10T11:35:34.888402+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:45.665424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000988+0000","last_change":"2024-09-10T11:35:32.881414+0000","last_active":"2024-09-10T11:35:35.000988+0000","last_peered":"2024-09-10T11:35:35.000988+0000","last_clean":"2024-09-10T11:35:35.000988+0000","last_became_active":"2024-09-10T11:35:32.880678+0000","last_became_peered":"2024-09-10T11:35:32.880678+0000","last_unstale":"2024-09-10T11:35:35.000988+0000","last_undegraded":"2024-09-10T11:35:35.000988+0000","last_fullsized":"2024-09-10T11:35:35.000988+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:09.710515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000491+0000","last_change":"2024-09-10T11:35:33.880514+0000","last_active":"2024-09-10T11:35:35.000491+0000","last_peered":"2024-09-10T11:35:35.000491+0000","last_clean":"2024-09-10T11:35:35.000491+0000","last_became_active":"2024-09-10T11:35:33.879971+0000","last_became_peered":"2024-09-10T11:35:33.879971+0000","last_unstale":"2024-09-10T11:35:35.000491+0000","last_undegraded":"2024-09-10T11:35:35.000491+0000","last_fullsized":"2024-09-10T11:35:35.000491+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:52:59.494596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870850+0000","last_change":"2024-09-10T11:35:32.879436+0000","last_active":"2024-09-10T11:35:34.870850+0000","last_peered":"2024-09-10T11:35:34.870850+0000","last_clean":"2024-09-10T11:35:34.870850+0000","last_became_active":"2024-09-10T11:35:32.878826+0000","last_became_peered":"2024-09-10T11:35:32.878826+0000","last_unstale":"2024-09-10T11:35:34.870850+0000","last_undegraded":"2024-09-10T11:35:34.870850+0000","last_fullsized":"2024-09-10T11:35:34.870850+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:48.551774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000468+0000","last_change":"2024-09-10T11:35:33.882414+0000","last_active":"2024-09-10T11:35:35.000468+0000","last_peered":"2024-09-10T11:35:35.000468+0000","last_clean":"2024-09-10T11:35:35.000468+0000","last_became_active":"2024-09-10T11:35:33.882198+0000","last_became_peered":"2024-09-10T11:35:33.882198+0000","last_unstale":"2024-09-10T11:35:35.000468+0000","last_undegraded":"2024-09-10T11:35:35.000468+0000","last_fullsized":"2024-09-10T11:35:35.000468+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:11.589259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000194+0000","last_change":"2024-09-10T11:35:32.873498+0000","last_active":"2024-09-10T11:35:35.000194+0000","last_peered":"2024-09-10T11:35:35.000194+0000","last_clean":"2024-09-10T11:35:35.000194+0000","last_became_active":"2024-09-10T11:35:32.873252+0000","last_became_peered":"2024-09-10T11:35:32.873252+0000","last_unstale":"2024-09-10T11:35:35.000194+0000","last_undegraded":"2024-09-10T11:35:35.000194+0000","last_fullsized":"2024-09-10T11:35:35.000194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:19:16.885741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.889869+0000","last_change":"2024-09-10T11:35:33.882902+0000","last_active":"2024-09-10T11:35:34.889869+0000","last_peered":"2024-09-10T11:35:34.889869+0000","last_clean":"2024-09-10T11:35:34.889869+0000","last_became_active":"2024-09-10T11:35:33.882658+0000","last_became_peered":"2024-09-10T11:35:33.882658+0000","last_unstale":"2024-09-10T11:35:34.889869+0000","last_undegraded":"2024-09-10T11:35:34.889869+0000","last_fullsized":"2024-09-10T11:35:34.889869+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:38.429844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000094+0000","last_change":"2024-09-10T11:35:32.881226+0000","last_active":"2024-09-10T11:35:35.000094+0000","last_peered":"2024-09-10T11:35:35.000094+0000","last_clean":"2024-09-10T11:35:35.000094+0000","last_became_active":"2024-09-10T11:35:32.880772+0000","last_became_peered":"2024-09-10T11:35:32.880772+0000","last_unstale":"2024-09-10T11:35:35.000094+0000","last_undegraded":"2024-09-10T11:35:35.000094+0000","last_fullsized":"2024-09-10T11:35:35.000094+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:41.263553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888678+0000","last_change":"2024-09-10T11:35:33.876728+0000","last_active":"2024-09-10T11:35:34.888678+0000","last_peered":"2024-09-10T11:35:34.888678+0000","last_clean":"2024-09-10T11:35:34.888678+0000","last_became_active":"2024-09-10T11:35:33.876442+0000","last_became_peered":"2024-09-10T11:35:33.876442+0000","last_unstale":"2024-09-10T11:35:34.888678+0000","last_undegraded":"2024-09-10T11:35:34.888678+0000","last_fullsized":"2024-09-10T11:35:34.888678+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:03.306411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":20,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870272+0000","last_change":"2024-09-10T11:35:32.872121+0000","last_active":"2024-09-10T11:35:34.870272+0000","last_peered":"2024-09-10T11:35:34.870272+0000","last_clean":"2024-09-10T11:35:34.870272+0000","last_became_active":"2024-09-10T11:35:32.871573+0000","last_became_peered":"2024-09-10T11:35:32.871573+0000","last_unstale":"2024-09-10T11:35:34.870272+0000","last_undegraded":"2024-09-10T11:35:34.870272+0000","last_fullsized":"2024-09-10T11:35:34.870272+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:02.801824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000446+0000","last_change":"2024-09-10T11:35:33.873742+0000","last_active":"2024-09-10T11:35:35.000446+0000","last_peered":"2024-09-10T11:35:35.000446+0000","last_clean":"2024-09-10T11:35:35.000446+0000","last_became_active":"2024-09-10T11:35:33.873003+0000","last_became_peered":"2024-09-10T11:35:33.873003+0000","last_unstale":"2024-09-10T11:35:35.000446+0000","last_undegraded":"2024-09-10T11:35:35.000446+0000","last_fullsized":"2024-09-10T11:35:35.000446+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:03.089349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.001236+0000","last_change":"2024-09-10T11:35:32.880271+0000","last_active":"2024-09-10T11:35:35.001236+0000","last_peered":"2024-09-10T11:35:35.001236+0000","last_clean":"2024-09-10T11:35:35.001236+0000","last_became_active":"2024-09-10T11:35:32.878429+0000","last_became_peered":"2024-09-10T11:35:32.878429+0000","last_unstale":"2024-09-10T11:35:35.001236+0000","last_undegraded":"2024-09-10T11:35:35.001236+0000","last_fullsized":"2024-09-10T11:35:35.001236+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:37.339641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870881+0000","last_change":"2024-09-10T11:35:33.883809+0000","last_active":"2024-09-10T11:35:34.870881+0000","last_peered":"2024-09-10T11:35:34.870881+0000","last_clean":"2024-09-10T11:35:34.870881+0000","last_became_active":"2024-09-10T11:35:33.883155+0000","last_became_peered":"2024-09-10T11:35:33.883155+0000","last_unstale":"2024-09-10T11:35:34.870881+0000","last_undegraded":"2024-09-10T11:35:34.870881+0000","last_fullsized":"2024-09-10T11:35:34.870881+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:23.755402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.868965+0000","last_change":"2024-09-10T11:35:32.872791+0000","last_active":"2024-09-10T11:35:34.868965+0000","last_peered":"2024-09-10T11:35:34.868965+0000","last_clean":"2024-09-10T11:35:34.868965+0000","last_became_active":"2024-09-10T11:35:32.872561+0000","last_became_peered":"2024-09-10T11:35:32.872561+0000","last_unstale":"2024-09-10T11:35:34.868965+0000","last_undegraded":"2024-09-10T11:35:34.868965+0000","last_fullsized":"2024-09-10T11:35:34.868965+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:42.830780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870868+0000","last_change":"2024-09-10T11:35:33.882368+0000","last_active":"2024-09-10T11:35:34.870868+0000","last_peered":"2024-09-10T11:35:34.870868+0000","last_clean":"2024-09-10T11:35:34.870868+0000","last_became_active":"2024-09-10T11:35:33.882068+0000","last_became_peered":"2024-09-10T11:35:33.882068+0000","last_unstale":"2024-09-10T11:35:34.870868+0000","last_undegraded":"2024-09-10T11:35:34.870868+0000","last_fullsized":"2024-09-10T11:35:34.870868+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:42.752700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":22,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869425+0000","last_change":"2024-09-10T11:35:32.873403+0000","last_active":"2024-09-10T11:35:34.869425+0000","last_peered":"2024-09-10T11:35:34.869425+0000","last_clean":"2024-09-10T11:35:34.869425+0000","last_became_active":"2024-09-10T11:35:32.872432+0000","last_became_peered":"2024-09-10T11:35:32.872432+0000","last_unstale":"2024-09-10T11:35:34.869425+0000","last_undegraded":"2024-09-10T11:35:34.869425+0000","last_fullsized":"2024-09-10T11:35:34.869425+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:09:14.131421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870384+0000","last_change":"2024-09-10T11:35:33.874820+0000","last_active":"2024-09-10T11:35:34.870384+0000","last_peered":"2024-09-10T11:35:34.870384+0000","last_clean":"2024-09-10T11:35:34.870384+0000","last_became_active":"2024-09-10T11:35:33.873125+0000","last_became_peered":"2024-09-10T11:35:33.873125+0000","last_unstale":"2024-09-10T11:35:34.870384+0000","last_undegraded":"2024-09-10T11:35:34.870384+0000","last_fullsized":"2024-09-10T11:35:34.870384+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:08.154655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.889936+0000","last_change":"2024-09-10T11:35:32.880016+0000","last_active":"2024-09-10T11:35:34.889936+0000","last_peered":"2024-09-10T11:35:34.889936+0000","last_clean":"2024-09-10T11:35:34.889936+0000","last_became_active":"2024-09-10T11:35:32.879737+0000","last_became_peered":"2024-09-10T11:35:32.879737+0000","last_unstale":"2024-09-10T11:35:34.889936+0000","last_undegraded":"2024-09-10T11:35:34.889936+0000","last_fullsized":"2024-09-10T11:35:34.889936+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:47.485611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869333+0000","last_change":"2024-09-10T11:35:33.879412+0000","last_active":"2024-09-10T11:35:34.869333+0000","last_peered":"2024-09-10T11:35:34.869333+0000","last_clean":"2024-09-10T11:35:34.869333+0000","last_became_active":"2024-09-10T11:35:33.879068+0000","last_became_peered":"2024-09-10T11:35:33.879068+0000","last_unstale":"2024-09-10T11:35:34.869333+0000","last_undegraded":"2024-09-10T11:35:34.869333+0000","last_fullsized":"2024-09-10T11:35:34.869333+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:40:29.152159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.001348+0000","last_change":"2024-09-10T11:35:32.880446+0000","last_active":"2024-09-10T11:35:35.001348+0000","last_peered":"2024-09-10T11:35:35.001348+0000","last_clean":"2024-09-10T11:35:35.001348+0000","last_became_active":"2024-09-10T11:35:32.878749+0000","last_became_peered":"2024-09-10T11:35:32.878749+0000","last_unstale":"2024-09-10T11:35:35.001348+0000","last_undegraded":"2024-09-10T11:35:35.001348+0000","last_fullsized":"2024-09-10T11:35:35.001348+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:03.836607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888634+0000","last_change":"2024-09-10T11:35:33.871093+0000","last_active":"2024-09-10T11:35:34.888634+0000","last_peered":"2024-09-10T11:35:34.888634+0000","last_clean":"2024-09-10T11:35:34.888634+0000","last_became_active":"2024-09-10T11:35:33.870956+0000","last_became_peered":"2024-09-10T11:35:33.870956+0000","last_unstale":"2024-09-10T11:35:34.888634+0000","last_undegraded":"2024-09-10T11:35:34.888634+0000","last_fullsized":"2024-09-10T11:35:34.888634+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:51.284019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.001083+0000","last_change":"2024-09-10T11:35:32.876822+0000","last_active":"2024-09-10T11:35:35.001083+0000","last_peered":"2024-09-10T11:35:35.001083+0000","last_clean":"2024-09-10T11:35:35.001083+0000","last_became_active":"2024-09-10T11:35:32.876628+0000","last_became_peered":"2024-09-10T11:35:32.876628+0000","last_unstale":"2024-09-10T11:35:35.001083+0000","last_undegraded":"2024-09-10T11:35:35.001083+0000","last_fullsized":"2024-09-10T11:35:35.001083+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:31:12.436291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000656+0000","last_change":"2024-09-10T11:35:33.878637+0000","last_active":"2024-09-10T11:35:35.000656+0000","last_peered":"2024-09-10T11:35:35.000656+0000","last_clean":"2024-09-10T11:35:35.000656+0000","last_became_active":"2024-09-10T11:35:33.877877+0000","last_became_peered":"2024-09-10T11:35:33.877877+0000","last_unstale":"2024-09-10T11:35:35.000656+0000","last_undegraded":"2024-09-10T11:35:35.000656+0000","last_fullsized":"2024-09-10T11:35:35.000656+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:44.365596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000156+0000","last_change":"2024-09-10T11:35:32.874486+0000","last_active":"2024-09-10T11:35:35.000156+0000","last_peered":"2024-09-10T11:35:35.000156+0000","last_clean":"2024-09-10T11:35:35.000156+0000","last_became_active":"2024-09-10T11:35:32.874321+0000","last_became_peered":"2024-09-10T11:35:32.874321+0000","last_unstale":"2024-09-10T11:35:35.000156+0000","last_undegraded":"2024-09-10T11:35:35.000156+0000","last_fullsized":"2024-09-10T11:35:35.000156+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:04.718702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869035+0000","last_change":"2024-09-10T11:35:33.881732+0000","last_active":"2024-09-10T11:35:34.869035+0000","last_peered":"2024-09-10T11:35:34.869035+0000","last_clean":"2024-09-10T11:35:34.869035+0000","last_became_active":"2024-09-10T11:35:33.881527+0000","last_became_peered":"2024-09-10T11:35:33.881527+0000","last_unstale":"2024-09-10T11:35:34.869035+0000","last_undegraded":"2024-09-10T11:35:34.869035+0000","last_fullsized":"2024-09-10T11:35:34.869035+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:28.999486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000236+0000","last_change":"2024-09-10T11:35:32.875247+0000","last_active":"2024-09-10T11:35:35.000236+0000","last_peered":"2024-09-10T11:35:35.000236+0000","last_clean":"2024-09-10T11:35:35.000236+0000","last_became_active":"2024-09-10T11:35:32.874999+0000","last_became_peered":"2024-09-10T11:35:32.874999+0000","last_unstale":"2024-09-10T11:35:35.000236+0000","last_undegraded":"2024-09-10T11:35:35.000236+0000","last_fullsized":"2024-09-10T11:35:35.000236+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:18.098455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869365+0000","last_change":"2024-09-10T11:35:33.886066+0000","last_active":"2024-09-10T11:35:34.869365+0000","last_peered":"2024-09-10T11:35:34.869365+0000","last_clean":"2024-09-10T11:35:34.869365+0000","last_became_active":"2024-09-10T11:35:33.885686+0000","last_became_peered":"2024-09-10T11:35:33.885686+0000","last_unstale":"2024-09-10T11:35:34.869365+0000","last_undegraded":"2024-09-10T11:35:34.869365+0000","last_fullsized":"2024-09-10T11:35:34.869365+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:22.458696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869371+0000","last_change":"2024-09-10T11:35:32.876222+0000","last_active":"2024-09-10T11:35:34.869371+0000","last_peered":"2024-09-10T11:35:34.869371+0000","last_clean":"2024-09-10T11:35:34.869371+0000","last_became_active":"2024-09-10T11:35:32.875083+0000","last_became_peered":"2024-09-10T11:35:32.875083+0000","last_unstale":"2024-09-10T11:35:34.869371+0000","last_undegraded":"2024-09-10T11:35:34.869371+0000","last_fullsized":"2024-09-10T11:35:34.869371+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:26:22.467387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000818+0000","last_change":"2024-09-10T11:35:33.878446+0000","last_active":"2024-09-10T11:35:35.000818+0000","last_peered":"2024-09-10T11:35:35.000818+0000","last_clean":"2024-09-10T11:35:35.000818+0000","last_became_active":"2024-09-10T11:35:33.877524+0000","last_became_peered":"2024-09-10T11:35:33.877524+0000","last_unstale":"2024-09-10T11:35:35.000818+0000","last_undegraded":"2024-09-10T11:35:35.000818+0000","last_fullsized":"2024-09-10T11:35:35.000818+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:23.777345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.871099+0000","last_change":"2024-09-10T11:35:32.876899+0000","last_active":"2024-09-10T11:35:34.871099+0000","last_peered":"2024-09-10T11:35:34.871099+0000","last_clean":"2024-09-10T11:35:34.871099+0000","last_became_active":"2024-09-10T11:35:32.876659+0000","last_became_peered":"2024-09-10T11:35:32.876659+0000","last_unstale":"2024-09-10T11:35:34.871099+0000","last_undegraded":"2024-09-10T11:35:34.871099+0000","last_fullsized":"2024-09-10T11:35:34.871099+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:01.176502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000867+0000","last_change":"2024-09-10T11:35:33.884897+0000","last_active":"2024-09-10T11:35:35.000867+0000","last_peered":"2024-09-10T11:35:35.000867+0000","last_clean":"2024-09-10T11:35:35.000867+0000","last_became_active":"2024-09-10T11:35:33.884424+0000","last_became_peered":"2024-09-10T11:35:33.884424+0000","last_unstale":"2024-09-10T11:35:35.000867+0000","last_undegraded":"2024-09-10T11:35:35.000867+0000","last_fullsized":"2024-09-10T11:35:35.000867+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:56.344785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870961+0000","last_change":"2024-09-10T11:35:32.871297+0000","last_active":"2024-09-10T11:35:34.870961+0000","last_peered":"2024-09-10T11:35:34.870961+0000","last_clean":"2024-09-10T11:35:34.870961+0000","last_became_active":"2024-09-10T11:35:32.871047+0000","last_became_peered":"2024-09-10T11:35:32.871047+0000","last_unstale":"2024-09-10T11:35:34.870961+0000","last_undegraded":"2024-09-10T11:35:34.870961+0000","last_fullsized":"2024-09-10T11:35:34.870961+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:01.614186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000149+0000","last_change":"2024-09-10T11:35:33.875257+0000","last_active":"2024-09-10T11:35:35.000149+0000","last_peered":"2024-09-10T11:35:35.000149+0000","last_clean":"2024-09-10T11:35:35.000149+0000","last_became_active":"2024-09-10T11:35:33.874933+0000","last_became_peered":"2024-09-10T11:35:33.874933+0000","last_unstale":"2024-09-10T11:35:35.000149+0000","last_undegraded":"2024-09-10T11:35:35.000149+0000","last_fullsized":"2024-09-10T11:35:35.000149+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:01.588513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.871005+0000","last_change":"2024-09-10T11:35:32.873272+0000","last_active":"2024-09-10T11:35:34.871005+0000","last_peered":"2024-09-10T11:35:34.871005+0000","last_clean":"2024-09-10T11:35:34.871005+0000","last_became_active":"2024-09-10T11:35:32.872538+0000","last_became_peered":"2024-09-10T11:35:32.872538+0000","last_unstale":"2024-09-10T11:35:34.871005+0000","last_undegraded":"2024-09-10T11:35:34.871005+0000","last_fullsized":"2024-09-10T11:35:34.871005+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:51.011554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890066+0000","last_change":"2024-09-10T11:35:33.882954+0000","last_active":"2024-09-10T11:35:34.890066+0000","last_peered":"2024-09-10T11:35:34.890066+0000","last_clean":"2024-09-10T11:35:34.890066+0000","last_became_active":"2024-09-10T11:35:33.882688+0000","last_became_peered":"2024-09-10T11:35:33.882688+0000","last_unstale":"2024-09-10T11:35:34.890066+0000","last_undegraded":"2024-09-10T11:35:34.890066+0000","last_fullsized":"2024-09-10T11:35:34.890066+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:05.768616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"138'1","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890113+0000","last_change":"2024-09-10T11:35:32.876892+0000","last_active":"2024-09-10T11:35:34.890113+0000","last_peered":"2024-09-10T11:35:34.890113+0000","last_clean":"2024-09-10T11:35:34.890113+0000","last_became_active":"2024-09-10T11:35:32.876652+0000","last_became_peered":"2024-09-10T11:35:32.876652+0000","last_unstale":"2024-09-10T11:35:34.890113+0000","last_undegraded":"2024-09-10T11:35:34.890113+0000","last_fullsized":"2024-09-10T11:35:34.890113+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:16:25.979136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870529+0000","last_change":"2024-09-10T11:35:33.875138+0000","last_active":"2024-09-10T11:35:34.870529+0000","last_peered":"2024-09-10T11:35:34.870529+0000","last_clean":"2024-09-10T11:35:34.870529+0000","last_became_active":"2024-09-10T11:35:33.873916+0000","last_became_peered":"2024-09-10T11:35:33.873916+0000","last_unstale":"2024-09-10T11:35:34.870529+0000","last_undegraded":"2024-09-10T11:35:34.870529+0000","last_fullsized":"2024-09-10T11:35:34.870529+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:26.934668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.870385+0000","last_change":"2024-09-10T11:35:32.871814+0000","last_active":"2024-09-10T11:35:34.870385+0000","last_peered":"2024-09-10T11:35:34.870385+0000","last_clean":"2024-09-10T11:35:34.870385+0000","last_became_active":"2024-09-10T11:35:32.871263+0000","last_became_peered":"2024-09-10T11:35:32.871263+0000","last_unstale":"2024-09-10T11:35:34.870385+0000","last_undegraded":"2024-09-10T11:35:34.870385+0000","last_fullsized":"2024-09-10T11:35:34.870385+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:00.176235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000139+0000","last_change":"2024-09-10T11:35:33.880769+0000","last_active":"2024-09-10T11:35:35.000139+0000","last_peered":"2024-09-10T11:35:35.000139+0000","last_clean":"2024-09-10T11:35:35.000139+0000","last_became_active":"2024-09-10T11:35:33.880078+0000","last_became_peered":"2024-09-10T11:35:33.880078+0000","last_unstale":"2024-09-10T11:35:35.000139+0000","last_undegraded":"2024-09-10T11:35:35.000139+0000","last_fullsized":"2024-09-10T11:35:35.000139+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:10.346142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.871190+0000","last_change":"2024-09-10T11:35:32.877914+0000","last_active":"2024-09-10T11:35:34.871190+0000","last_peered":"2024-09-10T11:35:34.871190+0000","last_clean":"2024-09-10T11:35:34.871190+0000","last_became_active":"2024-09-10T11:35:32.877237+0000","last_became_peered":"2024-09-10T11:35:32.877237+0000","last_unstale":"2024-09-10T11:35:34.871190+0000","last_undegraded":"2024-09-10T11:35:34.871190+0000","last_fullsized":"2024-09-10T11:35:34.871190+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:42:24.182779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869032+0000","last_change":"2024-09-10T11:35:33.878914+0000","last_active":"2024-09-10T11:35:34.869032+0000","last_peered":"2024-09-10T11:35:34.869032+0000","last_clean":"2024-09-10T11:35:34.869032+0000","last_became_active":"2024-09-10T11:35:33.878692+0000","last_became_peered":"2024-09-10T11:35:33.878692+0000","last_unstale":"2024-09-10T11:35:34.869032+0000","last_undegraded":"2024-09-10T11:35:34.869032+0000","last_fullsized":"2024-09-10T11:35:34.869032+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:54.695796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.888570+0000","last_change":"2024-09-10T11:35:32.875745+0000","last_active":"2024-09-10T11:35:34.888570+0000","last_peered":"2024-09-10T11:35:34.888570+0000","last_clean":"2024-09-10T11:35:34.888570+0000","last_became_active":"2024-09-10T11:35:32.875389+0000","last_became_peered":"2024-09-10T11:35:32.875389+0000","last_unstale":"2024-09-10T11:35:34.888570+0000","last_undegraded":"2024-09-10T11:35:34.888570+0000","last_fullsized":"2024-09-10T11:35:34.888570+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:23:33.306462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000974+0000","last_change":"2024-09-10T11:35:33.878812+0000","last_active":"2024-09-10T11:35:35.000974+0000","last_peered":"2024-09-10T11:35:35.000974+0000","last_clean":"2024-09-10T11:35:35.000974+0000","last_became_active":"2024-09-10T11:35:33.878179+0000","last_became_peered":"2024-09-10T11:35:33.878179+0000","last_unstale":"2024-09-10T11:35:35.000974+0000","last_undegraded":"2024-09-10T11:35:35.000974+0000","last_fullsized":"2024-09-10T11:35:35.000974+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:48.444878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000246+0000","last_change":"2024-09-10T11:35:32.874061+0000","last_active":"2024-09-10T11:35:35.000246+0000","last_peered":"2024-09-10T11:35:35.000246+0000","last_clean":"2024-09-10T11:35:35.000246+0000","last_became_active":"2024-09-10T11:35:32.873819+0000","last_became_peered":"2024-09-10T11:35:32.873819+0000","last_unstale":"2024-09-10T11:35:35.000246+0000","last_undegraded":"2024-09-10T11:35:35.000246+0000","last_fullsized":"2024-09-10T11:35:35.000246+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:03.325739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000273+0000","last_change":"2024-09-10T11:35:33.880948+0000","last_active":"2024-09-10T11:35:35.000273+0000","last_peered":"2024-09-10T11:35:35.000273+0000","last_clean":"2024-09-10T11:35:35.000273+0000","last_became_active":"2024-09-10T11:35:33.880392+0000","last_became_peered":"2024-09-10T11:35:33.880392+0000","last_unstale":"2024-09-10T11:35:35.000273+0000","last_undegraded":"2024-09-10T11:35:35.000273+0000","last_fullsized":"2024-09-10T11:35:35.000273+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:44.221770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869530+0000","last_change":"2024-09-10T11:35:32.873052+0000","last_active":"2024-09-10T11:35:34.869530+0000","last_peered":"2024-09-10T11:35:34.869530+0000","last_clean":"2024-09-10T11:35:34.869530+0000","last_became_active":"2024-09-10T11:35:32.871102+0000","last_became_peered":"2024-09-10T11:35:32.871102+0000","last_unstale":"2024-09-10T11:35:34.869530+0000","last_undegraded":"2024-09-10T11:35:34.869530+0000","last_fullsized":"2024-09-10T11:35:34.869530+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:47.496355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890086+0000","last_change":"2024-09-10T11:35:33.875416+0000","last_active":"2024-09-10T11:35:34.890086+0000","last_peered":"2024-09-10T11:35:34.890086+0000","last_clean":"2024-09-10T11:35:34.890086+0000","last_became_active":"2024-09-10T11:35:33.875194+0000","last_became_peered":"2024-09-10T11:35:33.875194+0000","last_unstale":"2024-09-10T11:35:34.890086+0000","last_undegraded":"2024-09-10T11:35:34.890086+0000","last_fullsized":"2024-09-10T11:35:34.890086+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:37.637073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.890060+0000","last_change":"2024-09-10T11:35:32.877402+0000","last_active":"2024-09-10T11:35:34.890060+0000","last_peered":"2024-09-10T11:35:34.890060+0000","last_clean":"2024-09-10T11:35:34.890060+0000","last_became_active":"2024-09-10T11:35:32.877081+0000","last_became_peered":"2024-09-10T11:35:32.877081+0000","last_unstale":"2024-09-10T11:35:34.890060+0000","last_undegraded":"2024-09-10T11:35:34.890060+0000","last_fullsized":"2024-09-10T11:35:34.890060+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:06.471985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869606+0000","last_change":"2024-09-10T11:35:33.878556+0000","last_active":"2024-09-10T11:35:34.869606+0000","last_peered":"2024-09-10T11:35:34.869606+0000","last_clean":"2024-09-10T11:35:34.869606+0000","last_became_active":"2024-09-10T11:35:33.878269+0000","last_became_peered":"2024-09-10T11:35:33.878269+0000","last_unstale":"2024-09-10T11:35:34.869606+0000","last_undegraded":"2024-09-10T11:35:34.869606+0000","last_fullsized":"2024-09-10T11:35:34.869606+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:32.861773+0000","last_clean_scrub_stamp":"2024-09-10T11:35:32.861773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:27.004847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:35.000538+0000","last_change":"2024-09-10T11:35:32.873520+0000","last_active":"2024-09-10T11:35:35.000538+0000","last_peered":"2024-09-10T11:35:35.000538+0000","last_clean":"2024-09-10T11:35:35.000538+0000","last_became_active":"2024-09-10T11:35:32.873266+0000","last_became_peered":"2024-09-10T11:35:32.873266+0000","last_unstale":"2024-09-10T11:35:35.000538+0000","last_undegraded":"2024-09-10T11:35:35.000538+0000","last_fullsized":"2024-09-10T11:35:35.000538+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:35:31.857526+0000","last_clean_scrub_stamp":"2024-09-10T11:35:31.857526+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:32:47.796639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":409,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-10T11:35:34.869082+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:35:34.869082+0000","last_peered":"2024-09-10T11:35:34.869082+0000","last_clean":"2024-09-10T11:35:34.869082+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:35:34.869082+0000","last_undegraded":"2024-09-10T11:35:34.869082+0000","last_fullsized":"2024-09-10T11:35:34.869082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":748,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194564,"statfs":{"total":200000000,"available":199233883,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574915,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1323,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193989,"statfs":{"total":200000000,"available":198645159,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":748,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194564,"statfs":{"total":200000000,"available":199233917,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":746,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194566,"statfs":{"total":200000000,"available":199235619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199234981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1322,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193989,"statfs":{"total":200000000,"available":198645251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":747,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194565,"statfs":{"total":200000000,"available":199234967,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:35:37.146 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:35:37.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:35:37.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.287+0000 7f2643cba640 1 Processor -- start 2024-09-10T11:35:37.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.287+0000 7f2643cba640 1 -- start start 2024-09-10T11:35:37.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2643cba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f263c06ece0 0x7f263c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2643cba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f263c15c060 0x7f263c15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2643cba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c152e60 con 0x7f263c0d1200 2024-09-10T11:35:37.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2643cba640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f263c0d15e0 con 0x7f263c06ece0 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2643cba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f263c072680 con 0x7f263c15c060 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f263c06ece0 0x7f263c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f263c06ece0 0x7f263c073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37628/0 (socket says 172.21.15.99:37628) 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 -- 172.21.15.99:0/3477377535 learned_addr learned my addr 172.21.15.99:0/3477377535 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2642257640 1 --2- 172.21.15.99:0/3477377535 >> v2:172.21.15.196:3300/0 conn(0x7f263c15c060 0x7f263c15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 -- 172.21.15.99:0/3477377535 >> v2:172.21.15.196:3300/0 conn(0x7f263c15c060 msgr2=0x7f263c15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 --2- 172.21.15.99:0/3477377535 >> v2:172.21.15.196:3300/0 conn(0x7f263c15c060 0x7f263c15c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 -- 172.21.15.99:0/3477377535 >> v1:172.21.15.196:6789/0 conn(0x7f263c0d1200 legacy=0x7f263c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:37.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.288+0000 7f2641255640 1 -- 172.21.15.99:0/3477377535 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c072960 con 0x7f263c06ece0 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.289+0000 7f2641255640 1 --2- 172.21.15.99:0/3477377535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c073440 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f263c0cd570 tx=0x7f262c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a984cc4b74f06b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.289+0000 7f2640a54640 1 -- 172.21.15.99:0/3477377535 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c037070 con 0x7f263c06ece0 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.289+0000 7f2640a54640 1 -- 172.21.15.99:0/3477377535 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c032440 con 0x7f263c06ece0 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.289+0000 7f2640a54640 1 -- 172.21.15.99:0/3477377535 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c032810 con 0x7f263c06ece0 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 -- 172.21.15.99:0/3477377535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 msgr2=0x7f263c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 --2- 172.21.15.99:0/3477377535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c073440 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f263c0cd570 tx=0x7f262c02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 -- 172.21.15.99:0/3477377535 shutdown_connections 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 --2- 172.21.15.99:0/3477377535 >> v2:172.21.15.196:3300/0 conn(0x7f263c15c060 0x7f263c15c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 --2- 172.21.15.99:0/3477377535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c073440 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 -- 172.21.15.99:0/3477377535 >> 172.21.15.99:0/3477377535 conn(0x7f263c06bc10 msgr2=0x7f263c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:37.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 -- 172.21.15.99:0/3477377535 shutdown_connections 2024-09-10T11:35:37.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.290+0000 7f2643cba640 1 -- 172.21.15.99:0/3477377535 wait complete. 2024-09-10T11:35:37.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.291+0000 7f2643cba640 1 Processor -- start 2024-09-10T11:35:37.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.291+0000 7f2643cba640 1 -- start start 2024-09-10T11:35:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0d1200 0x7f263c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c15c060 0x7f263c1f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f263c15cd90 con 0x7f263c06ece0 2024-09-10T11:35:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f263c072f20 con 0x7f263c15c060 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f263c15ca30 con 0x7f263c0d1200 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37642/0 (socket says 172.21.15.99:37642) 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 -- 172.21.15.99:0/3088528549 learned_addr learned my addr 172.21.15.99:0/3088528549 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2642257640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c15c060 0x7f263c1f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641255640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0d1200 0x7f263c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 -- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0d1200 msgr2=0x7f263c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0d1200 0x7f263c154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 -- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c15c060 msgr2=0x7f263c1f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c15c060 0x7f263c1f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 -- 172.21.15.99:0/3088528549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c1f2910 con 0x7f263c06ece0 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2641a56640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f263800b420 tx=0x7f263800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638011830 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2638016070 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1f2bf0 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638011c10 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.292+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c1f31b0 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.293+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c073440 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.294+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f263801b460 con 0x7f263c06ece0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.294+0000 7f26327fc640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.295+0000 7f2641255640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.295+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f263808ab70 con 0x7f263c06ece0 2024-09-10T11:35:37.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.295+0000 7f2641255640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 0x7f262006e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.299+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2638059520 con 0x7f263c06ece0 2024-09-10T11:35:37.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.525+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f263c0761b0 con 0x7f263c06ece0 2024-09-10T11:35:37.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.527+0000 7f26327fc640 1 -- 172.21.15.99:0/3088528549 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f2638058e20 con 0x7f263c06ece0 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:35:37.529 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 msgr2=0x7f262006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 0x7f262006e2f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f262c034000 tx=0x7f262c02e000 comp rx=0 tx=0).stop 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 msgr2=0x7f263c155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f263800b420 tx=0x7f263800b8e0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 shutdown_connections 2024-09-10T11:35:37.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f262006be30 0x7f262006e2f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f263c15c060 0x7f263c1f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f263c0d1200 0x7f263c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 --2- 172.21.15.99:0/3088528549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f263c06ece0 0x7f263c155d40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 >> 172.21.15.99:0/3088528549 conn(0x7f263c06bc10 msgr2=0x7f263c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 shutdown_connections 2024-09-10T11:35:37.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.529+0000 7f2643cba640 1 -- 172.21.15.99:0/3088528549 wait complete. 2024-09-10T11:35:37.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:37.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.639+0000 7fa7e7c1e640 1 Processor -- start 2024-09-10T11:35:37.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.639+0000 7fa7e7c1e640 1 -- start start 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e7c1e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7e00d1200 0x7fa7e0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e7c1e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7e015c770 0x7fa7e006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e7c1e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e0152e60 con 0x7fa7e006e5d0 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e7c1e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e006ef80 con 0x7fa7e00d1200 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e7c1e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e00d15e0 con 0x7fa7e015c770 2024-09-10T11:35:37.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e59ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7e00d1200 0x7fa7e0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e51b9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa7e006e5d0 0x7fa7e015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46744/0 (socket says 172.21.15.99:46744) 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e51b9640 1 -- 172.21.15.99:0/1468577680 learned_addr learned my addr 172.21.15.99:0/1468577680 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e61bb640 1 --2- 172.21.15.99:0/1468577680 >> v2:172.21.15.196:3300/0 conn(0x7fa7e015c770 0x7fa7e006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e59ba640 1 -- 172.21.15.99:0/1468577680 >> v2:172.21.15.196:3300/0 conn(0x7fa7e015c770 msgr2=0x7fa7e006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/1468577680 >> v2:172.21.15.196:3300/0 conn(0x7fa7e015c770 0x7fa7e006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e59ba640 1 -- 172.21.15.99:0/1468577680 >> v1:172.21.15.99:6789/0 conn(0x7fa7e006e5d0 legacy=0x7fa7e015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e59ba640 1 -- 172.21.15.99:0/1468577680 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0072680 con 0x7fa7e00d1200 2024-09-10T11:35:37.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.640+0000 7fa7e49b8640 1 -- 172.21.15.99:0/1468577680 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3503661337 0 0) 0x7fa7e0152e60 con 0x7fa7e006e5d0 2024-09-10T11:35:37.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/1468577680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e00d1200 0x7fa7e0073440 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00720c0 tx=0x7fa7dc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf01f1b9d6d80f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e49b8640 1 -- 172.21.15.99:0/1468577680 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc007740 con 0x7fa7e00d1200 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e49b8640 1 -- 172.21.15.99:0/1468577680 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0078e0 con 0x7fa7e00d1200 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e49b8640 1 -- 172.21.15.99:0/1468577680 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc007cb0 con 0x7fa7e00d1200 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/1468577680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e00d1200 msgr2=0x7fa7e0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.641+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/1468577680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e00d1200 0x7fa7e0073440 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00720c0 tx=0x7fa7dc02a5f0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/1468577680 shutdown_connections 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/1468577680 >> v2:172.21.15.196:3300/0 conn(0x7fa7e015c770 0x7fa7e006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/1468577680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e00d1200 0x7fa7e0073440 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/1468577680 >> 172.21.15.99:0/1468577680 conn(0x7fa7e006bc10 msgr2=0x7fa7e006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/1468577680 shutdown_connections 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.642+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/1468577680 wait complete. 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 Processor -- start 2024-09-10T11:35:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 -- start start 2024-09-10T11:35:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 0x7fa7e01f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 0x7fa7e015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.643+0000 7fa7e7c1e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0072f20 con 0x7fa7e015c770 2024-09-10T11:35:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e7c1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e0072c40 con 0x7fa7e00d1200 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e7c1e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e015cd90 con 0x7fa7e006e5d0 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37660/0 (socket says 172.21.15.99:37660) 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 -- 172.21.15.99:0/3288232274 learned_addr learned my addr 172.21.15.99:0/3288232274 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 0x7fa7e01f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e51b9640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 0x7fa7e015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 -- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 msgr2=0x7fa7e01f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 0x7fa7e01f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 -- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 msgr2=0x7fa7e015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 0x7fa7e015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 -- 172.21.15.99:0/3288232274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e0141ec0 con 0x7fa7e015c770 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 0x7fa7e01f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e51b9640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 0x7fa7e015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e61bb640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800d800 tx=0x7fa7d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800b660 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d8019070 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800ba60 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e01421a0 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.644+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e0142760 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.645+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e0073440 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.646+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7d800bc00 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.646+0000 7fa7d67fc640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 0x7fa7c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.646+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7fa7d808b290 con 0x7fa7e015c770 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.646+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 0x7fa7c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.647+0000 7fa7e59ba640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 0x7fa7c406e320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc034000 tx=0x7fa7dc033040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:37.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.650+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7d8059c40 con 0x7fa7e015c770 2024-09-10T11:35:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.850+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7e00b4600 con 0x7fa7e015c770 2024-09-10T11:35:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.851+0000 7fa7d67fc640 1 -- 172.21.15.99:0/3288232274 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7fa7d8059540 con 0x7fa7e015c770 2024-09-10T11:35:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:37.854 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:35:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:35:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 msgr2=0x7fa7c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 0x7fa7c406e320 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc034000 tx=0x7fa7dc033040 comp rx=0 tx=0).stop 2024-09-10T11:35:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 msgr2=0x7fa7e015b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800d800 tx=0x7fa7d800dcc0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 shutdown_connections 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa7c406be60 0x7fa7c406e320 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7e015c770 0x7fa7e015b050 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7e00d1200 0x7fa7e015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 --2- 172.21.15.99:0/3288232274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7e006e5d0 0x7fa7e01f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 >> 172.21.15.99:0/3288232274 conn(0x7fa7e006bc10 msgr2=0x7fa7e00b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 shutdown_connections 2024-09-10T11:35:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:37.853+0000 7fa7e7c1e640 1 -- 172.21.15.99:0/3288232274 wait complete. 2024-09-10T11:35:38.269 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:35:33.202363+0000', 'modified': '2024-09-10T11:35:35.207223+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/3835517741', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6833', 'nonce': 3835517741}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3835517741}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6990, 'qdb_cluster': [6990]} 2024-09-10T11:35:38.270 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:35:38.270 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T11:35:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.372+0000 7f1cd8077640 1 Processor -- start 2024-09-10T11:35:38.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.372+0000 7f1cd8077640 1 -- start start 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd8077640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1cd00d1200 0x7f1cd0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd8077640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 0x7f1cd006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd8077640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd0152e60 con 0x7f1cd006e5d0 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd8077640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd006ef80 con 0x7f1cd00d1200 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd8077640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd00d15e0 con 0x7f1cd015c770 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1cd00d1200 0x7f1cd0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1cd00d1200 0x7f1cd0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37676/0 (socket says 172.21.15.99:37676) 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3261357223 learned_addr learned my addr 172.21.15.99:0/3261357223 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3261357223 >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 0x7f1cd006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3261357223 >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 msgr2=0x7f1cd006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3261357223 >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 0x7f1cd006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3261357223 >> v1:172.21.15.196:6789/0 conn(0x7f1cd006e5d0 legacy=0x7f1cd015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3261357223 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd0072680 con 0x7f1cd00d1200 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.373+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3261357223 >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 0x7f1cd006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.374+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3261357223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd00d1200 0x7f1cd0073440 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00720c0 tx=0x7f1ccc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5dc4669a9bd779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.374+0000 7f1cd4e11640 1 -- 172.21.15.99:0/3261357223 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc037070 con 0x7f1cd00d1200 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.374+0000 7f1cd4e11640 1 -- 172.21.15.99:0/3261357223 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc032440 con 0x7f1cd00d1200 2024-09-10T11:35:38.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.374+0000 7f1cd4e11640 1 -- 172.21.15.99:0/3261357223 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc032810 con 0x7f1cd00d1200 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 -- 172.21.15.99:0/3261357223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd00d1200 msgr2=0x7f1cd0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3261357223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd00d1200 0x7f1cd0073440 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00720c0 tx=0x7f1ccc02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 -- 172.21.15.99:0/3261357223 shutdown_connections 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3261357223 >> v2:172.21.15.196:3300/0 conn(0x7f1cd015c770 0x7f1cd006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3261357223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd00d1200 0x7f1cd0073440 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 -- 172.21.15.99:0/3261357223 >> 172.21.15.99:0/3261357223 conn(0x7f1cd006bc10 msgr2=0x7f1cd006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 -- 172.21.15.99:0/3261357223 shutdown_connections 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.375+0000 7f1cd8077640 1 -- 172.21.15.99:0/3261357223 wait complete. 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.376+0000 7f1cd8077640 1 Processor -- start 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.376+0000 7f1cd8077640 1 -- start start 2024-09-10T11:35:38.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cd00d1200 0x7f1cd01f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 0x7f1cd01e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0072f20 con 0x7f1cd006e5d0 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd0072c40 con 0x7f1cd015c770 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd015cd90 con 0x7f1cd00d1200 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37688/0 (socket says 172.21.15.99:37688) 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3794180960 learned_addr learned my addr 172.21.15.99:0/3794180960 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 0x7f1cd01e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5612640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cd00d1200 0x7f1cd01f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cd00d1200 msgr2=0x7f1cd01f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cd00d1200 0x7f1cd01f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 msgr2=0x7f1cd01e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 0x7f1cd01e6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 -- 172.21.15.99:0/3794180960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd01f2990 con 0x7f1cd006e5d0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 0x7f1cd01e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc000c00 tx=0x7f1ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cc67fc640 1 -- 172.21.15.99:0/3794180960 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc041020 con 0x7f1cd006e5d0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cc67fc640 1 -- 172.21.15.99:0/3794180960 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc030970 con 0x7f1cd006e5d0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cc67fc640 1 -- 172.21.15.99:0/3794180960 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc037040 con 0x7f1cd006e5d0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.377+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd01569d0 con 0x7f1cd006e5d0 2024-09-10T11:35:38.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.378+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd0156f90 con 0x7f1cd006e5d0 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.379+0000 7f1cc67fc640 1 -- 172.21.15.99:0/3794180960 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ccc044020 con 0x7f1cd006e5d0 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.379+0000 7f1cc67fc640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.380+0000 7f1cc67fc640 1 -- 172.21.15.99:0/3794180960 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f1ccc0ab930 con 0x7f1cd006e5d0 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.380+0000 7f1cd5612640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.380+0000 7f1cd5612640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 0x7f1cb406e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.380+0000 7f1cdae0d740 1 Processor -- start 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.380+0000 7f1cdae0d740 1 -- start start 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 0x5621e94736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621e93d5520 con 0x5621e9474a50 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621e937d3e0 con 0x5621e9470a80 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5621e93d14e0 con 0x5621e94732c0 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37702/0 (socket says 172.21.15.99:37702) 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd6614640 1 -- 172.21.15.99:0/1323024809 learned_addr learned my addr 172.21.15.99:0/1323024809 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd5612640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 0x5621e94736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd6614640 1 -- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 msgr2=0x5621e94b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.381+0000 7f1cd6614640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cd6614640 1 -- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 msgr2=0x5621e94736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cd6614640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 0x5621e94736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cd6614640 1 -- 172.21.15.99:0/1323024809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e93982f0 con 0x5621e9474a50 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cd5612640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 0x5621e94736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cd6614640 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e620 tx=0x7f1cc800eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48460f16be36742 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cb37fe640 1 -- 172.21.15.99:0/1323024809 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800ca70 con 0x5621e9474a50 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cb37fe640 1 -- 172.21.15.99:0/1323024809 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc800cc10 con 0x5621e9474a50 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cb37fe640 1 -- 172.21.15.99:0/1323024809 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800a410 con 0x5621e9474a50 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 -- 172.21.15.99:0/1323024809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 msgr2=0x5621e946d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e620 tx=0x7f1cc800eae0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 -- 172.21.15.99:0/1323024809 shutdown_connections 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e946d8b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9470a80 0x5621e94736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/1323024809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.382+0000 7f1cdae0d740 1 -- 172.21.15.99:0/1323024809 >> 172.21.15.99:0/1323024809 conn(0x5621e946fa80 msgr2=0x5621e94b7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.383+0000 7f1cdae0d740 1 -- 172.21.15.99:0/1323024809 shutdown_connections 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.383+0000 7f1cdae0d740 1 -- 172.21.15.99:0/1323024809 wait complete. 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.383+0000 7f1cdae0d740 1 -- start start 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.383+0000 7f1cdae0d740 1 Processor -- start 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e9496ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621e946ef20 con 0x5621e9497420 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621e935e450 con 0x5621e9474a50 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5621e94b9290 con 0x5621e94732c0 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cdae0d740 10 client.0 _ll_register_callbacks cb 0x5621e90e7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:35:38.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37712/0 (socket says 172.21.15.99:37712) 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 -- 172.21.15.99:0/3361363078 learned_addr learned my addr 172.21.15.99:0/3361363078 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd5612640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e9496ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 -- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 msgr2=0x5621e94b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 -- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 msgr2=0x5621e9496ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.384+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e9496ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cd6614640 1 -- 172.21.15.99:0/3361363078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e90e82d0 con 0x5621e9497420 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cd6614640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8010010 tx=0x7f1cc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b53ab80b6f2f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8027070 con 0x5621e9497420 2024-09-10T11:35:38.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f1cc8003da0 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc800cc10 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1cc8032050 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 10 client.7113 fetch_fsmap learned FSMap version 55 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5621e91b26b0 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc800cc10 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f1cc8035020 con 0x5621e9497420 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 10 client.7113 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 10 client.7113 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 20 client.7113 populate_metadata read hostname 'smithi099' 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e94bcea0 0x5621e94bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 4 client.7113 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.385+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 --> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e93e2c50 con 0x5621e94bcea0 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.386+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e94bcea0 0x5621e94bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.386+0000 7f1cb0ff9640 10 client.7113 ms_handle_connect on v2:172.21.15.196:6833/3835517741 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.386+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e94bcea0 0x5621e94bd280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cecb9e2978ce591b server_cookie=9ca3cd4319654c3b in_seq=0 out_seq=0 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/3361363078 <== mds.0 v2:172.21.15.196:6833/3835517741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621e93e2c50 con 0x5621e94bcea0 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cb0ff9640 10 client.7113 handle_command_reply: tid=0 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 client.7113 shutdown 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 msgr2=0x5621e94be3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8010010 tx=0x7f1cc8004ab0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 shutdown_connections 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e94be3e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e94bcea0 0x5621e94bd280 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e9496ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/3361363078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e94b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 >> 172.21.15.99:0/3361363078 conn(0x5621e9470a80 msgr2=0x5621e94b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cb0ff9640 0 client.7113 ms_handle_reset on v2:172.21.15.196:6833/3835517741 2024-09-10T11:35:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 shutdown_connections 2024-09-10T11:35:38.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 1 -- 172.21.15.99:0/3361363078 wait complete. 2024-09-10T11:35:38.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.387+0000 7f1cdae0d740 20 client.7113 trim_cache size 0 max 16384 2024-09-10T11:35:38.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.405+0000 7f1cdae0d740 1 Processor -- start 2024-09-10T11:35:38.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.405+0000 7f1cdae0d740 1 -- start start 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.405+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 0x5621e94bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.405+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 0x5621e9498260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd01fe100 con 0x5621e9474a50 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd015aff0 con 0x5621e94732c0 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd015b640 con 0x5621e9497420 2024-09-10T11:35:38.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37720/0 (socket says 172.21.15.99:37720) 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 -- 172.21.15.99:0/499502069 learned_addr learned my addr 172.21.15.99:0/499502069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 0x5621e94bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd6614640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 0x5621e9498260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 -- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 msgr2=0x5621e9498260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 0x5621e9498260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 -- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 msgr2=0x5621e94bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 0x5621e94bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 -- 172.21.15.99:0/499502069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e935bfb0 con 0x5621e9474a50 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 0x5621e94bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd6614640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 0x5621e9498260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cd5612640 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000da40 tx=0x7f1cc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf347c2920cec49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cb2ffd640 1 -- 172.21.15.99:0/499502069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0014c80 con 0x5621e9474a50 2024-09-10T11:35:38.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cb2ffd640 1 -- 172.21.15.99:0/499502069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc0027070 con 0x5621e9474a50 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.406+0000 7f1cb2ffd640 1 -- 172.21.15.99:0/499502069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0014e20 con 0x5621e9474a50 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- 172.21.15.99:0/499502069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 msgr2=0x5621e94bb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000da40 tx=0x7f1cc0013000 comp rx=0 tx=0).stop 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- 172.21.15.99:0/499502069 shutdown_connections 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e9497420 0x5621e9498260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9474a50 0x5621e94bb980 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/499502069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e94732c0 0x5621e94bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- 172.21.15.99:0/499502069 >> 172.21.15.99:0/499502069 conn(0x5621e9470a80 msgr2=0x5621e94c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- 172.21.15.99:0/499502069 shutdown_connections 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- 172.21.15.99:0/499502069 wait complete. 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 -- start start 2024-09-10T11:35:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.407+0000 7f1cdae0d740 1 Processor -- start 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e953de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e95207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5621e94b5a60 con 0x5621e9497420 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5621e949aca0 con 0x5621e9474a50 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x5621e94cba10 con 0x5621e94732c0 2024-09-10T11:35:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cdae0d740 10 client.0 _ll_register_callbacks cb 0x7f1cc800e440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cd6614640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37736/0 (socket says 172.21.15.99:37736) 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cd6614640 1 -- 172.21.15.99:0/489148460 learned_addr learned my addr 172.21.15.99:0/489148460 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cd5612640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e95207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.408+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e953de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 -- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 msgr2=0x5621e953de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e953de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 -- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 msgr2=0x5621e95207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e95207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 -- 172.21.15.99:0/489148460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e9520ce0 con 0x5621e9497420 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cd6614640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000f80 tx=0x7f1cc8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c70550693fd42fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:35:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc803b060 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f1cc80046b0 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc0027070 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1cc8027040 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 10 client.7119 fetch_fsmap learned FSMap version 55 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5621e94bce60 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc0027070 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f1cc8035040 con 0x5621e9497420 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 10 client.7119 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 10 client.7119 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 20 client.7119 populate_metadata read hostname 'smithi099' 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e9495bc0 0x5621e94ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 4 client.7119 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:35:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.409+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 --> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] -- command(tid 0: {"prefix": "status"}) -- 0x5621e93dcfa0 con 0x5621e9495bc0 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e9495bc0 0x5621e94ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cd5e13640 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e9495bc0 0x5621e94ca0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac25caea99c5650e server_cookie=e8e154d31f8e3f01 in_seq=0 out_seq=0 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cb0ff9640 10 client.7119 ms_handle_connect on v2:172.21.15.196:6833/3835517741 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cb0ff9640 1 -- 172.21.15.99:0/489148460 <== mds.0 v2:172.21.15.196:6833/3835517741 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5621e93dcfa0 con 0x5621e9495bc0 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cb0ff9640 10 client.7119 handle_command_reply: tid=0 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 client.7119 shutdown 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 msgr2=0x5621e953af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8000f80 tx=0x7f1cc8007a00 comp rx=0 tx=0).stop 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 shutdown_connections 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5621e9497420 0x5621e953af50 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:6833/3835517741,v1:172.21.15.196:6835/3835517741] conn(0x5621e9495bc0 0x5621e94ca0f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5621e9474a50 0x5621e95207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cb0ff9640 0 client.7119 ms_handle_reset on v2:172.21.15.196:6833/3835517741 2024-09-10T11:35:38.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 --2- 172.21.15.99:0/489148460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5621e94732c0 0x5621e953de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 >> 172.21.15.99:0/489148460 conn(0x5621e9470a80 msgr2=0x5621e94bb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 shutdown_connections 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.410+0000 7f1cdae0d740 1 -- 172.21.15.99:0/489148460 wait complete. 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cdae0d740 20 client.7119 trim_cache size 0 max 16384 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6990, 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:35:38.419 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5301013360000004, 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 55, 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 138, 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 138, 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 8.5557035900000002 2024-09-10T11:35:38.420 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 msgr2=0x7f1cb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 0x7f1cb406e2f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0006fd0 tx=0x7f1cc0008040 comp rx=0 tx=0).stop 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 msgr2=0x7f1cd0145190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc000c00 tx=0x7f1ccc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 shutdown_connections 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cd015c770 0x7f1cd01e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cd00d1200 0x7f1cd01f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 --2- 172.21.15.99:0/3794180960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cd006e5d0 0x7f1cd0145190 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 >> 172.21.15.99:0/3794180960 conn(0x7f1cd006bc10 msgr2=0x7f1cd015f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 shutdown_connections 2024-09-10T11:35:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.411+0000 7f1cd8077640 1 -- 172.21.15.99:0/3794180960 wait complete. 2024-09-10T11:35:38.424 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:35:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.520+0000 7ff4fc18d640 1 Processor -- start 2024-09-10T11:35:38.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.520+0000 7ff4fc18d640 1 -- start start 2024-09-10T11:35:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4fc18d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4f41610b0 0x7ff4f415e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4fc18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4fc18d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f40d1bd0 con 0x7ff4f41605c0 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4fc18d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f4075620 con 0x7ff4f41610b0 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4fc18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f40752b0 con 0x7ff4f40ae020 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4f9728640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4f41610b0 0x7ff4f415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4f9728640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4f41610b0 0x7ff4f415e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37746/0 (socket says 172.21.15.99:37746) 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.521+0000 7ff4f9728640 1 -- 172.21.15.99:0/3312103621 learned_addr learned my addr 172.21.15.99:0/3312103621 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4fa72a640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4f9728640 1 -- 172.21.15.99:0/3312103621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 msgr2=0x7ff4f415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4f9728640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4f9728640 1 -- 172.21.15.99:0/3312103621 >> v1:172.21.15.196:6789/0 conn(0x7ff4f41605c0 legacy=0x7ff4f41609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4f9728640 1 -- 172.21.15.99:0/3312103621 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f40d1eb0 con 0x7ff4f41610b0 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.522+0000 7ff4f9728640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41610b0 0x7ff4f415e200 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009870 tx=0x7ff4e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f8f6a3362fbc96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4f8f27640 1 -- 172.21.15.99:0/3312103621 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4037070 con 0x7ff4f41610b0 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4f8f27640 1 -- 172.21.15.99:0/3312103621 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e4032440 con 0x7ff4f41610b0 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4f8f27640 1 -- 172.21.15.99:0/3312103621 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4032810 con 0x7ff4f41610b0 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3312103621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41610b0 msgr2=0x7ff4f415e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41610b0 0x7ff4f415e200 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009870 tx=0x7ff4e402a270 comp rx=0 tx=0).stop 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3312103621 shutdown_connections 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f415e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3312103621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41610b0 0x7ff4f415e200 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3312103621 >> 172.21.15.99:0/3312103621 conn(0x7ff4f40730c0 msgr2=0x7ff4f40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3312103621 shutdown_connections 2024-09-10T11:35:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.523+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3312103621 wait complete. 2024-09-10T11:35:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.524+0000 7ff4fc18d640 1 Processor -- start 2024-09-10T11:35:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.524+0000 7ff4fc18d640 1 -- start start 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f41f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f41610b0 0x7ff4f40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4073e80 con 0x7ff4f41605c0 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4073d00 con 0x7ff4f41610b0 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fc18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f415ec80 con 0x7ff4f40ae020 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37762/0 (socket says 172.21.15.99:37762) 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 -- 172.21.15.99:0/3482740361 learned_addr learned my addr 172.21.15.99:0/3482740361 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4fa72a640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f41610b0 0x7ff4f40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9f29640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f41f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 -- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 msgr2=0x7ff4f41f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f41f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 -- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f41610b0 msgr2=0x7ff4f40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f41610b0 0x7ff4f40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.525+0000 7ff4f9728640 1 -- 172.21.15.99:0/3482740361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f40b5dd0 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4f9728640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4000c00 tx=0x7ff4e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4041020 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e40306e0 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f41f88d0 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4037040 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.526+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f41f8e90 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.527+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f415efb0 con 0x7ff4f41605c0 2024-09-10T11:35:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.527+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4e4044020 con 0x7ff4f41605c0 2024-09-10T11:35:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.528+0000 7ff4ea7fc640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 0x7ff4d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.528+0000 7ff4f9f29640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 0x7ff4d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.528+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7ff4e40ab620 con 0x7ff4f41605c0 2024-09-10T11:35:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.528+0000 7ff4f9f29640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 0x7ff4d806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0001640 tx=0x7ff4f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:38.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.532+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4e404b030 con 0x7ff4f41605c0 2024-09-10T11:35:38.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.729+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff4f415dc30 con 0x7ff4f41605c0 2024-09-10T11:35:38.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.730+0000 7ff4ea7fc640 1 -- 172.21.15.99:0/3482740361 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7ff4e4079fd0 con 0x7ff4f41605c0 2024-09-10T11:35:38.732 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 msgr2=0x7ff4d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 0x7ff4d806e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0001640 tx=0x7ff4f0009040 comp rx=0 tx=0).stop 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 msgr2=0x7ff4f40b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4000c00 tx=0x7ff4e40046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 shutdown_connections 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff4d806be30 0x7ff4d806e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f41610b0 0x7ff4f40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4f41605c0 0x7ff4f40b5740 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 --2- 172.21.15.99:0/3482740361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff4f40ae020 0x7ff4f41f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 >> 172.21.15.99:0/3482740361 conn(0x7ff4f40730c0 msgr2=0x7ff4f406dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 shutdown_connections 2024-09-10T11:35:38.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38.732+0000 7ff4fc18d640 1 -- 172.21.15.99:0/3482740361 wait complete. 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:35:38.745 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:38.746 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:35:38.761 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42510sec preferred_lft 42510sec 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:38.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:38.764 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:35:38.825 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:35:38.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:38.826 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:35:38.882 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:35:38.882 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:35:38.882 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:35:38.882 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:35:38.883 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:35:38.883 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:35:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:35:39.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:39.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.008 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:35:39.023 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:35:39.023 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:35:39.024 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:39.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:35:39.088 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:35:39.088 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:39.089 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:39.144 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:35:39.144 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:39.144 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:39.198 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:35:39.263 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:35:39.320 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:35:39.321 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:35:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:35:39.323 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:39.323 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:39.323 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:39.323 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:39.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.323 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:35:39.376 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:35:39.376 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:35:39.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:35:39.443 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:39.499 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:35:39.500 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:39.501 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:35:39.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.503 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:35:39.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:35:39.498+0000 7fa5a10f5480 -1 init, newargv = 0x557a6a86bf90 newargc=15 2024-09-10T11:35:39.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[55213]: starting ceph client 2024-09-10T11:35:39.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[55213]: starting fuse 2024-09-10T11:35:39.518 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:35:39.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:35:39.519 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:35:39.520 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:35:39.521 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:35:39.636 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.55213.asok 2024-09-10T11:35:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:39.646 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55213.asok 2024-09-10T11:35:39.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55213.asok status 2024-09-10T11:35:39.779 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:39.779 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:35:39.779 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:39.779 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:39.779 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "55213", 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7134, 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 7134 2024-09-10T11:35:39.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 207178722 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 207178722 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.7134 192.168.0.1:0/207178722", 2024-09-10T11:35:39.781 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/207178722", 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 55, 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 138, 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:35:39.782 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:39.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.788 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:35:39.788 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:39.803 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:35:39.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:39.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:39.804 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:35:39.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:35:40.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 Processor -- start 2024-09-10T11:35:40.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 -- start start 2024-09-10T11:35:40.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d0406ece0 0x7f9d04073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 0x7f9d0415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d04152e60 con 0x7f9d040d1200 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9d040d15e0 con 0x7f9d0406ece0 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.007+0000 7f9d08b59640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d04072680 con 0x7f9d0415c060 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d0406ece0 0x7f9d04073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d0406ece0 0x7f9d04073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37764/0 (socket says 172.21.15.99:37764) 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 -- 172.21.15.99:0/3332178830 learned_addr learned my addr 172.21.15.99:0/3332178830 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9d02d9d640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 0x7f9d0415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 -- 172.21.15.99:0/3332178830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 msgr2=0x7f9d0415c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 0x7f9d0415c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 -- 172.21.15.99:0/3332178830 >> v1:172.21.15.196:6789/0 conn(0x7f9d040d1200 legacy=0x7f9d0406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9cfbfff640 1 -- 172.21.15.99:0/3332178830 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d04072960 con 0x7f9d0406ece0 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.008+0000 7f9d02d9d640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 0x7f9d0415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9cfbfff640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406ece0 0x7f9d04073440 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009870 tx=0x7f9cec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5966db677de1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:40.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d01d9b640 1 -- 172.21.15.99:0/3332178830 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec037070 con 0x7f9d0406ece0 2024-09-10T11:35:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d01d9b640 1 -- 172.21.15.99:0/3332178830 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cec032440 con 0x7f9d0406ece0 2024-09-10T11:35:40.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d01d9b640 1 -- 172.21.15.99:0/3332178830 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec032810 con 0x7f9d0406ece0 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 -- 172.21.15.99:0/3332178830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406ece0 msgr2=0x7f9d04073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406ece0 0x7f9d04073440 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009870 tx=0x7f9cec02a180 comp rx=0 tx=0).stop 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 -- 172.21.15.99:0/3332178830 shutdown_connections 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0415c060 0x7f9d0415c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 --2- 172.21.15.99:0/3332178830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0406ece0 0x7f9d04073440 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 -- 172.21.15.99:0/3332178830 >> 172.21.15.99:0/3332178830 conn(0x7f9d0406bc10 msgr2=0x7f9d0406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:40.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 -- 172.21.15.99:0/3332178830 shutdown_connections 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.009+0000 7f9d08b59640 1 -- 172.21.15.99:0/3332178830 wait complete. 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 Processor -- start 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 -- start start 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0406ece0 0x7f9d04147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d040d1200 0x7f9d040c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0415cd90 con 0x7f9d0415c060 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d04072f20 con 0x7f9d040d1200 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.011+0000 7f9d08b59640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0415ca00 con 0x7f9d0406ece0 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37780/0 (socket says 172.21.15.99:37780) 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 -- 172.21.15.99:0/4171289866 learned_addr learned my addr 172.21.15.99:0/4171289866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:40.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9cfbfff640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d040d1200 0x7f9d040c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d0259c640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0406ece0 0x7f9d04147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 -- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0406ece0 msgr2=0x7f9d04147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0406ece0 0x7f9d04147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 -- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d040d1200 msgr2=0x7f9d040c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d040d1200 0x7f9d040c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 -- 172.21.15.99:0/4171289866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d040c6f10 con 0x7f9d0415c060 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d02d9d640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00cf20 tx=0x7f9cfc00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01a070 con 0x7f9d0415c060 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc00d840 con 0x7f9d0415c060 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d04147780 con 0x7f9d0415c060 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00dc30 con 0x7f9d0415c060 2024-09-10T11:35:40.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.012+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d0415a580 con 0x7f9d0415c060 2024-09-10T11:35:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.013+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d04073440 con 0x7f9d0415c060 2024-09-10T11:35:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.014+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9cfc01d070 con 0x7f9d0415c060 2024-09-10T11:35:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.014+0000 7f9cfaffd640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:40.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.014+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f9cfc08b5d0 con 0x7f9d0415c060 2024-09-10T11:35:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.014+0000 7f9d0259c640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:40.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.014+0000 7f9d0259c640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 0x7f9ce406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:40.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.018+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d04147780 con 0x7f9d0415c060 2024-09-10T11:35:40.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.215+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9d0406dd80 con 0x7f9d0415c060 2024-09-10T11:35:40.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.216+0000 7f9cfaffd640 1 -- 172.21.15.99:0/4171289866 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7f9d040c6f10 con 0x7f9d0415c060 2024-09-10T11:35:40.218 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:35:40.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 msgr2=0x7f9ce406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:40.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 0x7f9ce406e2f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4009060 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 msgr2=0x7f9d040c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc00cf20 tx=0x7f9cfc00a7d0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 shutdown_connections 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9ce406be30 0x7f9ce406e2f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d0415c060 0x7f9d040c6570 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d040d1200 0x7f9d040c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 --2- 172.21.15.99:0/4171289866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d0406ece0 0x7f9d04147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.217+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 >> 172.21.15.99:0/4171289866 conn(0x7f9d0406bc10 msgr2=0x7f9d040cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.218+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 shutdown_connections 2024-09-10T11:35:40.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:40.218+0000 7f9d08b59640 1 -- 172.21.15.99:0/4171289866 wait complete. 2024-09-10T11:35:40.230 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:35:40.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:35:40.231 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:40.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.231 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:35:40.251 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:35:40.251 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:35:40.251 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:35:40.251 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.251 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:35:40.252 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.253 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:35:40.254 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:40.254 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:35:40.254 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:35:40.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.254 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:35:40.319 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:35:40.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.322 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:35:40.380 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:35:40.380 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:35:40.380 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:35:40.380 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:35:40.380 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:35:40.381 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:35:40.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:35:40.509 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:40.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.515 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:35:40.531 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:35:40.531 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:35:40.531 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:40.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.532 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:35:40.597 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:35:40.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:40.597 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:40.653 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:35:40.653 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:35:40.653 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:40.708 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:35:40.774 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:40.829 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:35:40.830 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:35:40.831 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:40.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:40.832 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:35:40.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:35:40.885 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:35:40.928 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:35:40.952 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:35:41.008 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:35:41.009 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:41.010 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:35:41.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:41.011 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:35:41.012 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:35:41.008+0000 7f381651d480 -1 init, newargv = 0x56203e8da360 newargc=15 2024-09-10T11:35:41.012 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38177]: starting ceph client 2024-09-10T11:35:41.012 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38177]: starting fuse 2024-09-10T11:35:41.026 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:35:41.027 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:35:41.028 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:35:41.029 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:35:41.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.38177.asok 2024-09-10T11:35:41.147 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:35:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:35:41.154 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38177.asok 2024-09-10T11:35:41.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:41.155 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38177.asok status 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:35:41.287 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "38177", 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 5873, 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:35:41.288 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 5873 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3080045177 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:35:41.289 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3080045177 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.5873 192.168.0.1:0/3080045177", 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/3080045177", 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 55, 2024-09-10T11:35:41.290 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 138, 2024-09-10T11:35:41.291 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:35:41.291 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:35:41.291 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:35:41.291 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:35:41.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:41.296 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:35:41.296 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:41.312 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:35:41.312 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:41.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:41.312 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:35:41.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:41.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.484+0000 7f18c9015640 1 Processor -- start 2024-09-10T11:35:41.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 -- start start 2024-09-10T11:35:41.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c415ff70 con 0x7f18c415eb80 2024-09-10T11:35:41.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f18c40d30c0 con 0x7f18c41626c0 2024-09-10T11:35:41.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.485+0000 7f18c9015640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f18c40d52d0 con 0x7f18c40d4e40 2024-09-10T11:35:41.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c259c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c2d9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f18c415eb80 0x7f18c40d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:59656/0 (socket says 172.21.15.99:59656) 2024-09-10T11:35:41.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c259c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:37678/0 (socket says 172.21.15.99:37678) 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:36770/0 (socket says 172.21.15.99:36770) 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c2d9d640 1 -- 172.21.15.99:0/3061066537 learned_addr learned my addr 172.21.15.99:0/3061066537 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 -- 172.21.15.99:0/3061066537 >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 msgr2=0x7f18c40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 --2- 172.21.15.99:0/3061066537 >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 -- 172.21.15.99:0/3061066537 >> v1:172.21.15.196:6789/0 conn(0x7f18c415eb80 legacy=0x7f18c40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c1d9b640 1 -- 172.21.15.99:0/3061066537 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705058460 0 0) 0x7f18c415ff70 con 0x7f18c415eb80 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c359e640 1 -- 172.21.15.99:0/3061066537 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c40d1620 con 0x7f18c40d4e40 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.486+0000 7f18c259c640 1 --2- 172.21.15.99:0/3061066537 >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.487+0000 7f18c359e640 1 --2- 172.21.15.99:0/3061066537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009dc0 tx=0x7f18bc02a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63fd779c1b7b15e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.487+0000 7f18c1d9b640 1 -- 172.21.15.99:0/3061066537 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc007740 con 0x7f18c40d4e40 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.487+0000 7f18c1d9b640 1 -- 172.21.15.99:0/3061066537 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc0078e0 con 0x7f18c40d4e40 2024-09-10T11:35:41.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.487+0000 7f18c1d9b640 1 -- 172.21.15.99:0/3061066537 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc007cb0 con 0x7f18c40d4e40 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 -- 172.21.15.99:0/3061066537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 msgr2=0x7f18c40d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 --2- 172.21.15.99:0/3061066537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009dc0 tx=0x7f18bc02a5f0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 -- 172.21.15.99:0/3061066537 shutdown_connections 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 --2- 172.21.15.99:0/3061066537 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c40d4e40 0x7f18c40d1b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 --2- 172.21.15.99:0/3061066537 >> v2:172.21.15.196:3300/0 conn(0x7f18c41626c0 0x7f18c40d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 -- 172.21.15.99:0/3061066537 >> 172.21.15.99:0/3061066537 conn(0x7f18c408e680 msgr2=0x7f18c409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 -- 172.21.15.99:0/3061066537 shutdown_connections 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.488+0000 7f18c9015640 1 -- 172.21.15.99:0/3061066537 wait complete. 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.489+0000 7f18c9015640 1 Processor -- start 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.489+0000 7f18c9015640 1 -- start start 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c415eb80 0x7f18c4152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c41626c0 0x7f18c415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4163180 con 0x7f18c40d4e40 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18c4162ea0 con 0x7f18c415eb80 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c9015640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f18c4163460 con 0x7f18c41626c0 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37796/0 (socket says 172.21.15.99:37796) 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 -- 172.21.15.99:0/1992646463 learned_addr learned my addr 172.21.15.99:0/1992646463 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c359e640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c41626c0 0x7f18c415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c259c640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c415eb80 0x7f18c4152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 -- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c41626c0 msgr2=0x7f18c415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c41626c0 0x7f18c415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 -- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c415eb80 msgr2=0x7f18c4152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.490+0000 7f18c2d9d640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c415eb80 0x7f18c4152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18c2d9d640 1 -- 172.21.15.99:0/1992646463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c415ae10 con 0x7f18c40d4e40 2024-09-10T11:35:41.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18c2d9d640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c430 tx=0x7f18b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40074e0 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b40047e0 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c415b0f0 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4004bd0 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.491+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c415b6b0 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.492+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c40d4900 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.492+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18b4011070 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.493+0000 7f18bb7fe640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.493+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f18b408a930 con 0x7f18c40d4e40 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.493+0000 7f18c259c640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.493+0000 7f18c259c640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 0x7f18a406e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009560 tx=0x7f18ac009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.497+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18b4058d20 con 0x7f18c40d4e40 2024-09-10T11:35:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.696+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18c406d9f0 con 0x7f18c40d4e40 2024-09-10T11:35:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.696+0000 7f18bb7fe640 1 -- 172.21.15.99:0/1992646463 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f18b405f020 con 0x7f18c40d4e40 2024-09-10T11:35:41.698 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:41.698 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:35:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:35:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 msgr2=0x7f18a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 0x7f18a406e2f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009560 tx=0x7f18ac009190 comp rx=0 tx=0).stop 2024-09-10T11:35:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 msgr2=0x7f18c40b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c430 tx=0x7f18b400c8f0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 shutdown_connections 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f18a406be30 0x7f18a406e2f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18c41626c0 0x7f18c415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c415eb80 0x7f18c4152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 --2- 172.21.15.99:0/1992646463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18c40d4e40 0x7f18c40b3760 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 >> 172.21.15.99:0/1992646463 conn(0x7f18c408e680 msgr2=0x7f18c40b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 shutdown_connections 2024-09-10T11:35:41.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.698+0000 7f18c9015640 1 -- 172.21.15.99:0/1992646463 wait complete. 2024-09-10T11:35:41.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:35:41.827 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6990,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9363337119999997,"mdsmap_epoch":55,"osdmap_epoch":138,"osdmap_epoch_barrier":138,"uptime":11.961937387000001} 2024-09-10T11:35:41.827 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6990, "mdsmap_epoch": 55, "osdmap_epoch": 138, "osdmap_epoch_barrier": 138, "rank_uptime": 7.936333712, "state": "up:active", "uptime": 11.961937387, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:35:41.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-09-10T11:35:41.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.931+0000 7f53e34af640 1 Processor -- start 2024-09-10T11:35:41.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.931+0000 7f53e34af640 1 -- start start 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e34af640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53dc06bc10 0x7f53dc163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e34af640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f53dc160030 0x7f53dc06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e34af640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc06c530 con 0x7f53dc0b4930 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e34af640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f53dc0b5420 con 0x7f53dc06bc10 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e34af640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f53dc0b5140 con 0x7f53dc160030 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e124b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53dc06bc10 0x7f53dc163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e1a4c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f53dc0b4930 0x7f53dc06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46746/0 (socket says 172.21.15.99:46746) 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e124b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f53dc06bc10 0x7f53dc163620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37812/0 (socket says 172.21.15.99:37812) 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.932+0000 7f53e1a4c640 1 -- 172.21.15.99:0/3866358783 learned_addr learned my addr 172.21.15.99:0/3866358783 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e0a4a640 1 --2- 172.21.15.99:0/3866358783 >> v2:172.21.15.196:3300/0 conn(0x7f53dc160030 0x7f53dc06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e124b640 1 -- 172.21.15.99:0/3866358783 >> v2:172.21.15.196:3300/0 conn(0x7f53dc160030 msgr2=0x7f53dc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:41.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e124b640 1 --2- 172.21.15.99:0/3866358783 >> v2:172.21.15.196:3300/0 conn(0x7f53dc160030 0x7f53dc06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e124b640 1 -- 172.21.15.99:0/3866358783 >> v1:172.21.15.99:6789/0 conn(0x7f53dc0b4930 legacy=0x7f53dc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e124b640 1 -- 172.21.15.99:0/3866358783 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc162b40 con 0x7f53dc06bc10 2024-09-10T11:35:41.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53d3fff640 1 -- 172.21.15.99:0/3866358783 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3910736565 0 0) 0x7f53dc06c530 con 0x7f53dc0b4930 2024-09-10T11:35:41.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.933+0000 7f53e124b640 1 --2- 172.21.15.99:0/3866358783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc163620 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f53dc148c00 tx=0x7f53d802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91fad2438c58bba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53d3fff640 1 -- 172.21.15.99:0/3866358783 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8037070 con 0x7f53dc06bc10 2024-09-10T11:35:41.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53d3fff640 1 -- 172.21.15.99:0/3866358783 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8032440 con 0x7f53dc06bc10 2024-09-10T11:35:41.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53d3fff640 1 -- 172.21.15.99:0/3866358783 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d802ae60 con 0x7f53dc06bc10 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 -- 172.21.15.99:0/3866358783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 msgr2=0x7f53dc163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 --2- 172.21.15.99:0/3866358783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc163620 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f53dc148c00 tx=0x7f53d802a110 comp rx=0 tx=0).stop 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 -- 172.21.15.99:0/3866358783 shutdown_connections 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 --2- 172.21.15.99:0/3866358783 >> v2:172.21.15.196:3300/0 conn(0x7f53dc160030 0x7f53dc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 --2- 172.21.15.99:0/3866358783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc163620 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 -- 172.21.15.99:0/3866358783 >> 172.21.15.99:0/3866358783 conn(0x7f53dc0b3630 msgr2=0x7f53dc0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 -- 172.21.15.99:0/3866358783 shutdown_connections 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.934+0000 7f53e34af640 1 -- 172.21.15.99:0/3866358783 wait complete. 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.935+0000 7f53e34af640 1 Processor -- start 2024-09-10T11:35:41.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.935+0000 7f53e34af640 1 -- start start 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53dc0b4930 0x7f53dc0ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53dc160030 0x7f53dc0c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc0b4dc0 con 0x7f53dc06bc10 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc163100 con 0x7f53dc0b4930 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e34af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc0d1560 con 0x7f53dc160030 2024-09-10T11:35:41.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e124b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e124b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37818/0 (socket says 172.21.15.99:37818) 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e124b640 1 -- 172.21.15.99:0/1371496912 learned_addr learned my addr 172.21.15.99:0/1371496912 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e0a4a640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53dc0b4930 0x7f53dc0ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.936+0000 7f53e1a4c640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53dc160030 0x7f53dc0c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 -- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53dc160030 msgr2=0x7f53dc0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53dc160030 0x7f53dc0c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 -- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53dc0b4930 msgr2=0x7f53dc0ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53dc0b4930 0x7f53dc0ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 -- 172.21.15.99:0/1371496912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc0cf330 con 0x7f53dc06bc10 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e124b640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000c00 tx=0x7f53d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8037040 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53d8042070 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d8030960 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc0cf610 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.937+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc0cfbd0 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.938+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc15ef10 con 0x7f53dc06bc10 2024-09-10T11:35:41.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.939+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f53d8030b00 con 0x7f53dc06bc10 2024-09-10T11:35:41.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.939+0000 7f53d1ffb640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 0x7f53c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:41.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.939+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5709+0+0 (secure 0 0 0) 0x7f53d80ab660 con 0x7f53dc06bc10 2024-09-10T11:35:41.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.939+0000 7f53e0a4a640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 0x7f53c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:41.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.939+0000 7f53e0a4a640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 0x7f53c006e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009b40 tx=0x7f53cc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:41.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:41.943+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53d80799c0 con 0x7f53dc06bc10 2024-09-10T11:35:42.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.111+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f53dc163620 con 0x7f53dc06bc10 2024-09-10T11:35:42.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.899+0000 7f53d1ffb640 1 -- 172.21.15.99:0/1371496912 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v139) ==== 181+0+0 (secure 0 0 0) 0x7f53d807dcc0 con 0x7f53dc06bc10 2024-09-10T11:35:42.901 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-09-10T11:35:42.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.901+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 msgr2=0x7f53c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.901+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 0x7f53c006e2f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f53cc009b40 tx=0x7f53cc009190 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.901+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 msgr2=0x7f53dc15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.901+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000c00 tx=0x7f53d80046c0 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 shutdown_connections 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f53c006be30 0x7f53c006e2f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53dc160030 0x7f53dc0c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53dc0b4930 0x7f53dc0ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 --2- 172.21.15.99:0/1371496912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f53dc06bc10 0x7f53dc15e9d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 >> 172.21.15.99:0/1371496912 conn(0x7f53dc0b3630 msgr2=0x7f53dc161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 shutdown_connections 2024-09-10T11:35:42.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:42.902+0000 7f53e34af640 1 -- 172.21.15.99:0/1371496912 wait complete. 2024-09-10T11:35:42.918 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:35:43.033 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:35:43.034 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:35:43.034 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-09-10T11:35:43.034 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:43.034 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:43.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:43.050 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-09-10T11:35:43.189 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records in 2024-09-10T11:35:43.190 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records out 2024-09-10T11:35:43.190 INFO:teuthology.orchestra.run.smithi099.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0773487 s, 217 MB/s 2024-09-10T11:35:43.190 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:43.190 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:43.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:43.206 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-09-10T11:35:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records in 2024-09-10T11:35:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records out 2024-09-10T11:35:43.343 INFO:teuthology.orchestra.run.smithi099.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0793956 s, 225 MB/s 2024-09-10T11:35:43.344 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-09-10T11:35:43.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:35:43.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 Processor -- start 2024-09-10T11:35:43.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 -- start start 2024-09-10T11:35:43.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 0x7fe2dc0718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe2dc160d70 0x7fe2dc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2dc071e30 con 0x7fe2dc161270 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe2dc06f2c0 con 0x7fe2dc160d70 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.441+0000 7fe2e196f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe2dc06f5a0 con 0x7fe2dc146920 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe2dc160d70 0x7fe2dc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe2dc160d70 0x7fe2dc0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59844/0 (socket says 172.21.15.99:59844) 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 -- 172.21.15.99:0/2205838960 learned_addr learned my addr 172.21.15.99:0/2205838960 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 -- 172.21.15.99:0/2205838960 >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 msgr2=0x7fe2dc0718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2daffd640 1 --2- 172.21.15.99:0/2205838960 >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 0x7fe2dc0718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 --2- 172.21.15.99:0/2205838960 >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 0x7fe2dc0718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 -- 172.21.15.99:0/2205838960 >> v1:172.21.15.196:6789/0 conn(0x7fe2dc161270 legacy=0x7fe2dc160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2db7fe640 1 -- 172.21.15.99:0/2205838960 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc0ced70 con 0x7fe2dc160d70 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.442+0000 7fe2daffd640 1 --2- 172.21.15.99:0/2205838960 >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 0x7fe2dc0718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2db7fe640 1 --2- 172.21.15.99:0/2205838960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc160d70 0x7fe2dc0cf570 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc067a80 tx=0x7fe2cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857844561fbe174f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2d9ffb640 1 -- 172.21.15.99:0/2205838960 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc037070 con 0x7fe2dc160d70 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2d9ffb640 1 -- 172.21.15.99:0/2205838960 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2cc032440 con 0x7fe2dc160d70 2024-09-10T11:35:43.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2d9ffb640 1 -- 172.21.15.99:0/2205838960 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc032810 con 0x7fe2dc160d70 2024-09-10T11:35:43.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 -- 172.21.15.99:0/2205838960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc160d70 msgr2=0x7fe2dc0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 --2- 172.21.15.99:0/2205838960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc160d70 0x7fe2dc0cf570 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc067a80 tx=0x7fe2cc02a1a0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 -- 172.21.15.99:0/2205838960 shutdown_connections 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 --2- 172.21.15.99:0/2205838960 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc160d70 0x7fe2dc0cf570 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 --2- 172.21.15.99:0/2205838960 >> v2:172.21.15.196:3300/0 conn(0x7fe2dc146920 0x7fe2dc0718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 -- 172.21.15.99:0/2205838960 >> 172.21.15.99:0/2205838960 conn(0x7fe2dc0b2a80 msgr2=0x7fe2dc0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 -- 172.21.15.99:0/2205838960 shutdown_connections 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.443+0000 7fe2e196f640 1 -- 172.21.15.99:0/2205838960 wait complete. 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 Processor -- start 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 -- start start 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2dc160d70 0x7fe2dc1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2dc161270 0x7fe2dc09e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc0adee0 con 0x7fe2dc146920 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc0add60 con 0x7fe2dc161270 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.445+0000 7fe2e196f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe2dc0cfab0 con 0x7fe2dc160d70 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59850/0 (socket says 172.21.15.99:59850) 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 -- 172.21.15.99:0/3799969024 learned_addr learned my addr 172.21.15.99:0/3799969024 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 -- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2dc160d70 msgr2=0x7fe2dc1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2dc160d70 0x7fe2dc1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 -- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2dc161270 msgr2=0x7fe2dc09e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2dc161270 0x7fe2dc09e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 -- 172.21.15.99:0/3799969024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1f5b90 con 0x7fe2dc146920 2024-09-10T11:35:43.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2daffd640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400cf60 tx=0x7fe2d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d401a070 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2d400d840 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc09ee10 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d400dc10 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.446+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1f2c80 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.447+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2dc0718f0 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.448+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2d401d070 con 0x7fe2dc146920 2024-09-10T11:35:43.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.448+0000 7fe2d3fff640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 0x7fe2bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:43.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.448+0000 7fe2da7fc640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 0x7fe2bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:43.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.448+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fe2d408b600 con 0x7fe2dc146920 2024-09-10T11:35:43.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.449+0000 7fe2da7fc640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 0x7fe2bc06e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006fd0 tx=0x7fe2c4006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:43.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.452+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe2d40599f0 con 0x7fe2dc146920 2024-09-10T11:35:43.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.617+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fe2dc0d0710 con 0x7fe2dc146920 2024-09-10T11:35:43.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.618+0000 7fe2d3fff640 1 -- 172.21.15.99:0/3799969024 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) ==== 81+0+34282 (secure 0 0 0) 0x7fe2d405dcf0 con 0x7fe2dc146920 2024-09-10T11:35:43.620 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 139, 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:35:42.898378+0000", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:35:43.621 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:35:43.622 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:43.623 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:43.624 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:43.625 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:43.626 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:43.627 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:35:43.628 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:43.629 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:35:43.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:43.631 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:43.632 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:43.633 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.634 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:35:43.635 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:35:43.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "139", 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:43.637 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:43.638 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:43.639 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:43.641 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.642 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.643 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:43.647 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.649 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.650 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.651 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:35:43.652 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.654 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.660 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:35:43.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:35:43.664 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.666 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.667 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.668 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.671 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:35:43.672 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:35:43.676 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.677 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.678 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.682 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:43.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:43.687 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.688 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:43.689 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:35:43.690 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.691 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:35:43.692 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:35:43.693 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:43.694 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:43.695 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 msgr2=0x7fe2bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 0x7fe2bc06e2f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006fd0 tx=0x7fe2c4006bc0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 msgr2=0x7fe2dc09ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400cf60 tx=0x7fe2d400cf90 comp rx=0 tx=0).stop 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 shutdown_connections 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe2bc06be30 0x7fe2bc06e2f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2dc161270 0x7fe2dc09e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2dc160d70 0x7fe2dc1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 --2- 172.21.15.99:0/3799969024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2dc146920 0x7fe2dc09ba30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 >> 172.21.15.99:0/3799969024 conn(0x7fe2dc0b2a80 msgr2=0x7fe2dc0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 shutdown_connections 2024-09-10T11:35:43.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:43.620+0000 7fe2e196f640 1 -- 172.21.15.99:0/3799969024 wait complete. 2024-09-10T11:35:43.698 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:35:48.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:35:48.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.808+0000 7f6543186640 1 Processor -- start 2024-09-10T11:35:48.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 -- start start 2024-09-10T11:35:48.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f653c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f653c0d5010 0x7f653c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653c15ff70 con 0x7f653c1626c0 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f653c0d1620 con 0x7f653c0d5010 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f6543186640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f653c162bc0 con 0x7f653c15eb80 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f653bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f653c1626c0 0x7f653c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56648/0 (socket says 172.21.15.99:56648) 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.809+0000 7f653bfff640 1 -- 172.21.15.99:0/4239507195 learned_addr learned my addr 172.21.15.99:0/4239507195 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f6540f22640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f653c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f6541723640 1 --2- 172.21.15.99:0/4239507195 >> v2:172.21.15.196:3300/0 conn(0x7f653c0d5010 0x7f653c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2172171955 0 0) 0x7f653c15ff70 con 0x7f653c1626c0 2024-09-10T11:35:48.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6524003520 con 0x7f653c1626c0 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2913497560 0 0) 0x7f6524003520 con 0x7f653c1626c0 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 msgr2=0x7f653c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f653c0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 >> v2:172.21.15.196:3300/0 conn(0x7f653c0d5010 msgr2=0x7f653c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 --2- 172.21.15.99:0/4239507195 >> v2:172.21.15.196:3300/0 conn(0x7f653c0d5010 0x7f653c0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c162ea0 con 0x7f653c1626c0 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f65300019f0 con 0x7f653c1626c0 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 >> v1:172.21.15.99:6789/0 conn(0x7f653c1626c0 legacy=0x7f653c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f65240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f6524004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f653c0d1620 con 0x7f6524003f30 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f653c162bc0 con 0x7f6524004d40 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f652400ba10 con 0x7f653c15eb80 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f653bfff640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f6524004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f6541723640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.810+0000 7f6540f22640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f65240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 -- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 msgr2=0x7f65240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c15eb80 0x7f65240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 -- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6524004d40 msgr2=0x7f65240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6524004d40 0x7f65240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 -- 172.21.15.99:0/4239507195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524003520 con 0x7f6524003f30 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653bfff640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f6524004800 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f653000a890 tx=0x7f6524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231618fff32e9fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653000a4e0 con 0x7f6524003f30 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653002f070 con 0x7f6524003f30 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.811+0000 7f653b7fe640 1 -- 172.21.15.99:0/4239507195 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653002bcf0 con 0x7f6524003f30 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 -- 172.21.15.99:0/4239507195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 msgr2=0x7f6524004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 --2- 172.21.15.99:0/4239507195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f6524004800 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f653000a890 tx=0x7f6524005830 comp rx=0 tx=0).stop 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6541723640 1 -- 172.21.15.99:0/4239507195 reap_dead start 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 -- 172.21.15.99:0/4239507195 shutdown_connections 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 -- 172.21.15.99:0/4239507195 >> 172.21.15.99:0/4239507195 conn(0x7f653c08e680 msgr2=0x7f653c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 -- 172.21.15.99:0/4239507195 shutdown_connections 2024-09-10T11:35:48.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.812+0000 7f6543186640 1 -- 172.21.15.99:0/4239507195 wait complete. 2024-09-10T11:35:48.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.813+0000 7f6543186640 1 Processor -- start 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.813+0000 7f6543186640 1 -- start start 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 0x7f653c0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 0x7f653c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f653c0d2060 con 0x7f6524003f30 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f653c163460 con 0x7f653c152cf0 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6543186640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f653c0d21e0 con 0x7f653c0b7af0 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6540f22640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6540f22640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59862/0 (socket says 172.21.15.99:59862) 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f6540f22640 1 -- 172.21.15.99:0/3729962864 learned_addr learned my addr 172.21.15.99:0/3729962864 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:48.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.814+0000 7f653bfff640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 0x7f653c0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6541723640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 0x7f653c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 -- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 msgr2=0x7f653c0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 0x7f653c0b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 -- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 msgr2=0x7f653c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 0x7f653c0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 -- 172.21.15.99:0/3729962864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c0ccf60 con 0x7f6524003f30 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6541723640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 0x7f653c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f653bfff640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 0x7f653c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6540f22640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f652c011600 tx=0x7f652c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c00f920 con 0x7f6524003f30 2024-09-10T11:35:48.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f652c00fac0 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c0b4600 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c00fe70 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.815+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c0b4bc0 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.816+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c0d18b0 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.817+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f652c015460 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.817+0000 7f65397fa640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 0x7f652806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.817+0000 7f653bfff640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 0x7f652806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.817+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f652c08b660 con 0x7f6524003f30 2024-09-10T11:35:48.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.818+0000 7f653bfff640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 0x7f652806e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6530033000 tx=0x7f6530007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:48.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.822+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f653c0d18b0 con 0x7f6524003f30 2024-09-10T11:35:48.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.993+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f653c0d1b20 con 0x7f6524003f30 2024-09-10T11:35:48.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.994+0000 7f65397fa640 1 -- 172.21.15.99:0/3729962864 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) ==== 81+0+34282 (secure 0 0 0) 0x7f652c019030 con 0x7f6524003f30 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 139, 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:35:42.898378+0000", 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:35:48.997 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:35:48.998 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:35:48.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:49.000 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:35:49.001 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:49.002 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:49.003 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:49.004 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:49.005 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:35:49.006 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:35:49.007 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:49.008 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:49.009 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:49.010 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:49.011 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:49.012 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:35:49.013 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:35:49.014 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:35:49.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:49.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:49.017 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "139", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:49.018 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:49.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:49.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:49.021 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:49.022 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.023 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:35:49.025 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.027 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:49.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.030 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:35:49.032 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:49.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.037 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:35:49.042 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.043 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:35:49.044 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.046 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.049 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.053 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.054 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:49.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.060 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.065 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.066 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.072 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:49.073 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:49.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.075 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.077 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:49.079 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.080 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.081 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:49.083 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.085 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:49.086 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:35:49.087 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:35:49.088 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:49.089 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:49.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.996+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 msgr2=0x7f652806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:49.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.996+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 0x7f652806e320 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6530033000 tx=0x7f6530007000 comp rx=0 tx=0).stop 2024-09-10T11:35:49.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.996+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 msgr2=0x7f653c1534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.996+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f652c011600 tx=0x7f652c011ac0 comp rx=0 tx=0).stop 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 shutdown_connections 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f652806be60 0x7f652806e320 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f653c152cf0 0x7f653c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f653c0b7af0 0x7f653c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 --2- 172.21.15.99:0/3729962864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524003f30 0x7f653c1534a0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 >> 172.21.15.99:0/3729962864 conn(0x7f653c08e680 msgr2=0x7f653c0cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 shutdown_connections 2024-09-10T11:35:49.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:48.997+0000 7f6543186640 1 -- 172.21.15.99:0/3729962864 wait complete. 2024-09-10T11:35:49.093 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:35:54.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:35:54.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.205+0000 7f4d1eeb7640 1 Processor -- start 2024-09-10T11:35:54.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.205+0000 7f4d1eeb7640 1 -- start start 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1eeb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1eeb7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d180b00c0 0x7f4d18071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1eeb7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d180b4ed0 con 0x7f4d180721f0 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1eeb7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4d180b5d20 con 0x7f4d180b00c0 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1eeb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d180b3e30 con 0x7f4d1815c950 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d0ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d180b00c0 0x7f4d18071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1cc53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1cc53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:41414/0 (socket says 172.21.15.99:41414) 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1cc53640 1 -- 172.21.15.99:0/37346051 learned_addr learned my addr 172.21.15.99:0/37346051 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:54.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.206+0000 7f4d1d454640 1 --1- 172.21.15.99:0/37346051 >> v1:172.21.15.196:6789/0 conn(0x7f4d180721f0 0x7f4d180ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:40872/0 (socket says 172.21.15.99:40872) 2024-09-10T11:35:54.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.207+0000 7f4d0ffff640 1 -- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 msgr2=0x7f4d180b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:54.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.207+0000 7f4d0ffff640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.207+0000 7f4d0ffff640 1 -- 172.21.15.99:0/37346051 >> v1:172.21.15.196:6789/0 conn(0x7f4d180721f0 legacy=0x7f4d180ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.207+0000 7f4d0ffff640 1 -- 172.21.15.99:0/37346051 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d180b4110 con 0x7f4d180b00c0 2024-09-10T11:35:54.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.207+0000 7f4d1cc53640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:54.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d0ffff640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d180b00c0 0x7f4d18071cb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d71350da0df85a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d0f7fe640 1 -- 172.21.15.99:0/37346051 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08037070 con 0x7f4d180b00c0 2024-09-10T11:35:54.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d0f7fe640 1 -- 172.21.15.99:0/37346051 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08032440 con 0x7f4d180b00c0 2024-09-10T11:35:54.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d0f7fe640 1 -- 172.21.15.99:0/37346051 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08032810 con 0x7f4d180b00c0 2024-09-10T11:35:54.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d180b00c0 msgr2=0x7f4d18071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d180b00c0 0x7f4d18071cb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d080097a0 tx=0x7f4d0802a210 comp rx=0 tx=0).stop 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/37346051 shutdown_connections 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d180b00c0 0x7f4d18071cb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/37346051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d180b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/37346051 >> 172.21.15.99:0/37346051 conn(0x7f4d1806bc10 msgr2=0x7f4d1806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/37346051 shutdown_connections 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.208+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/37346051 wait complete. 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.210+0000 7f4d1eeb7640 1 Processor -- start 2024-09-10T11:35:54.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.210+0000 7f4d1eeb7640 1 -- start start 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.210+0000 7f4d1eeb7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.210+0000 7f4d1eeb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d18161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d18144f80 0x7f4d181442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180b49b0 con 0x7f4d180721f0 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d180b46d0 con 0x7f4d18144f80 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4d180b3a50 con 0x7f4d1815c950 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38684/0 (socket says 172.21.15.99:38684) 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 -- 172.21.15.99:0/735543049 learned_addr learned my addr 172.21.15.99:0/735543049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1d454640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d18144f80 0x7f4d181442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d0ffff640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d18161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 -- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 msgr2=0x7f4d18161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d18161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 -- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d18144f80 msgr2=0x7f4d181442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d18144f80 0x7f4d181442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d18153600 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1cc53640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e9e0 tx=0x7f4d1400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14017070 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1400c9d0 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d18152840 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400cd80 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.211+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d18152e00 con 0x7f4d180721f0 2024-09-10T11:35:54.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.213+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d18071cb0 con 0x7f4d180721f0 2024-09-10T11:35:54.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.213+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d1401a070 con 0x7f4d180721f0 2024-09-10T11:35:54.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.213+0000 7f4d0d7fa640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 0x7f4d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.213+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f4d1408ace0 con 0x7f4d180721f0 2024-09-10T11:35:54.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.213+0000 7f4d0ffff640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 0x7f4d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.214+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7f4d04073960 con 0x7f4d180721f0 2024-09-10T11:35:54.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.214+0000 7f4d0ffff640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 0x7f4d0406e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.218+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4d14058fb0 con 0x7f4d180721f0 2024-09-10T11:35:54.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.383+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f4d180b1d90 con 0x7f4d180721f0 2024-09-10T11:35:54.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.384+0000 7f4d0d7fa640 1 -- 172.21.15.99:0/735543049 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34301 (secure 0 0 0) 0x7f4d1405d2b0 con 0x7f4d180721f0 2024-09-10T11:35:54.385 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:54.385 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:54.385 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 140, 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:35:50.678965+0000", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:35:54.386 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:35:54.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.388 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.389 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.390 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.391 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.392 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:35:54.393 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:35:54.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.395 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.397 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.399 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:35:54.400 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:35:54.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:35:54.402 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "140", 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.404 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.405 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.406 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.407 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:35:54.413 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.414 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:35:54.419 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.420 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:35:54.425 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.426 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:35:54.429 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.430 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:35:54.431 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.432 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.438 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:35:54.441 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.443 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.444 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.448 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.450 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.451 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.452 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.455 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.456 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:54.458 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.459 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:35:54.461 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.463 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:54.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 msgr2=0x7f4d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 0x7f4d0406e2f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4d08034000 tx=0x7f4d0802e000 comp rx=0 tx=0).stop 2024-09-10T11:35:54.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 msgr2=0x7f4d18154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e9e0 tx=0x7f4d1400eea0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 shutdown_connections 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4d0406be30 0x7f4d0406e2f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d18144f80 0x7f4d181442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d1815c950 0x7f4d18161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 --2- 172.21.15.99:0/735543049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d180721f0 0x7f4d18154450 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 >> 172.21.15.99:0/735543049 conn(0x7f4d1806bc10 msgr2=0x7f4d180b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 shutdown_connections 2024-09-10T11:35:54.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.386+0000 7f4d1eeb7640 1 -- 172.21.15.99:0/735543049 wait complete. 2024-09-10T11:35:54.467 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-09-10T11:35:54.467 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:35:54.467 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:35:54.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:54.484 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-09-10T11:35:54.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:35:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:dd: error writing 'large_file_b': No space left on device 2024-09-10T11:35:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:dd: closing output file 'large_file_b': No space left on device 2024-09-10T11:35:54.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:35:54.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.680+0000 7fc7718d7640 1 Processor -- start 2024-09-10T11:35:54.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.680+0000 7fc7718d7640 1 -- start start 2024-09-10T11:35:54.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc7718d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc7718d7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc76c0b3160 0x7fc76c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc7718d7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc76c0b3620 con 0x7fc76c0b3960 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc7718d7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc76c06d960 con 0x7fc76c0b3160 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc7718d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c06dc40 con 0x7fc76c163580 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc76affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc76b7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc76c0b3160 0x7fc76c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc76affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:41432/0 (socket says 172.21.15.99:41432) 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc76b7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc76c0b3160 0x7fc76c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:44358/0 (socket says 172.21.15.99:44358) 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.681+0000 7fc76affd640 1 -- 172.21.15.99:0/3481369213 learned_addr learned my addr 172.21.15.99:0/3481369213 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76b7fe640 1 -- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 msgr2=0x7fc76c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76b7fe640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76b7fe640 1 -- 172.21.15.99:0/3481369213 >> v1:172.21.15.196:6789/0 conn(0x7fc76c0b3960 legacy=0x7fc76c0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76b7fe640 1 -- 172.21.15.99:0/3481369213 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c06df20 con 0x7fc76c0b3160 2024-09-10T11:35:54.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76affd640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.682+0000 7fc76b7fe640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3160 0x7fc76c06cd20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d90 tx=0x7fc76402a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc432118c55a2a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc769ffb640 1 -- 172.21.15.99:0/3481369213 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764007740 con 0x7fc76c0b3160 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc769ffb640 1 -- 172.21.15.99:0/3481369213 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7640078e0 con 0x7fc76c0b3160 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc769ffb640 1 -- 172.21.15.99:0/3481369213 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764007cb0 con 0x7fc76c0b3160 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 -- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3160 msgr2=0x7fc76c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3160 0x7fc76c06cd20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc764009d90 tx=0x7fc76402a530 comp rx=0 tx=0).stop 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 -- 172.21.15.99:0/3481369213 shutdown_connections 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3160 0x7fc76c06cd20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3481369213 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 -- 172.21.15.99:0/3481369213 >> 172.21.15.99:0/3481369213 conn(0x7fc76c0b4570 msgr2=0x7fc76c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:54.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 -- 172.21.15.99:0/3481369213 shutdown_connections 2024-09-10T11:35:54.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.683+0000 7fc7718d7640 1 -- 172.21.15.99:0/3481369213 wait complete. 2024-09-10T11:35:54.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.684+0000 7fc7718d7640 1 Processor -- start 2024-09-10T11:35:54.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 -- start start 2024-09-10T11:35:54.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3960 0x7fc76c0b7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c06d3e0 con 0x7fc76c0b3160 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c06d260 con 0x7fc76c0b3960 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc7718d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c06d560 con 0x7fc76c163580 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc76affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.685+0000 7fc76affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38694/0 (socket says 172.21.15.99:38694) 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 -- 172.21.15.99:0/3440093342 learned_addr learned my addr 172.21.15.99:0/3440093342 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76b7fe640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76a7fc640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3960 0x7fc76c0b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 -- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 msgr2=0x7fc76c0b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 -- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3960 msgr2=0x7fc76c0b7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3960 0x7fc76c0b7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c155270 con 0x7fc76c0b3160 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc76affd640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9d0 tx=0x7fc75c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c017070 con 0x7fc76c0b3160 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c00ca60 con 0x7fc76c0b3160 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c00ce10 con 0x7fc76c0b3160 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1559c0 con 0x7fc76c0b3160 2024-09-10T11:35:54.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.686+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c155f80 con 0x7fc76c0b3160 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.687+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c06cd20 con 0x7fc76c0b3160 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.688+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc75c01a070 con 0x7fc76c0b3160 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.688+0000 7fc753fff640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 0x7fc75806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.688+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc75c08acf0 con 0x7fc76c0b3160 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.688+0000 7fc76a7fc640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 0x7fc75806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:54.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.688+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7fc758073990 con 0x7fc76c0b3160 2024-09-10T11:35:54.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.689+0000 7fc76a7fc640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 0x7fc75806e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc76c070de0 tx=0x7fc760009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:54.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.692+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc75c08f070 con 0x7fc76c0b3160 2024-09-10T11:35:54.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.858+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fc76c0b89a0 con 0x7fc76c0b3160 2024-09-10T11:35:54.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.859+0000 7fc753fff640 1 -- 172.21.15.99:0/3440093342 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34301 (secure 0 0 0) 0x7fc75c059730 con 0x7fc76c0b3160 2024-09-10T11:35:54.860 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 140, 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:35:50.678965+0000", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:35:54.861 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:35:54.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.863 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.864 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.865 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.866 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.867 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:35:54.868 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.869 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:35:54.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.871 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.872 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.873 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:35:54.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.875 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:35:54.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:35:54.877 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "140", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:54.879 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:54.880 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.882 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.883 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.884 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.887 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:54.888 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.891 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.895 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.900 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.901 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.903 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:35:54.906 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.907 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:35:54.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:35:54.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:35:54.912 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.913 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.914 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.915 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.918 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.919 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:35:54.920 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.921 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.922 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.923 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:54.925 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.926 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:54.927 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:54.928 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:35:54.929 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.930 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.931 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.932 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.934 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:35:54.935 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:35:54.936 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:35:54.937 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:35:54.938 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:35:54.938 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:35:54.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:54.938 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 msgr2=0x7fc75806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 0x7fc75806e320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc76c070de0 tx=0x7fc760009040 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 msgr2=0x7fc76c154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00e9d0 tx=0x7fc75c00ee90 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 shutdown_connections 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc75806be60 0x7fc75806e320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc76c163580 0x7fc76c0b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc76c0b3960 0x7fc76c0b7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 --2- 172.21.15.99:0/3440093342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc76c0b3160 0x7fc76c154d30 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 >> 172.21.15.99:0/3440093342 conn(0x7fc76c0b4570 msgr2=0x7fc76c0b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 shutdown_connections 2024-09-10T11:35:54.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:54.861+0000 7fc7718d7640 1 -- 172.21.15.99:0/3440093342 wait complete. 2024-09-10T11:35:54.941 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:35:55.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.039+0000 7fcbd3518640 1 Processor -- start 2024-09-10T11:35:55.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.039+0000 7fcbd3518640 1 -- start start 2024-09-10T11:35:55.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd3518640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd3518640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 0x7fcbcc0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd3518640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbcc0b40e0 con 0x7fcbcc15ee00 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd3518640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcbcc0b5ab0 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd3518640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc073c50 con 0x7fcbcc0b0090 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38700/0 (socket says 172.21.15.99:38700) 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 -- 172.21.15.99:0/570368231 learned_addr learned my addr 172.21.15.99:0/570368231 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd1ab5640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 0x7fcbcc0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 -- 172.21.15.99:0/570368231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 msgr2=0x7fcbcc0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 0x7fcbcc0b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 -- 172.21.15.99:0/570368231 >> v1:172.21.15.99:6789/0 conn(0x7fcbcc15ee00 legacy=0x7fcbcc0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd0ab3640 1 -- 172.21.15.99:0/570368231 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc0b4380 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbcbfff640 1 -- 172.21.15.99:0/570368231 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737250030 0 0) 0x7fcbcc0b40e0 con 0x7fcbcc15ee00 2024-09-10T11:35:55.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.040+0000 7fcbd1ab5640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 0x7fcbcc0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:55.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd0ab3640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009870 tx=0x7fcbc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409a5955a855dc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbcbfff640 1 -- 172.21.15.99:0/570368231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0037070 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbcbfff640 1 -- 172.21.15.99:0/570368231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc0032440 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbcbfff640 1 -- 172.21.15.99:0/570368231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0032810 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 -- 172.21.15.99:0/570368231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc15f1e0 msgr2=0x7fcbcc0b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009870 tx=0x7fcbc002a270 comp rx=0 tx=0).stop 2024-09-10T11:35:55.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 -- 172.21.15.99:0/570368231 shutdown_connections 2024-09-10T11:35:55.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc0b0090 0x7fcbcc0b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 --2- 172.21.15.99:0/570368231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc15f1e0 0x7fcbcc0b5f90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.041+0000 7fcbd3518640 1 -- 172.21.15.99:0/570368231 >> 172.21.15.99:0/570368231 conn(0x7fcbcc0730c0 msgr2=0x7fcbcc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.042+0000 7fcbd3518640 1 -- 172.21.15.99:0/570368231 shutdown_connections 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.042+0000 7fcbd3518640 1 -- 172.21.15.99:0/570368231 wait complete. 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 Processor -- start 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 -- start start 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc15ee00 0x7fcbcc1f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 0x7fcbcc1f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b21b0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0afd60 con 0x7fcbcc15ee00 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.043+0000 7fcbd3518640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b0600 con 0x7fcbcc15f1e0 2024-09-10T11:35:55.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38712/0 (socket says 172.21.15.99:38712) 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 -- 172.21.15.99:0/1476932255 learned_addr learned my addr 172.21.15.99:0/1476932255 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd1ab5640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 0x7fcbcc1f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd0ab3640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc15ee00 0x7fcbcc1f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 -- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 msgr2=0x7fcbcc1f7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 0x7fcbcc1f7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 -- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc15ee00 msgr2=0x7fcbcc1f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc15ee00 0x7fcbcc1f1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc0d08d0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd1ab5640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 0x7fcbcc1f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd12b4640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c4e0 tx=0x7fcbc400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4017070 con 0x7fcbcc0b0090 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc40035f0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40039e0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc0d0bb0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.044+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc0d1170 con 0x7fcbcc0b0090 2024-09-10T11:35:55.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.045+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc0d1950 con 0x7fcbcc0b0090 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.046+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcbc401a070 con 0x7fcbcc0b0090 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.046+0000 7fcbc9ffb640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 0x7fcbb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.046+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fcbc408a140 con 0x7fcbcc0b0090 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.046+0000 7fcbd0ab3640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 0x7fcbb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.046+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7fcbb80739c0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.047+0000 7fcbd0ab3640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 0x7fcbb806e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0034000 tx=0x7fcbc002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.051+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcbcc0d1950 con 0x7fcbcc0b0090 2024-09-10T11:35:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.249+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbcc0b5f90 con 0x7fcbcc0b0090 2024-09-10T11:35:55.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.249+0000 7fcbc9ffb640 1 -- 172.21.15.99:0/1476932255 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7fcbc40584a0 con 0x7fcbcc0b0090 2024-09-10T11:35:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:55.251 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:35:55.251 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 msgr2=0x7fcbb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 0x7fcbb806e2f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0034000 tx=0x7fcbc002e000 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 msgr2=0x7fcbcc0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400c4e0 tx=0x7fcbc400c9a0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 shutdown_connections 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcbb806be30 0x7fcbb806e2f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc15f1e0 0x7fcbcc1f7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc15ee00 0x7fcbcc1f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 --2- 172.21.15.99:0/1476932255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbcc0b0090 0x7fcbcc0cff80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.251+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 >> 172.21.15.99:0/1476932255 conn(0x7fcbcc0730c0 msgr2=0x7fcbcc0aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:55.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.252+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 shutdown_connections 2024-09-10T11:35:55.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.252+0000 7fcbd3518640 1 -- 172.21.15.99:0/1476932255 wait complete. 2024-09-10T11:35:55.263 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:35:55.391 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6990,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":21.499835907000001,"mdsmap_epoch":55,"osdmap_epoch":140,"osdmap_epoch_barrier":138,"uptime":25.525440825} 2024-09-10T11:35:55.392 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6990, "mdsmap_epoch": 55, "osdmap_epoch": 140, "osdmap_epoch_barrier": 138, "rank_uptime": 21.499835907, "state": "up:active", "uptime": 25.525440825, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:35:55.392 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:35:55.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:55.392 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-09-10T11:35:55.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:35:55.411 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-09-10T11:35:55.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:35:55.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.609+0000 7f12c6376640 1 Processor -- start 2024-09-10T11:35:55.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.609+0000 7f12c6376640 1 -- start start 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12c6376640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f12c006ece0 0x7f12c0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12c6376640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c0152e60 con 0x7f12c00d1200 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12c6376640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c00d15e0 con 0x7f12c015c060 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12c6376640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f12c0072680 con 0x7f12c006ece0 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12bffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f12c00d1200 0x7f12c006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48598/0 (socket says 172.21.15.99:48598) 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.610+0000 7f12bffff640 1 -- 172.21.15.99:0/3556029648 learned_addr learned my addr 172.21.15.99:0/3556029648 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12bf7fe640 1 --2- 172.21.15.99:0/3556029648 >> v2:172.21.15.196:3300/0 conn(0x7f12c006ece0 0x7f12c0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488025320 0 0) 0x7f12c0152e60 con 0x7f12c00d1200 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12a8003520 con 0x7f12c00d1200 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 38742192 0 0) 0x7f12a8003520 con 0x7f12c00d1200 2024-09-10T11:35:55.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 >> v2:172.21.15.196:3300/0 conn(0x7f12c006ece0 msgr2=0x7f12c0073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 --2- 172.21.15.99:0/3556029648 >> v2:172.21.15.196:3300/0 conn(0x7f12c006ece0 0x7f12c0073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 >> v1:172.21.15.196:6789/0 conn(0x7f12c015c060 legacy=0x7f12c015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c0072960 con 0x7f12c00d1200 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f12b0001670 con 0x7f12c00d1200 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 >> v1:172.21.15.99:6789/0 conn(0x7f12c00d1200 legacy=0x7f12c006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80039b0 0x7f12a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a8004800 0x7f12a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 0x7f12a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0072680 con 0x7f12a80051c0 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12a800c160 con 0x7f12a8004800 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12a800c360 con 0x7f12a80039b0 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 0x7f12a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 -- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80039b0 msgr2=0x7f12a8003dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80039b0 0x7f12a8003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 -- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a8004800 msgr2=0x7f12a8004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12a8004800 0x7f12a8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.611+0000 7f12c4913640 1 -- 172.21.15.99:0/3556029648 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a8003520 con 0x7f12a80051c0 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.612+0000 7f12c4913640 1 -- 172.21.15.99:0/3556029648 reap_dead start 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.612+0000 7f12c4913640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 0x7f12a800b9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f12b8001e50 tx=0x7f12a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8505c37666dd8cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.612+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8001a70 con 0x7f12a80051c0 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.612+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8003970 con 0x7f12a80051c0 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.612+0000 7f12beffd640 1 -- 172.21.15.99:0/3556029648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8003d60 con 0x7f12a80051c0 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 -- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 msgr2=0x7f12a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 0x7f12a800b9c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f12b8001e50 tx=0x7f12a8005830 comp rx=0 tx=0).stop 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 -- 172.21.15.99:0/3556029648 shutdown_connections 2024-09-10T11:35:55.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 --2- 172.21.15.99:0/3556029648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12a80051c0 0x7f12a800b9c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 -- 172.21.15.99:0/3556029648 >> 172.21.15.99:0/3556029648 conn(0x7f12c006bc10 msgr2=0x7f12c006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:55.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 -- 172.21.15.99:0/3556029648 shutdown_connections 2024-09-10T11:35:55.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.613+0000 7f12c6376640 1 -- 172.21.15.99:0/3556029648 wait complete. 2024-09-10T11:35:55.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.614+0000 7f12c6376640 1 Processor -- start 2024-09-10T11:35:55.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.614+0000 7f12c6376640 1 -- start start 2024-09-10T11:35:55.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80051c0 0x7f12c00cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c00ae670 0x7f12c014a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c015cd90 con 0x7f12c00aff60 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0072f20 con 0x7f12c00ae670 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c6376640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12c0073440 con 0x7f12a80051c0 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38742/0 (socket says 172.21.15.99:38742) 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 -- 172.21.15.99:0/1862486836 learned_addr learned my addr 172.21.15.99:0/1862486836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12bffff640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80051c0 0x7f12c00cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 -- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80051c0 msgr2=0x7f12c00cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80051c0 0x7f12c00cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 -- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c00ae670 msgr2=0x7f12c014a140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c00ae670 0x7f12c014a140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c00aeaf0 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12c4913640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f12b8001f70 tx=0x7f12b80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8001a70 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.615+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b8006b50 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.616+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c00b0d30 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.616+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8007030 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.616+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c00b12f0 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.617+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c00b16d0 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.617+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12b80071d0 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.617+0000 7f12bcff9640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 0x7f12ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.618+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f12b80aabc0 con 0x7f12c00aff60 2024-09-10T11:35:55.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.618+0000 7f12bffff640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 0x7f12ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:55.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.618+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7f12ac073960 con 0x7f12c00aff60 2024-09-10T11:35:55.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.618+0000 7f12bffff640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 0x7f12ac06e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12b00011e0 tx=0x7f12b0001a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:55.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.622+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12c00b16d0 con 0x7f12c00aff60 2024-09-10T11:35:55.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.791+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f12c006e5d0 con 0x7f12c00aff60 2024-09-10T11:35:55.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.792+0000 7f12bcff9640 1 -- 172.21.15.99:0/1862486836 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34301 (secure 0 0 0) 0x7f12b8043060 con 0x7f12c00aff60 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 140, 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:35:50.678965+0000", 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:35:55.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:55.795 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:35:55.796 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:55.797 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.798 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:55.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:55.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:55.801 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:55.802 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:55.803 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:35:55.804 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:55.805 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:35:55.808 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:55.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:55.810 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:55.811 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:55.812 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:35:55.813 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:35:55.814 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:35:55.815 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "140", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:35:55.816 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:35:55.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:35:55.818 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.819 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:55.820 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.821 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:35:55.824 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.825 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.827 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:35:55.829 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.833 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:35:55.836 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.837 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.839 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:55.844 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.846 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:35:55.849 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.851 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.853 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.856 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.857 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:35:55.859 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.860 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:35:55.865 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:35:55.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:35:55.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.869 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.870 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.872 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.873 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:35:55.874 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:35:55.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:35:55.876 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:35:55.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:35:55.877 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:35:55.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 msgr2=0x7f12ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 0x7f12ac06e2f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12b00011e0 tx=0x7f12b0001a40 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 msgr2=0x7f12c00b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f12b8001f70 tx=0x7f12b80014a0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 shutdown_connections 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12ac06be30 0x7f12ac06e2f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00aff60 0x7f12c00b03e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c00ae670 0x7f12c014a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 --2- 172.21.15.99:0/1862486836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12a80051c0 0x7f12c00cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.794+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 >> 172.21.15.99:0/1862486836 conn(0x7f12c006bc10 msgr2=0x7f12c015c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.795+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 shutdown_connections 2024-09-10T11:35:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:35:55.795+0000 7f12c6376640 1 -- 172.21.15.99:0/1862486836 wait complete. 2024-09-10T11:35:55.880 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:35:59.020 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:35:59.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:36:00.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:36:00.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.991+0000 7fb4290d0640 1 Processor -- start 2024-09-10T11:36:00.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.991+0000 7fb4290d0640 1 -- start start 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb4290d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb42415ee00 0x7fb4240b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb4290d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb4240b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb4290d0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4240b40e0 con 0x7fb4240b0090 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb4290d0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb4240b5ab0 con 0x7fb42415ee00 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb4290d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb424073c50 con 0x7fb42415f1e0 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb422d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb42415ee00 0x7fb4240b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb422d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb42415ee00 0x7fb4240b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38748/0 (socket says 172.21.15.99:38748) 2024-09-10T11:36:00.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb42359e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb4240b0090 0x7fb4240b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48604/0 (socket says 172.21.15.99:48604) 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb422d9d640 1 -- 172.21.15.99:0/3991322017 learned_addr learned my addr 172.21.15.99:0/3991322017 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.992+0000 7fb42259c640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb4240b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb422d9d640 1 -- 172.21.15.99:0/3991322017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 msgr2=0x7fb4240b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb422d9d640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb4240b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb422d9d640 1 -- 172.21.15.99:0/3991322017 >> v1:172.21.15.99:6789/0 conn(0x7fb4240b0090 legacy=0x7fb4240b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb421d9b640 1 -- 172.21.15.99:0/3991322017 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3692454795 0 0) 0x7fb4240b40e0 con 0x7fb4240b0090 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb422d9d640 1 -- 172.21.15.99:0/3991322017 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4240b4380 con 0x7fb42415ee00 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb422d9d640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42415ee00 0x7fb4240b5f90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb41402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d90db32ae4a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb421d9b640 1 -- 172.21.15.99:0/3991322017 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414037070 con 0x7fb42415ee00 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb421d9b640 1 -- 172.21.15.99:0/3991322017 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb414032440 con 0x7fb42415ee00 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.993+0000 7fb421d9b640 1 -- 172.21.15.99:0/3991322017 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414032810 con 0x7fb42415ee00 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 -- 172.21.15.99:0/3991322017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42415ee00 msgr2=0x7fb4240b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:00.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42415ee00 0x7fb4240b5f90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb41402a270 comp rx=0 tx=0).stop 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 -- 172.21.15.99:0/3991322017 shutdown_connections 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb4240b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 --2- 172.21.15.99:0/3991322017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb42415ee00 0x7fb4240b5f90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 -- 172.21.15.99:0/3991322017 >> 172.21.15.99:0/3991322017 conn(0x7fb4240730c0 msgr2=0x7fb4240734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 -- 172.21.15.99:0/3991322017 shutdown_connections 2024-09-10T11:36:00.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.994+0000 7fb4290d0640 1 -- 172.21.15.99:0/3991322017 wait complete. 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 Processor -- start 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 -- start start 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 0x7fb4241505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb42409b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4240b21b0 con 0x7fb4240b0090 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4240afd60 con 0x7fb42415ee00 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.996+0000 7fb4290d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4240b0600 con 0x7fb42415f1e0 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38758/0 (socket says 172.21.15.99:38758) 2024-09-10T11:36:01.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 -- 172.21.15.99:0/2947633743 learned_addr learned my addr 172.21.15.99:0/2947633743 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb42359e640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb42409b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 -- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 msgr2=0x7fb42409b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb42259c640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 0x7fb4241505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb42409b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 -- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 msgr2=0x7fb4241505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 0x7fb4241505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 -- 172.21.15.99:0/2947633743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb424150af0 con 0x7fb4240b0090 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb42359e640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb42409b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb42259c640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 0x7fb4241505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb422d9d640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb4140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414041070 con 0x7fb4240b0090 2024-09-10T11:36:01.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41403c890 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414037040 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42409bc40 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.997+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42409c200 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.998+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4240b5f90 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.999+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb414044070 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.999+0000 7fb40b7fe640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 0x7fb41006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.999+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7fb4140aba40 con 0x7fb4240b0090 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:00.999+0000 7fb42259c640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 0x7fb41006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.000+0000 7fb42259c640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 0x7fb41006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.004+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb41407a510 con 0x7fb4240b0090 2024-09-10T11:36:01.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.171+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fb42415c040 con 0x7fb4240b0090 2024-09-10T11:36:01.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.172+0000 7fb40b7fe640 1 -- 172.21.15.99:0/2947633743 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34282 (secure 0 0 0) 0x7fb414079e10 con 0x7fb4240b0090 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:36:00.680818+0000", 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:36:01.174 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:36:01.175 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:36:01.176 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.177 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.178 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:36:01.179 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.180 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.181 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:36:01.182 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:36:01.183 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.184 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.185 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.186 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.187 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:36:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:36:01.189 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.190 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:36:01.191 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.192 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.193 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.194 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:36:01.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.196 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.197 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:36:01.198 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:36:01.200 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.201 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:36:01.203 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.204 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.205 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.210 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:36:01.211 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.213 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.214 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:36:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:36:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:36:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:36:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:36:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:36:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:36:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:36:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.256 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 msgr2=0x7fb41006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 0x7fb41006e2f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb418006fd0 tx=0x7fb418008040 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 msgr2=0x7fb42409cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb414009870 tx=0x7fb4140046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 shutdown_connections 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb41006be30 0x7fb41006e2f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb42415f1e0 0x7fb42409b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.174+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb42415ee00 0x7fb4241505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.175+0000 7fb4290d0640 1 --2- 172.21.15.99:0/2947633743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4240b0090 0x7fb42409cce0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.175+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 >> 172.21.15.99:0/2947633743 conn(0x7fb4240730c0 msgr2=0x7fb42406e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:01.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.175+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 shutdown_connections 2024-09-10T11:36:01.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.175+0000 7fb4290d0640 1 -- 172.21.15.99:0/2947633743 wait complete. 2024-09-10T11:36:01.259 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:36:01.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:36:01.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.357+0000 7fa764677640 1 Processor -- start 2024-09-10T11:36:01.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.357+0000 7fa764677640 1 -- start start 2024-09-10T11:36:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa764677640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa75c15b2d0 0x7fa75c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa764677640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75c0b0020 con 0x7fa75c15aef0 2024-09-10T11:36:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa764677640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75c06ee10 con 0x7fa75c0afbb0 2024-09-10T11:36:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa764677640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa75c0cd9a0 con 0x7fa75c15b2d0 2024-09-10T11:36:01.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa762413640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa75c15aef0 0x7fa75c0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48618/0 (socket says 172.21.15.99:48618) 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa762413640 1 -- 172.21.15.99:0/1089238231 learned_addr learned my addr 172.21.15.99:0/1089238231 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2300637174 0 0) 0x7fa75c0b0020 con 0x7fa75c15aef0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa748003520 con 0x7fa75c15aef0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761c12640 1 --2- 172.21.15.99:0/1089238231 >> v2:172.21.15.196:3300/0 conn(0x7fa75c15b2d0 0x7fa75c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3198312715 0 0) 0x7fa748003520 con 0x7fa75c15aef0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 >> v2:172.21.15.196:3300/0 conn(0x7fa75c15b2d0 msgr2=0x7fa75c0b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 --2- 172.21.15.99:0/1089238231 >> v2:172.21.15.196:3300/0 conn(0x7fa75c15b2d0 0x7fa75c0b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 >> v1:172.21.15.196:6789/0 conn(0x7fa75c0afbb0 legacy=0x7fa75c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c0b0290 con 0x7fa75c15aef0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa7580019f0 con 0x7fa75c15aef0 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 >> v1:172.21.15.99:6789/0 conn(0x7fa75c15aef0 legacy=0x7fa75c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7480039b0 0x7fa748003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa748004800 0x7fa748004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa74800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c0cd9a0 con 0x7fa7480051c0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa74800c060 con 0x7fa7480039b0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.358+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa74800c260 con 0x7fa748004800 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa74800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762413640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7480039b0 0x7fa748003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa761c12640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa748004800 0x7fa748004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 -- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa748004800 msgr2=0x7fa748004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa748004800 0x7fa748004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 -- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7480039b0 msgr2=0x7fa748003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7480039b0 0x7fa748003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 -- 172.21.15.99:0/1089238231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa748003520 con 0x7fa7480051c0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.359+0000 7fa762c14640 1 -- 172.21.15.99:0/1089238231 reap_dead start 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa762c14640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa74800b9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa754001750 tx=0x7fa748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bd42a5b7d2480a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa754001370 con 0x7fa7480051c0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa754005310 con 0x7fa7480051c0 2024-09-10T11:36:01.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa761411640 1 -- 172.21.15.99:0/1089238231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75402abe0 con 0x7fa7480051c0 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 -- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 msgr2=0x7fa74800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa74800b9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa754001750 tx=0x7fa748005830 comp rx=0 tx=0).stop 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 -- 172.21.15.99:0/1089238231 shutdown_connections 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 --2- 172.21.15.99:0/1089238231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa74800b9c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 -- 172.21.15.99:0/1089238231 >> 172.21.15.99:0/1089238231 conn(0x7fa75c06bc10 msgr2=0x7fa75c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 -- 172.21.15.99:0/1089238231 shutdown_connections 2024-09-10T11:36:01.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.360+0000 7fa764677640 1 -- 172.21.15.99:0/1089238231 wait complete. 2024-09-10T11:36:01.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.361+0000 7fa764677640 1 Processor -- start 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.361+0000 7fa764677640 1 -- start start 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa75c1357e0 0x7fa75c14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa75c14a690 0x7fa75c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c06eb30 con 0x7fa7480051c0 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c06e850 con 0x7fa75c14a690 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa764677640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa75c0b4510 con 0x7fa75c1357e0 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38780/0 (socket says 172.21.15.99:38780) 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 -- 172.21.15.99:0/2036396678 learned_addr learned my addr 172.21.15.99:0/2036396678 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 -- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa75c1357e0 msgr2=0x7fa75c14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa75c1357e0 0x7fa75c14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 -- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa75c14a690 msgr2=0x7fa75c14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.362+0000 7fa762413640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa75c14a690 0x7fa75c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa762413640 1 -- 172.21.15.99:0/2036396678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c14c980 con 0x7fa7480051c0 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa762413640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa7580077f0 tx=0x7fa75800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758017020 con 0x7fa7480051c0 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7580133f0 con 0x7fa7480051c0 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c14cc60 con 0x7fa7480051c0 2024-09-10T11:36:01.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7580137e0 con 0x7fa7480051c0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.363+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c14d220 con 0x7fa7480051c0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.364+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75c0b48a0 con 0x7fa7480051c0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.364+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa75801a020 con 0x7fa7480051c0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.365+0000 7fa752ffd640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 0x7fa74006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.365+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7fa75808b9d0 con 0x7fa7480051c0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.365+0000 7fa761c12640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 0x7fa74006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.365+0000 7fa761c12640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 0x7fa74006e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa75c159350 tx=0x7fa74c009320 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.369+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa758021020 con 0x7fa7480051c0 2024-09-10T11:36:01.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.535+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fa75c151860 con 0x7fa7480051c0 2024-09-10T11:36:01.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.536+0000 7fa752ffd640 1 -- 172.21.15.99:0/2036396678 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34282 (secure 0 0 0) 0x7fa75805a380 con 0x7fa7480051c0 2024-09-10T11:36:01.537 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:01.537 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 141, 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:36:00.680818+0000", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:36:01.538 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 17, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:36:01.539 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.540 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.541 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.542 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.543 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.544 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.545 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 16, 2024-09-10T11:36:01.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:31.809882+0000", 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.547 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.548 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "137", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:36:01.549 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.550 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.551 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:36:01.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 2.25, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 2.25, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2.25, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2.25, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 17, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:35:32.181001+0000", 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:36:01.554 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:36:01.555 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "141", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:36:01.556 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:36:01.557 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:36:01.558 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:36:01.559 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.560 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.561 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:36:01.565 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.566 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.567 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.572 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.573 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:36:01.574 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.576 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.577 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.578 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.580 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.581 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:36:01.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.585 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.589 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:36:01.590 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.591 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:36:01.593 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.594 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.595 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:36:01.596 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.597 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:36:01.598 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:36:01.599 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:36:01.600 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.601 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:36:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 136, 2024-09-10T11:36:01.603 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:36:01.605 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.606 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:01.607 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.608 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.609 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.610 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.612 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.613 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:36:01.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:36:01.615 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:36:01.616 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:36:01.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:01.617 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:36:01.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 msgr2=0x7fa74006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 0x7fa74006e2f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa75c159350 tx=0x7fa74c009320 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 msgr2=0x7fa75c0aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa7580077f0 tx=0x7fa75800a730 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 shutdown_connections 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa74006be30 0x7fa74006e2f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa75c14a690 0x7fa75c14ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa75c1357e0 0x7fa75c14a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 --2- 172.21.15.99:0/2036396678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7480051c0 0x7fa75c0aa490 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 >> 172.21.15.99:0/2036396678 conn(0x7fa75c06bc10 msgr2=0x7fa75c0cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 shutdown_connections 2024-09-10T11:36:01.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.538+0000 7fa764677640 1 -- 172.21.15.99:0/2036396678 wait complete. 2024-09-10T11:36:01.620 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:01.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.719+0000 7f8d594b7640 1 Processor -- start 2024-09-10T11:36:01.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.719+0000 7f8d594b7640 1 -- start start 2024-09-10T11:36:01.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d594b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d5415eb80 0x7f8d540d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d594b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d540d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d594b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d5415ff70 con 0x7f8d541626c0 2024-09-10T11:36:01.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d594b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8d540d1620 con 0x7f8d5415eb80 2024-09-10T11:36:01.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d594b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8d54162bc0 con 0x7f8d540d5010 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d4bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8d541626c0 0x7f8d540d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48622/0 (socket says 172.21.15.99:48622) 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d4bfff640 1 -- 172.21.15.99:0/2531994992 learned_addr learned my addr 172.21.15.99:0/2531994992 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d537fe640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d540d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d52ffd640 1 --2- 172.21.15.99:0/2531994992 >> v2:172.21.15.196:3300/0 conn(0x7f8d5415eb80 0x7f8d540d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2083028337 0 0) 0x7f8d5415ff70 con 0x7f8d541626c0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d40003520 con 0x7f8d541626c0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 560232023 0 0) 0x7f8d40003520 con 0x7f8d541626c0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 msgr2=0x7f8d540d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d540d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 >> v2:172.21.15.196:3300/0 conn(0x7f8d5415eb80 msgr2=0x7f8d540d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 --2- 172.21.15.99:0/2531994992 >> v2:172.21.15.196:3300/0 conn(0x7f8d5415eb80 0x7f8d540d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d54162ea0 con 0x7f8d541626c0 2024-09-10T11:36:01.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8d3c001670 con 0x7f8d541626c0 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 >> v1:172.21.15.99:6789/0 conn(0x7f8d541626c0 legacy=0x7f8d540d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.720+0000 7f8d527fc640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40003f30 0x7f8d40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d527fc640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d40004d40 0x7f8d400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540d1620 con 0x7f8d40003f30 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54162bc0 con 0x7f8d40004d40 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4000ba10 con 0x7f8d540d5010 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40003f30 0x7f8d40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d52ffd640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 -- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 msgr2=0x7f8d400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d540d5010 0x7f8d400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 -- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d40004d40 msgr2=0x7f8d400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d40004d40 0x7f8d400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.721+0000 7f8d4bfff640 1 -- 172.21.15.99:0/2531994992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d40003520 con 0x7f8d40003f30 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d4bfff640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40003f30 0x7f8d40004800 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c210 tx=0x7f8d40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ae430308adb1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c007d30 con 0x7f8d40003f30 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c005af0 con 0x7f8d40003f30 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d527fc640 1 -- 172.21.15.99:0/2531994992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c007070 con 0x7f8d40003f30 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 -- 172.21.15.99:0/2531994992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40003f30 msgr2=0x7f8d40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 --2- 172.21.15.99:0/2531994992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d40003f30 0x7f8d40004800 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c210 tx=0x7f8d40005830 comp rx=0 tx=0).stop 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d537fe640 1 -- 172.21.15.99:0/2531994992 reap_dead start 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 -- 172.21.15.99:0/2531994992 shutdown_connections 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 -- 172.21.15.99:0/2531994992 >> 172.21.15.99:0/2531994992 conn(0x7f8d5408e680 msgr2=0x7f8d5409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 -- 172.21.15.99:0/2531994992 shutdown_connections 2024-09-10T11:36:01.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.722+0000 7f8d594b7640 1 -- 172.21.15.99:0/2531994992 wait complete. 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.723+0000 7f8d594b7640 1 Processor -- start 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 -- start start 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d40003f30 0x7f8d54159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d540cd380 0x7f8d540cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d540d2ef0 con 0x7f8d541f4b40 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d54163460 con 0x7f8d540cd380 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d594b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8d540d3070 con 0x7f8d40003f30 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d537fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38804/0 (socket says 172.21.15.99:38804) 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.724+0000 7f8d537fe640 1 -- 172.21.15.99:0/938736677 learned_addr learned my addr 172.21.15.99:0/938736677 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 -- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d40003f30 msgr2=0x7f8d54159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d40003f30 0x7f8d54159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 -- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d540cd380 msgr2=0x7f8d540cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d540cd380 0x7f8d540cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 -- 172.21.15.99:0/938736677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d540b7b90 con 0x7f8d541f4b40 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d537fe640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b44a0 tx=0x7f8d4c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c00bce0 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d4c01a070 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d540b7e70 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4c014640 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.725+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d540b8430 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.726+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d540b8630 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.727+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d4c005d10 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.727+0000 7f8d4b7fe640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 0x7f8d3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.727+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f8d4c08afe0 con 0x7f8d541f4b40 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.727+0000 7f8d52ffd640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 0x7f8d3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:01.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.727+0000 7f8d52ffd640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 0x7f8d3406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009a40 tx=0x7f8d44009320 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:01.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.731+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d540b8630 con 0x7f8d541f4b40 2024-09-10T11:36:01.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.933+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d540d29b0 con 0x7f8d541f4b40 2024-09-10T11:36:01.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.933+0000 7f8d4b7fe640 1 -- 172.21.15.99:0/938736677 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f8d4c059900 con 0x7f8d541f4b40 2024-09-10T11:36:01.934 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:01.934 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:36:01.935 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 msgr2=0x7f8d3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 0x7f8d3406e2f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d44009a40 tx=0x7f8d44009320 comp rx=0 tx=0).stop 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 msgr2=0x7f8d541f4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b44a0 tx=0x7f8d4c00b480 comp rx=0 tx=0).stop 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 shutdown_connections 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d3406be30 0x7f8d3406e2f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d541f4b40 0x7f8d541f4fc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d540cd380 0x7f8d540cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 --2- 172.21.15.99:0/938736677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d40003f30 0x7f8d54159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 >> 172.21.15.99:0/938736677 conn(0x7f8d5408e680 msgr2=0x7f8d540b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 shutdown_connections 2024-09-10T11:36:01.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:01.936+0000 7f8d594b7640 1 -- 172.21.15.99:0/938736677 wait complete. 2024-09-10T11:36:01.948 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-09-10T11:36:02.077 INFO:teuthology.orchestra.run.smithi196.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":6990,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.186338377999999,"mdsmap_epoch":55,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":32.211942778000001} 2024-09-10T11:36:02.078 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 6990, "mdsmap_epoch": 55, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 28.186338378, "state": "up:active", "uptime": 32.211942778, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:36:02.078 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:36:02.078 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:36:02.078 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:02.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:02.094 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-09-10T11:36:02.364 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2024-09-10T11:36:02.364 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2024-09-10T11:36:02.364 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.21046 s, 249 MB/s 2024-09-10T11:36:02.364 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:36:02.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:36:02.409 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:36:02.451 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:36:02.451 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:36:02.451 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:36:02.451 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:02.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:36:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 Processor -- start 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 -- start start 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077c0d52f0 con 0x7f077c15eb80 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f077c0d30c0 con 0x7f077c0d4e40 2024-09-10T11:36:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.606+0000 7f0782cd9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f077c15ff70 con 0x7f077c1626c0 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f0781276640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f0780a75640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f077c15eb80 0x7f077c0d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41858/0 (socket says 172.21.15.99:41858) 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f0781276640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:49678/0 (socket says 172.21.15.99:49678) 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f0780a75640 1 -- 172.21.15.99:0/1022049243 learned_addr learned my addr 172.21.15.99:0/1022049243 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:48066/0 (socket says 172.21.15.99:48066) 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 -- 172.21.15.99:0/1022049243 >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 msgr2=0x7f077c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:02.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 --2- 172.21.15.99:0/1022049243 >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 -- 172.21.15.99:0/1022049243 >> v1:172.21.15.196:6789/0 conn(0x7f077c15eb80 legacy=0x7f077c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077bfff640 1 -- 172.21.15.99:0/1022049243 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c0d1620 con 0x7f077c1626c0 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f077b7fe640 1 -- 172.21.15.99:0/1022049243 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2308140548 0 0) 0x7f0770001110 con 0x7f077c15eb80 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.607+0000 7f0781276640 1 --2- 172.21.15.99:0/1022049243 >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.608+0000 7f077bfff640 1 --2- 172.21.15.99:0/1022049243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f076802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28e7dc2af8b052b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.608+0000 7f077b7fe640 1 -- 172.21.15.99:0/1022049243 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768037070 con 0x7f077c1626c0 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.608+0000 7f077b7fe640 1 -- 172.21.15.99:0/1022049243 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076802ae50 con 0x7f077c1626c0 2024-09-10T11:36:02.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.608+0000 7f077b7fe640 1 -- 172.21.15.99:0/1022049243 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768032660 con 0x7f077c1626c0 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 -- 172.21.15.99:0/1022049243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 msgr2=0x7f077c0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 --2- 172.21.15.99:0/1022049243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f076802a100 comp rx=0 tx=0).stop 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 -- 172.21.15.99:0/1022049243 shutdown_connections 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 --2- 172.21.15.99:0/1022049243 >> v2:172.21.15.196:3300/0 conn(0x7f077c0d4e40 0x7f077c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 --2- 172.21.15.99:0/1022049243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c1626c0 0x7f077c0d4900 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 -- 172.21.15.99:0/1022049243 >> 172.21.15.99:0/1022049243 conn(0x7f077c08e680 msgr2=0x7f077c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 -- 172.21.15.99:0/1022049243 shutdown_connections 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.609+0000 7f0782cd9640 1 -- 172.21.15.99:0/1022049243 wait complete. 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.610+0000 7f0782cd9640 1 Processor -- start 2024-09-10T11:36:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.610+0000 7f0782cd9640 1 -- start start 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c0d4e40 0x7f077c15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 0x7f077c0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f077c163180 con 0x7f077c15eb80 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f077c162ea0 con 0x7f077c1626c0 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0782cd9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f077c163460 con 0x7f077c0d4e40 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56266/0 (socket says 172.21.15.99:56266) 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 -- 172.21.15.99:0/2147809190 learned_addr learned my addr 172.21.15.99:0/2147809190 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0780a75640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c0d4e40 0x7f077c15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0781276640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 0x7f077c0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 -- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c0d4e40 msgr2=0x7f077c15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c0d4e40 0x7f077c15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 -- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 msgr2=0x7f077c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 0x7f077c0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 -- 172.21.15.99:0/2147809190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c15a760 con 0x7f077c15eb80 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f0781276640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 0x7f077c0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.611+0000 7f077bfff640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f07680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.612+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768037040 con 0x7f077c15eb80 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.612+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0768004030 con 0x7f077c15eb80 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.612+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c1f7bf0 con 0x7f077c15eb80 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.612+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768006e10 con 0x7f077c15eb80 2024-09-10T11:36:02.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.612+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1f81b0 con 0x7f077c15eb80 2024-09-10T11:36:02.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.613+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c0d4900 con 0x7f077c15eb80 2024-09-10T11:36:02.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.613+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0768003610 con 0x7f077c15eb80 2024-09-10T11:36:02.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.614+0000 7f07797fa640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 0x7f076006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:02.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.614+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f07680ab590 con 0x7f077c15eb80 2024-09-10T11:36:02.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.614+0000 7f0780a75640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 0x7f076006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:02.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.614+0000 7f0780a75640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 0x7f076006e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f0770007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:02.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.618+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f077c15a760 con 0x7f077c15eb80 2024-09-10T11:36:02.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.796+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f077c15db80 con 0x7f077c15eb80 2024-09-10T11:36:02.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.796+0000 7f07797fa640 1 -- 172.21.15.99:0/2147809190 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v141) ==== 80+0+0 (secure 0 0 0) 0x7f0768042020 con 0x7f077c15eb80 2024-09-10T11:36:02.797 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:36:02.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 msgr2=0x7f076006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 0x7f076006e2f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0770000f60 tx=0x7f0770007c00 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 msgr2=0x7f077c0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f07680097a0 tx=0x7f07680046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 shutdown_connections 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f076006be30 0x7f076006e2f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f077c1626c0 0x7f077c0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f077c15eb80 0x7f077c0cc710 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 --2- 172.21.15.99:0/2147809190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f077c0d4e40 0x7f077c15a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.798+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 >> 172.21.15.99:0/2147809190 conn(0x7f077c08e680 msgr2=0x7f077c15cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.799+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 shutdown_connections 2024-09-10T11:36:02.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:02.799+0000 7f0782cd9640 1 -- 172.21.15.99:0/2147809190 wait complete. 2024-09-10T11:36:02.810 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:36:02.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:02.810 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:02.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[55213]: fuse finished with error 0 and tester_r 0 2024-09-10T11:36:02.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:36:02.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:02.839 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:36:02.918 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:36:02.918 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:36:02.918 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:02.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:36:03.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 Processor -- start 2024-09-10T11:36:03.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 -- start start 2024-09-10T11:36:03.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30b4160030 0x7f30b40b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30b40d1760 0x7f30b40cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30b40ce430 con 0x7f30b40d0130 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f30b40ce200 con 0x7f30b40d1760 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.058+0000 7f30ba78d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30b40ce710 con 0x7f30b4160030 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30b40d1760 0x7f30b40cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30b40d1760 0x7f30b40cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56276/0 (socket says 172.21.15.99:56276) 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 -- 172.21.15.99:0/3007674861 learned_addr learned my addr 172.21.15.99:0/3007674861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b3fff640 1 --2- 172.21.15.99:0/3007674861 >> v2:172.21.15.196:3300/0 conn(0x7f30b4160030 0x7f30b40b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 -- 172.21.15.99:0/3007674861 >> v2:172.21.15.196:3300/0 conn(0x7f30b4160030 msgr2=0x7f30b40b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 --2- 172.21.15.99:0/3007674861 >> v2:172.21.15.196:3300/0 conn(0x7f30b4160030 0x7f30b40b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 -- 172.21.15.99:0/3007674861 >> v1:172.21.15.196:6789/0 conn(0x7f30b40d0130 legacy=0x7f30b40cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.059+0000 7f30b37fe640 1 -- 172.21.15.99:0/3007674861 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b40d1240 con 0x7f30b40d1760 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30b37fe640 1 --2- 172.21.15.99:0/3007674861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40cd5b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f30b40727b0 tx=0x7f309c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e454a36b51e2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30b2ffd640 1 -- 172.21.15.99:0/3007674861 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c037070 con 0x7f30b40d1760 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30b2ffd640 1 -- 172.21.15.99:0/3007674861 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c032440 con 0x7f30b40d1760 2024-09-10T11:36:03.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30b2ffd640 1 -- 172.21.15.99:0/3007674861 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c032810 con 0x7f30b40d1760 2024-09-10T11:36:03.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 -- 172.21.15.99:0/3007674861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 msgr2=0x7f30b40cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 --2- 172.21.15.99:0/3007674861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40cd5b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f30b40727b0 tx=0x7f309c02a230 comp rx=0 tx=0).stop 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 -- 172.21.15.99:0/3007674861 shutdown_connections 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 --2- 172.21.15.99:0/3007674861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40cd5b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 --2- 172.21.15.99:0/3007674861 >> v2:172.21.15.196:3300/0 conn(0x7f30b4160030 0x7f30b40b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 -- 172.21.15.99:0/3007674861 >> 172.21.15.99:0/3007674861 conn(0x7f30b40b4650 msgr2=0x7f30b40b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 -- 172.21.15.99:0/3007674861 shutdown_connections 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.060+0000 7f30ba78d640 1 -- 172.21.15.99:0/3007674861 wait complete. 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 Processor -- start 2024-09-10T11:36:03.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 -- start start 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30b40d0130 0x7f30b4149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30b4160030 0x7f30b40ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4160590 con 0x7f30b40d1760 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.062+0000 7f30ba78d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4160410 con 0x7f30b4160030 2024-09-10T11:36:03.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30ba78d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30b4160710 con 0x7f30b40d0130 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56288/0 (socket says 172.21.15.99:56288) 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 -- 172.21.15.99:0/2428156214 learned_addr learned my addr 172.21.15.99:0/2428156214 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b3fff640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30b40d0130 0x7f30b4149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b8d2a640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30b4160030 0x7f30b40ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 -- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30b40d0130 msgr2=0x7f30b4149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30b40d0130 0x7f30b4149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 -- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30b4160030 msgr2=0x7f30b40ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30b4160030 0x7f30b40ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 -- 172.21.15.99:0/2428156214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b40cc940 con 0x7f30b40d1760 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b37fe640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f309c000c00 tx=0x7f309c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c041020 con 0x7f30b40d1760 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f309c0306c0 con 0x7f30b40d1760 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b40ccc20 con 0x7f30b40d1760 2024-09-10T11:36:03.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c037040 con 0x7f30b40d1760 2024-09-10T11:36:03.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.063+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b40b0d90 con 0x7f30b40d1760 2024-09-10T11:36:03.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.064+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b40cfc80 con 0x7f30b40d1760 2024-09-10T11:36:03.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.065+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f309c044020 con 0x7f30b40d1760 2024-09-10T11:36:03.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.065+0000 7f30b0ff9640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 0x7f309406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.065+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f309c0ab520 con 0x7f30b40d1760 2024-09-10T11:36:03.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.065+0000 7f30b3fff640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 0x7f309406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.066+0000 7f30b3fff640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 0x7f309406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.070+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f309c04b030 con 0x7f30b40d1760 2024-09-10T11:36:03.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.248+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f30b40cf9a0 con 0x7f30b40d1760 2024-09-10T11:36:03.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.248+0000 7f30b0ff9640 1 -- 172.21.15.99:0/2428156214 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v141) ==== 80+0+0 (secure 0 0 0) 0x7f309c079910 con 0x7f30b40d1760 2024-09-10T11:36:03.249 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 msgr2=0x7f309406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 0x7f309406e2f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4008040 comp rx=0 tx=0).stop 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 msgr2=0x7f30b40c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f309c000c00 tx=0x7f309c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 shutdown_connections 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f309406be30 0x7f309406e2f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30b4160030 0x7f30b40ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30b40d1760 0x7f30b40c9b90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 --2- 172.21.15.99:0/2428156214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30b40d0130 0x7f30b4149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 >> 172.21.15.99:0/2428156214 conn(0x7f30b40b4650 msgr2=0x7f30b40b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 shutdown_connections 2024-09-10T11:36:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.250+0000 7f30ba78d640 1 -- 172.21.15.99:0/2428156214 wait complete. 2024-09-10T11:36:03.262 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:36:03.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:03.262 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:03.286 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38177]: fuse finished with error 0 and tester_r 0 2024-09-10T11:36:03.288 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:36:03.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:03.288 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:36:03.428 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:36:03.428 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:36:03.428 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:03.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:36:03.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:36:03.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:03.445 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:36:03.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:36:03.500 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:36:03.500 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:36:03.501 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:36:03.501 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:03.517 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:36:03.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:36:03.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:03.517 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:36:03.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:36:03.572 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:36:03.572 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:03.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.670+0000 7f9fdb227640 1 Processor -- start 2024-09-10T11:36:03.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.670+0000 7f9fdb227640 1 -- start start 2024-09-10T11:36:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fdb227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 0x7f9fd4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fdb227640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fd406e5d0 0x7f9fd406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fdb227640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd4152e60 con 0x7f9fd415c060 2024-09-10T11:36:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fdb227640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd406ef50 con 0x7f9fd406e5d0 2024-09-10T11:36:03.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fdb227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd40d15e0 con 0x7f9fd40d1200 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fcbfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fd406e5d0 0x7f9fd406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fcbfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9fd406e5d0 0x7f9fd406ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56298/0 (socket says 172.21.15.99:56298) 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fd97c4640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9fd415c060 0x7f9fd415c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55306/0 (socket says 172.21.15.99:55306) 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.671+0000 7f9fcbfff640 1 -- 172.21.15.99:0/3381125547 learned_addr learned my addr 172.21.15.99:0/3381125547 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcbfff640 1 -- 172.21.15.99:0/3381125547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 msgr2=0x7f9fd4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fd8fc3640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 0x7f9fd4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcbfff640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 0x7f9fd4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcbfff640 1 -- 172.21.15.99:0/3381125547 >> v1:172.21.15.99:6789/0 conn(0x7f9fd415c060 legacy=0x7f9fd415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcb7fe640 1 -- 172.21.15.99:0/3381125547 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038553022 0 0) 0x7f9fd4152e60 con 0x7f9fd415c060 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcbfff640 1 -- 172.21.15.99:0/3381125547 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd4072680 con 0x7f9fd406e5d0 2024-09-10T11:36:03.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fd8fc3640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 0x7f9fd4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcbfff640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd406e5d0 0x7f9fd406ea10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009870 tx=0x7f9fc402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838b0fdd2f52ff5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.672+0000 7f9fcb7fe640 1 -- 172.21.15.99:0/3381125547 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4037070 con 0x7f9fd406e5d0 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fcb7fe640 1 -- 172.21.15.99:0/3381125547 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fc4032440 con 0x7f9fd406e5d0 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fcb7fe640 1 -- 172.21.15.99:0/3381125547 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4032810 con 0x7f9fd406e5d0 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 -- 172.21.15.99:0/3381125547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd406e5d0 msgr2=0x7f9fd406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd406e5d0 0x7f9fd406ea10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009870 tx=0x7f9fc402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 -- 172.21.15.99:0/3381125547 shutdown_connections 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd406e5d0 0x7f9fd406ea10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 --2- 172.21.15.99:0/3381125547 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd40d1200 0x7f9fd4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 -- 172.21.15.99:0/3381125547 >> 172.21.15.99:0/3381125547 conn(0x7f9fd406bc10 msgr2=0x7f9fd406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 -- 172.21.15.99:0/3381125547 shutdown_connections 2024-09-10T11:36:03.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.673+0000 7f9fdb227640 1 -- 172.21.15.99:0/3381125547 wait complete. 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.674+0000 7f9fdb227640 1 Processor -- start 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.674+0000 7f9fdb227640 1 -- start start 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd406e5d0 0x7f9fd415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd40d1200 0x7f9fd415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4072f20 con 0x7f9fd415c060 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd4072c40 con 0x7f9fd40d1200 2024-09-10T11:36:03.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fdb227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd415cd90 con 0x7f9fd406e5d0 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56310/0 (socket says 172.21.15.99:56310) 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 -- 172.21.15.99:0/596893913 learned_addr learned my addr 172.21.15.99:0/596893913 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 -- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd406e5d0 msgr2=0x7f9fd415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd406e5d0 0x7f9fd415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 -- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd40d1200 msgr2=0x7f9fd415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd40d1200 0x7f9fd415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.675+0000 7f9fd97c4640 1 -- 172.21.15.99:0/596893913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd40c5950 con 0x7f9fd415c060 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fd97c4640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c7a0 tx=0x7f9fcc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc018070 con 0x7f9fd415c060 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc00a640 con 0x7f9fd415c060 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00aa70 con 0x7f9fd415c060 2024-09-10T11:36:03.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd40c5c30 con 0x7f9fd415c060 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.676+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd40c61f0 con 0x7f9fd415c060 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.677+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd406ea10 con 0x7f9fd415c060 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.677+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fcc01b070 con 0x7f9fd415c060 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.678+0000 7f9fc97fa640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 0x7f9fc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.678+0000 7f9fd8fc3640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 0x7f9fc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.678+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f9fcc08a4c0 con 0x7f9fd415c060 2024-09-10T11:36:03.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.678+0000 7f9fd8fc3640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 0x7f9fc006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006fd0 tx=0x7f9fd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.682+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fcc058e70 con 0x7f9fd415c060 2024-09-10T11:36:03.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.878+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fd4154830 con 0x7f9fd415c060 2024-09-10T11:36:03.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.879+0000 7f9fc97fa640 1 -- 172.21.15.99:0/596893913 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f9fcc058770 con 0x7f9fd415c060 2024-09-10T11:36:03.880 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:03.880 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:36:03.880 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:36:03.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 msgr2=0x7f9fc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 0x7f9fc006e2f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006fd0 tx=0x7f9fd0008040 comp rx=0 tx=0).stop 2024-09-10T11:36:03.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 msgr2=0x7f9fd4152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c7a0 tx=0x7f9fcc00cc60 comp rx=0 tx=0).stop 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 shutdown_connections 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9fc006be30 0x7f9fc006e2f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9fd415c060 0x7f9fd4152c30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9fd40d1200 0x7f9fd415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 --2- 172.21.15.99:0/596893913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9fd406e5d0 0x7f9fd415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 >> 172.21.15.99:0/596893913 conn(0x7f9fd406bc10 msgr2=0x7f9fd406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 shutdown_connections 2024-09-10T11:36:03.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.881+0000 7f9fdb227640 1 -- 172.21.15.99:0/596893913 wait complete. 2024-09-10T11:36:03.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:03.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.990+0000 7fb0b31be640 1 Processor -- start 2024-09-10T11:36:03.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.990+0000 7fb0b31be640 1 -- start start 2024-09-10T11:36:03.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b31be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b31be640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b31be640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ac0ceb10 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b31be640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ac06f2a0 con 0x7fb0ac06ee30 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b31be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b3840 con 0x7fb0ac0d1180 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56314/0 (socket says 172.21.15.99:56314) 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 -- 172.21.15.99:0/1793301945 learned_addr learned my addr 172.21.15.99:0/1793301945 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0a3fff640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 -- 172.21.15.99:0/1793301945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 msgr2=0x7fb0ac06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 -- 172.21.15.99:0/1793301945 >> v1:172.21.15.196:6789/0 conn(0x7fb0ac15c7b0 legacy=0x7fb0ac0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.991+0000 7fb0b175b640 1 -- 172.21.15.99:0/1793301945 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac0b3a90 con 0x7fb0ac06ee30 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.992+0000 7fb0b175b640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009e50 tx=0x7fb0a402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b1b9da397d9f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.992+0000 7fb0a37fe640 1 -- 172.21.15.99:0/1793301945 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4036070 con 0x7fb0ac06ee30 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.992+0000 7fb0a37fe640 1 -- 172.21.15.99:0/1793301945 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a4031600 con 0x7fb0ac06ee30 2024-09-10T11:36:03.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.992+0000 7fb0a37fe640 1 -- 172.21.15.99:0/1793301945 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40319d0 con 0x7fb0ac06ee30 2024-09-10T11:36:03.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 -- 172.21.15.99:0/1793301945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac06ee30 msgr2=0x7fb0ac06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009e50 tx=0x7fb0a402a6c0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 -- 172.21.15.99:0/1793301945 shutdown_connections 2024-09-10T11:36:03.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac06d4c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1793301945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 -- 172.21.15.99:0/1793301945 >> 172.21.15.99:0/1793301945 conn(0x7fb0ac06bc10 msgr2=0x7fb0ac06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 -- 172.21.15.99:0/1793301945 shutdown_connections 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.993+0000 7fb0b31be640 1 -- 172.21.15.99:0/1793301945 wait complete. 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.994+0000 7fb0b31be640 1 Processor -- start 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.994+0000 7fb0b31be640 1 -- start start 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac14b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b4330 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b4090 con 0x7fb0ac06ee30 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b31be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0b4610 con 0x7fb0ac0d1180 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56322/0 (socket says 172.21.15.99:56322) 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 -- 172.21.15.99:0/1359456396 learned_addr learned my addr 172.21.15.99:0/1359456396 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0a3fff640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b0f5a640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac14b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 -- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 msgr2=0x7fb0ac156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 -- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac06ee30 msgr2=0x7fb0ac14b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac14b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.995+0000 7fb0b175b640 1 -- 172.21.15.99:0/1359456396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac14cf20 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0b175b640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009f70 tx=0x7fb0a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4036050 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a4042070 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4009510 con 0x7fb0ac15c7b0 2024-09-10T11:36:03.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac14d200 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.996+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac14d7c0 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.997+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac15cd00 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.997+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0a4007bd0 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.998+0000 7fb0a17fa640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 0x7fb09806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.998+0000 7fb0b0f5a640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 0x7fb09806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.998+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7fb0a40abbe0 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:03.998+0000 7fb0b0f5a640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 0x7fb09806e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8000f60 tx=0x7fb0a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.002+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0ac0ceb10 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.204+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0ac158330 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.205+0000 7fb0a17fa640 1 -- 172.21.15.99:0/1359456396 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7fb0a4079f40 con 0x7fb0ac15c7b0 2024-09-10T11:36:04.206 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:04.206 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:36:04.206 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:36:04.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 msgr2=0x7fb09806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 0x7fb09806e2f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8000f60 tx=0x7fb0a8009040 comp rx=0 tx=0).stop 2024-09-10T11:36:04.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 msgr2=0x7fb0ac156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009f70 tx=0x7fb0a4004a40 comp rx=0 tx=0).stop 2024-09-10T11:36:04.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 shutdown_connections 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb09806be30 0x7fb09806e2f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0ac15c7b0 0x7fb0ac156680 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0ac0d1180 0x7fb0ac156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 --2- 172.21.15.99:0/1359456396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ac06ee30 0x7fb0ac14b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 >> 172.21.15.99:0/1359456396 conn(0x7fb0ac06bc10 msgr2=0x7fb0ac0b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 shutdown_connections 2024-09-10T11:36:04.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.207+0000 7fb0b31be640 1 -- 172.21.15.99:0/1359456396 wait complete. 2024-09-10T11:36:04.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.316+0000 7f9692fe4640 1 Processor -- start 2024-09-10T11:36:04.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9692fe4640 1 -- start start 2024-09-10T11:36:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9692fe4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f968c0d5010 0x7f968c0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9692fe4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968c15ff70 con 0x7f968c15eb80 2024-09-10T11:36:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9692fe4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968c0d1620 con 0x7f968c1626c0 2024-09-10T11:36:04.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9692fe4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f968c0d30c0 con 0x7f968c0d5010 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9690d80640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f968c15eb80 0x7f968c0d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55322/0 (socket says 172.21.15.99:55322) 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.317+0000 7f9690d80640 1 -- 172.21.15.99:0/2336143423 learned_addr learned my addr 172.21.15.99:0/2336143423 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065155354 0 0) 0x7f968c15ff70 con 0x7f968c15eb80 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9674003520 con 0x7f968c15eb80 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 --2- 172.21.15.99:0/2336143423 >> v2:172.21.15.196:3300/0 conn(0x7f968c0d5010 0x7f968c0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1477400195 0 0) 0x7f9674003520 con 0x7f968c15eb80 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 >> v2:172.21.15.196:3300/0 conn(0x7f968c0d5010 msgr2=0x7f968c0d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 --2- 172.21.15.99:0/2336143423 >> v2:172.21.15.196:3300/0 conn(0x7f968c0d5010 0x7f968c0d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 >> v1:172.21.15.196:6789/0 conn(0x7f968c1626c0 legacy=0x7f968c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c162bc0 con 0x7f968c15eb80 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f967c001670 con 0x7f968c15eb80 2024-09-10T11:36:04.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 >> v1:172.21.15.99:6789/0 conn(0x7f968c15eb80 legacy=0x7f968c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96740039b0 0x7f9674003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9674004800 0x7f9674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f967400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968c0d30c0 con 0x7f96740051c0 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f967400c160 con 0x7f96740039b0 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f967400c360 con 0x7f9674004800 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f967400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9690d80640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96740039b0 0x7f9674003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f968bfff640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9674004800 0x7f9674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 -- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9674004800 msgr2=0x7f9674004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9674004800 0x7f9674004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 -- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96740039b0 msgr2=0x7f9674003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:04.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96740039b0 0x7f9674003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.318+0000 7f9691581640 1 -- 172.21.15.99:0/2336143423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9674003520 con 0x7f96740051c0 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.319+0000 7f9691581640 1 -- 172.21.15.99:0/2336143423 reap_dead start 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.319+0000 7f9691581640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f967400b9c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9684009fc0 tx=0x7f9674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90766b58db2bf274 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.319+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035020 con 0x7f96740051c0 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.319+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9684003bd0 con 0x7f96740051c0 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.319+0000 7f968b7fe640 1 -- 172.21.15.99:0/2336143423 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684009c10 con 0x7f96740051c0 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 -- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 msgr2=0x7f967400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f967400b9c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9684009fc0 tx=0x7f9674005830 comp rx=0 tx=0).stop 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 -- 172.21.15.99:0/2336143423 shutdown_connections 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 --2- 172.21.15.99:0/2336143423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f967400b9c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 -- 172.21.15.99:0/2336143423 >> 172.21.15.99:0/2336143423 conn(0x7f968c08e680 msgr2=0x7f968c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 -- 172.21.15.99:0/2336143423 shutdown_connections 2024-09-10T11:36:04.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.320+0000 7f9692fe4640 1 -- 172.21.15.99:0/2336143423 wait complete. 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.321+0000 7f9692fe4640 1 Processor -- start 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.321+0000 7f9692fe4640 1 -- start start 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f968c0cbd90 0x7f968c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f968c150420 0x7f968c14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968c163460 con 0x7f96740051c0 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f968c163180 con 0x7f968c150420 2024-09-10T11:36:04.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9692fe4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f968c0d29b0 con 0x7f968c0cbd90 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56342/0 (socket says 172.21.15.99:56342) 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 -- 172.21.15.99:0/3211868818 learned_addr learned my addr 172.21.15.99:0/3211868818 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9691581640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f968c150420 0x7f968c14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f968bfff640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f968c0cbd90 0x7f968c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 -- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f968c0cbd90 msgr2=0x7f968c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f968c0cbd90 0x7f968c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 -- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f968c150420 msgr2=0x7f968c14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f968c150420 0x7f968c14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.322+0000 7f9690d80640 1 -- 172.21.15.99:0/3211868818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c0cc300 con 0x7f96740051c0 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f9690d80640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f967c007290 tx=0x7f967c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00f940 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967c018070 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00fce0 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c15b310 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.323+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c15b8d0 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.324+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c09e4d0 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.324+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f967c01d460 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.325+0000 7f96897fa640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 0x7f967806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.325+0000 7f968bfff640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 0x7f967806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.325+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f967c08ad40 con 0x7f96740051c0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.325+0000 7f968bfff640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 0x7f967806e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.329+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f968c09e4d0 con 0x7f96740051c0 2024-09-10T11:36:04.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.493+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f968c0d2d40 con 0x7f96740051c0 2024-09-10T11:36:04.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.494+0000 7f96897fa640 1 -- 172.21.15.99:0/3211868818 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16751 (secure 0 0 0) 0x7f967c059780 con 0x7f96740051c0 2024-09-10T11:36:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:04.495 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":141,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:00.680818+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:31.809882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:32.181001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 msgr2=0x7f967806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 0x7f967806e2f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9680000c00 tx=0x7f9680009040 comp rx=0 tx=0).stop 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 msgr2=0x7f968c14d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f967c007290 tx=0x7f967c007690 comp rx=0 tx=0).stop 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 shutdown_connections 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f967806be30 0x7f967806e2f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f968c150420 0x7f968c14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f968c0cbd90 0x7f968c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 --2- 172.21.15.99:0/3211868818 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96740051c0 0x7f968c14d900 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 >> 172.21.15.99:0/3211868818 conn(0x7f968c08e680 msgr2=0x7f968c15ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 shutdown_connections 2024-09-10T11:36:04.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.496+0000 7f9692fe4640 1 -- 172.21.15.99:0/3211868818 wait complete. 2024-09-10T11:36:04.508 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:36:04.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.609+0000 7f0f7e349640 1 Processor -- start 2024-09-10T11:36:04.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.609+0000 7f0f7e349640 1 -- start start 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f7e349640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f7e349640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0f7806e5d0 0x7f0f7806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f7e349640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f78152e60 con 0x7f0f7815c060 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f7e349640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0f7806ef50 con 0x7f0f7806e5d0 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f7e349640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0f780d15e0 con 0x7f0f780d1200 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f777fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0f7806e5d0 0x7f0f7806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:48150/0 (socket says 172.21.15.99:48150) 2024-09-10T11:36:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 -- 172.21.15.99:0/52276553 learned_addr learned my addr 172.21.15.99:0/52276553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 -- 172.21.15.99:0/52276553 >> v2:172.21.15.196:3300/0 conn(0x7f0f7806e5d0 msgr2=0x7f0f7806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 --2- 172.21.15.99:0/52276553 >> v2:172.21.15.196:3300/0 conn(0x7f0f7806e5d0 0x7f0f7806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 -- 172.21.15.99:0/52276553 >> v1:172.21.15.196:6789/0 conn(0x7f0f7815c060 legacy=0x7f0f7815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.610+0000 7f0f77fff640 1 -- 172.21.15.99:0/52276553 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f78072680 con 0x7f0f780d1200 2024-09-10T11:36:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.611+0000 7f0f77fff640 1 --2- 172.21.15.99:0/52276553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800bb10 tx=0x7f0f6802c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f49819b88cb072 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.611+0000 7f0f76ffd640 1 -- 172.21.15.99:0/52276553 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68032790 con 0x7f0f780d1200 2024-09-10T11:36:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.611+0000 7f0f76ffd640 1 -- 172.21.15.99:0/52276553 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f68038070 con 0x7f0f780d1200 2024-09-10T11:36:04.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.611+0000 7f0f76ffd640 1 -- 172.21.15.99:0/52276553 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68032b50 con 0x7f0f780d1200 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 -- 172.21.15.99:0/52276553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 msgr2=0x7f0f78073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 --2- 172.21.15.99:0/52276553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800bb10 tx=0x7f0f6802c890 comp rx=0 tx=0).stop 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 -- 172.21.15.99:0/52276553 shutdown_connections 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 --2- 172.21.15.99:0/52276553 >> v2:172.21.15.196:3300/0 conn(0x7f0f7806e5d0 0x7f0f7806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 --2- 172.21.15.99:0/52276553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f78073440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 -- 172.21.15.99:0/52276553 >> 172.21.15.99:0/52276553 conn(0x7f0f7806bc10 msgr2=0x7f0f7806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 -- 172.21.15.99:0/52276553 shutdown_connections 2024-09-10T11:36:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.612+0000 7f0f7e349640 1 -- 172.21.15.99:0/52276553 wait complete. 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.613+0000 7f0f7e349640 1 Processor -- start 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.613+0000 7f0f7e349640 1 -- start start 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f780aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 0x7f0f781f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78072f20 con 0x7f0f7806e5d0 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f78072c40 con 0x7f0f7815c060 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f7e349640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7815cd90 con 0x7f0f780d1200 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f77fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56358/0 (socket says 172.21.15.99:56358) 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.614+0000 7f0f77fff640 1 -- 172.21.15.99:0/1993585763 learned_addr learned my addr 172.21.15.99:0/1993585763 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f777fe640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f780aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f7c8e6640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 0x7f0f781f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 -- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 msgr2=0x7f0f780aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f780aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 -- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 msgr2=0x7f0f781f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 0x7f0f781f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 -- 172.21.15.99:0/1993585763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f781f12c0 con 0x7f0f7806e5d0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f7c8e6640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 0x7f0f781f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f77fff640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800b2c0 tx=0x7f0f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68033620 con 0x7f0f7806e5d0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f68033800 con 0x7f0f7806e5d0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f780b1520 con 0x7f0f7806e5d0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68033bf0 con 0x7f0f7806e5d0 2024-09-10T11:36:04.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.615+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f780adb00 con 0x7f0f7806e5d0 2024-09-10T11:36:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.616+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f7806ea10 con 0x7f0f7806e5d0 2024-09-10T11:36:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.616+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f6804d060 con 0x7f0f7806e5d0 2024-09-10T11:36:04.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.617+0000 7f0f74ff9640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 0x7f0f6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.617+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f0f680adad0 con 0x7f0f7806e5d0 2024-09-10T11:36:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.617+0000 7f0f777fe640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 0x7f0f6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.617+0000 7f0f777fe640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 0x7f0f6406e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f781f3700 tx=0x7f0f6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.621+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f68041070 con 0x7f0f7806e5d0 2024-09-10T11:36:04.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.819+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f780761b0 con 0x7f0f7806e5d0 2024-09-10T11:36:04.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.820+0000 7f0f74ff9640 1 -- 172.21.15.99:0/1993585763 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f0f6807c630 con 0x7f0f7806e5d0 2024-09-10T11:36:04.821 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:04.821 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:36:04.821 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 msgr2=0x7f0f6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 0x7f0f6406e2f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0f781f3700 tx=0x7f0f6c008040 comp rx=0 tx=0).stop 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 msgr2=0x7f0f780b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800b2c0 tx=0x7f0f680046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 shutdown_connections 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0f6406be30 0x7f0f6406e2f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f7815c060 0x7f0f781f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0f780d1200 0x7f0f780aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 --2- 172.21.15.99:0/1993585763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f7806e5d0 0x7f0f780b0fe0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 >> 172.21.15.99:0/1993585763 conn(0x7f0f7806bc10 msgr2=0x7f0f780cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 shutdown_connections 2024-09-10T11:36:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.822+0000 7f0f7e349640 1 -- 172.21.15.99:0/1993585763 wait complete. 2024-09-10T11:36:04.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:04.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.932+0000 7f4b69dbd640 1 Processor -- start 2024-09-10T11:36:04.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.932+0000 7f4b69dbd640 1 -- start start 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b69dbd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b640d1200 0x7f4b64073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b69dbd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b69dbd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b64152e60 con 0x7f4b6406e5d0 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b69dbd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b6406ef80 con 0x7f4b640d1200 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b69dbd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b640d15e0 con 0x7f4b6415c770 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:48164/0 (socket says 172.21.15.99:48164) 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 -- 172.21.15.99:0/3541237380 learned_addr learned my addr 172.21.15.99:0/3541237380 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b637fe640 1 --2- 172.21.15.99:0/3541237380 >> v2:172.21.15.196:3300/0 conn(0x7f4b640d1200 0x7f4b64073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 -- 172.21.15.99:0/3541237380 >> v2:172.21.15.196:3300/0 conn(0x7f4b640d1200 msgr2=0x7f4b64073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b627fc640 1 -- 172.21.15.99:0/3541237380 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2068742211 0 0) 0x7f4b64152e60 con 0x7f4b6406e5d0 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 --2- 172.21.15.99:0/3541237380 >> v2:172.21.15.196:3300/0 conn(0x7f4b640d1200 0x7f4b64073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 -- 172.21.15.99:0/3541237380 >> v1:172.21.15.196:6789/0 conn(0x7f4b6406e5d0 legacy=0x7f4b6415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.933+0000 7f4b63fff640 1 -- 172.21.15.99:0/3541237380 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b64072680 con 0x7f4b6415c770 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.934+0000 7f4b63fff640 1 --2- 172.21.15.99:0/3541237380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d150 tx=0x7f4b5c02c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=323995167bf052d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.934+0000 7f4b627fc640 1 -- 172.21.15.99:0/3541237380 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c039070 con 0x7f4b6415c770 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.934+0000 7f4b627fc640 1 -- 172.21.15.99:0/3541237380 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5c009600 con 0x7f4b6415c770 2024-09-10T11:36:04.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.934+0000 7f4b627fc640 1 -- 172.21.15.99:0/3541237380 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0099d0 con 0x7f4b6415c770 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3541237380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 msgr2=0x7f4b6406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3541237380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d150 tx=0x7f4b5c02c890 comp rx=0 tx=0).stop 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3541237380 shutdown_connections 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3541237380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b6406ea40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3541237380 >> v2:172.21.15.196:3300/0 conn(0x7f4b640d1200 0x7f4b64073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3541237380 >> 172.21.15.99:0/3541237380 conn(0x7f4b6406bc10 msgr2=0x7f4b6406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:04.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3541237380 shutdown_connections 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.935+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3541237380 wait complete. 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.936+0000 7f4b69dbd640 1 Processor -- start 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.936+0000 7f4b69dbd640 1 -- start start 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b640d1200 0x7f4b641429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b641f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64072f20 con 0x7f4b6406e5d0 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64072c40 con 0x7f4b640d1200 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b69dbd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b6415cd90 con 0x7f4b6415c770 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56364/0 (socket says 172.21.15.99:56364) 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 -- 172.21.15.99:0/3090556971 learned_addr learned my addr 172.21.15.99:0/3090556971 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:04.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b62ffd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b640d1200 0x7f4b641429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b63fff640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b641f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 -- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 msgr2=0x7f4b641f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b641f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 -- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b640d1200 msgr2=0x7f4b641429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b640d1200 0x7f4b641429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.937+0000 7f4b637fe640 1 -- 172.21.15.99:0/3090556971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641f8950 con 0x7f4b6406e5d0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b637fe640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e870 tx=0x7f4b5400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b54017070 con 0x7f4b6406e5d0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b5400c940 con 0x7f4b6406e5d0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400cd30 con 0x7f4b6406e5d0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b641f8390 con 0x7f4b6406e5d0 2024-09-10T11:36:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.938+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641421b0 con 0x7f4b6406e5d0 2024-09-10T11:36:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.939+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b64073440 con 0x7f4b6406e5d0 2024-09-10T11:36:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.939+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b5401a070 con 0x7f4b6406e5d0 2024-09-10T11:36:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.940+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 0x7f4b5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.940+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f4b5408af40 con 0x7f4b6406e5d0 2024-09-10T11:36:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.940+0000 7f4b62ffd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 0x7f4b5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.940+0000 7f4b62ffd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 0x7f4b5006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4b6415ba60 tx=0x7f4b58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:04.944+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b54059330 con 0x7f4b6406e5d0 2024-09-10T11:36:05.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.144+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b640b4b80 con 0x7f4b6406e5d0 2024-09-10T11:36:05.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.144+0000 7f4b4bfff640 1 -- 172.21.15.99:0/3090556971 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5225 (secure 0 0 0) 0x7f4b5405d630 con 0x7f4b6406e5d0 2024-09-10T11:36:05.145 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:05.145 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"btime":"2024-09-10T11:35:35:879740+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:35:33.202363+0000","modified":"2024-09-10T11:35:35.207223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3835517741","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":3835517741},{"type":"v1","addr":"172.21.15.196:6835","nonce":3835517741}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6990,"qdb_cluster":[6990]},"id":8}]} 2024-09-10T11:36:05.146 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 55 2024-09-10T11:36:05.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 msgr2=0x7f4b5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 0x7f4b5006e2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4b6415ba60 tx=0x7f4b58009040 comp rx=0 tx=0).stop 2024-09-10T11:36:05.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 msgr2=0x7f4b6413f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400e870 tx=0x7f4b5400ed30 comp rx=0 tx=0).stop 2024-09-10T11:36:05.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 shutdown_connections 2024-09-10T11:36:05.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b5006be30 0x7f4b5006e2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b6415c770 0x7f4b641f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b640d1200 0x7f4b641429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 --2- 172.21.15.99:0/3090556971 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b6406e5d0 0x7f4b6413f810 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 >> 172.21.15.99:0/3090556971 conn(0x7f4b6406bc10 msgr2=0x7f4b64151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 shutdown_connections 2024-09-10T11:36:05.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.146+0000 7f4b69dbd640 1 -- 172.21.15.99:0/3090556971 wait complete. 2024-09-10T11:36:05.158 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:05.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 Processor -- start 2024-09-10T11:36:05.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 -- start start 2024-09-10T11:36:05.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 0x7fbb1c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1c15ff70 con 0x7fbb1c1626c0 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c0d1620 con 0x7fbb1c15eb80 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.256+0000 7fbb22445640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c162bc0 con 0x7fbb1c0d5010 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1b7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbb1c1626c0 0x7fbb1c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55336/0 (socket says 172.21.15.99:55336) 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56370/0 (socket says 172.21.15.99:56370) 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1b7fe640 1 -- 172.21.15.99:0/3694393415 learned_addr learned my addr 172.21.15.99:0/3694393415 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 -- 172.21.15.99:0/3694393415 >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 msgr2=0x7fbb1c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb209e2640 1 --2- 172.21.15.99:0/3694393415 >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 0x7fbb1c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 --2- 172.21.15.99:0/3694393415 >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 0x7fbb1c0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 -- 172.21.15.99:0/3694393415 >> v1:172.21.15.99:6789/0 conn(0x7fbb1c1626c0 legacy=0x7fbb1c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1affd640 1 -- 172.21.15.99:0/3694393415 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 786044834 0 0) 0x7fbb1c15ff70 con 0x7fbb1c1626c0 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb1bfff640 1 -- 172.21.15.99:0/3694393415 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c162ea0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.257+0000 7fbb209e2640 1 --2- 172.21.15.99:0/3694393415 >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 0x7fbb1c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb1bfff640 1 --2- 172.21.15.99:0/3694393415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097a0 tx=0x7fbb0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca14343ec0bfbec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb1affd640 1 -- 172.21.15.99:0/3694393415 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037070 con 0x7fbb1c15eb80 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb1affd640 1 -- 172.21.15.99:0/3694393415 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c02ae50 con 0x7fbb1c15eb80 2024-09-10T11:36:05.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb1affd640 1 -- 172.21.15.99:0/3694393415 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c032660 con 0x7fbb1c15eb80 2024-09-10T11:36:05.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 -- 172.21.15.99:0/3694393415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 msgr2=0x7fbb1c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 --2- 172.21.15.99:0/3694393415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0097a0 tx=0x7fbb0c02a140 comp rx=0 tx=0).stop 2024-09-10T11:36:05.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 -- 172.21.15.99:0/3694393415 shutdown_connections 2024-09-10T11:36:05.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 --2- 172.21.15.99:0/3694393415 >> v2:172.21.15.196:3300/0 conn(0x7fbb1c0d5010 0x7fbb1c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 --2- 172.21.15.99:0/3694393415 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0d29b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 -- 172.21.15.99:0/3694393415 >> 172.21.15.99:0/3694393415 conn(0x7fbb1c08e680 msgr2=0x7fbb1c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.258+0000 7fbb22445640 1 -- 172.21.15.99:0/3694393415 shutdown_connections 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.259+0000 7fbb22445640 1 -- 172.21.15.99:0/3694393415 wait complete. 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.260+0000 7fbb22445640 1 Processor -- start 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.260+0000 7fbb22445640 1 -- start start 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.260+0000 7fbb22445640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1c0d5010 0x7fbb1c1598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c1626c0 0x7fbb1c0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c0d2ef0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c163460 con 0x7fbb1c1626c0 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1c0d3070 con 0x7fbb1c0d5010 2024-09-10T11:36:05.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56372/0 (socket says 172.21.15.99:56372) 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 -- 172.21.15.99:0/1313060673 learned_addr learned my addr 172.21.15.99:0/1313060673 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb209e2640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c1626c0 0x7fbb1c0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1bfff640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1c0d5010 0x7fbb1c1598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 -- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1c0d5010 msgr2=0x7fbb1c1598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1c0d5010 0x7fbb1c1598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 -- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c1626c0 msgr2=0x7fbb1c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c1626c0 0x7fbb1c0cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 -- 172.21.15.99:0/1313060673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c0b1e00 con 0x7fbb1c15eb80 2024-09-10T11:36:05.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb1b7fe640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c410 tx=0x7fbb1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100074e0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb100047e0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004bd0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.261+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c0b20e0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.262+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c0b2980 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.262+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c0b2b80 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.263+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb10011070 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.263+0000 7fbb18ff9640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 0x7fbb0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.263+0000 7fbb1bfff640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 0x7fbb0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.263+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7fbb1008ab00 con 0x7fbb1c15eb80 2024-09-10T11:36:05.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.264+0000 7fbb1bfff640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 0x7fbb0806e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c034000 tx=0x7fbb0c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.268+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb1c0b2b80 con 0x7fbb1c15eb80 2024-09-10T11:36:05.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.431+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb1c0d29b0 con 0x7fbb1c15eb80 2024-09-10T11:36:05.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.432+0000 7fbb18ff9640 1 -- 172.21.15.99:0/1313060673 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16751 (secure 0 0 0) 0x7fbb1000e280 con 0x7fbb1c15eb80 2024-09-10T11:36:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:05.433 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":141,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:00.680818+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:35:31.809882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T11:35:32.181001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 msgr2=0x7fbb0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 0x7fbb0806e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c034000 tx=0x7fbb0c02e000 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 msgr2=0x7fbb1c0ba440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000c410 tx=0x7fbb1000c8d0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 shutdown_connections 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb0806be30 0x7fbb0806e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb1c1626c0 0x7fbb1c0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb1c15eb80 0x7fbb1c0ba440 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 --2- 172.21.15.99:0/1313060673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1c0d5010 0x7fbb1c1598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 >> 172.21.15.99:0/1313060673 conn(0x7fbb1c08e680 msgr2=0x7fbb1c15cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 shutdown_connections 2024-09-10T11:36:05.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.434+0000 7fbb22445640 1 -- 172.21.15.99:0/1313060673 wait complete. 2024-09-10T11:36:05.446 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:36:05.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.544+0000 7f6c7ad68640 1 Processor -- start 2024-09-10T11:36:05.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.544+0000 7f6c7ad68640 1 -- start start 2024-09-10T11:36:05.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c7ad68640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6c74073440 0x7f6c7415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c7ad68640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c74152e60 con 0x7f6c7406ece0 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c7ad68640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c740d15e0 con 0x7f6c740d1200 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c7ad68640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6c74072680 con 0x7f6c74073440 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6c7406ece0 0x7f6c7415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55342/0 (socket says 172.21.15.99:55342) 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6bfff640 1 -- 172.21.15.99:0/128247778 learned_addr learned my addr 172.21.15.99:0/128247778 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c79305640 1 --2- 172.21.15.99:0/128247778 >> v2:172.21.15.196:3300/0 conn(0x7f6c74073440 0x7f6c7415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1008861296 0 0) 0x7f6c74152e60 con 0x7f6c7406ece0 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c5c003520 con 0x7f6c7406ece0 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2129976363 0 0) 0x7f6c5c003520 con 0x7f6c7406ece0 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 >> v2:172.21.15.196:3300/0 conn(0x7f6c74073440 msgr2=0x7f6c7415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 --2- 172.21.15.99:0/128247778 >> v2:172.21.15.196:3300/0 conn(0x7f6c74073440 0x7f6c7415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 >> v1:172.21.15.196:6789/0 conn(0x7f6c740d1200 legacy=0x7f6c7406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c74072960 con 0x7f6c7406ece0 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f6c64001670 con 0x7f6c7406ece0 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 >> v1:172.21.15.99:6789/0 conn(0x7f6c7406ece0 legacy=0x7f6c7415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c5c0039b0 0x7f6c5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c004800 0x7f6c5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 0x7f6c5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.545+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74072680 con 0x7f6c5c0051c0 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c00e370 con 0x7f6c5c004800 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c5c00e570 con 0x7f6c5c0039b0 2024-09-10T11:36:05.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 0x7f6c5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c78b04640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c5c0039b0 0x7f6c5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c6bfff640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c004800 0x7f6c5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 -- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c5c0039b0 msgr2=0x7f6c5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c5c0039b0 0x7f6c5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 -- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c004800 msgr2=0x7f6c5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c004800 0x7f6c5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.546+0000 7f6c79305640 1 -- 172.21.15.99:0/128247778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c003520 con 0x7f6c5c0051c0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c79305640 1 -- 172.21.15.99:0/128247778 reap_dead start 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c79305640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 0x7f6c5c00b9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fc0 tx=0x7f6c5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43813ecca6bd627 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c035020 con 0x7f6c5c0051c0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c003bd0 con 0x7f6c5c0051c0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c6b7fe640 1 -- 172.21.15.99:0/128247778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c009c10 con 0x7f6c5c0051c0 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 -- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 msgr2=0x7f6c5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 0x7f6c5c00b9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009fc0 tx=0x7f6c5c005830 comp rx=0 tx=0).stop 2024-09-10T11:36:05.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 -- 172.21.15.99:0/128247778 shutdown_connections 2024-09-10T11:36:05.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/128247778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c5c0051c0 0x7f6c5c00b9c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 -- 172.21.15.99:0/128247778 >> 172.21.15.99:0/128247778 conn(0x7f6c7406bc10 msgr2=0x7f6c7406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 -- 172.21.15.99:0/128247778 shutdown_connections 2024-09-10T11:36:05.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.547+0000 7f6c7ad68640 1 -- 172.21.15.99:0/128247778 wait complete. 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.548+0000 7f6c7ad68640 1 Processor -- start 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.548+0000 7f6c7ad68640 1 -- start start 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 0x7f6c741f1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c741f3090 0x7f6c74155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7415cd90 con 0x7f6c741555b0 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c74072f20 con 0x7f6c5c0051c0 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c7ad68640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7415c060 con 0x7f6c741f3090 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c79305640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c79305640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56386/0 (socket says 172.21.15.99:56386) 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.549+0000 7f6c79305640 1 -- 172.21.15.99:0/3851765129 learned_addr learned my addr 172.21.15.99:0/3851765129 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c6bfff640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c741f3090 0x7f6c74155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c78b04640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 0x7f6c741f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 -- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c741f3090 msgr2=0x7f6c74155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c741f3090 0x7f6c74155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 -- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 msgr2=0x7f6c741f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 0x7f6c741f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 -- 172.21.15.99:0/3851765129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741f3510 con 0x7f6c741555b0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c78b04640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 0x7f6c741f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c79305640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e230 tx=0x7f6c6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c035030 con 0x7f6c741555b0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c043070 con 0x7f6c741555b0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741f0f60 con 0x7f6c741555b0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c02d070 con 0x7f6c741555b0 2024-09-10T11:36:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.550+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c74158410 con 0x7f6c741555b0 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.551+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7415c3f0 con 0x7f6c741555b0 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.551+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c6c005d10 con 0x7f6c741555b0 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.552+0000 7f6c697fa640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 0x7f6c6006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.552+0000 7f6c78b04640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 0x7f6c6006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.552+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5709+0+0 (secure 0 0 0) 0x7f6c6c0aba90 con 0x7f6c741555b0 2024-09-10T11:36:05.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.552+0000 7f6c78b04640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 0x7f6c6006e320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6c70000c00 tx=0x7f6c70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.556+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c6c07a4d0 con 0x7f6c741555b0 2024-09-10T11:36:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.756+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6c7415c850 con 0x7f6c741555b0 2024-09-10T11:36:05.757 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:36:05.756+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:36:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.758+0000 7f6c697fa640 1 -- 172.21.15.99:0/3851765129 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) ==== 161+0+0 (secure 0 0 0) 0x7f6c6c079dd0 con 0x7f6c741555b0 2024-09-10T11:36:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:36:05.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 msgr2=0x7f6c6006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 0x7f6c6006e320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6c70000c00 tx=0x7f6c70009040 comp rx=0 tx=0).stop 2024-09-10T11:36:05.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 msgr2=0x7f6c741f01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e230 tx=0x7f6c6c009750 comp rx=0 tx=0).stop 2024-09-10T11:36:05.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 shutdown_connections 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6c6006be60 0x7f6c6006e320 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c741555b0 0x7f6c741f01c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c741f3090 0x7f6c74155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 --2- 172.21.15.99:0/3851765129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c5c0051c0 0x7f6c741f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 >> 172.21.15.99:0/3851765129 conn(0x7f6c7406bc10 msgr2=0x7f6c7406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 shutdown_connections 2024-09-10T11:36:05.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.761+0000 7f6c7ad68640 1 -- 172.21.15.99:0/3851765129 wait complete. 2024-09-10T11:36:05.766 INFO:tasks.ceph.mds.b.smithi196.stderr: -7681> 2024-09-10T11:35:36.844+0000 7f6e81d90640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:36:05.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:36:05.799 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:36:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.873+0000 7f4bd8959640 1 Processor -- start 2024-09-10T11:36:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.873+0000 7f4bd8959640 1 -- start start 2024-09-10T11:36:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd8959640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4bd415eb80 0x7f4bd40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd8959640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4bd40d5010 0x7f4bd40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd8959640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd415ff70 con 0x7f4bd41626c0 2024-09-10T11:36:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd8959640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd40d1620 con 0x7f4bd415eb80 2024-09-10T11:36:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd8959640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd4162bc0 con 0x7f4bd40d5010 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4bd415eb80 0x7f4bd40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4bd415eb80 0x7f4bd40d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56398/0 (socket says 172.21.15.99:56398) 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd259c640 1 -- 172.21.15.99:0/2987837821 learned_addr learned my addr 172.21.15.99:0/2987837821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.874+0000 7f4bd2d9d640 1 --2- 172.21.15.99:0/2987837821 >> v2:172.21.15.196:3300/0 conn(0x7f4bd40d5010 0x7f4bd40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd259c640 1 -- 172.21.15.99:0/2987837821 >> v2:172.21.15.196:3300/0 conn(0x7f4bd40d5010 msgr2=0x7f4bd40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd259c640 1 --2- 172.21.15.99:0/2987837821 >> v2:172.21.15.196:3300/0 conn(0x7f4bd40d5010 0x7f4bd40d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd259c640 1 -- 172.21.15.99:0/2987837821 >> v1:172.21.15.196:6789/0 conn(0x7f4bd41626c0 legacy=0x7f4bd40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd259c640 1 -- 172.21.15.99:0/2987837821 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd4162ea0 con 0x7f4bd415eb80 2024-09-10T11:36:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd259c640 1 --2- 172.21.15.99:0/2987837821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd40d29b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009870 tx=0x7f4bc402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd00335c3c74a8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd159a640 1 -- 172.21.15.99:0/2987837821 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4037070 con 0x7f4bd415eb80 2024-09-10T11:36:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.875+0000 7f4bd159a640 1 -- 172.21.15.99:0/2987837821 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4032440 con 0x7f4bd415eb80 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd159a640 1 -- 172.21.15.99:0/2987837821 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4032810 con 0x7f4bd415eb80 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 -- 172.21.15.99:0/2987837821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 msgr2=0x7f4bd40d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2987837821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd40d29b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009870 tx=0x7f4bc402a210 comp rx=0 tx=0).stop 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 -- 172.21.15.99:0/2987837821 shutdown_connections 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2987837821 >> v2:172.21.15.196:3300/0 conn(0x7f4bd40d5010 0x7f4bd40d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2987837821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd40d29b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 -- 172.21.15.99:0/2987837821 >> 172.21.15.99:0/2987837821 conn(0x7f4bd408e680 msgr2=0x7f4bd409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 -- 172.21.15.99:0/2987837821 shutdown_connections 2024-09-10T11:36:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.876+0000 7f4bd8959640 1 -- 172.21.15.99:0/2987837821 wait complete. 2024-09-10T11:36:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.878+0000 7f4bd8959640 1 Processor -- start 2024-09-10T11:36:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.878+0000 7f4bd8959640 1 -- start start 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 0x7f4bd415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bd41626c0 0x7f4bd41e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40d2ef0 con 0x7f4bd415eb80 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4163460 con 0x7f4bd40d5010 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd8959640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd40d3070 con 0x7f4bd41626c0 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56410/0 (socket says 172.21.15.99:56410) 2024-09-10T11:36:05.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd1d9b640 1 -- 172.21.15.99:0/2344091091 learned_addr learned my addr 172.21.15.99:0/2344091091 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 0x7f4bd415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd2d9d640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bd41626c0 0x7f4bd41e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 -- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bd41626c0 msgr2=0x7f4bd41e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bd41626c0 0x7f4bd41e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 -- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 msgr2=0x7f4bd41e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 -- 172.21.15.99:0/2344091091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40b32e0 con 0x7f4bd40d5010 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd1d9b640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:05.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.879+0000 7f4bd259c640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 0x7f4bd415a120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4031000 tx=0x7f4bc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.880+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4041020 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.880+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4030660 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.880+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd40b2760 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.880+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4037040 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.880+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd40b2d20 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.881+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd40d29b0 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.882+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bc4044020 con 0x7f4bd40d5010 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.882+0000 7f4bc2ffd640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 0x7f4bb406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.882+0000 7f4bd1d9b640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 0x7f4bb406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:05.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.882+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5795+0+0 (secure 0 0 0) 0x7f4bc40ab410 con 0x7f4bd40d5010 2024-09-10T11:36:05.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.882+0000 7f4bd1d9b640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 0x7f4bb406e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40c8590 tx=0x7f4bc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:05.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:05.887+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4bc404b030 con 0x7f4bd40d5010 2024-09-10T11:36:06.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.089+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4bd40b7b40 con 0x7f4bd40d5010 2024-09-10T11:36:06.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.762+0000 7f4bc2ffd640 1 -- 172.21.15.99:0/2344091091 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) ==== 104+0+0 (secure 0 0 0) 0x7f4bc4079710 con 0x7f4bd40d5010 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 msgr2=0x7f4bb406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 0x7f4bb406e320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40c8590 tx=0x7f4bc8009040 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 msgr2=0x7f4bd415a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 0x7f4bd415a120 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4031000 tx=0x7f4bc40046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 shutdown_connections 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4bb406be60 0x7f4bb406e320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bd41626c0 0x7f4bd41e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4bd415eb80 0x7f4bd41e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 --2- 172.21.15.99:0/2344091091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bd40d5010 0x7f4bd415a120 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 >> 172.21.15.99:0/2344091091 conn(0x7f4bd408e680 msgr2=0x7f4bd406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 shutdown_connections 2024-09-10T11:36:06.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.764+0000 7f4bd8959640 1 -- 172.21.15.99:0/2344091091 wait complete. 2024-09-10T11:36:06.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:36:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.884+0000 7f388461e640 1 Processor -- start 2024-09-10T11:36:06.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.884+0000 7f388461e640 1 -- start start 2024-09-10T11:36:06.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f388461e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f387c06e4d0 0x7f387c167610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f388461e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387c0ceb10 con 0x7f387c15c7b0 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f388461e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387c0b3a90 con 0x7f387c0d1180 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f388461e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f387c0b3840 con 0x7f387c06e4d0 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f3882bbb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f387c06e4d0 0x7f387c167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f38823ba640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f387c15c7b0 0x7f387c0d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55344/0 (socket says 172.21.15.99:55344) 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.885+0000 7f38823ba640 1 -- 172.21.15.99:0/3230309763 learned_addr learned my addr 172.21.15.99:0/3230309763 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f3882bbb640 1 -- 172.21.15.99:0/3230309763 >> v1:172.21.15.196:6789/0 conn(0x7f387c0d1180 legacy=0x7f387c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f3882bbb640 1 -- 172.21.15.99:0/3230309763 >> v1:172.21.15.99:6789/0 conn(0x7f387c15c7b0 legacy=0x7f387c0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f38813b8640 1 -- 172.21.15.99:0/3230309763 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286835191 0 0) 0x7f3878001110 con 0x7f387c15c7b0 2024-09-10T11:36:06.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f3882bbb640 1 -- 172.21.15.99:0/3230309763 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c0b3d70 con 0x7f387c06e4d0 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f3882bbb640 1 --2- 172.21.15.99:0/3230309763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c06e4d0 0x7f387c167610 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3874009d90 tx=0x7f387402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45aa412042c6ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f38813b8640 1 -- 172.21.15.99:0/3230309763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874007740 con 0x7f387c06e4d0 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f38813b8640 1 -- 172.21.15.99:0/3230309763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38740078e0 con 0x7f387c06e4d0 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.886+0000 7f38813b8640 1 -- 172.21.15.99:0/3230309763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874007cb0 con 0x7f387c06e4d0 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 -- 172.21.15.99:0/3230309763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c06e4d0 msgr2=0x7f387c167610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 --2- 172.21.15.99:0/3230309763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c06e4d0 0x7f387c167610 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3874009d90 tx=0x7f387402a6a0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 -- 172.21.15.99:0/3230309763 shutdown_connections 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 --2- 172.21.15.99:0/3230309763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c06e4d0 0x7f387c167610 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 -- 172.21.15.99:0/3230309763 >> 172.21.15.99:0/3230309763 conn(0x7f387c06bc10 msgr2=0x7f387c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 -- 172.21.15.99:0/3230309763 shutdown_connections 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.887+0000 7f388461e640 1 -- 172.21.15.99:0/3230309763 wait complete. 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.888+0000 7f388461e640 1 Processor -- start 2024-09-10T11:36:06.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.888+0000 7f388461e640 1 -- start start 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f387c06e4d0 0x7f387c151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 0x7f387c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f387c0b4610 con 0x7f387c15c7b0 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f387c0b4370 con 0x7f387c06e4d0 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f388461e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f387c15cb90 con 0x7f387c0d1180 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:06.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56438/0 (socket says 172.21.15.99:56438) 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 -- 172.21.15.99:0/1991354268 learned_addr learned my addr 172.21.15.99:0/1991354268 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f38823ba640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f387c06e4d0 0x7f387c151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3881bb9640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 0x7f387c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 -- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 msgr2=0x7f387c14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 0x7f387c14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 -- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f387c06e4d0 msgr2=0x7f387c151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.889+0000 7f3882bbb640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f387c06e4d0 0x7f387c151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3882bbb640 1 -- 172.21.15.99:0/1991354268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c1545a0 con 0x7f387c15c7b0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3881bb9640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 0x7f387c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3882bbb640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3874009eb0 tx=0x7f3874004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874040070 con 0x7f387c15c7b0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3874031c40 con 0x7f387c15c7b0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874003990 con 0x7f387c15c7b0 2024-09-10T11:36:06.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c154880 con 0x7f387c15c7b0 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.890+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c154e40 con 0x7f387c15c7b0 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.891+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c0d0a70 con 0x7f387c15c7b0 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.891+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3874039040 con 0x7f387c15c7b0 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.892+0000 7f3872ffd640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 0x7f386006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.892+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5795+0+0 (secure 0 0 0) 0x7f38740abe20 con 0x7f387c15c7b0 2024-09-10T11:36:06.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.892+0000 7f38823ba640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 0x7f386006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:06.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.892+0000 7f38823ba640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 0x7f386006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3878007fd0 tx=0x7f3878007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:06.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:06.896+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f387407a120 con 0x7f387c15c7b0 2024-09-10T11:36:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:07.064+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f387c158e10 con 0x7f387c15c7b0 2024-09-10T11:36:08.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.056+0000 7f3872ffd640 1 -- 172.21.15.99:0/1991354268 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v143) ==== 181+0+0 (secure 0 0 0) 0x7f387407e420 con 0x7f387c15c7b0 2024-09-10T11:36:08.057 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:36:08.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 msgr2=0x7f386006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:08.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 0x7f386006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3878007fd0 tx=0x7f3878007de0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 msgr2=0x7f387c0b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:08.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3874009eb0 tx=0x7f3874004a40 comp rx=0 tx=0).stop 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 shutdown_connections 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f386006be30 0x7f386006e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f387c15c7b0 0x7f387c0b09d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f387c0d1180 0x7f387c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 --2- 172.21.15.99:0/1991354268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f387c06e4d0 0x7f387c151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.059+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 >> 172.21.15.99:0/1991354268 conn(0x7f387c06bc10 msgr2=0x7f387c159610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.060+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 shutdown_connections 2024-09-10T11:36:08.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.060+0000 7f388461e640 1 -- 172.21.15.99:0/1991354268 wait complete. 2024-09-10T11:36:08.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:36:08.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.176+0000 7f8d6f30a640 1 Processor -- start 2024-09-10T11:36:08.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 -- start start 2024-09-10T11:36:08.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 0x7f8d680d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d680d5740 0x7f8d68163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d680d2150 con 0x7f8d680b59b0 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8d680d0d30 con 0x7f8d680d5740 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.177+0000 7f8d6f30a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680d5bd0 con 0x7f8d680d16f0 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d680d5740 0x7f8d68163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d680d5740 0x7f8d68163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56440/0 (socket says 172.21.15.99:56440) 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 -- 172.21.15.99:0/1417931963 learned_addr learned my addr 172.21.15.99:0/1417931963 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 0x7f8d680d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 -- 172.21.15.99:0/1417931963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 msgr2=0x7f8d680d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 0x7f8d680d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 -- 172.21.15.99:0/1417931963 >> v1:172.21.15.196:6789/0 conn(0x7f8d680b59b0 legacy=0x7f8d68162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d8a7640 1 -- 172.21.15.99:0/1417931963 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680b5780 con 0x7f8d680d5740 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.178+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 0x7f8d680d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:08.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6d8a7640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d5740 0x7f8d68163420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009e50 tx=0x7f8d6002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85891f5144abd4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:08.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d5ffff640 1 -- 172.21.15.99:0/1417931963 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60036070 con 0x7f8d680d5740 2024-09-10T11:36:08.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d5ffff640 1 -- 172.21.15.99:0/1417931963 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60031600 con 0x7f8d680d5740 2024-09-10T11:36:08.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d5ffff640 1 -- 172.21.15.99:0/1417931963 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600319d0 con 0x7f8d680d5740 2024-09-10T11:36:08.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1417931963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d5740 msgr2=0x7f8d68163420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:08.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d5740 0x7f8d68163420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009e50 tx=0x7f8d6002a6c0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1417931963 shutdown_connections 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d5740 0x7f8d68163420 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1417931963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d16f0 0x7f8d680d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1417931963 >> 172.21.15.99:0/1417931963 conn(0x7f8d680d4c20 msgr2=0x7f8d680d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1417931963 shutdown_connections 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.179+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1417931963 wait complete. 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 Processor -- start 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 -- start start 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 0x7f8d68157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d5740 0x7f8d680b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680b52f0 con 0x7f8d680d16f0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d680b4d30 con 0x7f8d680b59b0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6f30a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8d680b4f80 con 0x7f8d680d5740 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6c8a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6c8a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56444/0 (socket says 172.21.15.99:56444) 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.181+0000 7f8d6c8a5640 1 -- 172.21.15.99:0/1944513235 learned_addr learned my addr 172.21.15.99:0/1944513235 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6d8a7640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d5740 0x7f8d680b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 0x7f8d68157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 -- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d5740 msgr2=0x7f8d680b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d5740 0x7f8d680b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 -- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 msgr2=0x7f8d68157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 0x7f8d68157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 -- 172.21.15.99:0/1944513235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d680b7d70 con 0x7f8d680d16f0 2024-09-10T11:36:08.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 0x7f8d68157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6c8a5640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800b3c0 tx=0x7f8d5800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58011830 con 0x7f8d680d16f0 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d58016070 con 0x7f8d680d16f0 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d680b9f80 con 0x7f8d680d16f0 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58011c50 con 0x7f8d680d16f0 2024-09-10T11:36:08.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.182+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d680ba540 con 0x7f8d680d16f0 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.183+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d68163ad0 con 0x7f8d680d16f0 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.184+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d5801b460 con 0x7f8d680d16f0 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.184+0000 7f8d5dffb640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.184+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.184+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5357+0+0 (secure 0 0 0) 0x7f8d5808a6f0 con 0x7f8d680d16f0 2024-09-10T11:36:08.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.185+0000 7f8d6d0a6640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 0x7f8d4c06e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8d64006fd0 tx=0x7f8d64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:08.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.189+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d58020020 con 0x7f8d680d16f0 2024-09-10T11:36:08.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:08.353+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8d68161da0 con 0x7f8d680d16f0 2024-09-10T11:36:09.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.060+0000 7f8d5dffb640 1 -- 172.21.15.99:0/1944513235 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v144) ==== 169+0+0 (secure 0 0 0) 0x7f8d58059290 con 0x7f8d680d16f0 2024-09-10T11:36:09.061 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 msgr2=0x7f8d4c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 0x7f8d4c06e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8d64006fd0 tx=0x7f8d64008040 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 msgr2=0x7f8d680b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800b3c0 tx=0x7f8d5800b880 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 shutdown_connections 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8d4c06be60 0x7f8d4c06e320 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d680d5740 0x7f8d680b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d680d16f0 0x7f8d680b2cb0 secure :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800b3c0 tx=0x7f8d5800b880 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 --2- 172.21.15.99:0/1944513235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d680b59b0 0x7f8d68157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 >> 172.21.15.99:0/1944513235 conn(0x7f8d680d4c20 msgr2=0x7f8d681612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:09.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 shutdown_connections 2024-09-10T11:36:09.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.064+0000 7f8d6f30a640 1 -- 172.21.15.99:0/1944513235 wait complete. 2024-09-10T11:36:09.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:36:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.179+0000 7f64c1986640 1 Processor -- start 2024-09-10T11:36:09.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.179+0000 7f64c1986640 1 -- start start 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64c1986640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f64bc15eb80 0x7f64bc0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64c1986640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc15ff70 con 0x7f64bc0d5010 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64c1986640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64bc0d1620 con 0x7f64bc1626c0 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64c1986640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f64bc162bc0 con 0x7f64bc15eb80 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64bb7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f64bc0d5010 0x7f64bc0d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55348/0 (socket says 172.21.15.99:55348) 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64bb7fe640 1 -- 172.21.15.99:0/1313529698 learned_addr learned my addr 172.21.15.99:0/1313529698 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64baffd640 1 --2- 172.21.15.99:0/1313529698 >> v2:172.21.15.196:3300/0 conn(0x7f64bc15eb80 0x7f64bc0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1977927102 0 0) 0x7f64bc15ff70 con 0x7f64bc0d5010 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.180+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64a8003520 con 0x7f64bc0d5010 2024-09-10T11:36:09.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1252414075 0 0) 0x7f64a8003520 con 0x7f64bc0d5010 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 >> v2:172.21.15.196:3300/0 conn(0x7f64bc15eb80 msgr2=0x7f64bc0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 --2- 172.21.15.99:0/1313529698 >> v2:172.21.15.196:3300/0 conn(0x7f64bc15eb80 0x7f64bc0d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 >> v1:172.21.15.196:6789/0 conn(0x7f64bc1626c0 legacy=0x7f64bc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc162ea0 con 0x7f64bc0d5010 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f64b40019f0 con 0x7f64bc0d5010 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 >> v1:172.21.15.99:6789/0 conn(0x7f64bc0d5010 legacy=0x7f64bc0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a8004800 0x7f64a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a80051c0 0x7f64a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc162bc0 con 0x7f64a80039b0 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64a800c160 con 0x7f64a80051c0 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64a800c360 con 0x7f64a8004800 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64bb7fe640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a80051c0 0x7f64a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64b27fc640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a8004800 0x7f64a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 -- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a8004800 msgr2=0x7f64a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a8004800 0x7f64a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 -- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a80051c0 msgr2=0x7f64a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64a80051c0 0x7f64a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64baffd640 1 -- 172.21.15.99:0/1313529698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a8003520 con 0x7f64a80039b0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.181+0000 7f64bb7fe640 1 -- 172.21.15.99:0/1313529698 reap_dead start 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64baffd640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009ac0 tx=0x7f64a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c961de01cae84ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac036070 con 0x7f64a80039b0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64ac031400 con 0x7f64a80039b0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64ba7fc640 1 -- 172.21.15.99:0/1313529698 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac009690 con 0x7f64a80039b0 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 -- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 msgr2=0x7f64a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f64ac009ac0 tx=0x7f64a8005830 comp rx=0 tx=0).stop 2024-09-10T11:36:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 -- 172.21.15.99:0/1313529698 shutdown_connections 2024-09-10T11:36:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 --2- 172.21.15.99:0/1313529698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64a80039b0 0x7f64a8003dd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 -- 172.21.15.99:0/1313529698 >> 172.21.15.99:0/1313529698 conn(0x7f64bc08e680 msgr2=0x7f64bc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.182+0000 7f64c1986640 1 -- 172.21.15.99:0/1313529698 shutdown_connections 2024-09-10T11:36:09.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.183+0000 7f64c1986640 1 -- 172.21.15.99:0/1313529698 wait complete. 2024-09-10T11:36:09.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.184+0000 7f64c1986640 1 Processor -- start 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.184+0000 7f64c1986640 1 -- start start 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.184+0000 7f64c1986640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a80039b0 0x7f64bc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.184+0000 7f64c1986640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64bc0b8950 0x7f64bc0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc0d4900 con 0x7f64bc0b5420 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc163460 con 0x7f64bc0b8950 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64bc0d4a80 con 0x7f64a80039b0 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56454/0 (socket says 172.21.15.99:56454) 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 -- 172.21.15.99:0/2440406356 learned_addr learned my addr 172.21.15.99:0/2440406356 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64b27fc640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64bc0b8950 0x7f64bc0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64baffd640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a80039b0 0x7f64bc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 -- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a80039b0 msgr2=0x7f64bc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a80039b0 0x7f64bc1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 -- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64bc0b8950 msgr2=0x7f64bc0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64bc0b8950 0x7f64bc0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 -- 172.21.15.99:0/2440406356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc0b58c0 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64bb7fe640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f64bc162aa0 tx=0x7f64b400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b400ebb0 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b4018070 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b4013420 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc0b7b90 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.185+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc0b8150 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.186+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc0d29b0 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.187+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64b40135c0 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.187+0000 7f64b8ff9640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 0x7f649c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.187+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7f64b408acd0 con 0x7f64bc0b5420 2024-09-10T11:36:09.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.187+0000 7f64baffd640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 0x7f649c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.187+0000 7f64baffd640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 0x7f649c06e320 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64ac033000 tx=0x7f64ac008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.191+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64b40593c0 con 0x7f64bc0b5420 2024-09-10T11:36:09.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.358+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f64bc06c840 con 0x7f64bc0b5420 2024-09-10T11:36:09.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.361+0000 7f64b8ff9640 1 -- 172.21.15.99:0/2440406356 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f64b405d6c0 con 0x7f64bc0b5420 2024-09-10T11:36:09.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 msgr2=0x7f649c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 0x7f649c06e320 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f64ac033000 tx=0x7f64ac008030 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 msgr2=0x7f64bc0b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f64bc162aa0 tx=0x7f64b400e530 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 shutdown_connections 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f649c06be60 0x7f649c06e320 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f64bc0b5420 0x7f64bc0b29c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64bc0b8950 0x7f64bc0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 --2- 172.21.15.99:0/2440406356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64a80039b0 0x7f64bc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 >> 172.21.15.99:0/2440406356 conn(0x7f64bc08e680 msgr2=0x7f64bc06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 shutdown_connections 2024-09-10T11:36:09.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.363+0000 7f64c1986640 1 -- 172.21.15.99:0/2440406356 wait complete. 2024-09-10T11:36:09.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-09-10T11:36:09.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.471+0000 7fa2b9ce6640 1 Processor -- start 2024-09-10T11:36:09.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 -- start start 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 0x7fa2b40b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2b40b40e0 con 0x7fa2b40b0090 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b40b5ab0 con 0x7fa2b415ee00 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.472+0000 7fa2b9ce6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b4073c50 con 0x7fa2b415f1e0 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b37fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 0x7fa2b40b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b37fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:49886/0 (socket says 172.21.15.99:49886) 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b37fe640 1 -- 172.21.15.99:0/1373264245 learned_addr learned my addr 172.21.15.99:0/1373264245 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b3fff640 1 --1- 172.21.15.99:0/1373264245 >> v1:172.21.15.196:6789/0 conn(0x7fa2b40b0090 0x7fa2b40b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41978/0 (socket says 172.21.15.99:41978) 2024-09-10T11:36:09.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b2ffd640 1 -- 172.21.15.99:0/1373264245 >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 msgr2=0x7fa2b40b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:09.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b2ffd640 1 --2- 172.21.15.99:0/1373264245 >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b2ffd640 1 -- 172.21.15.99:0/1373264245 >> v1:172.21.15.196:6789/0 conn(0x7fa2b40b0090 legacy=0x7fa2b40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b2ffd640 1 -- 172.21.15.99:0/1373264245 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b40b4380 con 0x7fa2b415f1e0 2024-09-10T11:36:09.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.473+0000 7fa2b37fe640 1 --2- 172.21.15.99:0/1373264245 >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.474+0000 7fa2b2ffd640 1 --2- 172.21.15.99:0/1373264245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 0x7fa2b40b1560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ffb473d344507c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.474+0000 7fa2b27fc640 1 -- 172.21.15.99:0/1373264245 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8037070 con 0x7fa2b415f1e0 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.474+0000 7fa2b27fc640 1 -- 172.21.15.99:0/1373264245 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a8032440 con 0x7fa2b415f1e0 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.474+0000 7fa2b27fc640 1 -- 172.21.15.99:0/1373264245 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8032810 con 0x7fa2b415f1e0 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/1373264245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 msgr2=0x7fa2b40b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/1373264245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 0x7fa2b40b1560 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80097a0 tx=0x7fa2a802a180 comp rx=0 tx=0).stop 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/1373264245 shutdown_connections 2024-09-10T11:36:09.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/1373264245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415f1e0 0x7fa2b40b1560 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/1373264245 >> v2:172.21.15.196:3300/0 conn(0x7fa2b415ee00 0x7fa2b40b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/1373264245 >> 172.21.15.99:0/1373264245 conn(0x7fa2b40730c0 msgr2=0x7fa2b40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:09.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/1373264245 shutdown_connections 2024-09-10T11:36:09.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.475+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/1373264245 wait complete. 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.476+0000 7fa2b9ce6640 1 Processor -- start 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.476+0000 7fa2b9ce6640 1 -- start start 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415ee00 0x7fa2b414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2b415f1e0 0x7fa2b41f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40b21b0 con 0x7fa2b40b0090 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40afd60 con 0x7fa2b415f1e0 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b40b0600 con 0x7fa2b415ee00 2024-09-10T11:36:09.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56470/0 (socket says 172.21.15.99:56470) 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 -- 172.21.15.99:0/2404288840 learned_addr learned my addr 172.21.15.99:0/2404288840 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b2ffd640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415ee00 0x7fa2b414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b3fff640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2b415f1e0 0x7fa2b41f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 -- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415ee00 msgr2=0x7fa2b414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415ee00 0x7fa2b414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 -- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2b415f1e0 msgr2=0x7fa2b41f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2b415f1e0 0x7fa2b41f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 -- 172.21.15.99:0/2404288840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b40ca9a0 con 0x7fa2b40b0090 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b37fe640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9d0 tx=0x7fa2a400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4017070 con 0x7fa2b40b0090 2024-09-10T11:36:09.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a400ca60 con 0x7fa2b40b0090 2024-09-10T11:36:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b40cac80 con 0x7fa2b40b0090 2024-09-10T11:36:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400ce10 con 0x7fa2b40b0090 2024-09-10T11:36:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.477+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b40cb240 con 0x7fa2b40b0090 2024-09-10T11:36:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.478+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b40728c0 con 0x7fa2b40b0090 2024-09-10T11:36:09.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.479+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2a401a070 con 0x7fa2b40b0090 2024-09-10T11:36:09.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.479+0000 7fa29bfff640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:09.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.479+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7fa2a408aaa0 con 0x7fa2b40b0090 2024-09-10T11:36:09.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.479+0000 7fa2b2ffd640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:09.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.480+0000 7fa2b2ffd640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8034000 tx=0x7fa2a802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:09.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.484+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2b40728c0 con 0x7fa2b40b0090 2024-09-10T11:36:09.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:09.712+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7fa2b40b5f90 con 0x7fa2b40b0090 2024-09-10T11:36:10.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.061+0000 7fa29bfff640 1 -- 172.21.15.99:0/2404288840 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v308) ==== 140+0+0 (secure 0 0 0) 0x7fa2a4059080 con 0x7fa2b40b0090 2024-09-10T11:36:10.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 msgr2=0x7fa2a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8034000 tx=0x7fa2a802e000 comp rx=0 tx=0).stop 2024-09-10T11:36:10.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 msgr2=0x7fa2b40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9d0 tx=0x7fa2a400ee90 comp rx=0 tx=0).stop 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 shutdown_connections 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2b415f1e0 0x7fa2b41f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2b415ee00 0x7fa2b414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 --2- 172.21.15.99:0/2404288840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b40b0090 0x7fa2b40ca3c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.063+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 >> 172.21.15.99:0/2404288840 conn(0x7fa2b40730c0 msgr2=0x7fa2b415c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.064+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 shutdown_connections 2024-09-10T11:36:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.064+0000 7fa2b9ce6640 1 -- 172.21.15.99:0/2404288840 wait complete. 2024-09-10T11:36:10.077 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-09-10T11:36:10.077 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-09-10T11:36:10.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:10.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.174+0000 7f975f882640 1 Processor -- start 2024-09-10T11:36:10.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 -- start start 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f975815c7b0 0x7f97580d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 0x7f975806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97580ceb10 con 0x7f97580d0fb0 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f97580d1420 con 0x7f975815c7b0 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.175+0000 7f975f882640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97580b3840 con 0x7f975806f000 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f975815c7b0 0x7f97580d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975ce1d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f97580d0fb0 0x7f975806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55356/0 (socket says 172.21.15.99:55356) 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f975815c7b0 0x7f97580d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56478/0 (socket says 172.21.15.99:56478) 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975ce1d640 1 -- 172.21.15.99:0/1129722259 learned_addr learned my addr 172.21.15.99:0/1129722259 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975de1f640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 0x7f975806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 -- 172.21.15.99:0/1129722259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 msgr2=0x7f975806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:10.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 0x7f975806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 -- 172.21.15.99:0/1129722259 >> v1:172.21.15.99:6789/0 conn(0x7f97580d0fb0 legacy=0x7f975806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f974ffff640 1 -- 172.21.15.99:0/1129722259 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940545853 0 0) 0x7f9748001110 con 0x7f97580d0fb0 2024-09-10T11:36:10.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975d61e640 1 -- 172.21.15.99:0/1129722259 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97580b3a90 con 0x7f975815c7b0 2024-09-10T11:36:10.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.176+0000 7f975de1f640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 0x7f975806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:10.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975d61e640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f97580d0a70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f97540097a0 tx=0x7f975402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d122b867b869ab6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f974ffff640 1 -- 172.21.15.99:0/1129722259 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754037070 con 0x7f975815c7b0 2024-09-10T11:36:10.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f974ffff640 1 -- 172.21.15.99:0/1129722259 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9754032440 con 0x7f975815c7b0 2024-09-10T11:36:10.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f974ffff640 1 -- 172.21.15.99:0/1129722259 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754032810 con 0x7f975815c7b0 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 -- 172.21.15.99:0/1129722259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 msgr2=0x7f97580d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f97580d0a70 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f97540097a0 tx=0x7f975402a180 comp rx=0 tx=0).stop 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 -- 172.21.15.99:0/1129722259 shutdown_connections 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f975806f000 0x7f975806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 --2- 172.21.15.99:0/1129722259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f97580d0a70 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 -- 172.21.15.99:0/1129722259 >> 172.21.15.99:0/1129722259 conn(0x7f975806bc10 msgr2=0x7f975806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 -- 172.21.15.99:0/1129722259 shutdown_connections 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.177+0000 7f975f882640 1 -- 172.21.15.99:0/1129722259 wait complete. 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.178+0000 7f975f882640 1 Processor -- start 2024-09-10T11:36:10.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 -- start start 2024-09-10T11:36:10.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975806f000 0x7f975813cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97580d0fb0 0x7f9758154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f97580b4330 con 0x7f975815c7b0 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97580b4050 con 0x7f975806f000 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975f882640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f97580b4610 con 0x7f97580d0fb0 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975de1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975de1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56482/0 (socket says 172.21.15.99:56482) 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.179+0000 7f975de1f640 1 -- 172.21.15.99:0/1556000438 learned_addr learned my addr 172.21.15.99:0/1556000438 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975ce1d640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97580d0fb0 0x7f9758154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975d61e640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975806f000 0x7f975813cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 -- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97580d0fb0 msgr2=0x7f9758154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97580d0fb0 0x7f9758154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 -- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975806f000 msgr2=0x7f975813cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975806f000 0x7f975813cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 -- 172.21.15.99:0/1556000438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581531d0 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975de1f640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f975000cf20 tx=0x7f975000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975001a070 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f975000d840 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581534b0 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975000dc30 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.180+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9758153a70 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.181+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975815cd00 con 0x7f975815c7b0 2024-09-10T11:36:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.182+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f975001d070 con 0x7f975815c7b0 2024-09-10T11:36:10.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.182+0000 7f974dffb640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.182+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7f975008b4a0 con 0x7f975815c7b0 2024-09-10T11:36:10.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.182+0000 7f975d61e640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.183+0000 7f975d61e640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 0x7f973c06e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9754034000 tx=0x7f975402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.186+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f975005a150 con 0x7f975815c7b0 2024-09-10T11:36:10.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.350+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f975806ed20 con 0x7f975815c7b0 2024-09-10T11:36:10.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.351+0000 7f974dffb640 1 -- 172.21.15.99:0/1556000438 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+13045 (secure 0 0 0) 0x7f9750059a50 con 0x7f975815c7b0 2024-09-10T11:36:10.353 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:10.353 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":144,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:09.057107+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6835/3835517741":"2024-09-11T11:36:05.756957+0000","172.21.15.196:6833/3835517741":"2024-09-11T11:36:05.756957+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:10.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 msgr2=0x7f973c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 0x7f973c06e2f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9754034000 tx=0x7f975402e000 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 msgr2=0x7f975813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f975000cf20 tx=0x7f975000a7d0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 shutdown_connections 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f973c06be30 0x7f973c06e2f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f975815c7b0 0x7f975813d490 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97580d0fb0 0x7f9758154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 --2- 172.21.15.99:0/1556000438 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f975806f000 0x7f975813cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 >> 172.21.15.99:0/1556000438 conn(0x7f975806bc10 msgr2=0x7f97581573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 shutdown_connections 2024-09-10T11:36:10.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.353+0000 7f975f882640 1 -- 172.21.15.99:0/1556000438 wait complete. 2024-09-10T11:36:10.366 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:36:10.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:36:10.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.465+0000 7fd9cdf6d640 1 Processor -- start 2024-09-10T11:36:10.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.465+0000 7fd9cdf6d640 1 -- start start 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.466+0000 7fd9cdf6d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9c80cf410 0x7fd9c80cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.466+0000 7fd9cdf6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.466+0000 7fd9cdf6d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c80d1770 con 0x7fd9c815cc30 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.466+0000 7fd9cdf6d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd9c80cd810 con 0x7fd9c80cf410 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.466+0000 7fd9cdf6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c80cdaf0 con 0x7fd9c80d1000 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c77fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd9c80cf410 0x7fd9c80cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:48298/0 (socket says 172.21.15.99:48298) 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c7fff640 1 -- 172.21.15.99:0/772744670 learned_addr learned my addr 172.21.15.99:0/772744670 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c6ffd640 1 --1- 172.21.15.99:0/772744670 >> v1:172.21.15.196:6789/0 conn(0x7fd9c815cc30 0x7fd9c80cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41994/0 (socket says 172.21.15.99:41994) 2024-09-10T11:36:10.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c67fc640 1 -- 172.21.15.99:0/772744670 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317947149 0 0) 0x7fd9c80d1770 con 0x7fd9c815cc30 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c67fc640 1 -- 172.21.15.99:0/772744670 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9b0003520 con 0x7fd9c815cc30 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c77fe640 1 -- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 msgr2=0x7fd9c80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c77fe640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c77fe640 1 -- 172.21.15.99:0/772744670 >> v1:172.21.15.196:6789/0 conn(0x7fd9c815cc30 legacy=0x7fd9c80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c77fe640 1 -- 172.21.15.99:0/772744670 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c80cd4c0 con 0x7fd9c80cf410 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.467+0000 7fd9c7fff640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9c77fe640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80cf410 0x7fd9c80cee30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80097a0 tx=0x7fd9b802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934f69aab8096f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9c67fc640 1 -- 172.21.15.99:0/772744670 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8037070 con 0x7fd9c80cf410 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9c67fc640 1 -- 172.21.15.99:0/772744670 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b802adb0 con 0x7fd9c80cf410 2024-09-10T11:36:10.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9c67fc640 1 -- 172.21.15.99:0/772744670 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8032660 con 0x7fd9c80cf410 2024-09-10T11:36:10.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80cf410 msgr2=0x7fd9c80cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.468+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80cf410 0x7fd9c80cee30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80097a0 tx=0x7fd9b802a060 comp rx=0 tx=0).stop 2024-09-10T11:36:10.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/772744670 shutdown_connections 2024-09-10T11:36:10.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c80d1000 0x7fd9c80ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/772744670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80cf410 0x7fd9c80cee30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/772744670 >> 172.21.15.99:0/772744670 conn(0x7fd9c80b4570 msgr2=0x7fd9c80b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/772744670 shutdown_connections 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.469+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/772744670 wait complete. 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.470+0000 7fd9cdf6d640 1 Processor -- start 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.470+0000 7fd9cdf6d640 1 -- start start 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80d1000 0x7fd9c80cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c815cc30 0x7fd9c81f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c80ce9b0 con 0x7fd9c80cf410 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c80ce830 con 0x7fd9c80d1000 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c80b5870 con 0x7fd9c815cc30 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56486/0 (socket says 172.21.15.99:56486) 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 -- 172.21.15.99:0/3900287074 learned_addr learned my addr 172.21.15.99:0/3900287074 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c6ffd640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80d1000 0x7fd9c80cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c7fff640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c815cc30 0x7fd9c81f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 -- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c815cc30 msgr2=0x7fd9c81f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c815cc30 0x7fd9c81f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 -- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80d1000 msgr2=0x7fd9c80cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80d1000 0x7fd9c80cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 -- 172.21.15.99:0/3900287074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c80c73b0 con 0x7fd9c80cf410 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9c77fe640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8035040 tx=0x7fd9b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8041070 con 0x7fd9c80cf410 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b803c680 con 0x7fd9c80cf410 2024-09-10T11:36:10.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.471+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c80cb2b0 con 0x7fd9c80cf410 2024-09-10T11:36:10.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.472+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8037040 con 0x7fd9c80cf410 2024-09-10T11:36:10.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.472+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c80cb870 con 0x7fd9c80cf410 2024-09-10T11:36:10.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.472+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c80d1770 con 0x7fd9c80cf410 2024-09-10T11:36:10.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.473+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9b803a070 con 0x7fd9c80cf410 2024-09-10T11:36:10.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.473+0000 7fd9affff640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 0x7fd9b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.473+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7fd9b80aae20 con 0x7fd9c80cf410 2024-09-10T11:36:10.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.473+0000 7fd9c6ffd640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 0x7fd9b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.474+0000 7fd9c6ffd640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 0x7fd9b406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000f60 tx=0x7fd9bc007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.478+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9b8079400 con 0x7fd9c80cf410 2024-09-10T11:36:10.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.640+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9c80b89a0 con 0x7fd9c80cf410 2024-09-10T11:36:10.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.641+0000 7fd9affff640 1 -- 172.21.15.99:0/3900287074 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) ==== 93+0+10 (secure 0 0 0) 0x7fd9b807d700 con 0x7fd9c80cf410 2024-09-10T11:36:10.643 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 msgr2=0x7fd9b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 0x7fd9b406e2f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000f60 tx=0x7fd9bc007cf0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 msgr2=0x7fd9c81f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8035040 tx=0x7fd9b80046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 shutdown_connections 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd9b406be30 0x7fd9b406e2f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9c815cc30 0x7fd9c81f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9c80d1000 0x7fd9c80cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 --2- 172.21.15.99:0/3900287074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9c80cf410 0x7fd9c81f2350 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 >> 172.21.15.99:0/3900287074 conn(0x7fd9c80b4570 msgr2=0x7fd9c8070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 shutdown_connections 2024-09-10T11:36:10.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.643+0000 7fd9cdf6d640 1 -- 172.21.15.99:0/3900287074 wait complete. 2024-09-10T11:36:10.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-09-10T11:36:10.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.754+0000 7f781dc27640 1 Processor -- start 2024-09-10T11:36:10.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.754+0000 7f781dc27640 1 -- start start 2024-09-10T11:36:10.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f781dc27640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 0x7f7818073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f781dc27640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 0x7f781806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f781dc27640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7818152e60 con 0x7f781815c060 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f781dc27640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f781806ef50 con 0x7f78180d1200 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f781dc27640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f78180d15e0 con 0x7f781806e5d0 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f7817fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f781815c060 0x7f781815c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55358/0 (socket says 172.21.15.99:55358) 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.755+0000 7f7817fff640 1 -- 172.21.15.99:0/4224125145 learned_addr learned my addr 172.21.15.99:0/4224125145 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f7816ffd640 1 --2- 172.21.15.99:0/4224125145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 0x7f781806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f78177fe640 1 --2- 172.21.15.99:0/4224125145 >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 0x7f7818073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f78167fc640 1 -- 172.21.15.99:0/4224125145 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1291872015 0 0) 0x7f7818152e60 con 0x7f781815c060 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f78167fc640 1 -- 172.21.15.99:0/4224125145 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7800003520 con 0x7f781815c060 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f7816ffd640 1 -- 172.21.15.99:0/4224125145 >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 msgr2=0x7f7818073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f7816ffd640 1 --2- 172.21.15.99:0/4224125145 >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 0x7f7818073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f7816ffd640 1 -- 172.21.15.99:0/4224125145 >> v1:172.21.15.99:6789/0 conn(0x7f781815c060 legacy=0x7f781815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f7816ffd640 1 -- 172.21.15.99:0/4224125145 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7818072680 con 0x7f781806e5d0 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.756+0000 7f78177fe640 1 --2- 172.21.15.99:0/4224125145 >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 0x7f7818073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f7816ffd640 1 --2- 172.21.15.99:0/4224125145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 0x7f781806ea10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4200c4047888411a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f78167fc640 1 -- 172.21.15.99:0/4224125145 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c037070 con 0x7f781806e5d0 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f78167fc640 1 -- 172.21.15.99:0/4224125145 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c02ae50 con 0x7f781806e5d0 2024-09-10T11:36:10.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f78167fc640 1 -- 172.21.15.99:0/4224125145 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c032640 con 0x7f781806e5d0 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 -- 172.21.15.99:0/4224125145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 msgr2=0x7f781806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 --2- 172.21.15.99:0/4224125145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 0x7f781806ea10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f780c0097a0 tx=0x7f780c02a100 comp rx=0 tx=0).stop 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 -- 172.21.15.99:0/4224125145 shutdown_connections 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 --2- 172.21.15.99:0/4224125145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781806e5d0 0x7f781806ea10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 --2- 172.21.15.99:0/4224125145 >> v2:172.21.15.196:3300/0 conn(0x7f78180d1200 0x7f7818073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 -- 172.21.15.99:0/4224125145 >> 172.21.15.99:0/4224125145 conn(0x7f781806bc10 msgr2=0x7f781806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 -- 172.21.15.99:0/4224125145 shutdown_connections 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.757+0000 7f781dc27640 1 -- 172.21.15.99:0/4224125145 wait complete. 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.759+0000 7f781dc27640 1 Processor -- start 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.759+0000 7f781dc27640 1 -- start start 2024-09-10T11:36:10.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f781806e5d0 0x7f78180cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781815c060 0x7f78181503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7818072f20 con 0x7f78180d1200 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7818072c40 con 0x7f781806e5d0 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f781815cd90 con 0x7f781815c060 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56494/0 (socket says 172.21.15.99:56494) 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 -- 172.21.15.99:0/338110921 learned_addr learned my addr 172.21.15.99:0/338110921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f78177fe640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f781806e5d0 0x7f78180cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7817fff640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781815c060 0x7f78181503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 -- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781815c060 msgr2=0x7f78181503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781815c060 0x7f78181503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 -- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f781806e5d0 msgr2=0x7f78180cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f781806e5d0 0x7f78180cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 -- 172.21.15.99:0/338110921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78180d04c0 con 0x7f78180d1200 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f7816ffd640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f780c02a140 tx=0x7f780c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c041070 con 0x7f78180d1200 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780c03c460 con 0x7f78180d1200 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78180d07a0 con 0x7f78180d1200 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c037040 con 0x7f78180d1200 2024-09-10T11:36:10.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.760+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78180c5b90 con 0x7f78180d1200 2024-09-10T11:36:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.761+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78180d0970 con 0x7f78180d1200 2024-09-10T11:36:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.762+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f780c044070 con 0x7f78180d1200 2024-09-10T11:36:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.762+0000 7f77fffff640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 0x7f780406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:10.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.762+0000 7f78177fe640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 0x7f780406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:10.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.762+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7f780c0ab100 con 0x7f78180d1200 2024-09-10T11:36:10.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.763+0000 7f78177fe640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 0x7f780406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7808009950 tx=0x7f7808008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:10.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.767+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f780c079e40 con 0x7f78180d1200 2024-09-10T11:36:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:10.995+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7f781806ea10 con 0x7f78180d1200 2024-09-10T11:36:11.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.065+0000 7f77fffff640 1 -- 172.21.15.99:0/338110921 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v309) ==== 135+0+0 (secure 0 0 0) 0x7f780c079740 con 0x7f78180d1200 2024-09-10T11:36:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.067+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 msgr2=0x7f780406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.067+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 0x7f780406e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f7808009950 tx=0x7f7808008040 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.067+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 msgr2=0x7f78180ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.067+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f780c02a140 tx=0x7f780c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 shutdown_connections 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f780406be30 0x7f780406e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f781815c060 0x7f78181503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78180d1200 0x7f78180ce2a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 --2- 172.21.15.99:0/338110921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f781806e5d0 0x7f78180cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 >> 172.21.15.99:0/338110921 conn(0x7f781806bc10 msgr2=0x7f781815f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 shutdown_connections 2024-09-10T11:36:11.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.068+0000 7f781dc27640 1 -- 172.21.15.99:0/338110921 wait complete. 2024-09-10T11:36:11.081 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:36:11.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.179+0000 7f584b6dd640 1 Processor -- start 2024-09-10T11:36:11.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.179+0000 7f584b6dd640 1 -- start start 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f584b6dd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f584415b2d0 0x7f58440b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f584b6dd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58440b0020 con 0x7f58440afbb0 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f584b6dd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584406ee10 con 0x7f584415aef0 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f584b6dd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f58440cd9a0 con 0x7f584415b2d0 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5848c78640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f584415b2d0 0x7f58440b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5849c7a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f58440afbb0 0x7f58440714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55366/0 (socket says 172.21.15.99:55366) 2024-09-10T11:36:11.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5849c7a640 1 -- 172.21.15.99:0/940382365 learned_addr learned my addr 172.21.15.99:0/940382365 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:11.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5848c78640 1 -- 172.21.15.99:0/940382365 >> v1:172.21.15.196:6789/0 conn(0x7f584415aef0 legacy=0x7f58440cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:11.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5848c78640 1 -- 172.21.15.99:0/940382365 >> v1:172.21.15.99:6789/0 conn(0x7f58440afbb0 legacy=0x7f58440714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.180+0000 7f5848c78640 1 -- 172.21.15.99:0/940382365 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58440b0290 con 0x7f584415b2d0 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.181+0000 7f5848c78640 1 --2- 172.21.15.99:0/940382365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415b2d0 0x7f58440b4510 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f58380097d0 tx=0x7f583802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb46b7e5728ef9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.181+0000 7f5843fff640 1 -- 172.21.15.99:0/940382365 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838037070 con 0x7f584415b2d0 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.181+0000 7f5843fff640 1 -- 172.21.15.99:0/940382365 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5838032440 con 0x7f584415b2d0 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.181+0000 7f5843fff640 1 -- 172.21.15.99:0/940382365 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838032810 con 0x7f584415b2d0 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 -- 172.21.15.99:0/940382365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415b2d0 msgr2=0x7f58440b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 --2- 172.21.15.99:0/940382365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415b2d0 0x7f58440b4510 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f58380097d0 tx=0x7f583802a210 comp rx=0 tx=0).stop 2024-09-10T11:36:11.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 -- 172.21.15.99:0/940382365 shutdown_connections 2024-09-10T11:36:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 --2- 172.21.15.99:0/940382365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415b2d0 0x7f58440b4510 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 -- 172.21.15.99:0/940382365 >> 172.21.15.99:0/940382365 conn(0x7f584406bc10 msgr2=0x7f584406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 -- 172.21.15.99:0/940382365 shutdown_connections 2024-09-10T11:36:11.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.182+0000 7f584b6dd640 1 -- 172.21.15.99:0/940382365 wait complete. 2024-09-10T11:36:11.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.183+0000 7f584b6dd640 1 Processor -- start 2024-09-10T11:36:11.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.183+0000 7f584b6dd640 1 -- start start 2024-09-10T11:36:11.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 0x7f5844132910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f584415b2d0 0x7f5844149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f584406eb30 con 0x7f584415aef0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f584406e850 con 0x7f58440afbb0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f584b6dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5844071bb0 con 0x7f584415b2d0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56508/0 (socket says 172.21.15.99:56508) 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 -- 172.21.15.99:0/2251284390 learned_addr learned my addr 172.21.15.99:0/2251284390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5849479640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 0x7f5844132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5849c7a640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f584415b2d0 0x7f5844149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 -- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f584415b2d0 msgr2=0x7f5844149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f584415b2d0 0x7f5844149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 -- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 msgr2=0x7f5844132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 0x7f5844132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 -- 172.21.15.99:0/2251284390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5844134f10 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5849479640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 0x7f5844132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5848c78640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f58380098f0 tx=0x7f58380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838037040 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.184+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5838004230 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.185+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58441351f0 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.185+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838030770 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.185+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58441357b0 con 0x7f584415aef0 2024-09-10T11:36:11.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.186+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58440b4510 con 0x7f584415aef0 2024-09-10T11:36:11.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.186+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5838030910 con 0x7f584415aef0 2024-09-10T11:36:11.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.187+0000 7f5841ffb640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 0x7f583006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.187+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7f58380ab630 con 0x7f584415aef0 2024-09-10T11:36:11.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.187+0000 7f5849479640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 0x7f583006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.187+0000 7f5849479640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 0x7f583006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58340045b0 tx=0x7f5834009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.191+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5838032e20 con 0x7f584415aef0 2024-09-10T11:36:11.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.355+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5844148d10 con 0x7f584415aef0 2024-09-10T11:36:11.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.358+0000 7f5841ffb640 1 -- 172.21.15.99:0/2251284390 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f583807a2e0 con 0x7f584415aef0 2024-09-10T11:36:11.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 msgr2=0x7f583006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 0x7f583006e2f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58340045b0 tx=0x7f5834009040 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 msgr2=0x7f58441487d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f58380098f0 tx=0x7f58380046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 shutdown_connections 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f583006be30 0x7f583006e2f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f584415b2d0 0x7f5844149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f584415aef0 0x7f58441487d0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.360+0000 7f584b6dd640 1 --2- 172.21.15.99:0/2251284390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58440afbb0 0x7f5844132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.361+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 >> 172.21.15.99:0/2251284390 conn(0x7f584406bc10 msgr2=0x7f5844159a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.361+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 shutdown_connections 2024-09-10T11:36:11.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.361+0000 7f584b6dd640 1 -- 172.21.15.99:0/2251284390 wait complete. 2024-09-10T11:36:11.373 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:36:11.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:36:11.411 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:36:11.450 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:36:11.450 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:36:11.451 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:11.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:36:11.467 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:36:11.467 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:11.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:36:11.510 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:36:11.510 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:11.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:36:11.526 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:36:11.526 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:36:11.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:36:11.567 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:36:11.567 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:11.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.663+0000 7ff61d6ed640 1 Processor -- start 2024-09-10T11:36:11.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.663+0000 7ff61d6ed640 1 -- start start 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff61d6ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 0x7ff6181609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff61d6ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff618160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff61d6ed640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6180d1bd0 con 0x7ff61815cfd0 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff61d6ed640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff618075620 con 0x7ff6181605c0 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff61d6ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff6180752b0 con 0x7ff6180ae020 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff616ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 0x7ff6181609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff60e7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff618160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff6177fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff61815cfd0 0x7ff61815d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:42004/0 (socket says 172.21.15.99:42004) 2024-09-10T11:36:11.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.664+0000 7ff6177fe640 1 -- 172.21.15.99:0/4093174988 learned_addr learned my addr 172.21.15.99:0/4093174988 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:11.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff60e7fc640 1 -- 172.21.15.99:0/4093174988 >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 msgr2=0x7ff6181609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:11.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff60e7fc640 1 --2- 172.21.15.99:0/4093174988 >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 0x7ff6181609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff60e7fc640 1 -- 172.21.15.99:0/4093174988 >> v1:172.21.15.196:6789/0 conn(0x7ff61815cfd0 legacy=0x7ff61815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff60e7fc640 1 -- 172.21.15.99:0/4093174988 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6180d1eb0 con 0x7ff6180ae020 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff6167fc640 1 -- 172.21.15.99:0/4093174988 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588297686 0 0) 0x7ff6180d1bd0 con 0x7ff61815cfd0 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.665+0000 7ff616ffd640 1 --2- 172.21.15.99:0/4093174988 >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 0x7ff6181609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff60e7fc640 1 --2- 172.21.15.99:0/4093174988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff618160ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097a0 tx=0x7ff60002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd36f3d650c092b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff6167fc640 1 -- 172.21.15.99:0/4093174988 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600037070 con 0x7ff6180ae020 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff6167fc640 1 -- 172.21.15.99:0/4093174988 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff600032440 con 0x7ff6180ae020 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff6167fc640 1 -- 172.21.15.99:0/4093174988 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600032810 con 0x7ff6180ae020 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 -- 172.21.15.99:0/4093174988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 msgr2=0x7ff618160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/4093174988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff618160ee0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097a0 tx=0x7ff60002a180 comp rx=0 tx=0).stop 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 -- 172.21.15.99:0/4093174988 shutdown_connections 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/4093174988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff618160ee0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/4093174988 >> v2:172.21.15.196:3300/0 conn(0x7ff6181605c0 0x7ff6181609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 -- 172.21.15.99:0/4093174988 >> 172.21.15.99:0/4093174988 conn(0x7ff6180730c0 msgr2=0x7ff6180734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:11.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 -- 172.21.15.99:0/4093174988 shutdown_connections 2024-09-10T11:36:11.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.666+0000 7ff61d6ed640 1 -- 172.21.15.99:0/4093174988 wait complete. 2024-09-10T11:36:11.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.667+0000 7ff61d6ed640 1 Processor -- start 2024-09-10T11:36:11.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 -- start start 2024-09-10T11:36:11.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff61814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6181605c0 0x7ff6180b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff618073d00 con 0x7ff61815cfd0 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff61815a9e0 con 0x7ff6181605c0 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff61d6ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff618073e80 con 0x7ff6180ae020 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff60e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff60e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56510/0 (socket says 172.21.15.99:56510) 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.668+0000 7ff60e7fc640 1 -- 172.21.15.99:0/728450080 learned_addr learned my addr 172.21.15.99:0/728450080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff616ffd640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff61814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff6177fe640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6181605c0 0x7ff6180b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 -- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 msgr2=0x7ff61814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff61814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 -- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6181605c0 msgr2=0x7ff6180b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6181605c0 0x7ff6180b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 -- 172.21.15.99:0/728450080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6180cad00 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60e7fc640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097a0 tx=0x7ff6000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600037050 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff600038040 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6180cafe0 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600004430 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.669+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6180cb5a0 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.670+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6181609a0 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.670+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff60003c6d0 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.671+0000 7ff60ffff640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 0x7ff5f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.671+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7ff6000ab3e0 con 0x7ff61815cfd0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.671+0000 7ff616ffd640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 0x7ff5f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:11.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.671+0000 7ff616ffd640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 0x7ff5f806e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff608009560 tx=0x7ff608009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:11.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.675+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff600079ad0 con 0x7ff61815cfd0 2024-09-10T11:36:11.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.874+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6180d0e00 con 0x7ff61815cfd0 2024-09-10T11:36:11.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.874+0000 7ff60ffff640 1 -- 172.21.15.99:0/728450080 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+3911 (secure 0 0 0) 0x7ff60007ddd0 con 0x7ff61815cfd0 2024-09-10T11:36:11.876 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:11.877 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":57,"btime":"2024-09-10T11:36:06:758519+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6400,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/38465171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":38465171},{"type":"v1","addr":"172.21.15.196:6834","nonce":38465171}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2310487519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2310487519},{"type":"v1","addr":"172.21.15.196:6835","nonce":2310487519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6972,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1728247559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1728247559},{"type":"v1","addr":"172.21.15.99:6836","nonce":1728247559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6975,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/1728263237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1728263237},{"type":"v1","addr":"172.21.15.99:6837","nonce":1728263237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[]} 2024-09-10T11:36:11.877 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 57 2024-09-10T11:36:11.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 msgr2=0x7ff5f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 0x7ff5f806e2f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff608009560 tx=0x7ff608009190 comp rx=0 tx=0).stop 2024-09-10T11:36:11.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 msgr2=0x7ff6180cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff6000097a0 tx=0x7ff6000046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 shutdown_connections 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff5f806be30 0x7ff5f806e2f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6181605c0 0x7ff6180b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff61815cfd0 0x7ff6180cc080 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 --2- 172.21.15.99:0/728450080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6180ae020 0x7ff61814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 >> 172.21.15.99:0/728450080 conn(0x7ff6180730c0 msgr2=0x7ff61815e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 shutdown_connections 2024-09-10T11:36:11.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:11.876+0000 7ff61d6ed640 1 -- 172.21.15.99:0/728450080 wait complete. 2024-09-10T11:36:11.890 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:36:11.890 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:36:11.890 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:36:11.890 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:36:11.890 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:36:11.890 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:36:11.890 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:36:11.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:36:11.891 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:36:11.891 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:36:11.891 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:36:11.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:36:11.891 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:36:11.891 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:36:11.892 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:36:11.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:36:11.900 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:36:11.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:36:11.901 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:36:11.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:36:11.903 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:36:11.903 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:36:11.904 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:36:11.904 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:36:11.942 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:36:11.942 DEBUG:teuthology.parallel:result is None 2024-09-10T11:36:11.943 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:36:11.943 DEBUG:teuthology.parallel:result is None 2024-09-10T11:36:11.946 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:36:11.947 DEBUG:teuthology.parallel:result is None 2024-09-10T11:36:11.947 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:36:11.948 DEBUG:teuthology.parallel:result is None 2024-09-10T11:36:11.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:36:12.017 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:36:12.018 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:36:12.032 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:36:12.032 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:36:12.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.051+0000 7f91650b7640 1 Processor -- start 2024-09-10T11:36:12.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.051+0000 7f91650b7640 1 -- start start 2024-09-10T11:36:12.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f91650b7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f91600d5010 0x7f91600d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:12.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f91650b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f916015ff70 con 0x7f91601626c0 2024-09-10T11:36:12.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f91650b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91600d1620 con 0x7f916015eb80 2024-09-10T11:36:12.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f91650b7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f91600d30c0 con 0x7f91600d5010 2024-09-10T11:36:12.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915f59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f91600d5010 0x7f91600d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:12.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915e59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f91601626c0 0x7f91600d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:55378/0 (socket says 172.21.15.99:55378) 2024-09-10T11:36:12.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915e59c640 1 -- 172.21.15.99:0/1783156912 learned_addr learned my addr 172.21.15.99:0/1783156912 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:12.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915f59e640 1 -- 172.21.15.99:0/1783156912 >> v1:172.21.15.196:6789/0 conn(0x7f916015eb80 legacy=0x7f91600d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:12.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915f59e640 1 -- 172.21.15.99:0/1783156912 >> v1:172.21.15.99:6789/0 conn(0x7f91601626c0 legacy=0x7f91600d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:12.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.052+0000 7f915f59e640 1 -- 172.21.15.99:0/1783156912 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9160162bc0 con 0x7f91600d5010 2024-09-10T11:36:12.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.053+0000 7f915f59e640 1 --2- 172.21.15.99:0/1783156912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91600d5010 0x7f91600d0770 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9158009d90 tx=0x7f915802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55553c5748afe5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:12.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.053+0000 7f915dd9b640 1 -- 172.21.15.99:0/1783156912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158031600 con 0x7f91600d5010 2024-09-10T11:36:12.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.053+0000 7f915dd9b640 1 -- 172.21.15.99:0/1783156912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91580317a0 con 0x7f91600d5010 2024-09-10T11:36:12.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.053+0000 7f915dd9b640 1 -- 172.21.15.99:0/1783156912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158031b70 con 0x7f91600d5010 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 -- 172.21.15.99:0/1783156912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91600d5010 msgr2=0x7f91600d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 --2- 172.21.15.99:0/1783156912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91600d5010 0x7f91600d0770 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9158009d90 tx=0x7f915802a610 comp rx=0 tx=0).stop 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 -- 172.21.15.99:0/1783156912 shutdown_connections 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 --2- 172.21.15.99:0/1783156912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f91600d5010 0x7f91600d0770 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 -- 172.21.15.99:0/1783156912 >> 172.21.15.99:0/1783156912 conn(0x7f916008e680 msgr2=0x7f916009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 -- 172.21.15.99:0/1783156912 shutdown_connections 2024-09-10T11:36:12.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.054+0000 7f91650b7640 1 -- 172.21.15.99:0/1783156912 wait complete. 2024-09-10T11:36:12.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.055+0000 7f91650b7640 1 Processor -- start 2024-09-10T11:36:12.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.055+0000 7f91650b7640 1 -- start start 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91600d5010 0x7f91600cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91601626c0 0x7f91601f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9160163460 con 0x7f916015eb80 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9160163180 con 0x7f91601626c0 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f91650b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f91600d0cb0 con 0x7f91600d5010 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f915e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f915e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56562/0 (socket says 172.21.15.99:56562) 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.056+0000 7f915e59c640 1 -- 172.21.15.99:0/2658657566 learned_addr learned my addr 172.21.15.99:0/2658657566 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915ed9d640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91600d5010 0x7f91600cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915f59e640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91601626c0 0x7f91601f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:12.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 -- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91600d5010 msgr2=0x7f91600cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91600d5010 0x7f91600cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 -- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91601626c0 msgr2=0x7f91601f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91601626c0 0x7f91601f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 -- 172.21.15.99:0/2658657566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91600cd260 con 0x7f916015eb80 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f915e59c640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f915400c440 tx=0x7f9154001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154017070 con 0x7f916015eb80 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9154005ce0 con 0x7f916015eb80 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91540072f0 con 0x7f916015eb80 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91600cc6e0 con 0x7f916015eb80 2024-09-10T11:36:12.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.057+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91600ccca0 con 0x7f916015eb80 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.058+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91600d29b0 con 0x7f916015eb80 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.059+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f915401a070 con 0x7f916015eb80 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.059+0000 7f91477fe640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.059+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 4943+0+0 (secure 0 0 0) 0x7f9154089f90 con 0x7f916015eb80 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.059+0000 7f915ed9d640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:12.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.060+0000 7f915ed9d640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 0x7f914c06e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9150006fd0 tx=0x7f9150008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:12.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.063+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9154058c40 con 0x7f916015eb80 2024-09-10T11:36:12.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-10T11:36:12.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:12.241+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f91600d4060 con 0x7f916015eb80 2024-09-10T11:36:13.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.078+0000 7f91477fe640 1 -- 172.21.15.99:0/2658657566 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v149) ==== 97+0+0 (secure 0 0 0) 0x7f9154058540 con 0x7f916015eb80 2024-09-10T11:36:13.080 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:36:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 msgr2=0x7f914c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 0x7f914c06e2f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9150006fd0 tx=0x7f9150008040 comp rx=0 tx=0).stop 2024-09-10T11:36:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 msgr2=0x7f91601f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f915400c440 tx=0x7f9154001e70 comp rx=0 tx=0).stop 2024-09-10T11:36:13.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 shutdown_connections 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91601626c0 0x7f91601f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f916015eb80 0x7f91601f4fe0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 --2- 172.21.15.99:0/2658657566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91600d5010 0x7f91600cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 >> 172.21.15.99:0/2658657566 conn(0x7f916008e680 msgr2=0x7f916006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 shutdown_connections 2024-09-10T11:36:13.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.081+0000 7f91650b7640 1 -- 172.21.15.99:0/2658657566 wait complete. 2024-09-10T11:36:13.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:36:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.199+0000 7fa89d768640 1 Processor -- start 2024-09-10T11:36:13.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.199+0000 7fa89d768640 1 -- start start 2024-09-10T11:36:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa89d768640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa89806bc10 0x7fa898163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa89d768640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 0x7fa89806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa89d768640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8980b5140 con 0x7fa898160030 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa89d768640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa8980b5420 con 0x7fa89806bc10 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa89d768640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa898162b40 con 0x7fa8980b4930 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa89806bc10 0x7fa898163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa89806bc10 0x7fa898163620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34274/0 (socket says 172.21.15.99:34274) 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 -- 172.21.15.99:0/2156684073 learned_addr learned my addr 172.21.15.99:0/2156684073 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa8977fe640 1 --2- 172.21.15.99:0/2156684073 >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 0x7fa89806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 -- 172.21.15.99:0/2156684073 >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 msgr2=0x7fa89806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 --2- 172.21.15.99:0/2156684073 >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 0x7fa89806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 -- 172.21.15.99:0/2156684073 >> v1:172.21.15.196:6789/0 conn(0x7fa898160030 legacy=0x7fa89806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa896ffd640 1 -- 172.21.15.99:0/2156684073 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa898162e20 con 0x7fa89806bc10 2024-09-10T11:36:13.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.200+0000 7fa8977fe640 1 --2- 172.21.15.99:0/2156684073 >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 0x7fa89806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:13.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.201+0000 7fa896ffd640 1 --2- 172.21.15.99:0/2156684073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89806bc10 0x7fa898163620 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa8980721b0 tx=0x7fa88802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa14599eb3f41a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.201+0000 7fa8967fc640 1 -- 172.21.15.99:0/2156684073 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888037070 con 0x7fa89806bc10 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.201+0000 7fa8967fc640 1 -- 172.21.15.99:0/2156684073 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa888032440 con 0x7fa89806bc10 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.201+0000 7fa8967fc640 1 -- 172.21.15.99:0/2156684073 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888032810 con 0x7fa89806bc10 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 -- 172.21.15.99:0/2156684073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89806bc10 msgr2=0x7fa898163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 --2- 172.21.15.99:0/2156684073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89806bc10 0x7fa898163620 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa8980721b0 tx=0x7fa88802a1a0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 -- 172.21.15.99:0/2156684073 shutdown_connections 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 --2- 172.21.15.99:0/2156684073 >> v2:172.21.15.196:3300/0 conn(0x7fa8980b4930 0x7fa89806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 --2- 172.21.15.99:0/2156684073 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa89806bc10 0x7fa898163620 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 -- 172.21.15.99:0/2156684073 >> 172.21.15.99:0/2156684073 conn(0x7fa8980b3630 msgr2=0x7fa8980b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 -- 172.21.15.99:0/2156684073 shutdown_connections 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.202+0000 7fa89d768640 1 -- 172.21.15.99:0/2156684073 wait complete. 2024-09-10T11:36:13.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.203+0000 7fa89d768640 1 Processor -- start 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.203+0000 7fa89d768640 1 -- start start 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa89806bc10 0x7fa89815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898160030 0x7fa8981eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980d1560 con 0x7fa8980b4930 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980b4dc0 con 0x7fa898160030 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa89d768640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa8980d16e0 con 0x7fa89806bc10 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa88e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34286/0 (socket says 172.21.15.99:34286) 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa88e7fc640 1 -- 172.21.15.99:0/2783156729 learned_addr learned my addr 172.21.15.99:0/2783156729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa896ffd640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa89806bc10 0x7fa89815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa8977fe640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898160030 0x7fa8981eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.204+0000 7fa88e7fc640 1 -- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa89806bc10 msgr2=0x7fa89815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88e7fc640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa89806bc10 0x7fa89815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88e7fc640 1 -- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898160030 msgr2=0x7fa8981eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88e7fc640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898160030 0x7fa8981eaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88e7fc640 1 -- 172.21.15.99:0/2783156729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89815eed0 con 0x7fa8980b4930 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88e7fc640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa88000b3f0 tx=0x7fa88000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880015020 con 0x7fa8980b4930 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa880011830 con 0x7fa8980b4930 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8980c6e80 con 0x7fa8980b4930 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880011c20 con 0x7fa8980b4930 2024-09-10T11:36:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.205+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8980c7440 con 0x7fa8980b4930 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.206+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898163620 con 0x7fa8980b4930 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.206+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa880018020 con 0x7fa8980b4930 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.207+0000 7fa88ffff640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.207+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4857+0+0 (secure 0 0 0) 0x7fa88008a5c0 con 0x7fa8980b4930 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.207+0000 7fa896ffd640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.207+0000 7fa896ffd640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 0x7fa87806e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.211+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa88001f030 con 0x7fa8980b4930 2024-09-10T11:36:13.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.437+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa8981627f0 con 0x7fa8980b4930 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.438+0000 7fa88ffff640 1 -- 172.21.15.99:0/2783156729 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fa880058c70 con 0x7fa8980b4930 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.440 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:36:13.442 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.444 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.446 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:36:13.447 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:36:13.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:36:13.449 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:36:13.450 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.451 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:36:13.452 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:36:13.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:36:13.453 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:36:13.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 msgr2=0x7fa87806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 0x7fa87806e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa888034000 tx=0x7fa88802e000 comp rx=0 tx=0).stop 2024-09-10T11:36:13.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 msgr2=0x7fa8981eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa88000b3f0 tx=0x7fa88000b8b0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 shutdown_connections 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898160030 0x7fa8981eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8980b4930 0x7fa8981eaa00 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 --2- 172.21.15.99:0/2783156729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa89806bc10 0x7fa89815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 >> 172.21.15.99:0/2783156729 conn(0x7fa8980b3630 msgr2=0x7fa8980b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 shutdown_connections 2024-09-10T11:36:13.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.440+0000 7fa89d768640 1 -- 172.21.15.99:0/2783156729 wait complete. 2024-09-10T11:36:13.455 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:36:13.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.551+0000 7f5133c77640 1 Processor -- start 2024-09-10T11:36:13.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.552+0000 7f5133c77640 1 -- start start 2024-09-10T11:36:13.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.552+0000 7f5133c77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f512c15ee00 0x7f512c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.552+0000 7f5133c77640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c0b40e0 con 0x7f512c0b0090 2024-09-10T11:36:13.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.552+0000 7f5133c77640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c0b5ab0 con 0x7f512c15f1e0 2024-09-10T11:36:13.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.552+0000 7f5133c77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f512c073c50 con 0x7f512c15ee00 2024-09-10T11:36:13.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5132214640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f512c0b0090 0x7f512c0b1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51250/0 (socket says 172.21.15.99:51250) 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5132214640 1 -- 172.21.15.99:0/1305239858 learned_addr learned my addr 172.21.15.99:0/1305239858 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5131a13640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f512c15ee00 0x7f512c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240854077 0 0) 0x7f512c0b40e0 con 0x7f512c0b0090 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5114003520 con 0x7f512c0b0090 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106539765 0 0) 0x7f5114003520 con 0x7f512c0b0090 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f512c15ee00 msgr2=0x7f512c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f512c15ee00 0x7f512c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 >> v1:172.21.15.196:6789/0 conn(0x7f512c15f1e0 legacy=0x7f512c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0b4380 con 0x7f512c0b0090 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f511c0019f0 con 0x7f512c0b0090 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 >> v1:172.21.15.99:6789/0 conn(0x7f512c0b0090 legacy=0x7f512c0b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51140039e0 0x7f5114003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114004390 0x7f5114004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5114005010 0x7f511400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f512c073c50 con 0x7f5114005010 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f511400bf30 con 0x7f51140039e0 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f511400c130 con 0x7f5114004390 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5131a13640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f512c15ee00 0x7f512c0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5132214640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5114005010 0x7f511400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5131212640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114004390 0x7f5114004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.553+0000 7f5131a13640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51140039e0 0x7f5114003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 -- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114004390 msgr2=0x7f5114004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114004390 0x7f5114004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 -- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51140039e0 msgr2=0x7f5114003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51140039e0 0x7f5114003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 -- 172.21.15.99:0/1305239858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5114003520 con 0x7f5114005010 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5131a13640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51140039e0 0x7f5114003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:13.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5132214640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5114005010 0x7f511400b9c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f511c00a920 tx=0x7f5114005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712beb58b0220d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c00a4c0 con 0x7f5114005010 2024-09-10T11:36:13.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c02f070 con 0x7f5114005010 2024-09-10T11:36:13.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.554+0000 7f5130a11640 1 -- 172.21.15.99:0/1305239858 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c02bce0 con 0x7f5114005010 2024-09-10T11:36:13.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 -- 172.21.15.99:0/1305239858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5114005010 msgr2=0x7f511400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 --2- 172.21.15.99:0/1305239858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5114005010 0x7f511400b9c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f511c00a920 tx=0x7f5114005830 comp rx=0 tx=0).stop 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5132214640 1 -- 172.21.15.99:0/1305239858 reap_dead start 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 -- 172.21.15.99:0/1305239858 shutdown_connections 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 -- 172.21.15.99:0/1305239858 >> 172.21.15.99:0/1305239858 conn(0x7f512c0730c0 msgr2=0x7f512c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 -- 172.21.15.99:0/1305239858 shutdown_connections 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.555+0000 7f5133c77640 1 -- 172.21.15.99:0/1305239858 wait complete. 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.556+0000 7f5133c77640 1 Processor -- start 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.556+0000 7f5133c77640 1 -- start start 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114005010 0x7f512c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 0x7f512c145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0b0600 con 0x7f512c06e530 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0afd60 con 0x7f512c1455e0 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5133c77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0b0780 con 0x7f5114005010 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34300/0 (socket says 172.21.15.99:34300) 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 -- 172.21.15.99:0/4010242990 learned_addr learned my addr 172.21.15.99:0/4010242990 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5132214640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 0x7f512c145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131a13640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114005010 0x7f512c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 -- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114005010 msgr2=0x7f512c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114005010 0x7f512c152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 -- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 msgr2=0x7f512c145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 0x7f512c145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5131212640 1 -- 172.21.15.99:0/4010242990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c1f5ba0 con 0x7f512c06e530 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.557+0000 7f5132214640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 0x7f512c145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f5131212640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f512000f7f0 tx=0x7f512000fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200050d0 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5120018070 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000d730 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1f4de0 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.558+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1f53a0 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.559+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c0b5f90 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.559+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f512000d8d0 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.560+0000 7f512a7fc640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 0x7f511806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.560+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4857+0+0 (secure 0 0 0) 0x7f512008a900 con 0x7f512c06e530 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.560+0000 7f5131a13640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 0x7f511806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.560+0000 7f5131a13640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 0x7f511806e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f512c162a00 tx=0x7f5124008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.564+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5120059690 con 0x7f512c06e530 2024-09-10T11:36:13.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.728+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f512c0b15b0 con 0x7f512c06e530 2024-09-10T11:36:13.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.728+0000 7f512a7fc640 1 -- 172.21.15.99:0/4010242990 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v149) ==== 84+0+65 (secure 0 0 0) 0x7f5120058f90 con 0x7f512c06e530 2024-09-10T11:36:13.730 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:13.730 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:36:13.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:36:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:36:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:36:13.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:13.731 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 msgr2=0x7f511806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 0x7f511806e2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f512c162a00 tx=0x7f5124008040 comp rx=0 tx=0).stop 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 msgr2=0x7f512c06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f512000f7f0 tx=0x7f512000fbf0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 shutdown_connections 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f511806be30 0x7f511806e2f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c1455e0 0x7f512c145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f512c06e530 0x7f512c06e9b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 --2- 172.21.15.99:0/4010242990 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5114005010 0x7f512c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 >> 172.21.15.99:0/4010242990 conn(0x7f512c0730c0 msgr2=0x7f512c0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 shutdown_connections 2024-09-10T11:36:13.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.730+0000 7f5133c77640 1 -- 172.21.15.99:0/4010242990 wait complete. 2024-09-10T11:36:13.743 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:36:13.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:36:13.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.843+0000 7f2eccd31640 1 Processor -- start 2024-09-10T11:36:13.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.843+0000 7f2eccd31640 1 -- start start 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2eccd31640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ec815b2d0 0x7f2ec80b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2eccd31640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec80afbb0 0x7f2ec80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2eccd31640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec80b0020 con 0x7f2ec815aef0 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2eccd31640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec806ee10 con 0x7f2ec80afbb0 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2eccd31640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec80cd9a0 con 0x7f2ec815b2d0 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec80afbb0 0x7f2ec80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ec80afbb0 0x7f2ec80714a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34312/0 (socket says 172.21.15.99:34312) 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 -- 172.21.15.99:0/355546882 learned_addr learned my addr 172.21.15.99:0/355546882 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 -- 172.21.15.99:0/355546882 >> v2:172.21.15.196:3300/0 conn(0x7f2ec815b2d0 msgr2=0x7f2ec80b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 --2- 172.21.15.99:0/355546882 >> v2:172.21.15.196:3300/0 conn(0x7f2ec815b2d0 0x7f2ec80b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 -- 172.21.15.99:0/355546882 >> v1:172.21.15.196:6789/0 conn(0x7f2ec815aef0 legacy=0x7f2ec80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.844+0000 7f2ec6d9d640 1 -- 172.21.15.99:0/355546882 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec80b0290 con 0x7f2ec80afbb0 2024-09-10T11:36:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2ec6d9d640 1 --2- 172.21.15.99:0/355546882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80714a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009e50 tx=0x7f2ec002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6030bc623e7e453d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2ec5d9b640 1 -- 172.21.15.99:0/355546882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0036070 con 0x7f2ec80afbb0 2024-09-10T11:36:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2ec5d9b640 1 -- 172.21.15.99:0/355546882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec0031600 con 0x7f2ec80afbb0 2024-09-10T11:36:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2ec5d9b640 1 -- 172.21.15.99:0/355546882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec00319d0 con 0x7f2ec80afbb0 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 -- 172.21.15.99:0/355546882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 msgr2=0x7f2ec80714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 --2- 172.21.15.99:0/355546882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80714a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009e50 tx=0x7f2ec002a710 comp rx=0 tx=0).stop 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 -- 172.21.15.99:0/355546882 shutdown_connections 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 --2- 172.21.15.99:0/355546882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80714a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 --2- 172.21.15.99:0/355546882 >> v2:172.21.15.196:3300/0 conn(0x7f2ec815b2d0 0x7f2ec80b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.845+0000 7f2eccd31640 1 -- 172.21.15.99:0/355546882 >> 172.21.15.99:0/355546882 conn(0x7f2ec806bc10 msgr2=0x7f2ec806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.846+0000 7f2eccd31640 1 -- 172.21.15.99:0/355546882 shutdown_connections 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.846+0000 7f2eccd31640 1 -- 172.21.15.99:0/355546882 wait complete. 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.847+0000 7f2eccd31640 1 Processor -- start 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.847+0000 7f2eccd31640 1 -- start start 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 0x7f2ec815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec815b2d0 0x7f2ec815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec806eb30 con 0x7f2ec80afbb0 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec806e850 con 0x7f2ec815b2d0 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec80719e0 con 0x7f2ec815aef0 2024-09-10T11:36:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34314/0 (socket says 172.21.15.99:34314) 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 -- 172.21.15.99:0/3759331083 learned_addr learned my addr 172.21.15.99:0/3759331083 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ebdd9b640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 0x7f2ec815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec6d9d640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec815b2d0 0x7f2ec815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 -- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 msgr2=0x7f2ec815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 0x7f2ec815cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 -- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec815b2d0 msgr2=0x7f2ec815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec815b2d0 0x7f2ec815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 -- 172.21.15.99:0/3759331083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec81571a0 con 0x7f2ec80afbb0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ebdd9b640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 0x7f2ec815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ec659c640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c410 tx=0x7f2eb800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8017070 con 0x7f2ec80afbb0 2024-09-10T11:36:13.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb80035f0 con 0x7f2ec80afbb0 2024-09-10T11:36:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.848+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec8157480 con 0x7f2ec80afbb0 2024-09-10T11:36:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.849+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003a20 con 0x7f2ec80afbb0 2024-09-10T11:36:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.849+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec8157a40 con 0x7f2ec80afbb0 2024-09-10T11:36:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.849+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8071da0 con 0x7f2ec80afbb0 2024-09-10T11:36:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.850+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2eb801a070 con 0x7f2ec80afbb0 2024-09-10T11:36:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.850+0000 7f2ebf7fe640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.850+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4857+0+0 (secure 0 0 0) 0x7f2eb8089f60 con 0x7f2ec80afbb0 2024-09-10T11:36:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.850+0000 7f2ebdd9b640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.851+0000 7f2ebdd9b640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 0x7f2ea806e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80b4f50 tx=0x7f2eb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:13.855+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2eb8058cf0 con 0x7f2ec80afbb0 2024-09-10T11:36:14.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.020+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2ec8073150 con 0x7f2ec80afbb0 2024-09-10T11:36:14.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.361+0000 7f2ebf7fe640 1 -- 172.21.15.99:0/3759331083 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v150) ==== 138+0+0 (secure 0 0 0) 0x7f2eb80585f0 con 0x7f2ec80afbb0 2024-09-10T11:36:14.363 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:36:14.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 msgr2=0x7f2ea806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 0x7f2ea806e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80b4f50 tx=0x7f2eb0008040 comp rx=0 tx=0).stop 2024-09-10T11:36:14.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 msgr2=0x7f2ec80aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800c410 tx=0x7f2eb800c8d0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 shutdown_connections 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ea806be30 0x7f2ea806e2f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ec815b2d0 0x7f2ec815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ec815aef0 0x7f2ec815cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 --2- 172.21.15.99:0/3759331083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ec80afbb0 0x7f2ec80aae50 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 >> 172.21.15.99:0/3759331083 conn(0x7f2ec806bc10 msgr2=0x7f2ec806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 shutdown_connections 2024-09-10T11:36:14.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.364+0000 7f2eccd31640 1 -- 172.21.15.99:0/3759331083 wait complete. 2024-09-10T11:36:14.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:36:14.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.485+0000 7f304b453640 1 Processor -- start 2024-09-10T11:36:14.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.485+0000 7f304b453640 1 -- start start 2024-09-10T11:36:14.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f304b453640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30440d1180 0x7f304406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f304b453640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f304406ee30 0x7f304406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f304b453640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30440ceb10 con 0x7f304415c7b0 2024-09-10T11:36:14.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f304b453640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f304406f280 con 0x7f304406ee30 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f304b453640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f30440b3840 con 0x7f30440d1180 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f304406ee30 0x7f304406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30491ef640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f304415c7b0 0x7f30440d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51258/0 (socket says 172.21.15.99:51258) 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f304406ee30 0x7f304406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34326/0 (socket says 172.21.15.99:34326) 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30491ef640 1 -- 172.21.15.99:0/551193936 learned_addr learned my addr 172.21.15.99:0/551193936 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 -- 172.21.15.99:0/551193936 >> v2:172.21.15.196:3300/0 conn(0x7f30440d1180 msgr2=0x7f304406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 --2- 172.21.15.99:0/551193936 >> v2:172.21.15.196:3300/0 conn(0x7f30440d1180 0x7f304406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 -- 172.21.15.99:0/551193936 >> v1:172.21.15.99:6789/0 conn(0x7f304415c7b0 legacy=0x7f30440d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f303bfff640 1 -- 172.21.15.99:0/551193936 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431674064 0 0) 0x7f3040001110 con 0x7f304415c7b0 2024-09-10T11:36:14.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.486+0000 7f30499f0640 1 -- 172.21.15.99:0/551193936 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30440b3a90 con 0x7f304406ee30 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.487+0000 7f30499f0640 1 --2- 172.21.15.99:0/551193936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f304406d4c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f30440d0280 tx=0x7f303c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d9e7e394a7194f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.487+0000 7f303bfff640 1 -- 172.21.15.99:0/551193936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c007740 con 0x7f304406ee30 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.487+0000 7f303bfff640 1 -- 172.21.15.99:0/551193936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f303c0078e0 con 0x7f304406ee30 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.487+0000 7f303bfff640 1 -- 172.21.15.99:0/551193936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c007cb0 con 0x7f304406ee30 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 -- 172.21.15.99:0/551193936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 msgr2=0x7f304406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 --2- 172.21.15.99:0/551193936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f304406d4c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f30440d0280 tx=0x7f303c02a680 comp rx=0 tx=0).stop 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 -- 172.21.15.99:0/551193936 shutdown_connections 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 --2- 172.21.15.99:0/551193936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f304406d4c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 --2- 172.21.15.99:0/551193936 >> v2:172.21.15.196:3300/0 conn(0x7f30440d1180 0x7f304406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 -- 172.21.15.99:0/551193936 >> 172.21.15.99:0/551193936 conn(0x7f304406bc10 msgr2=0x7f304406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 -- 172.21.15.99:0/551193936 shutdown_connections 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.488+0000 7f304b453640 1 -- 172.21.15.99:0/551193936 wait complete. 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.489+0000 7f304b453640 1 Processor -- start 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.489+0000 7f304b453640 1 -- start start 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30440d1180 0x7f304414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304415c7b0 0x7f30440b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f30440b4330 con 0x7f304406ee30 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f30440b4050 con 0x7f304415c7b0 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f30440b4610 con 0x7f30440d1180 2024-09-10T11:36:14.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34336/0 (socket says 172.21.15.99:34336) 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 -- 172.21.15.99:0/53158853 learned_addr learned my addr 172.21.15.99:0/53158853 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30499f0640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304415c7b0 0x7f30440b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30489ee640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30440d1180 0x7f304414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 -- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30440d1180 msgr2=0x7f304414d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30440d1180 0x7f304414d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 -- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304415c7b0 msgr2=0x7f30440b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304415c7b0 0x7f30440b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 -- 172.21.15.99:0/53158853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f304414df50 con 0x7f304406ee30 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f30491ef640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f304000c460 tx=0x7f304000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:14.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040018070 con 0x7f304406ee30 2024-09-10T11:36:14.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3040007500 con 0x7f304406ee30 2024-09-10T11:36:14.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040004a20 con 0x7f304406ee30 2024-09-10T11:36:14.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.490+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3044154d30 con 0x7f304406ee30 2024-09-10T11:36:14.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.491+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30441f6330 con 0x7f304406ee30 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.491+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3044069640 con 0x7f304406ee30 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.492+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f304001b070 con 0x7f304406ee30 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.492+0000 7f3039ffb640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 0x7f302806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.492+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5227+0+0 (secure 0 0 0) 0x7f3040089e50 con 0x7f304406ee30 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.492+0000 7f30489ee640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 0x7f302806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:14.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.493+0000 7f30489ee640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 0x7f302806e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f304415b230 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:14.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.497+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3044069640 con 0x7f304406ee30 2024-09-10T11:36:14.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:14.661+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f30440b3270 con 0x7f304406ee30 2024-09-10T11:36:15.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.369+0000 7f3039ffb640 1 -- 172.21.15.99:0/53158853 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v151) ==== 170+0+0 (secure 0 0 0) 0x7f30440ceb10 con 0x7f304406ee30 2024-09-10T11:36:15.371 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.373+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 msgr2=0x7f302806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.373+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 0x7f302806e320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f304415b230 tx=0x7f3034008040 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.373+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 msgr2=0x7f30441f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.373+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f304000c460 tx=0x7f304000c920 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 shutdown_connections 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f302806be60 0x7f302806e320 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304415c7b0 0x7f30440b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f30440d1180 0x7f304414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 --2- 172.21.15.99:0/53158853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f304406ee30 0x7f30441f3290 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.376+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 >> 172.21.15.99:0/53158853 conn(0x7f304406bc10 msgr2=0x7f304406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:15.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.377+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 shutdown_connections 2024-09-10T11:36:15.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.377+0000 7f304b453640 1 -- 172.21.15.99:0/53158853 wait complete. 2024-09-10T11:36:15.397 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:36:15.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.499+0000 7f7bf6711640 1 Processor -- start 2024-09-10T11:36:15.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.499+0000 7f7bf6711640 1 -- start start 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7bf6711640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7bf0163140 0x7f7bf0163520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7bf6711640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bf00d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7bf6711640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bf00b5d30 con 0x7f7bf00714a0 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7bf6711640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf00d10d0 con 0x7f7bf0163140 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7bf6711640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf0071980 con 0x7f7bf00d0740 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7be7fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7bf00714a0 0x7f7bf00b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51270/0 (socket says 172.21.15.99:51270) 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.500+0000 7f7be7fff640 1 -- 172.21.15.99:0/345608204 learned_addr learned my addr 172.21.15.99:0/345608204 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bf4cae640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bf00d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972402564 0 0) 0x7f7bf00b5d30 con 0x7f7bf00714a0 2024-09-10T11:36:15.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7beffff640 1 --2- 172.21.15.99:0/345608204 >> v2:172.21.15.196:3300/0 conn(0x7f7bf0163140 0x7f7bf0163520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bdc003520 con 0x7f7bf00714a0 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3841726690 0 0) 0x7f7bdc003520 con 0x7f7bf00714a0 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 msgr2=0x7f7bf00d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bf00d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 >> v2:172.21.15.196:3300/0 conn(0x7f7bf0163140 msgr2=0x7f7bf0163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 --2- 172.21.15.99:0/345608204 >> v2:172.21.15.196:3300/0 conn(0x7f7bf0163140 0x7f7bf0163520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf0071c60 con 0x7f7bf00714a0 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7bd8001690 con 0x7f7bf00714a0 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 >> v1:172.21.15.99:6789/0 conn(0x7f7bf00714a0 legacy=0x7f7bf00b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc0039b0 0x7f7bdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc004800 0x7f7bdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf00d10d0 con 0x7f7bdc004800 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0071980 con 0x7f7bdc0039b0 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc00b9c0 con 0x7f7bf00d0740 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc004800 0x7f7bdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7beffff640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc0039b0 0x7f7bdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7bf4cae640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 -- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 msgr2=0x7f7bdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bf00d0740 0x7f7bdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 -- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc0039b0 msgr2=0x7f7bdc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bdc0039b0 0x7f7bdc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.501+0000 7f7be7fff640 1 -- 172.21.15.99:0/345608204 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc003520 con 0x7f7bdc004800 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.502+0000 7f7be7fff640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc004800 0x7f7bdc004c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00b5f00 tx=0x7f7bdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ecad601f22f64f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.502+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8007d60 con 0x7f7bdc004800 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.502+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd80058d0 con 0x7f7bdc004800 2024-09-10T11:36:15.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.502+0000 7f7bef7fe640 1 -- 172.21.15.99:0/345608204 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8005cc0 con 0x7f7bdc004800 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 -- 172.21.15.99:0/345608204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc004800 msgr2=0x7f7bdc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 --2- 172.21.15.99:0/345608204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bdc004800 0x7f7bdc004c80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00b5f00 tx=0x7f7bdc005830 comp rx=0 tx=0).stop 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf4cae640 1 -- 172.21.15.99:0/345608204 reap_dead start 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 -- 172.21.15.99:0/345608204 shutdown_connections 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 -- 172.21.15.99:0/345608204 >> 172.21.15.99:0/345608204 conn(0x7f7bf006bc10 msgr2=0x7f7bf006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 -- 172.21.15.99:0/345608204 shutdown_connections 2024-09-10T11:36:15.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.503+0000 7f7bf6711640 1 -- 172.21.15.99:0/345608204 wait complete. 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.504+0000 7f7bf6711640 1 Processor -- start 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.504+0000 7f7bf6711640 1 -- start start 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 0x7f7bf0148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 0x7f7bf0157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0072500 con 0x7f7bf015f060 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0072220 con 0x7f7bf0157030 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf6711640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf00d0b90 con 0x7f7bdc004800 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34354/0 (socket says 172.21.15.99:34354) 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 -- 172.21.15.99:0/1676354839 learned_addr learned my addr 172.21.15.99:0/1676354839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7be7fff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 0x7f7bf0157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7beffff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 0x7f7bf0148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 -- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 msgr2=0x7f7bf0148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 0x7f7bf0148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 -- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 msgr2=0x7f7bf0157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 0x7f7bf0157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 -- 172.21.15.99:0/1676354839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf0157b60 con 0x7f7bf015f060 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7be7fff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 0x7f7bf0157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7beffff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 0x7f7bf0148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bf4cae640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00d18b0 tx=0x7f7be800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.505+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be800fde0 con 0x7f7bf015f060 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.506+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be801d070 con 0x7f7bf015f060 2024-09-10T11:36:15.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.506+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8015670 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.506+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf0157e40 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.506+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf0158400 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.506+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf0163520 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.507+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7be8015810 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.507+0000 7f7bed7fa640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 0x7f7bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.508+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5609+0+0 (secure 0 0 0) 0x7f7be808b660 con 0x7f7bf015f060 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.508+0000 7f7beffff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 0x7f7bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.508+0000 7f7beffff640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 0x7f7bd006e2f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009bc0 tx=0x7f7be0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:15.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.512+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7be8010040 con 0x7f7bf015f060 2024-09-10T11:36:15.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:15.710+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7bf015e860 con 0x7f7bf015f060 2024-09-10T11:36:16.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.370+0000 7f7bed7fa640 1 -- 172.21.15.99:0/1676354839 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v59) ==== 174+0+0 (secure 0 0 0) 0x7f7be805a070 con 0x7f7bf015f060 2024-09-10T11:36:16.373 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 18 and data pool 19 2024-09-10T11:36:16.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 msgr2=0x7f7bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 0x7f7bd006e2f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7be0009bc0 tx=0x7f7be0008040 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 msgr2=0x7f7bf015f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00d18b0 tx=0x7f7be800f580 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 shutdown_connections 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7bd006be30 0x7f7bd006e2f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.375+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bf015f060 0x7f7bf015f510 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.376+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bf0157030 0x7f7bf0157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.376+0000 7f7bf6711640 1 --2- 172.21.15.99:0/1676354839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bdc004800 0x7f7bf0148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.376+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 >> 172.21.15.99:0/1676354839 conn(0x7f7bf006bc10 msgr2=0x7f7bf0161430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:16.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.377+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 shutdown_connections 2024-09-10T11:36:16.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.377+0000 7f7bf6711640 1 -- 172.21.15.99:0/1676354839 wait complete. 2024-09-10T11:36:16.406 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:16.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.503+0000 7f86e294b640 1 Processor -- start 2024-09-10T11:36:16.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.503+0000 7f86e294b640 1 -- start start 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e294b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 0x7f86dc073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e294b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86dc15c060 0x7f86dc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e294b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86dc152e60 con 0x7f86dc0d1200 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e294b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f86dc0d15e0 con 0x7f86dc15c060 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e294b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc072680 con 0x7f86dc06ece0 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86dc15c060 0x7f86dc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86dc15c060 0x7f86dc15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34364/0 (socket says 172.21.15.99:34364) 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 -- 172.21.15.99:0/1888687182 learned_addr learned my addr 172.21.15.99:0/1888687182 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:16.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86db7fe640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 0x7f86dc073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 -- 172.21.15.99:0/1888687182 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 msgr2=0x7f86dc073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 0x7f86dc073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 -- 172.21.15.99:0/1888687182 >> v1:172.21.15.196:6789/0 conn(0x7f86dc0d1200 legacy=0x7f86dc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86e0ee8640 1 -- 172.21.15.99:0/1888687182 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc072960 con 0x7f86dc15c060 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.504+0000 7f86db7fe640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 0x7f86dc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.505+0000 7f86e0ee8640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc15c060 0x7f86dc15c4a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009e50 tx=0x7f86d402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8908ebc3ad3b891e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.505+0000 7f86daffd640 1 -- 172.21.15.99:0/1888687182 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4036070 con 0x7f86dc15c060 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.505+0000 7f86daffd640 1 -- 172.21.15.99:0/1888687182 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4031600 con 0x7f86dc15c060 2024-09-10T11:36:16.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.505+0000 7f86daffd640 1 -- 172.21.15.99:0/1888687182 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d40319d0 con 0x7f86dc15c060 2024-09-10T11:36:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 -- 172.21.15.99:0/1888687182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc15c060 msgr2=0x7f86dc15c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc15c060 0x7f86dc15c4a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009e50 tx=0x7f86d402a630 comp rx=0 tx=0).stop 2024-09-10T11:36:16.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 -- 172.21.15.99:0/1888687182 shutdown_connections 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc15c060 0x7f86dc15c4a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 --2- 172.21.15.99:0/1888687182 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc06ece0 0x7f86dc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 -- 172.21.15.99:0/1888687182 >> 172.21.15.99:0/1888687182 conn(0x7f86dc06bc10 msgr2=0x7f86dc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 -- 172.21.15.99:0/1888687182 shutdown_connections 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.506+0000 7f86e294b640 1 -- 172.21.15.99:0/1888687182 wait complete. 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.507+0000 7f86e294b640 1 Processor -- start 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.507+0000 7f86e294b640 1 -- start start 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc0d1200 0x7f86dc143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 0x7f86dc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc15cd90 con 0x7f86dc06ece0 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc072f20 con 0x7f86dc15c060 2024-09-10T11:36:16.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e294b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc15c9e0 con 0x7f86dc0d1200 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34370/0 (socket says 172.21.15.99:34370) 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 -- 172.21.15.99:0/219612097 learned_addr learned my addr 172.21.15.99:0/219612097 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e0ee8640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 0x7f86dc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86db7fe640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc0d1200 0x7f86dc143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 -- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc0d1200 msgr2=0x7f86dc143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc0d1200 0x7f86dc143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 -- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 msgr2=0x7f86dc0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 0x7f86dc0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 -- 172.21.15.99:0/219612097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc13fef0 con 0x7f86dc06ece0 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86e0ee8640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 0x7f86dc0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.508+0000 7f86dbfff640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b3f0 tx=0x7f86cc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.509+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc015020 con 0x7f86dc06ece0 2024-09-10T11:36:16.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.509+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86cc011830 con 0x7f86dc06ece0 2024-09-10T11:36:16.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.509+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc1401d0 con 0x7f86dc06ece0 2024-09-10T11:36:16.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.509+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc011c20 con 0x7f86dc06ece0 2024-09-10T11:36:16.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.509+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc140790 con 0x7f86dc06ece0 2024-09-10T11:36:16.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.510+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc073440 con 0x7f86dc06ece0 2024-09-10T11:36:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.510+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86cc018020 con 0x7f86dc06ece0 2024-09-10T11:36:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.511+0000 7f86d8ff9640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 0x7f86c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.511+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5709+0+0 (secure 0 0 0) 0x7f86cc08aa60 con 0x7f86dc06ece0 2024-09-10T11:36:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.511+0000 7f86db7fe640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 0x7f86c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.511+0000 7f86db7fe640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 0x7f86c806e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86dc1463b0 tx=0x7f86d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.515+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86cc01f030 con 0x7f86dc06ece0 2024-09-10T11:36:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.682+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86dc15bc30 con 0x7f86dc06ece0 2024-09-10T11:36:16.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.683+0000 7f86d8ff9640 1 -- 172.21.15.99:0/219612097 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16701 (secure 0 0 0) 0x7f86cc0172c0 con 0x7f86dc06ece0 2024-09-10T11:36:16.685 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:16.686 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":152,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:15.712282+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:36:14.040327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T11:36:14.680742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 msgr2=0x7f86c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 0x7f86c806e320 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86dc1463b0 tx=0x7f86d0008040 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 msgr2=0x7f86dc15b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00b3f0 tx=0x7f86cc00b8b0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 shutdown_connections 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f86c806be60 0x7f86c806e320 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86dc15c060 0x7f86dc0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f86dc0d1200 0x7f86dc143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 --2- 172.21.15.99:0/219612097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86dc06ece0 0x7f86dc15b6f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 >> 172.21.15.99:0/219612097 conn(0x7f86dc06bc10 msgr2=0x7f86dc0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 shutdown_connections 2024-09-10T11:36:16.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.685+0000 7f86e294b640 1 -- 172.21.15.99:0/219612097 wait complete. 2024-09-10T11:36:16.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:16.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.795+0000 7fcd74cd2640 1 Processor -- start 2024-09-10T11:36:16.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.795+0000 7fcd74cd2640 1 -- start start 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd7015c7b0 0x7fcd700d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 0x7fcd7006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7006ee30 0x7fcd701604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcd700d1420 con 0x7fcd7015c7b0 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7006f280 con 0x7fcd700d0fb0 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd74cd2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700b3840 con 0x7fcd7006ee30 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd7015c7b0 0x7fcd700d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd7015c7b0 0x7fcd700d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34378/0 (socket says 172.21.15.99:34378) 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 -- 172.21.15.99:0/2526428435 learned_addr learned my addr 172.21.15.99:0/2526428435 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd67fff640 1 --2- 172.21.15.99:0/2526428435 >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 0x7fcd7006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6ed9d640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7006ee30 0x7fcd701604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 -- 172.21.15.99:0/2526428435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7006ee30 msgr2=0x7fcd701604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7006ee30 0x7fcd701604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 -- 172.21.15.99:0/2526428435 >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 msgr2=0x7fcd7006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/2526428435 >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 0x7fcd7006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd6e59c640 1 -- 172.21.15.99:0/2526428435 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700ceb10 con 0x7fcd7015c7b0 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.796+0000 7fcd67fff640 1 --2- 172.21.15.99:0/2526428435 >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 0x7fcd7006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.797+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7015c7b0 0x7fcd700d0a70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcd700d0250 tx=0x7fcd6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db001234d759d373 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.797+0000 7fcd6dd9b640 1 -- 172.21.15.99:0/2526428435 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60037070 con 0x7fcd7015c7b0 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.797+0000 7fcd6dd9b640 1 -- 172.21.15.99:0/2526428435 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd60032440 con 0x7fcd7015c7b0 2024-09-10T11:36:16.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.797+0000 7fcd6dd9b640 1 -- 172.21.15.99:0/2526428435 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60032810 con 0x7fcd7015c7b0 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 -- 172.21.15.99:0/2526428435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7015c7b0 msgr2=0x7fcd700d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7015c7b0 0x7fcd700d0a70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcd700d0250 tx=0x7fcd6002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 -- 172.21.15.99:0/2526428435 shutdown_connections 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7006ee30 0x7fcd701604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/2526428435 >> v2:172.21.15.196:3300/0 conn(0x7fcd700d0fb0 0x7fcd7006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/2526428435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7015c7b0 0x7fcd700d0a70 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 -- 172.21.15.99:0/2526428435 >> 172.21.15.99:0/2526428435 conn(0x7fcd7006bc10 msgr2=0x7fcd7006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 -- 172.21.15.99:0/2526428435 shutdown_connections 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.798+0000 7fcd74cd2640 1 -- 172.21.15.99:0/2526428435 wait complete. 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.799+0000 7fcd74cd2640 1 Processor -- start 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.799+0000 7fcd74cd2640 1 -- start start 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 0x7fcd70072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 0x7fcd7013eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b4050 con 0x7fcd7006ee30 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd700b3d70 con 0x7fcd700d0fb0 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd74cd2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcd700b4330 con 0x7fcd7015c7b0 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34390/0 (socket says 172.21.15.99:34390) 2024-09-10T11:36:16.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 -- 172.21.15.99:0/4239262698 learned_addr learned my addr 172.21.15.99:0/4239262698 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6ed9d640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 0x7fcd7013eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd67fff640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 0x7fcd70072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 -- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 msgr2=0x7fcd7013eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 0x7fcd7013eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 -- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 msgr2=0x7fcd70072b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 0x7fcd70072b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 -- 172.21.15.99:0/4239262698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd700730a0 con 0x7fcd7006ee30 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6ed9d640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 0x7fcd7013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd67fff640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 0x7fcd70072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.800+0000 7fcd6e59c640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002a220 tx=0x7fcd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.801+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60037040 con 0x7fcd7006ee30 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.801+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd60042070 con 0x7fcd7006ee30 2024-09-10T11:36:16.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.801+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60030bb0 con 0x7fcd7006ee30 2024-09-10T11:36:16.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.801+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd70073320 con 0x7fcd7006ee30 2024-09-10T11:36:16.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.801+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd70150230 con 0x7fcd7006ee30 2024-09-10T11:36:16.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.802+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7015cb90 con 0x7fcd7006ee30 2024-09-10T11:36:16.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.802+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd60047460 con 0x7fcd7006ee30 2024-09-10T11:36:16.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.803+0000 7fcd66ffd640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 0x7fcd5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:16.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.803+0000 7fcd67fff640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 0x7fcd5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:16.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.803+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5709+0+0 (secure 0 0 0) 0x7fcd600abaa0 con 0x7fcd7006ee30 2024-09-10T11:36:16.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.803+0000 7fcd67fff640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 0x7fcd5006e320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd7014d480 tx=0x7fcd58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:16.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.807+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd700730a0 con 0x7fcd7006ee30 2024-09-10T11:36:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.972+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd70158250 con 0x7fcd7006ee30 2024-09-10T11:36:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.973+0000 7fcd66ffd640 1 -- 172.21.15.99:0/4239262698 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+16701 (secure 0 0 0) 0x7fcd700ceb10 con 0x7fcd7006ee30 2024-09-10T11:36:16.975 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:16.976 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":152,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:15.712282+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:36:14.040327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T11:36:14.680742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 msgr2=0x7fcd5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 0x7fcd5006e320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd7014d480 tx=0x7fcd58008040 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 msgr2=0x7fcd7013f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcd6002a220 tx=0x7fcd600046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 shutdown_connections 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcd5006be60 0x7fcd5006e320 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcd7015c7b0 0x7fcd7013eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd700d0fb0 0x7fcd70072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.975+0000 7fcd74cd2640 1 --2- 172.21.15.99:0/4239262698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd7006ee30 0x7fcd7013f420 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.976+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 >> 172.21.15.99:0/4239262698 conn(0x7fcd7006bc10 msgr2=0x7fcd700afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.976+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 shutdown_connections 2024-09-10T11:36:16.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:16.976+0000 7fcd74cd2640 1 -- 172.21.15.99:0/4239262698 wait complete. 2024-09-10T11:36:16.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:36:17.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.089+0000 7f89fa523640 1 Processor -- start 2024-09-10T11:36:17.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.089+0000 7f89fa523640 1 -- start start 2024-09-10T11:36:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89fa523640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89fa523640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89fa523640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f4152e60 con 0x7f89f415c060 2024-09-10T11:36:17.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89fa523640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f89f406ef50 con 0x7f89f40d1200 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89fa523640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f89f40d15e0 con 0x7f89f406e5d0 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f3fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f3fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:42010/0 (socket says 172.21.15.99:42010) 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:52188/0 (socket says 172.21.15.99:52188) 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f3fff640 1 -- 172.21.15.99:0/3336159257 learned_addr learned my addr 172.21.15.99:0/3336159257 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f8ac0640 1 --1- 172.21.15.99:0/3336159257 >> v1:172.21.15.196:6789/0 conn(0x7f89f415c060 0x7f89f415c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:42802/0 (socket says 172.21.15.99:42802) 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 -- 172.21.15.99:0/3336159257 >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 msgr2=0x7f89f4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 --2- 172.21.15.99:0/3336159257 >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 -- 172.21.15.99:0/3336159257 >> v1:172.21.15.196:6789/0 conn(0x7f89f415c060 legacy=0x7f89f415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f37fe640 1 -- 172.21.15.99:0/3336159257 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f4072680 con 0x7f89f406e5d0 2024-09-10T11:36:17.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.090+0000 7f89f3fff640 1 --2- 172.21.15.99:0/3336159257 >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:17.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.091+0000 7f89f37fe640 1 --2- 172.21.15.99:0/3336159257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097a0 tx=0x7f89e402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99dedd03858716e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.091+0000 7f89f2ffd640 1 -- 172.21.15.99:0/3336159257 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4037070 con 0x7f89f406e5d0 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.091+0000 7f89f2ffd640 1 -- 172.21.15.99:0/3336159257 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e402ae50 con 0x7f89f406e5d0 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.091+0000 7f89f2ffd640 1 -- 172.21.15.99:0/3336159257 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e4032660 con 0x7f89f406e5d0 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 -- 172.21.15.99:0/3336159257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 msgr2=0x7f89f406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 --2- 172.21.15.99:0/3336159257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097a0 tx=0x7f89e402a100 comp rx=0 tx=0).stop 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 -- 172.21.15.99:0/3336159257 shutdown_connections 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 --2- 172.21.15.99:0/3336159257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f406ea10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 --2- 172.21.15.99:0/3336159257 >> v2:172.21.15.196:3300/0 conn(0x7f89f40d1200 0x7f89f4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 -- 172.21.15.99:0/3336159257 >> 172.21.15.99:0/3336159257 conn(0x7f89f406bc10 msgr2=0x7f89f406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 -- 172.21.15.99:0/3336159257 shutdown_connections 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.092+0000 7f89fa523640 1 -- 172.21.15.99:0/3336159257 wait complete. 2024-09-10T11:36:17.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.093+0000 7f89fa523640 1 Processor -- start 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.093+0000 7f89fa523640 1 -- start start 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f4151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f40d1200 0x7f89f415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4072f20 con 0x7f89f415c060 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4072c40 con 0x7f89f40d1200 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89fa523640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f89f415cd90 con 0x7f89f406e5d0 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89f8ac0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89f8ac0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34398/0 (socket says 172.21.15.99:34398) 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89f8ac0640 1 -- 172.21.15.99:0/455618279 learned_addr learned my addr 172.21.15.99:0/455618279 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89f3fff640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f4151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.094+0000 7f89f37fe640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f40d1200 0x7f89f415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 -- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 msgr2=0x7f89f4151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f4151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 -- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f40d1200 msgr2=0x7f89f415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f40d1200 0x7f89f415ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 -- 172.21.15.99:0/455618279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f40c5cb0 con 0x7f89f415c060 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f3fff640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f4151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f8ac0640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c780 tx=0x7f89ec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec017070 con 0x7f89f415c060 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ec00a640 con 0x7f89f415c060 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f4159bf0 con 0x7f89f415c060 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00a9f0 con 0x7f89f415c060 2024-09-10T11:36:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.095+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f415a1b0 con 0x7f89f415c060 2024-09-10T11:36:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.096+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f40cfb20 con 0x7f89f415c060 2024-09-10T11:36:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.096+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89ec01a070 con 0x7f89f415c060 2024-09-10T11:36:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.097+0000 7f89f0ff9640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 0x7f89d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.097+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5709+0+0 (secure 0 0 0) 0x7f89ec08a380 con 0x7f89f415c060 2024-09-10T11:36:17.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.097+0000 7f89f3fff640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 0x7f89d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.097+0000 7f89f3fff640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 0x7f89d406e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bcc0 tx=0x7f89dc00b450 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.101+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89f40cfb20 con 0x7f89f415c060 2024-09-10T11:36:17.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.301+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f89f406ea10 con 0x7f89f415c060 2024-09-10T11:36:17.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.385+0000 7f89f0ff9640 1 -- 172.21.15.99:0/455618279 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f89ec0586e0 con 0x7f89f415c060 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 msgr2=0x7f89d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 0x7f89d406e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00bcc0 tx=0x7f89dc00b450 comp rx=0 tx=0).stop 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 msgr2=0x7f89f415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c780 tx=0x7f89ec00cc40 comp rx=0 tx=0).stop 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 shutdown_connections 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f89d406be30 0x7f89d406e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89f415c060 0x7f89f415b0c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89f40d1200 0x7f89f415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 --2- 172.21.15.99:0/455618279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89f406e5d0 0x7f89f4151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 >> 172.21.15.99:0/455618279 conn(0x7f89f406bc10 msgr2=0x7f89f415dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:17.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 shutdown_connections 2024-09-10T11:36:17.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.388+0000 7f89fa523640 1 -- 172.21.15.99:0/455618279 wait complete. 2024-09-10T11:36:17.401 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:36:17.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:36:17.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.500+0000 7f65f44ae640 1 Processor -- start 2024-09-10T11:36:17.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.500+0000 7f65f44ae640 1 -- start start 2024-09-10T11:36:17.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f44ae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f65ec0b2460 0x7f65ec166700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f44ae640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ec073d70 con 0x7f65ec15ba90 2024-09-10T11:36:17.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f44ae640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65ec0b5360 con 0x7f65ec0b02a0 2024-09-10T11:36:17.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f44ae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f65ec0b55f0 con 0x7f65ec0b2460 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f224a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f65ec15ba90 0x7f65ec0ae780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51274/0 (socket says 172.21.15.99:51274) 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f224a640 1 -- 172.21.15.99:0/4056083778 learned_addr learned my addr 172.21.15.99:0/4056083778 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030362757 0 0) 0x7f65ec073d70 con 0x7f65ec15ba90 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65d8003520 con 0x7f65ec15ba90 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/4056083778 >> v2:172.21.15.196:3300/0 conn(0x7f65ec0b2460 0x7f65ec166700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2881996834 0 0) 0x7f65d8003520 con 0x7f65ec15ba90 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 >> v2:172.21.15.196:3300/0 conn(0x7f65ec0b2460 msgr2=0x7f65ec166700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 --2- 172.21.15.99:0/4056083778 >> v2:172.21.15.196:3300/0 conn(0x7f65ec0b2460 0x7f65ec166700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 >> v1:172.21.15.196:6789/0 conn(0x7f65ec0b02a0 legacy=0x7f65ec0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec0b0970 con 0x7f65ec15ba90 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f65e8001670 con 0x7f65ec15ba90 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 >> v1:172.21.15.99:6789/0 conn(0x7f65ec15ba90 legacy=0x7f65ec0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80039b0 0x7f65d8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65d8004800 0x7f65d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec0b55f0 con 0x7f65d80051c0 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65d800c0f0 con 0x7f65d8004800 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.501+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f65d800c2f0 con 0x7f65d80039b0 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f224a640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80039b0 0x7f65d8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f1a49640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65d8004800 0x7f65d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 -- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80039b0 msgr2=0x7f65d8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80039b0 0x7f65d8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 -- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65d8004800 msgr2=0x7f65d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65d8004800 0x7f65d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 -- 172.21.15.99:0/4056083778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d8003520 con 0x7f65d80051c0 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f1a49640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65d8004800 0x7f65d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 -- 172.21.15.99:0/4056083778 reap_dead start 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.502+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 0x7f65d800b9c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f65e4007330 tx=0x7f65d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26da3ba0e7994ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4009be0 con 0x7f65d80051c0 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e4003b90 con 0x7f65d80051c0 2024-09-10T11:36:17.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f1248640 1 -- 172.21.15.99:0/4056083778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004020 con 0x7f65d80051c0 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 -- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 msgr2=0x7f65d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 0x7f65d800b9c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f65e4007330 tx=0x7f65d8005830 comp rx=0 tx=0).stop 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 -- 172.21.15.99:0/4056083778 shutdown_connections 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 --2- 172.21.15.99:0/4056083778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d80051c0 0x7f65d800b9c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 -- 172.21.15.99:0/4056083778 >> 172.21.15.99:0/4056083778 conn(0x7f65ec0730c0 msgr2=0x7f65ec0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:17.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 -- 172.21.15.99:0/4056083778 shutdown_connections 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.503+0000 7f65f44ae640 1 -- 172.21.15.99:0/4056083778 wait complete. 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.504+0000 7f65f44ae640 1 Processor -- start 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 -- start start 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80051c0 0x7f65ec150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec14fe20 0x7f65ec151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec15be70 con 0x7f65ec09a5e0 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec0b0f30 con 0x7f65ec14fe20 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f44ae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec15bff0 con 0x7f65d80051c0 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f2a4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f2a4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34414/0 (socket says 172.21.15.99:34414) 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.505+0000 7f65f2a4b640 1 -- 172.21.15.99:0/3942892668 learned_addr learned my addr 172.21.15.99:0/3942892668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f224a640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80051c0 0x7f65ec150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 -- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80051c0 msgr2=0x7f65ec150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80051c0 0x7f65ec150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 -- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec14fe20 msgr2=0x7f65ec151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec14fe20 0x7f65ec151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 -- 172.21.15.99:0/3942892668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec1502a0 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f2a4b640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f65e402b070 tx=0x7f65e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4009be0 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e4041440 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4041810 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec09afa0 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.506+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec150f20 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.507+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec0b1450 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.507+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65e4036440 con 0x7f65ec09a5e0 2024-09-10T11:36:17.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.508+0000 7f65e2ffd640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 0x7f65d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:17.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.508+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f65e40ab8d0 con 0x7f65ec09a5e0 2024-09-10T11:36:17.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.508+0000 7f65f224a640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 0x7f65d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:17.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.508+0000 7f65f224a640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 0x7f65d006e320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001050 tx=0x7f65e8000f40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:17.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.512+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f65e4047060 con 0x7f65ec09a5e0 2024-09-10T11:36:17.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:17.709+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f65ec0b2b20 con 0x7f65ec09a5e0 2024-09-10T11:36:18.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.390+0000 7f65e2ffd640 1 -- 172.21.15.99:0/3942892668 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f65e407ae20 con 0x7f65ec09a5e0 2024-09-10T11:36:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 msgr2=0x7f65d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 0x7f65d006e320 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f65e8001050 tx=0x7f65e8000f40 comp rx=0 tx=0).stop 2024-09-10T11:36:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 msgr2=0x7f65ec09aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f65e402b070 tx=0x7f65e4004ab0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 shutdown_connections 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f65d006be60 0x7f65d006e320 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65ec09a5e0 0x7f65ec09aa60 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65ec14fe20 0x7f65ec151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 --2- 172.21.15.99:0/3942892668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d80051c0 0x7f65ec150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 >> 172.21.15.99:0/3942892668 conn(0x7f65ec0730c0 msgr2=0x7f65ec0d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 shutdown_connections 2024-09-10T11:36:18.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.393+0000 7f65f44ae640 1 -- 172.21.15.99:0/3942892668 wait complete. 2024-09-10T11:36:18.408 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:36:18.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:18.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.509+0000 7f76175b7640 1 Processor -- start 2024-09-10T11:36:18.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.509+0000 7f76175b7640 1 -- start start 2024-09-10T11:36:18.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f76175b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f7610073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f76175b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f761015c770 0x7f761006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f76175b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7610152e60 con 0x7f761006e5d0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f76175b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f761006ef50 con 0x7f761015c770 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f76175b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f76100d15e0 con 0x7f76100d1200 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7614b52640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f761006e5d0 0x7f761015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51282/0 (socket says 172.21.15.99:51282) 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7614b52640 1 -- 172.21.15.99:0/2064738796 learned_addr learned my addr 172.21.15.99:0/2064738796 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3764248368 0 0) 0x7f7610152e60 con 0x7f761006e5d0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7615353640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f7610073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75fc005730 con 0x7f761006e5d0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.510+0000 7f7615b54640 1 --2- 172.21.15.99:0/2064738796 >> v2:172.21.15.196:3300/0 conn(0x7f761015c770 0x7f761006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2178769320 0 0) 0x7f75fc005730 con 0x7f761006e5d0 2024-09-10T11:36:18.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 msgr2=0x7f7610073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f7610073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 >> v2:172.21.15.196:3300/0 conn(0x7f761015c770 msgr2=0x7f761006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 --2- 172.21.15.99:0/2064738796 >> v2:172.21.15.196:3300/0 conn(0x7f761015c770 0x7f761006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7610072680 con 0x7f761006e5d0 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7600001670 con 0x7f761006e5d0 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 >> v1:172.21.15.99:6789/0 conn(0x7f761006e5d0 legacy=0x7f761015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f75fc005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75fc006140 0x7f75fc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f75fc0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f761006ef50 con 0x7f75fc006f50 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f76100d15e0 con 0x7f75fc006140 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0034c0 con 0x7f76100d1200 2024-09-10T11:36:18.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f75fc0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615353640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f75fc005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7614b52640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75fc006140 0x7f75fc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 -- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 msgr2=0x7f75fc005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f75fc005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 -- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75fc006140 msgr2=0x7f75fc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75fc006140 0x7f75fc006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615b54640 1 -- 172.21.15.99:0/2064738796 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc005730 con 0x7f75fc006f50 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.511+0000 7f7615353640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76100d1200 0x7f75fc005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f7615b54640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f75fc0073d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7608009fc0 tx=0x7f75fc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e577a22a8d2606 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7608009c10 con 0x7f75fc006f50 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7608036070 con 0x7f75fc006f50 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f7607fff640 1 -- 172.21.15.99:0/2064738796 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760802d070 con 0x7f75fc006f50 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f76175b7640 1 -- 172.21.15.99:0/2064738796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 msgr2=0x7f75fc0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f76175b7640 1 --2- 172.21.15.99:0/2064738796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f75fc0073d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7608009fc0 tx=0x7f75fc007a40 comp rx=0 tx=0).stop 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f7615b54640 1 -- 172.21.15.99:0/2064738796 reap_dead start 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f76175b7640 1 -- 172.21.15.99:0/2064738796 shutdown_connections 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.512+0000 7f76175b7640 1 -- 172.21.15.99:0/2064738796 >> 172.21.15.99:0/2064738796 conn(0x7f761006bc10 msgr2=0x7f761006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.513+0000 7f76175b7640 1 -- 172.21.15.99:0/2064738796 shutdown_connections 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.513+0000 7f76175b7640 1 -- 172.21.15.99:0/2064738796 wait complete. 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.514+0000 7f76175b7640 1 Processor -- start 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.514+0000 7f76175b7640 1 -- start start 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.514+0000 7f76175b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.514+0000 7f76175b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761015dbd0 0x7f761014d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76101558f0 0x7f76101f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7610072f20 con 0x7f75fc006f50 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7610072c40 con 0x7f761015dbd0 2024-09-10T11:36:18.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f761015cd90 con 0x7f76101558f0 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34426/0 (socket says 172.21.15.99:34426) 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 -- 172.21.15.99:0/2004351763 learned_addr learned my addr 172.21.15.99:0/2004351763 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7614b52640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761015dbd0 0x7f761014d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615b54640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76101558f0 0x7f76101f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 -- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76101558f0 msgr2=0x7f76101f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76101558f0 0x7f76101f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 -- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761015dbd0 msgr2=0x7f761014d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761015dbd0 0x7f761014d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 -- 172.21.15.99:0/2004351763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761015e340 con 0x7f75fc006f50 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7615353640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f760c011630 tx=0x7f760c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c00f920 con 0x7f75fc006f50 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c00fac0 con 0x7f75fc006f50 2024-09-10T11:36:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761015e060 con 0x7f75fc006f50 2024-09-10T11:36:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c00fe90 con 0x7f75fc006f50 2024-09-10T11:36:18.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.515+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7610157350 con 0x7f75fc006f50 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.517+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761006ea10 con 0x7f75fc006f50 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.517+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f760c015460 con 0x7f75fc006f50 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.517+0000 7f7605ffb640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 0x7f75f406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.517+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f760c08b630 con 0x7f75fc006f50 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.517+0000 7f7614b52640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 0x7f75f406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.518+0000 7f7614b52640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 0x7f75f406e4c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f76100b3600 tx=0x7f7600007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.521+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f760c019030 con 0x7f75fc006f50 2024-09-10T11:36:18.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.723+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7610150c70 con 0x7f75fc006f50 2024-09-10T11:36:18.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.723+0000 7f7605ffb640 1 -- 172.21.15.99:0/2004351763 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f760c05a070 con 0x7f75fc006f50 2024-09-10T11:36:18.725 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:18.725 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:36:18.726 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:36:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 msgr2=0x7f75f406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 0x7f75f406e4c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f76100b3600 tx=0x7f7600007040 comp rx=0 tx=0).stop 2024-09-10T11:36:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 msgr2=0x7f76101565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f760c011630 tx=0x7f760c011af0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 shutdown_connections 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f75f406c000 0x7f75f406e4c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76101558f0 0x7f76101f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761015dbd0 0x7f761014d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 --2- 172.21.15.99:0/2004351763 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75fc006f50 0x7f76101565d0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 >> 172.21.15.99:0/2004351763 conn(0x7f761006bc10 msgr2=0x7f761006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 shutdown_connections 2024-09-10T11:36:18.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.725+0000 7f76175b7640 1 -- 172.21.15.99:0/2004351763 wait complete. 2024-09-10T11:36:18.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:36:18.825 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-10T11:36:18.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.835+0000 7fd8fe853640 1 Processor -- start 2024-09-10T11:36:18.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.835+0000 7fd8fe853640 1 -- start start 2024-09-10T11:36:18.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fe853640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8f80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fe853640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f80ceb10 con 0x7fd8f806f000 2024-09-10T11:36:18.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fe853640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f80b3a90 con 0x7fd8f80d0fb0 2024-09-10T11:36:18.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fe853640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f80b3840 con 0x7fd8f815c7b0 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fcdf0640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd8f806f000 0x7fd8f806d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51288/0 (socket says 172.21.15.99:51288) 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2131282446 learned_addr learned my addr 172.21.15.99:0/2131282446 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f7fff640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8f80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1216697833 0 0) 0x7fd8f0001490 con 0x7fd8f806f000 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e0003520 con 0x7fd8f806f000 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4179190171 0 0) 0x7fd8e0003520 con 0x7fd8f806f000 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 msgr2=0x7fd8f80d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8f80d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 >> v1:172.21.15.196:6789/0 conn(0x7fd8f80d0fb0 legacy=0x7fd8f806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f80b3d70 con 0x7fd8f806f000 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fd8f0001bc0 con 0x7fd8f806f000 2024-09-10T11:36:18.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 >> v1:172.21.15.99:6789/0 conn(0x7fd8f806f000 legacy=0x7fd8f806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8e00039e0 0x7fd8e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0004d40 0x7fd8e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80b3840 con 0x7fd8e0004d40 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e000ba70 con 0x7fd8e00039e0 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.836+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e000bc70 con 0x7fd8f815c7b0 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0004d40 0x7fd8e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8f7fff640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8e00039e0 0x7fd8e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8f77fe640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 msgr2=0x7fd8e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8f815c7b0 0x7fd8e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8e00039e0 msgr2=0x7fd8e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:18.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8e00039e0 0x7fd8e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2131282446 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0003520 con 0x7fd8e0004d40 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.837+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0004d40 0x7fd8e00051c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00013d0 tx=0x7fd8e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6f38890016e1af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000a650 con 0x7fd8e0004d40 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f002f070 con 0x7fd8e0004d40 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8f6ffd640 1 -- 172.21.15.99:0/2131282446 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f002be40 con 0x7fd8e0004d40 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 -- 172.21.15.99:0/2131282446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0004d40 msgr2=0x7fd8e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2131282446 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8e0004d40 0x7fd8e00051c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00013d0 tx=0x7fd8e0005830 comp rx=0 tx=0).stop 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2131282446 reap_dead start 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 -- 172.21.15.99:0/2131282446 shutdown_connections 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 -- 172.21.15.99:0/2131282446 >> 172.21.15.99:0/2131282446 conn(0x7fd8f806bc10 msgr2=0x7fd8f806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 -- 172.21.15.99:0/2131282446 shutdown_connections 2024-09-10T11:36:18.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.838+0000 7fd8fe853640 1 -- 172.21.15.99:0/2131282446 wait complete. 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.839+0000 7fd8fe853640 1 Processor -- start 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 -- start start 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8e0004d40 0x7fd8f8145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 0x7fd8f8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80b4330 con 0x7fd8f8150bb0 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f80d13f0 con 0x7fd8f81f2d10 2024-09-10T11:36:18.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fe853640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f80b4610 con 0x7fd8e0004d40 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fcdf0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fcdf0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34438/0 (socket says 172.21.15.99:34438) 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2215414017 learned_addr learned my addr 172.21.15.99:0/2215414017 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.840+0000 7fd8f7fff640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8e0004d40 0x7fd8f8145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8f77fe640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 0x7fd8f8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8e0004d40 msgr2=0x7fd8f8145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8e0004d40 0x7fd8f8145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 msgr2=0x7fd8f8150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 0x7fd8f8150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 -- 172.21.15.99:0/2215414017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f81514e0 con 0x7fd8f8150bb0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8f77fe640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 0x7fd8f8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fcdf0640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002b690 tx=0x7fd8f0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000a650 con 0x7fd8f8150bb0 2024-09-10T11:36:18.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f0001490 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0034670 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f8151d30 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.841+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f81522b0 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.842+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f815ccb0 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.843+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8f002bd10 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.843+0000 7fd8f4ff9640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 0x7fd8e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.843+0000 7fd8f7fff640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 0x7fd8e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.843+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7fd8f00ab2a0 con 0x7fd8f8150bb0 2024-09-10T11:36:18.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.844+0000 7fd8f7fff640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 0x7fd8e406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009bc0 tx=0x7fd8e8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:18.848+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd8f80ceb10 con 0x7fd8f8150bb0 2024-09-10T11:36:19.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.013+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8f81582e0 con 0x7fd8f8150bb0 2024-09-10T11:36:19.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.014+0000 7fd8f4ff9640 1 -- 172.21.15.99:0/2215414017 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16688 (secure 0 0 0) 0x7fd8f0079bc0 con 0x7fd8f8150bb0 2024-09-10T11:36:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:19.016 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":153,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:36:17.372339+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:36:14.040327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T11:36:14.680742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 msgr2=0x7fd8e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 0x7fd8e406e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009bc0 tx=0x7fd8e8009190 comp rx=0 tx=0).stop 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 msgr2=0x7fd8f815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd8f002b690 tx=0x7fd8f0001360 comp rx=0 tx=0).stop 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 shutdown_connections 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd8e406be30 0x7fd8e406e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8f8150bb0 0x7fd8f815a790 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8f81f2d10 0x7fd8f8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 --2- 172.21.15.99:0/2215414017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8e0004d40 0x7fd8f8145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 >> 172.21.15.99:0/2215414017 conn(0x7fd8f806bc10 msgr2=0x7fd8f80b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 shutdown_connections 2024-09-10T11:36:19.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.016+0000 7fd8fe853640 1 -- 172.21.15.99:0/2215414017 wait complete. 2024-09-10T11:36:19.029 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:36:19.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:36:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.125+0000 7f397baae640 1 Processor -- start 2024-09-10T11:36:19.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.125+0000 7f397baae640 1 -- start start 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f397415ee00 0x7f39740b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f397415f1e0 0x7f39740b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 0x7f39740b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f39740b40e0 con 0x7f397415ee00 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f39740b5ab0 con 0x7f397415f1e0 2024-09-10T11:36:19.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397baae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3974073c50 con 0x7f39740b0090 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397984a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f397415ee00 0x7f39740b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397984a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f397415ee00 0x7f39740b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34454/0 (socket says 172.21.15.99:34454) 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.126+0000 7f397984a640 1 -- 172.21.15.99:0/284568426 learned_addr learned my addr 172.21.15.99:0/284568426 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 -- 172.21.15.99:0/284568426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 msgr2=0x7f39740b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397a04b640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 0x7f39740b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 0x7f39740b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f3979049640 1 --2- 172.21.15.99:0/284568426 >> v2:172.21.15.196:3300/0 conn(0x7f397415f1e0 0x7f39740b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 -- 172.21.15.99:0/284568426 >> v2:172.21.15.196:3300/0 conn(0x7f397415f1e0 msgr2=0x7f39740b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 --2- 172.21.15.99:0/284568426 >> v2:172.21.15.196:3300/0 conn(0x7f397415f1e0 0x7f39740b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 -- 172.21.15.99:0/284568426 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39740b4380 con 0x7f397415ee00 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397a04b640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 0x7f39740b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.127+0000 7f397984a640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f397415ee00 0x7f39740b5f90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f397406da30 tx=0x7f396402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f177b1b50c8cc724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f3978848640 1 -- 172.21.15.99:0/284568426 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964036070 con 0x7f397415ee00 2024-09-10T11:36:19.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f3978848640 1 -- 172.21.15.99:0/284568426 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3964031600 con 0x7f397415ee00 2024-09-10T11:36:19.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f3978848640 1 -- 172.21.15.99:0/284568426 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39640319d0 con 0x7f397415ee00 2024-09-10T11:36:19.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 -- 172.21.15.99:0/284568426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f397415ee00 msgr2=0x7f39740b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f397415ee00 0x7f39740b5f90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f397406da30 tx=0x7f396402a6a0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 -- 172.21.15.99:0/284568426 shutdown_connections 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39740b0090 0x7f39740b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 --2- 172.21.15.99:0/284568426 >> v2:172.21.15.196:3300/0 conn(0x7f397415f1e0 0x7f39740b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 --2- 172.21.15.99:0/284568426 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f397415ee00 0x7f39740b5f90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 -- 172.21.15.99:0/284568426 >> 172.21.15.99:0/284568426 conn(0x7f39740730c0 msgr2=0x7f39740734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 -- 172.21.15.99:0/284568426 shutdown_connections 2024-09-10T11:36:19.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.128+0000 7f397baae640 1 -- 172.21.15.99:0/284568426 wait complete. 2024-09-10T11:36:19.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.129+0000 7f397baae640 1 Processor -- start 2024-09-10T11:36:19.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.129+0000 7f397baae640 1 -- start start 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 0x7f3974162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397415f1e0 0x7f397406e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39740b1fe0 con 0x7f39740b0090 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39740afd60 con 0x7f397415f1e0 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397baae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f39740b2160 con 0x7f397415ee00 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34460/0 (socket says 172.21.15.99:34460) 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 -- 172.21.15.99:0/2963192806 learned_addr learned my addr 172.21.15.99:0/2963192806 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f3979049640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 0x7f3974162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397a04b640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397415f1e0 0x7f397406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 -- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 msgr2=0x7f3974162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 0x7f3974162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 -- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397415f1e0 msgr2=0x7f397406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397415f1e0 0x7f397406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f397984a640 1 -- 172.21.15.99:0/2963192806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3974157330 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.130+0000 7f3979049640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 0x7f3974162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f397984a640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3964034040 tx=0x7f3964004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396403f020 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3964037040 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3974161080 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964036070 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.131+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3974161640 con 0x7f39740b0090 2024-09-10T11:36:19.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.132+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39740b0600 con 0x7f39740b0090 2024-09-10T11:36:19.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.132+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3964042020 con 0x7f39740b0090 2024-09-10T11:36:19.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.133+0000 7f39727fc640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.133+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f39640ab470 con 0x7f39740b0090 2024-09-10T11:36:19.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.133+0000 7f3979049640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.133+0000 7f3979049640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 0x7f396006e320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3968006fd0 tx=0x7f3968008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.137+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3964049030 con 0x7f39740b0090 2024-09-10T11:36:19.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.299+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3974073540 con 0x7f396006be60 2024-09-10T11:36:19.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.305+0000 7f39727fc640 1 -- 172.21.15.99:0/2963192806 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247921 (secure 0 0 0) 0x7f3974073540 con 0x7f396006be60 2024-09-10T11:36:19.310 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:19.311 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 msgr2=0x7f396006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 0x7f396006e320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3968006fd0 tx=0x7f3968008040 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 msgr2=0x7f3974159f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3964034040 tx=0x7f3964004a40 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 shutdown_connections 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f396006be60 0x7f396006e320 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f397415f1e0 0x7f397406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f397415ee00 0x7f3974162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 --2- 172.21.15.99:0/2963192806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39740b0090 0x7f3974159f90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 >> 172.21.15.99:0/2963192806 conn(0x7f39740730c0 msgr2=0x7f397415c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 shutdown_connections 2024-09-10T11:36:19.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.308+0000 7f397baae640 1 -- 172.21.15.99:0/2963192806 wait complete. 2024-09-10T11:36:19.321 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-10T11:36:18.982022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7755,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554744,"statfs":{"total":1600000000,"available":1592057993,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.618707"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392564+0000","last_change":"2024-09-10T11:36:16.392416+0000","last_active":"2024-09-10T11:36:16.392564+0000","last_peered":"2024-09-10T11:36:16.392564+0000","last_clean":"2024-09-10T11:36:16.392564+0000","last_became_active":"2024-09-10T11:36:16.391653+0000","last_became_peered":"2024-09-10T11:36:16.391653+0000","last_unstale":"2024-09-10T11:36:16.392564+0000","last_undegraded":"2024-09-10T11:36:16.392564+0000","last_fullsized":"2024-09-10T11:36:16.392564+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:03.624410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390327+0000","last_change":"2024-09-10T11:36:16.390219+0000","last_active":"2024-09-10T11:36:16.390327+0000","last_peered":"2024-09-10T11:36:16.390327+0000","last_clean":"2024-09-10T11:36:16.390327+0000","last_became_active":"2024-09-10T11:36:16.389546+0000","last_became_peered":"2024-09-10T11:36:16.389546+0000","last_unstale":"2024-09-10T11:36:16.390327+0000","last_undegraded":"2024-09-10T11:36:16.390327+0000","last_fullsized":"2024-09-10T11:36:16.390327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:29.623487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392814+0000","last_change":"2024-09-10T11:36:16.391679+0000","last_active":"2024-09-10T11:36:16.392814+0000","last_peered":"2024-09-10T11:36:16.392814+0000","last_clean":"2024-09-10T11:36:16.392814+0000","last_became_active":"2024-09-10T11:36:16.390954+0000","last_became_peered":"2024-09-10T11:36:16.390954+0000","last_unstale":"2024-09-10T11:36:16.392814+0000","last_undegraded":"2024-09-10T11:36:16.392814+0000","last_fullsized":"2024-09-10T11:36:16.392814+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:30.633037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395292+0000","last_change":"2024-09-10T11:36:16.395185+0000","last_active":"2024-09-10T11:36:16.395292+0000","last_peered":"2024-09-10T11:36:16.395292+0000","last_clean":"2024-09-10T11:36:16.395292+0000","last_became_active":"2024-09-10T11:36:16.395003+0000","last_became_peered":"2024-09-10T11:36:16.395003+0000","last_unstale":"2024-09-10T11:36:16.395292+0000","last_undegraded":"2024-09-10T11:36:16.395292+0000","last_fullsized":"2024-09-10T11:36:16.395292+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:13.833806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.393100+0000","last_change":"2024-09-10T11:36:16.393000+0000","last_active":"2024-09-10T11:36:16.393100+0000","last_peered":"2024-09-10T11:36:16.393100+0000","last_clean":"2024-09-10T11:36:16.393100+0000","last_became_active":"2024-09-10T11:36:16.392774+0000","last_became_peered":"2024-09-10T11:36:16.392774+0000","last_unstale":"2024-09-10T11:36:16.393100+0000","last_undegraded":"2024-09-10T11:36:16.393100+0000","last_fullsized":"2024-09-10T11:36:16.393100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:20.242917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388141+0000","last_change":"2024-09-10T11:36:16.388010+0000","last_active":"2024-09-10T11:36:16.388141+0000","last_peered":"2024-09-10T11:36:16.388141+0000","last_clean":"2024-09-10T11:36:16.388141+0000","last_became_active":"2024-09-10T11:36:16.387557+0000","last_became_peered":"2024-09-10T11:36:16.387557+0000","last_unstale":"2024-09-10T11:36:16.388141+0000","last_undegraded":"2024-09-10T11:36:16.388141+0000","last_fullsized":"2024-09-10T11:36:16.388141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:53.743553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389870+0000","last_change":"2024-09-10T11:36:16.389765+0000","last_active":"2024-09-10T11:36:16.389870+0000","last_peered":"2024-09-10T11:36:16.389870+0000","last_clean":"2024-09-10T11:36:16.389870+0000","last_became_active":"2024-09-10T11:36:16.389549+0000","last_became_peered":"2024-09-10T11:36:16.389549+0000","last_unstale":"2024-09-10T11:36:16.389870+0000","last_undegraded":"2024-09-10T11:36:16.389870+0000","last_fullsized":"2024-09-10T11:36:16.389870+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:34.891883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385141+0000","last_change":"2024-09-10T11:36:16.383526+0000","last_active":"2024-09-10T11:36:16.385141+0000","last_peered":"2024-09-10T11:36:16.385141+0000","last_clean":"2024-09-10T11:36:16.385141+0000","last_became_active":"2024-09-10T11:36:16.383298+0000","last_became_peered":"2024-09-10T11:36:16.383298+0000","last_unstale":"2024-09-10T11:36:16.385141+0000","last_undegraded":"2024-09-10T11:36:16.385141+0000","last_fullsized":"2024-09-10T11:36:16.385141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:53.934754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388453+0000","last_change":"2024-09-10T11:36:16.388017+0000","last_active":"2024-09-10T11:36:16.388453+0000","last_peered":"2024-09-10T11:36:16.388453+0000","last_clean":"2024-09-10T11:36:16.388453+0000","last_became_active":"2024-09-10T11:36:16.386907+0000","last_became_peered":"2024-09-10T11:36:16.386907+0000","last_unstale":"2024-09-10T11:36:16.388453+0000","last_undegraded":"2024-09-10T11:36:16.388453+0000","last_fullsized":"2024-09-10T11:36:16.388453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:23.572052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385040+0000","last_change":"2024-09-10T11:36:16.384553+0000","last_active":"2024-09-10T11:36:16.385040+0000","last_peered":"2024-09-10T11:36:16.385040+0000","last_clean":"2024-09-10T11:36:16.385040+0000","last_became_active":"2024-09-10T11:36:16.384325+0000","last_became_peered":"2024-09-10T11:36:16.384325+0000","last_unstale":"2024-09-10T11:36:16.385040+0000","last_undegraded":"2024-09-10T11:36:16.385040+0000","last_fullsized":"2024-09-10T11:36:16.385040+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:25.867523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387504+0000","last_change":"2024-09-10T11:36:16.386301+0000","last_active":"2024-09-10T11:36:16.387504+0000","last_peered":"2024-09-10T11:36:16.387504+0000","last_clean":"2024-09-10T11:36:16.387504+0000","last_became_active":"2024-09-10T11:36:16.383002+0000","last_became_peered":"2024-09-10T11:36:16.383002+0000","last_unstale":"2024-09-10T11:36:16.387504+0000","last_undegraded":"2024-09-10T11:36:16.387504+0000","last_fullsized":"2024-09-10T11:36:16.387504+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:01.537994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389506+0000","last_change":"2024-09-10T11:36:16.389358+0000","last_active":"2024-09-10T11:36:16.389506+0000","last_peered":"2024-09-10T11:36:16.389506+0000","last_clean":"2024-09-10T11:36:16.389506+0000","last_became_active":"2024-09-10T11:36:16.389112+0000","last_became_peered":"2024-09-10T11:36:16.389112+0000","last_unstale":"2024-09-10T11:36:16.389506+0000","last_undegraded":"2024-09-10T11:36:16.389506+0000","last_fullsized":"2024-09-10T11:36:16.389506+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:16.409948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390027+0000","last_change":"2024-09-10T11:36:16.389281+0000","last_active":"2024-09-10T11:36:16.390027+0000","last_peered":"2024-09-10T11:36:16.390027+0000","last_clean":"2024-09-10T11:36:16.390027+0000","last_became_active":"2024-09-10T11:36:16.389050+0000","last_became_peered":"2024-09-10T11:36:16.389050+0000","last_unstale":"2024-09-10T11:36:16.390027+0000","last_undegraded":"2024-09-10T11:36:16.390027+0000","last_fullsized":"2024-09-10T11:36:16.390027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:06.936329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380787+0000","last_change":"2024-09-10T11:36:16.380607+0000","last_active":"2024-09-10T11:36:16.380787+0000","last_peered":"2024-09-10T11:36:16.380787+0000","last_clean":"2024-09-10T11:36:16.380787+0000","last_became_active":"2024-09-10T11:36:16.380366+0000","last_became_peered":"2024-09-10T11:36:16.380366+0000","last_unstale":"2024-09-10T11:36:16.380787+0000","last_undegraded":"2024-09-10T11:36:16.380787+0000","last_fullsized":"2024-09-10T11:36:16.380787+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:36.972213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386991+0000","last_change":"2024-09-10T11:36:16.384493+0000","last_active":"2024-09-10T11:36:16.386991+0000","last_peered":"2024-09-10T11:36:16.386991+0000","last_clean":"2024-09-10T11:36:16.386991+0000","last_became_active":"2024-09-10T11:36:16.384226+0000","last_became_peered":"2024-09-10T11:36:16.384226+0000","last_unstale":"2024-09-10T11:36:16.386991+0000","last_undegraded":"2024-09-10T11:36:16.386991+0000","last_fullsized":"2024-09-10T11:36:16.386991+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:46.989391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391972+0000","last_change":"2024-09-10T11:36:16.391846+0000","last_active":"2024-09-10T11:36:16.391972+0000","last_peered":"2024-09-10T11:36:16.391972+0000","last_clean":"2024-09-10T11:36:16.391972+0000","last_became_active":"2024-09-10T11:36:16.391608+0000","last_became_peered":"2024-09-10T11:36:16.391608+0000","last_unstale":"2024-09-10T11:36:16.391972+0000","last_undegraded":"2024-09-10T11:36:16.391972+0000","last_fullsized":"2024-09-10T11:36:16.391972+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:02.988816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389084+0000","last_change":"2024-09-10T11:36:16.388697+0000","last_active":"2024-09-10T11:36:16.389084+0000","last_peered":"2024-09-10T11:36:16.389084+0000","last_clean":"2024-09-10T11:36:16.389084+0000","last_became_active":"2024-09-10T11:36:16.388399+0000","last_became_peered":"2024-09-10T11:36:16.388399+0000","last_unstale":"2024-09-10T11:36:16.389084+0000","last_undegraded":"2024-09-10T11:36:16.389084+0000","last_fullsized":"2024-09-10T11:36:16.389084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:18.496973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392750+0000","last_change":"2024-09-10T11:36:16.392643+0000","last_active":"2024-09-10T11:36:16.392750+0000","last_peered":"2024-09-10T11:36:16.392750+0000","last_clean":"2024-09-10T11:36:16.392750+0000","last_became_active":"2024-09-10T11:36:16.392352+0000","last_became_peered":"2024-09-10T11:36:16.392352+0000","last_unstale":"2024-09-10T11:36:16.392750+0000","last_undegraded":"2024-09-10T11:36:16.392750+0000","last_fullsized":"2024-09-10T11:36:16.392750+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:05.858064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390078+0000","last_change":"2024-09-10T11:36:16.389969+0000","last_active":"2024-09-10T11:36:16.390078+0000","last_peered":"2024-09-10T11:36:16.390078+0000","last_clean":"2024-09-10T11:36:16.390078+0000","last_became_active":"2024-09-10T11:36:16.389435+0000","last_became_peered":"2024-09-10T11:36:16.389435+0000","last_unstale":"2024-09-10T11:36:16.390078+0000","last_undegraded":"2024-09-10T11:36:16.390078+0000","last_fullsized":"2024-09-10T11:36:16.390078+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:42.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.383911+0000","last_change":"2024-09-10T11:36:16.383717+0000","last_active":"2024-09-10T11:36:16.383911+0000","last_peered":"2024-09-10T11:36:16.383911+0000","last_clean":"2024-09-10T11:36:16.383911+0000","last_became_active":"2024-09-10T11:36:16.382912+0000","last_became_peered":"2024-09-10T11:36:16.382912+0000","last_unstale":"2024-09-10T11:36:16.383911+0000","last_undegraded":"2024-09-10T11:36:16.383911+0000","last_fullsized":"2024-09-10T11:36:16.383911+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:51.369878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387904+0000","last_change":"2024-09-10T11:36:16.387801+0000","last_active":"2024-09-10T11:36:16.387904+0000","last_peered":"2024-09-10T11:36:16.387904+0000","last_clean":"2024-09-10T11:36:16.387904+0000","last_became_active":"2024-09-10T11:36:16.386938+0000","last_became_peered":"2024-09-10T11:36:16.386938+0000","last_unstale":"2024-09-10T11:36:16.387904+0000","last_undegraded":"2024-09-10T11:36:16.387904+0000","last_fullsized":"2024-09-10T11:36:16.387904+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:46.038152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384249+0000","last_change":"2024-09-10T11:36:16.384140+0000","last_active":"2024-09-10T11:36:16.384249+0000","last_peered":"2024-09-10T11:36:16.384249+0000","last_clean":"2024-09-10T11:36:16.384249+0000","last_became_active":"2024-09-10T11:36:16.383678+0000","last_became_peered":"2024-09-10T11:36:16.383678+0000","last_unstale":"2024-09-10T11:36:16.384249+0000","last_undegraded":"2024-09-10T11:36:16.384249+0000","last_fullsized":"2024-09-10T11:36:16.384249+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:46.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387012+0000","last_change":"2024-09-10T11:36:16.386901+0000","last_active":"2024-09-10T11:36:16.387012+0000","last_peered":"2024-09-10T11:36:16.387012+0000","last_clean":"2024-09-10T11:36:16.387012+0000","last_became_active":"2024-09-10T11:36:16.386367+0000","last_became_peered":"2024-09-10T11:36:16.386367+0000","last_unstale":"2024-09-10T11:36:16.387012+0000","last_undegraded":"2024-09-10T11:36:16.387012+0000","last_fullsized":"2024-09-10T11:36:16.387012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:39.874815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394111+0000","last_change":"2024-09-10T11:36:16.393994+0000","last_active":"2024-09-10T11:36:16.394111+0000","last_peered":"2024-09-10T11:36:16.394111+0000","last_clean":"2024-09-10T11:36:16.394111+0000","last_became_active":"2024-09-10T11:36:16.393749+0000","last_became_peered":"2024-09-10T11:36:16.393749+0000","last_unstale":"2024-09-10T11:36:16.394111+0000","last_undegraded":"2024-09-10T11:36:16.394111+0000","last_fullsized":"2024-09-10T11:36:16.394111+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:21.400597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390235+0000","last_change":"2024-09-10T11:36:16.390112+0000","last_active":"2024-09-10T11:36:16.390235+0000","last_peered":"2024-09-10T11:36:16.390235+0000","last_clean":"2024-09-10T11:36:16.390235+0000","last_became_active":"2024-09-10T11:36:16.389861+0000","last_became_peered":"2024-09-10T11:36:16.389861+0000","last_unstale":"2024-09-10T11:36:16.390235+0000","last_undegraded":"2024-09-10T11:36:16.390235+0000","last_fullsized":"2024-09-10T11:36:16.390235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:26.980952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380962+0000","last_change":"2024-09-10T11:36:16.380414+0000","last_active":"2024-09-10T11:36:16.380962+0000","last_peered":"2024-09-10T11:36:16.380962+0000","last_clean":"2024-09-10T11:36:16.380962+0000","last_became_active":"2024-09-10T11:36:16.379732+0000","last_became_peered":"2024-09-10T11:36:16.379732+0000","last_unstale":"2024-09-10T11:36:16.380962+0000","last_undegraded":"2024-09-10T11:36:16.380962+0000","last_fullsized":"2024-09-10T11:36:16.380962+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:01.157598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386385+0000","last_change":"2024-09-10T11:36:16.386258+0000","last_active":"2024-09-10T11:36:16.386385+0000","last_peered":"2024-09-10T11:36:16.386385+0000","last_clean":"2024-09-10T11:36:16.386385+0000","last_became_active":"2024-09-10T11:36:16.386031+0000","last_became_peered":"2024-09-10T11:36:16.386031+0000","last_unstale":"2024-09-10T11:36:16.386385+0000","last_undegraded":"2024-09-10T11:36:16.386385+0000","last_fullsized":"2024-09-10T11:36:16.386385+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:11.667942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.377188+0000","last_change":"2024-09-10T11:36:16.377081+0000","last_active":"2024-09-10T11:36:16.377188+0000","last_peered":"2024-09-10T11:36:16.377188+0000","last_clean":"2024-09-10T11:36:16.377188+0000","last_became_active":"2024-09-10T11:36:16.376816+0000","last_became_peered":"2024-09-10T11:36:16.376816+0000","last_unstale":"2024-09-10T11:36:16.377188+0000","last_undegraded":"2024-09-10T11:36:16.377188+0000","last_fullsized":"2024-09-10T11:36:16.377188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:38.807811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379523+0000","last_change":"2024-09-10T11:36:16.379262+0000","last_active":"2024-09-10T11:36:16.379523+0000","last_peered":"2024-09-10T11:36:16.379523+0000","last_clean":"2024-09-10T11:36:16.379523+0000","last_became_active":"2024-09-10T11:36:16.378978+0000","last_became_peered":"2024-09-10T11:36:16.378978+0000","last_unstale":"2024-09-10T11:36:16.379523+0000","last_undegraded":"2024-09-10T11:36:16.379523+0000","last_fullsized":"2024-09-10T11:36:16.379523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:28.817890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388169+0000","last_change":"2024-09-10T11:36:16.388010+0000","last_active":"2024-09-10T11:36:16.388169+0000","last_peered":"2024-09-10T11:36:16.388169+0000","last_clean":"2024-09-10T11:36:16.388169+0000","last_became_active":"2024-09-10T11:36:16.386747+0000","last_became_peered":"2024-09-10T11:36:16.386747+0000","last_unstale":"2024-09-10T11:36:16.388169+0000","last_undegraded":"2024-09-10T11:36:16.388169+0000","last_fullsized":"2024-09-10T11:36:16.388169+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:58.429058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382060+0000","last_change":"2024-09-10T11:36:16.379831+0000","last_active":"2024-09-10T11:36:16.382060+0000","last_peered":"2024-09-10T11:36:16.382060+0000","last_clean":"2024-09-10T11:36:16.382060+0000","last_became_active":"2024-09-10T11:36:16.379593+0000","last_became_peered":"2024-09-10T11:36:16.379593+0000","last_unstale":"2024-09-10T11:36:16.382060+0000","last_undegraded":"2024-09-10T11:36:16.382060+0000","last_fullsized":"2024-09-10T11:36:16.382060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:03.587480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392416+0000","last_change":"2024-09-10T11:36:16.392039+0000","last_active":"2024-09-10T11:36:16.392416+0000","last_peered":"2024-09-10T11:36:16.392416+0000","last_clean":"2024-09-10T11:36:16.392416+0000","last_became_active":"2024-09-10T11:36:16.389718+0000","last_became_peered":"2024-09-10T11:36:16.389718+0000","last_unstale":"2024-09-10T11:36:16.392416+0000","last_undegraded":"2024-09-10T11:36:16.392416+0000","last_fullsized":"2024-09-10T11:36:16.392416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:49.239185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387442+0000","last_change":"2024-09-10T11:36:16.387121+0000","last_active":"2024-09-10T11:36:16.387442+0000","last_peered":"2024-09-10T11:36:16.387442+0000","last_clean":"2024-09-10T11:36:16.387442+0000","last_became_active":"2024-09-10T11:36:16.386894+0000","last_became_peered":"2024-09-10T11:36:16.386894+0000","last_unstale":"2024-09-10T11:36:16.387442+0000","last_undegraded":"2024-09-10T11:36:16.387442+0000","last_fullsized":"2024-09-10T11:36:16.387442+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:09.287051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"152'4","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389049+0000","last_change":"2024-09-10T11:36:15.383439+0000","last_active":"2024-09-10T11:36:16.389049+0000","last_peered":"2024-09-10T11:36:16.389049+0000","last_clean":"2024-09-10T11:36:16.389049+0000","last_became_active":"2024-09-10T11:36:15.383201+0000","last_became_peered":"2024-09-10T11:36:15.383201+0000","last_unstale":"2024-09-10T11:36:16.389049+0000","last_undegraded":"2024-09-10T11:36:16.389049+0000","last_fullsized":"2024-09-10T11:36:16.389049+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:36.520495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.393035+0000","last_change":"2024-09-10T11:36:16.392931+0000","last_active":"2024-09-10T11:36:16.393035+0000","last_peered":"2024-09-10T11:36:16.393035+0000","last_clean":"2024-09-10T11:36:16.393035+0000","last_became_active":"2024-09-10T11:36:16.391332+0000","last_became_peered":"2024-09-10T11:36:16.391332+0000","last_unstale":"2024-09-10T11:36:16.393035+0000","last_undegraded":"2024-09-10T11:36:16.393035+0000","last_fullsized":"2024-09-10T11:36:16.393035+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:34.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.397313+0000","last_change":"2024-09-10T11:36:15.382380+0000","last_active":"2024-09-10T11:36:16.397313+0000","last_peered":"2024-09-10T11:36:16.397313+0000","last_clean":"2024-09-10T11:36:16.397313+0000","last_became_active":"2024-09-10T11:36:15.382159+0000","last_became_peered":"2024-09-10T11:36:15.382159+0000","last_unstale":"2024-09-10T11:36:16.397313+0000","last_undegraded":"2024-09-10T11:36:16.397313+0000","last_fullsized":"2024-09-10T11:36:16.397313+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:53.254381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395046+0000","last_change":"2024-09-10T11:36:16.394618+0000","last_active":"2024-09-10T11:36:16.395046+0000","last_peered":"2024-09-10T11:36:16.395046+0000","last_clean":"2024-09-10T11:36:16.395046+0000","last_became_active":"2024-09-10T11:36:16.394387+0000","last_became_peered":"2024-09-10T11:36:16.394387+0000","last_unstale":"2024-09-10T11:36:16.395046+0000","last_undegraded":"2024-09-10T11:36:16.395046+0000","last_fullsized":"2024-09-10T11:36:16.395046+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:35.482330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395324+0000","last_change":"2024-09-10T11:36:15.380559+0000","last_active":"2024-09-10T11:36:16.395324+0000","last_peered":"2024-09-10T11:36:16.395324+0000","last_clean":"2024-09-10T11:36:16.395324+0000","last_became_active":"2024-09-10T11:36:15.380067+0000","last_became_peered":"2024-09-10T11:36:15.380067+0000","last_unstale":"2024-09-10T11:36:16.395324+0000","last_undegraded":"2024-09-10T11:36:16.395324+0000","last_fullsized":"2024-09-10T11:36:16.395324+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.468483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387273+0000","last_change":"2024-09-10T11:36:16.382761+0000","last_active":"2024-09-10T11:36:16.387273+0000","last_peered":"2024-09-10T11:36:16.387273+0000","last_clean":"2024-09-10T11:36:16.387273+0000","last_became_active":"2024-09-10T11:36:16.381673+0000","last_became_peered":"2024-09-10T11:36:16.381673+0000","last_unstale":"2024-09-10T11:36:16.387273+0000","last_undegraded":"2024-09-10T11:36:16.387273+0000","last_fullsized":"2024-09-10T11:36:16.387273+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:15.822438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389923+0000","last_change":"2024-09-10T11:36:15.379624+0000","last_active":"2024-09-10T11:36:16.389923+0000","last_peered":"2024-09-10T11:36:16.389923+0000","last_clean":"2024-09-10T11:36:16.389923+0000","last_became_active":"2024-09-10T11:36:15.379160+0000","last_became_peered":"2024-09-10T11:36:15.379160+0000","last_unstale":"2024-09-10T11:36:16.389923+0000","last_undegraded":"2024-09-10T11:36:16.389923+0000","last_fullsized":"2024-09-10T11:36:16.389923+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:19.953466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385214+0000","last_change":"2024-09-10T11:36:16.384892+0000","last_active":"2024-09-10T11:36:16.385214+0000","last_peered":"2024-09-10T11:36:16.385214+0000","last_clean":"2024-09-10T11:36:16.385214+0000","last_became_active":"2024-09-10T11:36:16.384641+0000","last_became_peered":"2024-09-10T11:36:16.384641+0000","last_unstale":"2024-09-10T11:36:16.385214+0000","last_undegraded":"2024-09-10T11:36:16.385214+0000","last_fullsized":"2024-09-10T11:36:16.385214+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:09.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390362+0000","last_change":"2024-09-10T11:36:15.377277+0000","last_active":"2024-09-10T11:36:16.390362+0000","last_peered":"2024-09-10T11:36:16.390362+0000","last_clean":"2024-09-10T11:36:16.390362+0000","last_became_active":"2024-09-10T11:36:15.375763+0000","last_became_peered":"2024-09-10T11:36:15.375763+0000","last_unstale":"2024-09-10T11:36:16.390362+0000","last_undegraded":"2024-09-10T11:36:16.390362+0000","last_fullsized":"2024-09-10T11:36:16.390362+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:53.672885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391591+0000","last_change":"2024-09-10T11:36:16.391487+0000","last_active":"2024-09-10T11:36:16.391591+0000","last_peered":"2024-09-10T11:36:16.391591+0000","last_clean":"2024-09-10T11:36:16.391591+0000","last_became_active":"2024-09-10T11:36:16.390897+0000","last_became_peered":"2024-09-10T11:36:16.390897+0000","last_unstale":"2024-09-10T11:36:16.391591+0000","last_undegraded":"2024-09-10T11:36:16.391591+0000","last_fullsized":"2024-09-10T11:36:16.391591+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:03.967204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389627+0000","last_change":"2024-09-10T11:36:15.379738+0000","last_active":"2024-09-10T11:36:16.389627+0000","last_peered":"2024-09-10T11:36:16.389627+0000","last_clean":"2024-09-10T11:36:16.389627+0000","last_became_active":"2024-09-10T11:36:15.379474+0000","last_became_peered":"2024-09-10T11:36:15.379474+0000","last_unstale":"2024-09-10T11:36:16.389627+0000","last_undegraded":"2024-09-10T11:36:16.389627+0000","last_fullsized":"2024-09-10T11:36:16.389627+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:33.251117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387337+0000","last_change":"2024-09-10T11:36:16.383504+0000","last_active":"2024-09-10T11:36:16.387337+0000","last_peered":"2024-09-10T11:36:16.387337+0000","last_clean":"2024-09-10T11:36:16.387337+0000","last_became_active":"2024-09-10T11:36:16.383280+0000","last_became_peered":"2024-09-10T11:36:16.383280+0000","last_unstale":"2024-09-10T11:36:16.387337+0000","last_undegraded":"2024-09-10T11:36:16.387337+0000","last_fullsized":"2024-09-10T11:36:16.387337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:59.102987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384309+0000","last_change":"2024-09-10T11:36:15.376945+0000","last_active":"2024-09-10T11:36:16.384309+0000","last_peered":"2024-09-10T11:36:16.384309+0000","last_clean":"2024-09-10T11:36:16.384309+0000","last_became_active":"2024-09-10T11:36:15.376707+0000","last_became_peered":"2024-09-10T11:36:15.376707+0000","last_unstale":"2024-09-10T11:36:16.384309+0000","last_undegraded":"2024-09-10T11:36:16.384309+0000","last_fullsized":"2024-09-10T11:36:16.384309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:32.564882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384112+0000","last_change":"2024-09-10T11:36:16.383783+0000","last_active":"2024-09-10T11:36:16.384112+0000","last_peered":"2024-09-10T11:36:16.384112+0000","last_clean":"2024-09-10T11:36:16.384112+0000","last_became_active":"2024-09-10T11:36:16.383518+0000","last_became_peered":"2024-09-10T11:36:16.383518+0000","last_unstale":"2024-09-10T11:36:16.384112+0000","last_undegraded":"2024-09-10T11:36:16.384112+0000","last_fullsized":"2024-09-10T11:36:16.384112+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:01.893256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376015+0000","last_change":"2024-09-10T11:36:15.377956+0000","last_active":"2024-09-10T11:36:16.376015+0000","last_peered":"2024-09-10T11:36:16.376015+0000","last_clean":"2024-09-10T11:36:16.376015+0000","last_became_active":"2024-09-10T11:36:15.377656+0000","last_became_peered":"2024-09-10T11:36:15.377656+0000","last_unstale":"2024-09-10T11:36:16.376015+0000","last_undegraded":"2024-09-10T11:36:16.376015+0000","last_fullsized":"2024-09-10T11:36:16.376015+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:21.054760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384045+0000","last_change":"2024-09-10T11:36:16.383929+0000","last_active":"2024-09-10T11:36:16.384045+0000","last_peered":"2024-09-10T11:36:16.384045+0000","last_clean":"2024-09-10T11:36:16.384045+0000","last_became_active":"2024-09-10T11:36:16.383315+0000","last_became_peered":"2024-09-10T11:36:16.383315+0000","last_unstale":"2024-09-10T11:36:16.384045+0000","last_undegraded":"2024-09-10T11:36:16.384045+0000","last_fullsized":"2024-09-10T11:36:16.384045+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:55.611733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380919+0000","last_change":"2024-09-10T11:36:15.382121+0000","last_active":"2024-09-10T11:36:16.380919+0000","last_peered":"2024-09-10T11:36:16.380919+0000","last_clean":"2024-09-10T11:36:16.380919+0000","last_became_active":"2024-09-10T11:36:15.381755+0000","last_became_peered":"2024-09-10T11:36:15.381755+0000","last_unstale":"2024-09-10T11:36:16.380919+0000","last_undegraded":"2024-09-10T11:36:16.380919+0000","last_fullsized":"2024-09-10T11:36:16.380919+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:34.486042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389112+0000","last_change":"2024-09-10T11:36:16.388136+0000","last_active":"2024-09-10T11:36:16.389112+0000","last_peered":"2024-09-10T11:36:16.389112+0000","last_clean":"2024-09-10T11:36:16.389112+0000","last_became_active":"2024-09-10T11:36:16.387902+0000","last_became_peered":"2024-09-10T11:36:16.387902+0000","last_unstale":"2024-09-10T11:36:16.389112+0000","last_undegraded":"2024-09-10T11:36:16.389112+0000","last_fullsized":"2024-09-10T11:36:16.389112+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:03.664994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.377227+0000","last_change":"2024-09-10T11:36:15.379673+0000","last_active":"2024-09-10T11:36:16.377227+0000","last_peered":"2024-09-10T11:36:16.377227+0000","last_clean":"2024-09-10T11:36:16.377227+0000","last_became_active":"2024-09-10T11:36:15.378888+0000","last_became_peered":"2024-09-10T11:36:15.378888+0000","last_unstale":"2024-09-10T11:36:16.377227+0000","last_undegraded":"2024-09-10T11:36:16.377227+0000","last_fullsized":"2024-09-10T11:36:16.377227+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:11.589068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396586+0000","last_change":"2024-09-10T11:36:16.394841+0000","last_active":"2024-09-10T11:36:16.396586+0000","last_peered":"2024-09-10T11:36:16.396586+0000","last_clean":"2024-09-10T11:36:16.396586+0000","last_became_active":"2024-09-10T11:36:16.394391+0000","last_became_peered":"2024-09-10T11:36:16.394391+0000","last_unstale":"2024-09-10T11:36:16.396586+0000","last_undegraded":"2024-09-10T11:36:16.396586+0000","last_fullsized":"2024-09-10T11:36:16.396586+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.736908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"152'3","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388995+0000","last_change":"2024-09-10T11:36:15.381114+0000","last_active":"2024-09-10T11:36:16.388995+0000","last_peered":"2024-09-10T11:36:16.388995+0000","last_clean":"2024-09-10T11:36:16.388995+0000","last_became_active":"2024-09-10T11:36:15.380840+0000","last_became_peered":"2024-09-10T11:36:15.380840+0000","last_unstale":"2024-09-10T11:36:16.388995+0000","last_undegraded":"2024-09-10T11:36:16.388995+0000","last_fullsized":"2024-09-10T11:36:16.388995+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:28.335370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387313+0000","last_change":"2024-09-10T11:36:16.387093+0000","last_active":"2024-09-10T11:36:16.387313+0000","last_peered":"2024-09-10T11:36:16.387313+0000","last_clean":"2024-09-10T11:36:16.387313+0000","last_became_active":"2024-09-10T11:36:16.386646+0000","last_became_peered":"2024-09-10T11:36:16.386646+0000","last_unstale":"2024-09-10T11:36:16.387313+0000","last_undegraded":"2024-09-10T11:36:16.387313+0000","last_fullsized":"2024-09-10T11:36:16.387313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:35.682697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"152'6","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.398213+0000","last_change":"2024-09-10T11:36:15.379134+0000","last_active":"2024-09-10T11:36:16.398213+0000","last_peered":"2024-09-10T11:36:16.398213+0000","last_clean":"2024-09-10T11:36:16.398213+0000","last_became_active":"2024-09-10T11:36:15.378904+0000","last_became_peered":"2024-09-10T11:36:15.378904+0000","last_unstale":"2024-09-10T11:36:16.398213+0000","last_undegraded":"2024-09-10T11:36:16.398213+0000","last_fullsized":"2024-09-10T11:36:16.398213+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:37.970209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382404+0000","last_change":"2024-09-10T11:36:16.382254+0000","last_active":"2024-09-10T11:36:16.382404+0000","last_peered":"2024-09-10T11:36:16.382404+0000","last_clean":"2024-09-10T11:36:16.382404+0000","last_became_active":"2024-09-10T11:36:16.382007+0000","last_became_peered":"2024-09-10T11:36:16.382007+0000","last_unstale":"2024-09-10T11:36:16.382404+0000","last_undegraded":"2024-09-10T11:36:16.382404+0000","last_fullsized":"2024-09-10T11:36:16.382404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:13.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376150+0000","last_change":"2024-09-10T11:36:15.383585+0000","last_active":"2024-09-10T11:36:16.376150+0000","last_peered":"2024-09-10T11:36:16.376150+0000","last_clean":"2024-09-10T11:36:16.376150+0000","last_became_active":"2024-09-10T11:36:15.383286+0000","last_became_peered":"2024-09-10T11:36:15.383286+0000","last_unstale":"2024-09-10T11:36:16.376150+0000","last_undegraded":"2024-09-10T11:36:16.376150+0000","last_fullsized":"2024-09-10T11:36:16.376150+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:40.237745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392304+0000","last_change":"2024-09-10T11:36:16.392182+0000","last_active":"2024-09-10T11:36:16.392304+0000","last_peered":"2024-09-10T11:36:16.392304+0000","last_clean":"2024-09-10T11:36:16.392304+0000","last_became_active":"2024-09-10T11:36:16.391925+0000","last_became_peered":"2024-09-10T11:36:16.391925+0000","last_unstale":"2024-09-10T11:36:16.392304+0000","last_undegraded":"2024-09-10T11:36:16.392304+0000","last_fullsized":"2024-09-10T11:36:16.392304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:43.289706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381229+0000","last_change":"2024-09-10T11:36:15.381521+0000","last_active":"2024-09-10T11:36:16.381229+0000","last_peered":"2024-09-10T11:36:16.381229+0000","last_clean":"2024-09-10T11:36:16.381229+0000","last_became_active":"2024-09-10T11:36:15.381218+0000","last_became_peered":"2024-09-10T11:36:15.381218+0000","last_unstale":"2024-09-10T11:36:16.381229+0000","last_undegraded":"2024-09-10T11:36:16.381229+0000","last_fullsized":"2024-09-10T11:36:16.381229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:11.634535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387313+0000","last_change":"2024-09-10T11:36:16.383031+0000","last_active":"2024-09-10T11:36:16.387313+0000","last_peered":"2024-09-10T11:36:16.387313+0000","last_clean":"2024-09-10T11:36:16.387313+0000","last_became_active":"2024-09-10T11:36:16.382809+0000","last_became_peered":"2024-09-10T11:36:16.382809+0000","last_unstale":"2024-09-10T11:36:16.387313+0000","last_undegraded":"2024-09-10T11:36:16.387313+0000","last_fullsized":"2024-09-10T11:36:16.387313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:23:56.047494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"152'8","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394435+0000","last_change":"2024-09-10T11:36:15.383605+0000","last_active":"2024-09-10T11:36:16.394435+0000","last_peered":"2024-09-10T11:36:16.394435+0000","last_clean":"2024-09-10T11:36:16.394435+0000","last_became_active":"2024-09-10T11:36:15.383024+0000","last_became_peered":"2024-09-10T11:36:15.383024+0000","last_unstale":"2024-09-10T11:36:16.394435+0000","last_undegraded":"2024-09-10T11:36:16.394435+0000","last_fullsized":"2024-09-10T11:36:16.394435+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:01.603297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387051+0000","last_change":"2024-09-10T11:36:16.386191+0000","last_active":"2024-09-10T11:36:16.387051+0000","last_peered":"2024-09-10T11:36:16.387051+0000","last_clean":"2024-09-10T11:36:16.387051+0000","last_became_active":"2024-09-10T11:36:16.385962+0000","last_became_peered":"2024-09-10T11:36:16.385962+0000","last_unstale":"2024-09-10T11:36:16.387051+0000","last_undegraded":"2024-09-10T11:36:16.387051+0000","last_fullsized":"2024-09-10T11:36:16.387051+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:45.990807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396651+0000","last_change":"2024-09-10T11:36:15.382077+0000","last_active":"2024-09-10T11:36:16.396651+0000","last_peered":"2024-09-10T11:36:16.396651+0000","last_clean":"2024-09-10T11:36:16.396651+0000","last_became_active":"2024-09-10T11:36:15.381720+0000","last_became_peered":"2024-09-10T11:36:15.381720+0000","last_unstale":"2024-09-10T11:36:16.396651+0000","last_undegraded":"2024-09-10T11:36:16.396651+0000","last_fullsized":"2024-09-10T11:36:16.396651+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:38.022335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387033+0000","last_change":"2024-09-10T11:36:16.385653+0000","last_active":"2024-09-10T11:36:16.387033+0000","last_peered":"2024-09-10T11:36:16.387033+0000","last_clean":"2024-09-10T11:36:16.387033+0000","last_became_active":"2024-09-10T11:36:16.385344+0000","last_became_peered":"2024-09-10T11:36:16.385344+0000","last_unstale":"2024-09-10T11:36:16.387033+0000","last_undegraded":"2024-09-10T11:36:16.387033+0000","last_fullsized":"2024-09-10T11:36:16.387033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:40.382030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396058+0000","last_change":"2024-09-10T11:36:15.380319+0000","last_active":"2024-09-10T11:36:16.396058+0000","last_peered":"2024-09-10T11:36:16.396058+0000","last_clean":"2024-09-10T11:36:16.396058+0000","last_became_active":"2024-09-10T11:36:15.379678+0000","last_became_peered":"2024-09-10T11:36:15.379678+0000","last_unstale":"2024-09-10T11:36:16.396058+0000","last_undegraded":"2024-09-10T11:36:16.396058+0000","last_fullsized":"2024-09-10T11:36:16.396058+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:51.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390090+0000","last_change":"2024-09-10T11:36:16.390012+0000","last_active":"2024-09-10T11:36:16.390090+0000","last_peered":"2024-09-10T11:36:16.390090+0000","last_clean":"2024-09-10T11:36:16.390090+0000","last_became_active":"2024-09-10T11:36:16.389234+0000","last_became_peered":"2024-09-10T11:36:16.389234+0000","last_unstale":"2024-09-10T11:36:16.390090+0000","last_undegraded":"2024-09-10T11:36:16.390090+0000","last_fullsized":"2024-09-10T11:36:16.390090+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:06.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395791+0000","last_change":"2024-09-10T11:36:15.379482+0000","last_active":"2024-09-10T11:36:16.395791+0000","last_peered":"2024-09-10T11:36:16.395791+0000","last_clean":"2024-09-10T11:36:16.395791+0000","last_became_active":"2024-09-10T11:36:15.378991+0000","last_became_peered":"2024-09-10T11:36:15.378991+0000","last_unstale":"2024-09-10T11:36:16.395791+0000","last_undegraded":"2024-09-10T11:36:16.395791+0000","last_fullsized":"2024-09-10T11:36:16.395791+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:34.742364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386981+0000","last_change":"2024-09-10T11:36:16.385779+0000","last_active":"2024-09-10T11:36:16.386981+0000","last_peered":"2024-09-10T11:36:16.386981+0000","last_clean":"2024-09-10T11:36:16.386981+0000","last_became_active":"2024-09-10T11:36:16.385545+0000","last_became_peered":"2024-09-10T11:36:16.385545+0000","last_unstale":"2024-09-10T11:36:16.386981+0000","last_undegraded":"2024-09-10T11:36:16.386981+0000","last_fullsized":"2024-09-10T11:36:16.386981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:49.930668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384834+0000","last_change":"2024-09-10T11:36:15.381530+0000","last_active":"2024-09-10T11:36:16.384834+0000","last_peered":"2024-09-10T11:36:16.384834+0000","last_clean":"2024-09-10T11:36:16.384834+0000","last_became_active":"2024-09-10T11:36:15.381294+0000","last_became_peered":"2024-09-10T11:36:15.381294+0000","last_unstale":"2024-09-10T11:36:16.384834+0000","last_undegraded":"2024-09-10T11:36:16.384834+0000","last_fullsized":"2024-09-10T11:36:16.384834+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:56.652537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389818+0000","last_change":"2024-09-10T11:36:16.389688+0000","last_active":"2024-09-10T11:36:16.389818+0000","last_peered":"2024-09-10T11:36:16.389818+0000","last_clean":"2024-09-10T11:36:16.389818+0000","last_became_active":"2024-09-10T11:36:16.389079+0000","last_became_peered":"2024-09-10T11:36:16.389079+0000","last_unstale":"2024-09-10T11:36:16.389818+0000","last_undegraded":"2024-09-10T11:36:16.389818+0000","last_fullsized":"2024-09-10T11:36:16.389818+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:48.448566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.397234+0000","last_change":"2024-09-10T11:36:15.379605+0000","last_active":"2024-09-10T11:36:16.397234+0000","last_peered":"2024-09-10T11:36:16.397234+0000","last_clean":"2024-09-10T11:36:16.397234+0000","last_became_active":"2024-09-10T11:36:15.379229+0000","last_became_peered":"2024-09-10T11:36:15.379229+0000","last_unstale":"2024-09-10T11:36:16.397234+0000","last_undegraded":"2024-09-10T11:36:16.397234+0000","last_fullsized":"2024-09-10T11:36:16.397234+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:36.089134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394641+0000","last_change":"2024-09-10T11:36:16.394466+0000","last_active":"2024-09-10T11:36:16.394641+0000","last_peered":"2024-09-10T11:36:16.394641+0000","last_clean":"2024-09-10T11:36:16.394641+0000","last_became_active":"2024-09-10T11:36:16.394203+0000","last_became_peered":"2024-09-10T11:36:16.394203+0000","last_unstale":"2024-09-10T11:36:16.394641+0000","last_undegraded":"2024-09-10T11:36:16.394641+0000","last_fullsized":"2024-09-10T11:36:16.394641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:51.973289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379405+0000","last_change":"2024-09-10T11:36:15.382118+0000","last_active":"2024-09-10T11:36:16.379405+0000","last_peered":"2024-09-10T11:36:16.379405+0000","last_clean":"2024-09-10T11:36:16.379405+0000","last_became_active":"2024-09-10T11:36:15.381406+0000","last_became_peered":"2024-09-10T11:36:15.381406+0000","last_unstale":"2024-09-10T11:36:16.379405+0000","last_undegraded":"2024-09-10T11:36:16.379405+0000","last_fullsized":"2024-09-10T11:36:16.379405+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:13.674525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382466+0000","last_change":"2024-09-10T11:36:16.382352+0000","last_active":"2024-09-10T11:36:16.382466+0000","last_peered":"2024-09-10T11:36:16.382466+0000","last_clean":"2024-09-10T11:36:16.382466+0000","last_became_active":"2024-09-10T11:36:16.381616+0000","last_became_peered":"2024-09-10T11:36:16.381616+0000","last_unstale":"2024-09-10T11:36:16.382466+0000","last_undegraded":"2024-09-10T11:36:16.382466+0000","last_fullsized":"2024-09-10T11:36:16.382466+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:52.079847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376436+0000","last_change":"2024-09-10T11:36:15.382446+0000","last_active":"2024-09-10T11:36:16.376436+0000","last_peered":"2024-09-10T11:36:16.376436+0000","last_clean":"2024-09-10T11:36:16.376436+0000","last_became_active":"2024-09-10T11:36:15.382222+0000","last_became_peered":"2024-09-10T11:36:15.382222+0000","last_unstale":"2024-09-10T11:36:16.376436+0000","last_undegraded":"2024-09-10T11:36:16.376436+0000","last_fullsized":"2024-09-10T11:36:16.376436+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:09.660672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387296+0000","last_change":"2024-09-10T11:36:16.383018+0000","last_active":"2024-09-10T11:36:16.387296+0000","last_peered":"2024-09-10T11:36:16.387296+0000","last_clean":"2024-09-10T11:36:16.387296+0000","last_became_active":"2024-09-10T11:36:16.382065+0000","last_became_peered":"2024-09-10T11:36:16.382065+0000","last_unstale":"2024-09-10T11:36:16.387296+0000","last_undegraded":"2024-09-10T11:36:16.387296+0000","last_fullsized":"2024-09-10T11:36:16.387296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:00.304500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382230+0000","last_change":"2024-09-10T11:36:15.380816+0000","last_active":"2024-09-10T11:36:16.382230+0000","last_peered":"2024-09-10T11:36:16.382230+0000","last_clean":"2024-09-10T11:36:16.382230+0000","last_became_active":"2024-09-10T11:36:15.380480+0000","last_became_peered":"2024-09-10T11:36:15.380480+0000","last_unstale":"2024-09-10T11:36:16.382230+0000","last_undegraded":"2024-09-10T11:36:16.382230+0000","last_fullsized":"2024-09-10T11:36:16.382230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:51.206134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392486+0000","last_change":"2024-09-10T11:36:16.392291+0000","last_active":"2024-09-10T11:36:16.392486+0000","last_peered":"2024-09-10T11:36:16.392486+0000","last_clean":"2024-09-10T11:36:16.392486+0000","last_became_active":"2024-09-10T11:36:16.392032+0000","last_became_peered":"2024-09-10T11:36:16.392032+0000","last_unstale":"2024-09-10T11:36:16.392486+0000","last_undegraded":"2024-09-10T11:36:16.392486+0000","last_fullsized":"2024-09-10T11:36:16.392486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:12.297982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392281+0000","last_change":"2024-09-10T11:36:15.380841+0000","last_active":"2024-09-10T11:36:16.392281+0000","last_peered":"2024-09-10T11:36:16.392281+0000","last_clean":"2024-09-10T11:36:16.392281+0000","last_became_active":"2024-09-10T11:36:15.380583+0000","last_became_peered":"2024-09-10T11:36:15.380583+0000","last_unstale":"2024-09-10T11:36:16.392281+0000","last_undegraded":"2024-09-10T11:36:16.392281+0000","last_fullsized":"2024-09-10T11:36:16.392281+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:00.766804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389948+0000","last_change":"2024-09-10T11:36:16.389788+0000","last_active":"2024-09-10T11:36:16.389948+0000","last_peered":"2024-09-10T11:36:16.389948+0000","last_clean":"2024-09-10T11:36:16.389948+0000","last_became_active":"2024-09-10T11:36:16.388909+0000","last_became_peered":"2024-09-10T11:36:16.388909+0000","last_unstale":"2024-09-10T11:36:16.389948+0000","last_undegraded":"2024-09-10T11:36:16.389948+0000","last_fullsized":"2024-09-10T11:36:16.389948+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:09.622045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382981+0000","last_change":"2024-09-10T11:36:15.382141+0000","last_active":"2024-09-10T11:36:16.382981+0000","last_peered":"2024-09-10T11:36:16.382981+0000","last_clean":"2024-09-10T11:36:16.382981+0000","last_became_active":"2024-09-10T11:36:15.381702+0000","last_became_peered":"2024-09-10T11:36:15.381702+0000","last_unstale":"2024-09-10T11:36:16.382981+0000","last_undegraded":"2024-09-10T11:36:16.382981+0000","last_fullsized":"2024-09-10T11:36:16.382981+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:59.793080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.378262+0000","last_change":"2024-09-10T11:36:16.378145+0000","last_active":"2024-09-10T11:36:16.378262+0000","last_peered":"2024-09-10T11:36:16.378262+0000","last_clean":"2024-09-10T11:36:16.378262+0000","last_became_active":"2024-09-10T11:36:16.377899+0000","last_became_peered":"2024-09-10T11:36:16.377899+0000","last_unstale":"2024-09-10T11:36:16.378262+0000","last_undegraded":"2024-09-10T11:36:16.378262+0000","last_fullsized":"2024-09-10T11:36:16.378262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:00.811347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379556+0000","last_change":"2024-09-10T11:36:15.382395+0000","last_active":"2024-09-10T11:36:16.379556+0000","last_peered":"2024-09-10T11:36:16.379556+0000","last_clean":"2024-09-10T11:36:16.379556+0000","last_became_active":"2024-09-10T11:36:15.381863+0000","last_became_peered":"2024-09-10T11:36:15.381863+0000","last_unstale":"2024-09-10T11:36:16.379556+0000","last_undegraded":"2024-09-10T11:36:16.379556+0000","last_fullsized":"2024-09-10T11:36:16.379556+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:50.290083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388433+0000","last_change":"2024-09-10T11:36:16.387713+0000","last_active":"2024-09-10T11:36:16.388433+0000","last_peered":"2024-09-10T11:36:16.388433+0000","last_clean":"2024-09-10T11:36:16.388433+0000","last_became_active":"2024-09-10T11:36:16.386594+0000","last_became_peered":"2024-09-10T11:36:16.386594+0000","last_unstale":"2024-09-10T11:36:16.388433+0000","last_undegraded":"2024-09-10T11:36:16.388433+0000","last_fullsized":"2024-09-10T11:36:16.388433+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:46.750642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376611+0000","last_change":"2024-09-10T11:36:15.383769+0000","last_active":"2024-09-10T11:36:16.376611+0000","last_peered":"2024-09-10T11:36:16.376611+0000","last_clean":"2024-09-10T11:36:16.376611+0000","last_became_active":"2024-09-10T11:36:15.383513+0000","last_became_peered":"2024-09-10T11:36:15.383513+0000","last_unstale":"2024-09-10T11:36:16.376611+0000","last_undegraded":"2024-09-10T11:36:16.376611+0000","last_fullsized":"2024-09-10T11:36:16.376611+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:42.097339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391374+0000","last_change":"2024-09-10T11:36:16.391247+0000","last_active":"2024-09-10T11:36:16.391374+0000","last_peered":"2024-09-10T11:36:16.391374+0000","last_clean":"2024-09-10T11:36:16.391374+0000","last_became_active":"2024-09-10T11:36:16.390546+0000","last_became_peered":"2024-09-10T11:36:16.390546+0000","last_unstale":"2024-09-10T11:36:16.391374+0000","last_undegraded":"2024-09-10T11:36:16.391374+0000","last_fullsized":"2024-09-10T11:36:16.391374+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:26.944382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376830+0000","last_change":"2024-09-10T11:36:15.376778+0000","last_active":"2024-09-10T11:36:16.376830+0000","last_peered":"2024-09-10T11:36:16.376830+0000","last_clean":"2024-09-10T11:36:16.376830+0000","last_became_active":"2024-09-10T11:36:15.376295+0000","last_became_peered":"2024-09-10T11:36:15.376295+0000","last_unstale":"2024-09-10T11:36:16.376830+0000","last_undegraded":"2024-09-10T11:36:16.376830+0000","last_fullsized":"2024-09-10T11:36:16.376830+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:08.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392338+0000","last_change":"2024-09-10T11:36:16.392072+0000","last_active":"2024-09-10T11:36:16.392338+0000","last_peered":"2024-09-10T11:36:16.392338+0000","last_clean":"2024-09-10T11:36:16.392338+0000","last_became_active":"2024-09-10T11:36:16.391845+0000","last_became_peered":"2024-09-10T11:36:16.391845+0000","last_unstale":"2024-09-10T11:36:16.392338+0000","last_undegraded":"2024-09-10T11:36:16.392338+0000","last_fullsized":"2024-09-10T11:36:16.392338+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:24.677935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381994+0000","last_change":"2024-09-10T11:36:15.382708+0000","last_active":"2024-09-10T11:36:16.381994+0000","last_peered":"2024-09-10T11:36:16.381994+0000","last_clean":"2024-09-10T11:36:16.381994+0000","last_became_active":"2024-09-10T11:36:15.381919+0000","last_became_peered":"2024-09-10T11:36:15.381919+0000","last_unstale":"2024-09-10T11:36:16.381994+0000","last_undegraded":"2024-09-10T11:36:16.381994+0000","last_fullsized":"2024-09-10T11:36:16.381994+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:33.781561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394798+0000","last_change":"2024-09-10T11:36:16.394685+0000","last_active":"2024-09-10T11:36:16.394798+0000","last_peered":"2024-09-10T11:36:16.394798+0000","last_clean":"2024-09-10T11:36:16.394798+0000","last_became_active":"2024-09-10T11:36:16.394465+0000","last_became_peered":"2024-09-10T11:36:16.394465+0000","last_unstale":"2024-09-10T11:36:16.394798+0000","last_undegraded":"2024-09-10T11:36:16.394798+0000","last_fullsized":"2024-09-10T11:36:16.394798+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:41.872627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381266+0000","last_change":"2024-09-10T11:36:15.382257+0000","last_active":"2024-09-10T11:36:16.381266+0000","last_peered":"2024-09-10T11:36:16.381266+0000","last_clean":"2024-09-10T11:36:16.381266+0000","last_became_active":"2024-09-10T11:36:15.381948+0000","last_became_peered":"2024-09-10T11:36:15.381948+0000","last_unstale":"2024-09-10T11:36:16.381266+0000","last_undegraded":"2024-09-10T11:36:16.381266+0000","last_fullsized":"2024-09-10T11:36:16.381266+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:48.598882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.378034+0000","last_change":"2024-09-10T11:36:16.377893+0000","last_active":"2024-09-10T11:36:16.378034+0000","last_peered":"2024-09-10T11:36:16.378034+0000","last_clean":"2024-09-10T11:36:16.378034+0000","last_became_active":"2024-09-10T11:36:16.377508+0000","last_became_peered":"2024-09-10T11:36:16.377508+0000","last_unstale":"2024-09-10T11:36:16.378034+0000","last_undegraded":"2024-09-10T11:36:16.378034+0000","last_fullsized":"2024-09-10T11:36:16.378034+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:55.733317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.375186+0000","last_change":"2024-09-10T11:36:15.384761+0000","last_active":"2024-09-10T11:36:16.375186+0000","last_peered":"2024-09-10T11:36:16.375186+0000","last_clean":"2024-09-10T11:36:16.375186+0000","last_became_active":"2024-09-10T11:36:15.384431+0000","last_became_peered":"2024-09-10T11:36:15.384431+0000","last_unstale":"2024-09-10T11:36:16.375186+0000","last_undegraded":"2024-09-10T11:36:16.375186+0000","last_fullsized":"2024-09-10T11:36:16.375186+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:26.174416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392447+0000","last_change":"2024-09-10T11:36:16.390866+0000","last_active":"2024-09-10T11:36:16.392447+0000","last_peered":"2024-09-10T11:36:16.392447+0000","last_clean":"2024-09-10T11:36:16.392447+0000","last_became_active":"2024-09-10T11:36:16.390636+0000","last_became_peered":"2024-09-10T11:36:16.390636+0000","last_unstale":"2024-09-10T11:36:16.392447+0000","last_undegraded":"2024-09-10T11:36:16.392447+0000","last_fullsized":"2024-09-10T11:36:16.392447+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.440433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391096+0000","last_change":"2024-09-10T11:36:15.380862+0000","last_active":"2024-09-10T11:36:16.391096+0000","last_peered":"2024-09-10T11:36:16.391096+0000","last_clean":"2024-09-10T11:36:16.391096+0000","last_became_active":"2024-09-10T11:36:15.380644+0000","last_became_peered":"2024-09-10T11:36:15.380644+0000","last_unstale":"2024-09-10T11:36:16.391096+0000","last_undegraded":"2024-09-10T11:36:16.391096+0000","last_fullsized":"2024-09-10T11:36:16.391096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:54.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":435,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.374090+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:36:16.374090+0000","last_peered":"2024-09-10T11:36:16.374090+0000","last_clean":"2024-09-10T11:36:16.374090+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:36:16.374090+0000","last_undegraded":"2024-09-10T11:36:16.374090+0000","last_fullsized":"2024-09-10T11:36:16.374090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1395,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193916,"statfs":{"total":200000000,"available":198570911,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":826,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199153832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193908,"statfs":{"total":200000000,"available":198562502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:36:19.325 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:36:19.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.458+0000 7faf43f1f640 1 Processor -- start 2024-09-10T11:36:19.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.458+0000 7faf43f1f640 1 -- start start 2024-09-10T11:36:19.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf43f1f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c1605c0 0x7faf3c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf43f1f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf3c0ae020 0x7faf3c160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf43f1f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf3c0d1bd0 con 0x7faf3c15cfd0 2024-09-10T11:36:19.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf43f1f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7faf3c075620 con 0x7faf3c0ae020 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf43f1f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c0752b0 con 0x7faf3c1605c0 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf414ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faf3c0ae020 0x7faf3c160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf424bc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faf3c15cfd0 0x7faf3c15d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51300/0 (socket says 172.21.15.99:51300) 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf424bc640 1 -- 172.21.15.99:0/3383475829 learned_addr learned my addr 172.21.15.99:0/3383475829 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf41cbb640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c1605c0 0x7faf3c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf414ba640 1 -- 172.21.15.99:0/3383475829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c1605c0 msgr2=0x7faf3c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf414ba640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c1605c0 0x7faf3c1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf414ba640 1 -- 172.21.15.99:0/3383475829 >> v1:172.21.15.99:6789/0 conn(0x7faf3c15cfd0 legacy=0x7faf3c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf414ba640 1 -- 172.21.15.99:0/3383475829 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c0d1eb0 con 0x7faf3c0ae020 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.459+0000 7faf40cb9640 1 -- 172.21.15.99:0/3383475829 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873463285 0 0) 0x7faf3c0d1bd0 con 0x7faf3c15cfd0 2024-09-10T11:36:19.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf414ba640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c0ae020 0x7faf3c160ee0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097a0 tx=0x7faf2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f48fd8164d2edad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf40cb9640 1 -- 172.21.15.99:0/3383475829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c037070 con 0x7faf3c0ae020 2024-09-10T11:36:19.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf40cb9640 1 -- 172.21.15.99:0/3383475829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2c032440 con 0x7faf3c0ae020 2024-09-10T11:36:19.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf40cb9640 1 -- 172.21.15.99:0/3383475829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c032810 con 0x7faf3c0ae020 2024-09-10T11:36:19.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf43f1f640 1 -- 172.21.15.99:0/3383475829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c0ae020 msgr2=0x7faf3c160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.460+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c0ae020 0x7faf3c160ee0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0097a0 tx=0x7faf2c02a180 comp rx=0 tx=0).stop 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 -- 172.21.15.99:0/3383475829 shutdown_connections 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c0ae020 0x7faf3c160ee0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3383475829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c1605c0 0x7faf3c1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 -- 172.21.15.99:0/3383475829 >> 172.21.15.99:0/3383475829 conn(0x7faf3c0730c0 msgr2=0x7faf3c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 -- 172.21.15.99:0/3383475829 shutdown_connections 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.461+0000 7faf43f1f640 1 -- 172.21.15.99:0/3383475829 wait complete. 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 Processor -- start 2024-09-10T11:36:19.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 -- start start 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c0ae020 0x7faf3c0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 0x7faf3c1f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c073d00 con 0x7faf3c15cfd0 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c15a9e0 con 0x7faf3c1605c0 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.462+0000 7faf43f1f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c073e80 con 0x7faf3c0ae020 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34474/0 (socket says 172.21.15.99:34474) 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 -- 172.21.15.99:0/3530111329 learned_addr learned my addr 172.21.15.99:0/3530111329 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf41cbb640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c0ae020 0x7faf3c0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf424bc640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 0x7faf3c1f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 -- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c0ae020 msgr2=0x7faf3c0b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c0ae020 0x7faf3c0b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 -- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 msgr2=0x7faf3c1f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 0x7faf3c1f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 -- 172.21.15.99:0/3530111329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c1551e0 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf424bc640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 0x7faf3c1f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf414ba640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0096f0 tx=0x7faf2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c037040 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2c030770 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c1554c0 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c030b40 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.463+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c155a80 con 0x7faf3c15cfd0 2024-09-10T11:36:19.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.464+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c1609a0 con 0x7faf3c15cfd0 2024-09-10T11:36:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.465+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faf2c044020 con 0x7faf3c15cfd0 2024-09-10T11:36:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.465+0000 7faf327fc640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 0x7faf2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.465+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7faf2c0ab000 con 0x7faf3c15cfd0 2024-09-10T11:36:19.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.465+0000 7faf41cbb640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 0x7faf2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.466+0000 7faf41cbb640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 0x7faf2006e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.470+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf2c04b020 con 0x7faf3c15cfd0 2024-09-10T11:36:19.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.630+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf3c06e2a0 con 0x7faf2006be30 2024-09-10T11:36:19.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.636+0000 7faf327fc640 1 -- 172.21.15.99:0/3530111329 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247921 (secure 0 0 0) 0x7faf3c06e2a0 con 0x7faf2006be30 2024-09-10T11:36:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:19.642 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:36:19.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 msgr2=0x7faf2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 0x7faf2006e2f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7faf38006fd0 tx=0x7faf38008040 comp rx=0 tx=0).stop 2024-09-10T11:36:19.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 msgr2=0x7faf3c154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0096f0 tx=0x7faf2c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 shutdown_connections 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7faf2006be30 0x7faf2006e2f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf3c1605c0 0x7faf3c1f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf3c15cfd0 0x7faf3c154b20 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 --2- 172.21.15.99:0/3530111329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf3c0ae020 0x7faf3c0b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 >> 172.21.15.99:0/3530111329 conn(0x7faf3c0730c0 msgr2=0x7faf3c15e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 shutdown_connections 2024-09-10T11:36:19.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.639+0000 7faf43f1f640 1 -- 172.21.15.99:0/3530111329 wait complete. 2024-09-10T11:36:19.651 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-10T11:36:18.982022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7755,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554744,"statfs":{"total":1600000000,"available":1592057993,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.618707"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392564+0000","last_change":"2024-09-10T11:36:16.392416+0000","last_active":"2024-09-10T11:36:16.392564+0000","last_peered":"2024-09-10T11:36:16.392564+0000","last_clean":"2024-09-10T11:36:16.392564+0000","last_became_active":"2024-09-10T11:36:16.391653+0000","last_became_peered":"2024-09-10T11:36:16.391653+0000","last_unstale":"2024-09-10T11:36:16.392564+0000","last_undegraded":"2024-09-10T11:36:16.392564+0000","last_fullsized":"2024-09-10T11:36:16.392564+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:03.624410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390327+0000","last_change":"2024-09-10T11:36:16.390219+0000","last_active":"2024-09-10T11:36:16.390327+0000","last_peered":"2024-09-10T11:36:16.390327+0000","last_clean":"2024-09-10T11:36:16.390327+0000","last_became_active":"2024-09-10T11:36:16.389546+0000","last_became_peered":"2024-09-10T11:36:16.389546+0000","last_unstale":"2024-09-10T11:36:16.390327+0000","last_undegraded":"2024-09-10T11:36:16.390327+0000","last_fullsized":"2024-09-10T11:36:16.390327+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:29.623487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392814+0000","last_change":"2024-09-10T11:36:16.391679+0000","last_active":"2024-09-10T11:36:16.392814+0000","last_peered":"2024-09-10T11:36:16.392814+0000","last_clean":"2024-09-10T11:36:16.392814+0000","last_became_active":"2024-09-10T11:36:16.390954+0000","last_became_peered":"2024-09-10T11:36:16.390954+0000","last_unstale":"2024-09-10T11:36:16.392814+0000","last_undegraded":"2024-09-10T11:36:16.392814+0000","last_fullsized":"2024-09-10T11:36:16.392814+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:55:30.633037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395292+0000","last_change":"2024-09-10T11:36:16.395185+0000","last_active":"2024-09-10T11:36:16.395292+0000","last_peered":"2024-09-10T11:36:16.395292+0000","last_clean":"2024-09-10T11:36:16.395292+0000","last_became_active":"2024-09-10T11:36:16.395003+0000","last_became_peered":"2024-09-10T11:36:16.395003+0000","last_unstale":"2024-09-10T11:36:16.395292+0000","last_undegraded":"2024-09-10T11:36:16.395292+0000","last_fullsized":"2024-09-10T11:36:16.395292+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:13.833806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.393100+0000","last_change":"2024-09-10T11:36:16.393000+0000","last_active":"2024-09-10T11:36:16.393100+0000","last_peered":"2024-09-10T11:36:16.393100+0000","last_clean":"2024-09-10T11:36:16.393100+0000","last_became_active":"2024-09-10T11:36:16.392774+0000","last_became_peered":"2024-09-10T11:36:16.392774+0000","last_unstale":"2024-09-10T11:36:16.393100+0000","last_undegraded":"2024-09-10T11:36:16.393100+0000","last_fullsized":"2024-09-10T11:36:16.393100+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:30:20.242917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388141+0000","last_change":"2024-09-10T11:36:16.388010+0000","last_active":"2024-09-10T11:36:16.388141+0000","last_peered":"2024-09-10T11:36:16.388141+0000","last_clean":"2024-09-10T11:36:16.388141+0000","last_became_active":"2024-09-10T11:36:16.387557+0000","last_became_peered":"2024-09-10T11:36:16.387557+0000","last_unstale":"2024-09-10T11:36:16.388141+0000","last_undegraded":"2024-09-10T11:36:16.388141+0000","last_fullsized":"2024-09-10T11:36:16.388141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:53.743553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389870+0000","last_change":"2024-09-10T11:36:16.389765+0000","last_active":"2024-09-10T11:36:16.389870+0000","last_peered":"2024-09-10T11:36:16.389870+0000","last_clean":"2024-09-10T11:36:16.389870+0000","last_became_active":"2024-09-10T11:36:16.389549+0000","last_became_peered":"2024-09-10T11:36:16.389549+0000","last_unstale":"2024-09-10T11:36:16.389870+0000","last_undegraded":"2024-09-10T11:36:16.389870+0000","last_fullsized":"2024-09-10T11:36:16.389870+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:34.891883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385141+0000","last_change":"2024-09-10T11:36:16.383526+0000","last_active":"2024-09-10T11:36:16.385141+0000","last_peered":"2024-09-10T11:36:16.385141+0000","last_clean":"2024-09-10T11:36:16.385141+0000","last_became_active":"2024-09-10T11:36:16.383298+0000","last_became_peered":"2024-09-10T11:36:16.383298+0000","last_unstale":"2024-09-10T11:36:16.385141+0000","last_undegraded":"2024-09-10T11:36:16.385141+0000","last_fullsized":"2024-09-10T11:36:16.385141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:44:53.934754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388453+0000","last_change":"2024-09-10T11:36:16.388017+0000","last_active":"2024-09-10T11:36:16.388453+0000","last_peered":"2024-09-10T11:36:16.388453+0000","last_clean":"2024-09-10T11:36:16.388453+0000","last_became_active":"2024-09-10T11:36:16.386907+0000","last_became_peered":"2024-09-10T11:36:16.386907+0000","last_unstale":"2024-09-10T11:36:16.388453+0000","last_undegraded":"2024-09-10T11:36:16.388453+0000","last_fullsized":"2024-09-10T11:36:16.388453+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:23.572052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385040+0000","last_change":"2024-09-10T11:36:16.384553+0000","last_active":"2024-09-10T11:36:16.385040+0000","last_peered":"2024-09-10T11:36:16.385040+0000","last_clean":"2024-09-10T11:36:16.385040+0000","last_became_active":"2024-09-10T11:36:16.384325+0000","last_became_peered":"2024-09-10T11:36:16.384325+0000","last_unstale":"2024-09-10T11:36:16.385040+0000","last_undegraded":"2024-09-10T11:36:16.385040+0000","last_fullsized":"2024-09-10T11:36:16.385040+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:25.867523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387504+0000","last_change":"2024-09-10T11:36:16.386301+0000","last_active":"2024-09-10T11:36:16.387504+0000","last_peered":"2024-09-10T11:36:16.387504+0000","last_clean":"2024-09-10T11:36:16.387504+0000","last_became_active":"2024-09-10T11:36:16.383002+0000","last_became_peered":"2024-09-10T11:36:16.383002+0000","last_unstale":"2024-09-10T11:36:16.387504+0000","last_undegraded":"2024-09-10T11:36:16.387504+0000","last_fullsized":"2024-09-10T11:36:16.387504+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:56:01.537994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389506+0000","last_change":"2024-09-10T11:36:16.389358+0000","last_active":"2024-09-10T11:36:16.389506+0000","last_peered":"2024-09-10T11:36:16.389506+0000","last_clean":"2024-09-10T11:36:16.389506+0000","last_became_active":"2024-09-10T11:36:16.389112+0000","last_became_peered":"2024-09-10T11:36:16.389112+0000","last_unstale":"2024-09-10T11:36:16.389506+0000","last_undegraded":"2024-09-10T11:36:16.389506+0000","last_fullsized":"2024-09-10T11:36:16.389506+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:16.409948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390027+0000","last_change":"2024-09-10T11:36:16.389281+0000","last_active":"2024-09-10T11:36:16.390027+0000","last_peered":"2024-09-10T11:36:16.390027+0000","last_clean":"2024-09-10T11:36:16.390027+0000","last_became_active":"2024-09-10T11:36:16.389050+0000","last_became_peered":"2024-09-10T11:36:16.389050+0000","last_unstale":"2024-09-10T11:36:16.390027+0000","last_undegraded":"2024-09-10T11:36:16.390027+0000","last_fullsized":"2024-09-10T11:36:16.390027+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:06.936329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380787+0000","last_change":"2024-09-10T11:36:16.380607+0000","last_active":"2024-09-10T11:36:16.380787+0000","last_peered":"2024-09-10T11:36:16.380787+0000","last_clean":"2024-09-10T11:36:16.380787+0000","last_became_active":"2024-09-10T11:36:16.380366+0000","last_became_peered":"2024-09-10T11:36:16.380366+0000","last_unstale":"2024-09-10T11:36:16.380787+0000","last_undegraded":"2024-09-10T11:36:16.380787+0000","last_fullsized":"2024-09-10T11:36:16.380787+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:36.972213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386991+0000","last_change":"2024-09-10T11:36:16.384493+0000","last_active":"2024-09-10T11:36:16.386991+0000","last_peered":"2024-09-10T11:36:16.386991+0000","last_clean":"2024-09-10T11:36:16.386991+0000","last_became_active":"2024-09-10T11:36:16.384226+0000","last_became_peered":"2024-09-10T11:36:16.384226+0000","last_unstale":"2024-09-10T11:36:16.386991+0000","last_undegraded":"2024-09-10T11:36:16.386991+0000","last_fullsized":"2024-09-10T11:36:16.386991+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:46.989391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391972+0000","last_change":"2024-09-10T11:36:16.391846+0000","last_active":"2024-09-10T11:36:16.391972+0000","last_peered":"2024-09-10T11:36:16.391972+0000","last_clean":"2024-09-10T11:36:16.391972+0000","last_became_active":"2024-09-10T11:36:16.391608+0000","last_became_peered":"2024-09-10T11:36:16.391608+0000","last_unstale":"2024-09-10T11:36:16.391972+0000","last_undegraded":"2024-09-10T11:36:16.391972+0000","last_fullsized":"2024-09-10T11:36:16.391972+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:02.988816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389084+0000","last_change":"2024-09-10T11:36:16.388697+0000","last_active":"2024-09-10T11:36:16.389084+0000","last_peered":"2024-09-10T11:36:16.389084+0000","last_clean":"2024-09-10T11:36:16.389084+0000","last_became_active":"2024-09-10T11:36:16.388399+0000","last_became_peered":"2024-09-10T11:36:16.388399+0000","last_unstale":"2024-09-10T11:36:16.389084+0000","last_undegraded":"2024-09-10T11:36:16.389084+0000","last_fullsized":"2024-09-10T11:36:16.389084+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:18.496973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392750+0000","last_change":"2024-09-10T11:36:16.392643+0000","last_active":"2024-09-10T11:36:16.392750+0000","last_peered":"2024-09-10T11:36:16.392750+0000","last_clean":"2024-09-10T11:36:16.392750+0000","last_became_active":"2024-09-10T11:36:16.392352+0000","last_became_peered":"2024-09-10T11:36:16.392352+0000","last_unstale":"2024-09-10T11:36:16.392750+0000","last_undegraded":"2024-09-10T11:36:16.392750+0000","last_fullsized":"2024-09-10T11:36:16.392750+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:05.858064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390078+0000","last_change":"2024-09-10T11:36:16.389969+0000","last_active":"2024-09-10T11:36:16.390078+0000","last_peered":"2024-09-10T11:36:16.390078+0000","last_clean":"2024-09-10T11:36:16.390078+0000","last_became_active":"2024-09-10T11:36:16.389435+0000","last_became_peered":"2024-09-10T11:36:16.389435+0000","last_unstale":"2024-09-10T11:36:16.390078+0000","last_undegraded":"2024-09-10T11:36:16.390078+0000","last_fullsized":"2024-09-10T11:36:16.390078+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:42.420756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.383911+0000","last_change":"2024-09-10T11:36:16.383717+0000","last_active":"2024-09-10T11:36:16.383911+0000","last_peered":"2024-09-10T11:36:16.383911+0000","last_clean":"2024-09-10T11:36:16.383911+0000","last_became_active":"2024-09-10T11:36:16.382912+0000","last_became_peered":"2024-09-10T11:36:16.382912+0000","last_unstale":"2024-09-10T11:36:16.383911+0000","last_undegraded":"2024-09-10T11:36:16.383911+0000","last_fullsized":"2024-09-10T11:36:16.383911+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:19:51.369878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387904+0000","last_change":"2024-09-10T11:36:16.387801+0000","last_active":"2024-09-10T11:36:16.387904+0000","last_peered":"2024-09-10T11:36:16.387904+0000","last_clean":"2024-09-10T11:36:16.387904+0000","last_became_active":"2024-09-10T11:36:16.386938+0000","last_became_peered":"2024-09-10T11:36:16.386938+0000","last_unstale":"2024-09-10T11:36:16.387904+0000","last_undegraded":"2024-09-10T11:36:16.387904+0000","last_fullsized":"2024-09-10T11:36:16.387904+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:22:46.038152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384249+0000","last_change":"2024-09-10T11:36:16.384140+0000","last_active":"2024-09-10T11:36:16.384249+0000","last_peered":"2024-09-10T11:36:16.384249+0000","last_clean":"2024-09-10T11:36:16.384249+0000","last_became_active":"2024-09-10T11:36:16.383678+0000","last_became_peered":"2024-09-10T11:36:16.383678+0000","last_unstale":"2024-09-10T11:36:16.384249+0000","last_undegraded":"2024-09-10T11:36:16.384249+0000","last_fullsized":"2024-09-10T11:36:16.384249+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:46.260025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387012+0000","last_change":"2024-09-10T11:36:16.386901+0000","last_active":"2024-09-10T11:36:16.387012+0000","last_peered":"2024-09-10T11:36:16.387012+0000","last_clean":"2024-09-10T11:36:16.387012+0000","last_became_active":"2024-09-10T11:36:16.386367+0000","last_became_peered":"2024-09-10T11:36:16.386367+0000","last_unstale":"2024-09-10T11:36:16.387012+0000","last_undegraded":"2024-09-10T11:36:16.387012+0000","last_fullsized":"2024-09-10T11:36:16.387012+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:39.874815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394111+0000","last_change":"2024-09-10T11:36:16.393994+0000","last_active":"2024-09-10T11:36:16.394111+0000","last_peered":"2024-09-10T11:36:16.394111+0000","last_clean":"2024-09-10T11:36:16.394111+0000","last_became_active":"2024-09-10T11:36:16.393749+0000","last_became_peered":"2024-09-10T11:36:16.393749+0000","last_unstale":"2024-09-10T11:36:16.394111+0000","last_undegraded":"2024-09-10T11:36:16.394111+0000","last_fullsized":"2024-09-10T11:36:16.394111+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:21.400597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390235+0000","last_change":"2024-09-10T11:36:16.390112+0000","last_active":"2024-09-10T11:36:16.390235+0000","last_peered":"2024-09-10T11:36:16.390235+0000","last_clean":"2024-09-10T11:36:16.390235+0000","last_became_active":"2024-09-10T11:36:16.389861+0000","last_became_peered":"2024-09-10T11:36:16.389861+0000","last_unstale":"2024-09-10T11:36:16.390235+0000","last_undegraded":"2024-09-10T11:36:16.390235+0000","last_fullsized":"2024-09-10T11:36:16.390235+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:26.980952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380962+0000","last_change":"2024-09-10T11:36:16.380414+0000","last_active":"2024-09-10T11:36:16.380962+0000","last_peered":"2024-09-10T11:36:16.380962+0000","last_clean":"2024-09-10T11:36:16.380962+0000","last_became_active":"2024-09-10T11:36:16.379732+0000","last_became_peered":"2024-09-10T11:36:16.379732+0000","last_unstale":"2024-09-10T11:36:16.380962+0000","last_undegraded":"2024-09-10T11:36:16.380962+0000","last_fullsized":"2024-09-10T11:36:16.380962+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:01.157598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386385+0000","last_change":"2024-09-10T11:36:16.386258+0000","last_active":"2024-09-10T11:36:16.386385+0000","last_peered":"2024-09-10T11:36:16.386385+0000","last_clean":"2024-09-10T11:36:16.386385+0000","last_became_active":"2024-09-10T11:36:16.386031+0000","last_became_peered":"2024-09-10T11:36:16.386031+0000","last_unstale":"2024-09-10T11:36:16.386385+0000","last_undegraded":"2024-09-10T11:36:16.386385+0000","last_fullsized":"2024-09-10T11:36:16.386385+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:11.667942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.377188+0000","last_change":"2024-09-10T11:36:16.377081+0000","last_active":"2024-09-10T11:36:16.377188+0000","last_peered":"2024-09-10T11:36:16.377188+0000","last_clean":"2024-09-10T11:36:16.377188+0000","last_became_active":"2024-09-10T11:36:16.376816+0000","last_became_peered":"2024-09-10T11:36:16.376816+0000","last_unstale":"2024-09-10T11:36:16.377188+0000","last_undegraded":"2024-09-10T11:36:16.377188+0000","last_fullsized":"2024-09-10T11:36:16.377188+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:38.807811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379523+0000","last_change":"2024-09-10T11:36:16.379262+0000","last_active":"2024-09-10T11:36:16.379523+0000","last_peered":"2024-09-10T11:36:16.379523+0000","last_clean":"2024-09-10T11:36:16.379523+0000","last_became_active":"2024-09-10T11:36:16.378978+0000","last_became_peered":"2024-09-10T11:36:16.378978+0000","last_unstale":"2024-09-10T11:36:16.379523+0000","last_undegraded":"2024-09-10T11:36:16.379523+0000","last_fullsized":"2024-09-10T11:36:16.379523+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:28.817890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388169+0000","last_change":"2024-09-10T11:36:16.388010+0000","last_active":"2024-09-10T11:36:16.388169+0000","last_peered":"2024-09-10T11:36:16.388169+0000","last_clean":"2024-09-10T11:36:16.388169+0000","last_became_active":"2024-09-10T11:36:16.386747+0000","last_became_peered":"2024-09-10T11:36:16.386747+0000","last_unstale":"2024-09-10T11:36:16.388169+0000","last_undegraded":"2024-09-10T11:36:16.388169+0000","last_fullsized":"2024-09-10T11:36:16.388169+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:35:58.429058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382060+0000","last_change":"2024-09-10T11:36:16.379831+0000","last_active":"2024-09-10T11:36:16.382060+0000","last_peered":"2024-09-10T11:36:16.382060+0000","last_clean":"2024-09-10T11:36:16.382060+0000","last_became_active":"2024-09-10T11:36:16.379593+0000","last_became_peered":"2024-09-10T11:36:16.379593+0000","last_unstale":"2024-09-10T11:36:16.382060+0000","last_undegraded":"2024-09-10T11:36:16.382060+0000","last_fullsized":"2024-09-10T11:36:16.382060+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:03.587480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392416+0000","last_change":"2024-09-10T11:36:16.392039+0000","last_active":"2024-09-10T11:36:16.392416+0000","last_peered":"2024-09-10T11:36:16.392416+0000","last_clean":"2024-09-10T11:36:16.392416+0000","last_became_active":"2024-09-10T11:36:16.389718+0000","last_became_peered":"2024-09-10T11:36:16.389718+0000","last_unstale":"2024-09-10T11:36:16.392416+0000","last_undegraded":"2024-09-10T11:36:16.392416+0000","last_fullsized":"2024-09-10T11:36:16.392416+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:49.239185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387442+0000","last_change":"2024-09-10T11:36:16.387121+0000","last_active":"2024-09-10T11:36:16.387442+0000","last_peered":"2024-09-10T11:36:16.387442+0000","last_clean":"2024-09-10T11:36:16.387442+0000","last_became_active":"2024-09-10T11:36:16.386894+0000","last_became_peered":"2024-09-10T11:36:16.386894+0000","last_unstale":"2024-09-10T11:36:16.387442+0000","last_undegraded":"2024-09-10T11:36:16.387442+0000","last_fullsized":"2024-09-10T11:36:16.387442+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:09.287051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"152'4","reported_seq":16,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389049+0000","last_change":"2024-09-10T11:36:15.383439+0000","last_active":"2024-09-10T11:36:16.389049+0000","last_peered":"2024-09-10T11:36:16.389049+0000","last_clean":"2024-09-10T11:36:16.389049+0000","last_became_active":"2024-09-10T11:36:15.383201+0000","last_became_peered":"2024-09-10T11:36:15.383201+0000","last_unstale":"2024-09-10T11:36:16.389049+0000","last_undegraded":"2024-09-10T11:36:16.389049+0000","last_fullsized":"2024-09-10T11:36:16.389049+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:36.520495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.393035+0000","last_change":"2024-09-10T11:36:16.392931+0000","last_active":"2024-09-10T11:36:16.393035+0000","last_peered":"2024-09-10T11:36:16.393035+0000","last_clean":"2024-09-10T11:36:16.393035+0000","last_became_active":"2024-09-10T11:36:16.391332+0000","last_became_peered":"2024-09-10T11:36:16.391332+0000","last_unstale":"2024-09-10T11:36:16.393035+0000","last_undegraded":"2024-09-10T11:36:16.393035+0000","last_fullsized":"2024-09-10T11:36:16.393035+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:34.720859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.397313+0000","last_change":"2024-09-10T11:36:15.382380+0000","last_active":"2024-09-10T11:36:16.397313+0000","last_peered":"2024-09-10T11:36:16.397313+0000","last_clean":"2024-09-10T11:36:16.397313+0000","last_became_active":"2024-09-10T11:36:15.382159+0000","last_became_peered":"2024-09-10T11:36:15.382159+0000","last_unstale":"2024-09-10T11:36:16.397313+0000","last_undegraded":"2024-09-10T11:36:16.397313+0000","last_fullsized":"2024-09-10T11:36:16.397313+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:53.254381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395046+0000","last_change":"2024-09-10T11:36:16.394618+0000","last_active":"2024-09-10T11:36:16.395046+0000","last_peered":"2024-09-10T11:36:16.395046+0000","last_clean":"2024-09-10T11:36:16.395046+0000","last_became_active":"2024-09-10T11:36:16.394387+0000","last_became_peered":"2024-09-10T11:36:16.394387+0000","last_unstale":"2024-09-10T11:36:16.395046+0000","last_undegraded":"2024-09-10T11:36:16.395046+0000","last_fullsized":"2024-09-10T11:36:16.395046+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:57:35.482330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395324+0000","last_change":"2024-09-10T11:36:15.380559+0000","last_active":"2024-09-10T11:36:16.395324+0000","last_peered":"2024-09-10T11:36:16.395324+0000","last_clean":"2024-09-10T11:36:16.395324+0000","last_became_active":"2024-09-10T11:36:15.380067+0000","last_became_peered":"2024-09-10T11:36:15.380067+0000","last_unstale":"2024-09-10T11:36:16.395324+0000","last_undegraded":"2024-09-10T11:36:16.395324+0000","last_fullsized":"2024-09-10T11:36:16.395324+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.468483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387273+0000","last_change":"2024-09-10T11:36:16.382761+0000","last_active":"2024-09-10T11:36:16.387273+0000","last_peered":"2024-09-10T11:36:16.387273+0000","last_clean":"2024-09-10T11:36:16.387273+0000","last_became_active":"2024-09-10T11:36:16.381673+0000","last_became_peered":"2024-09-10T11:36:16.381673+0000","last_unstale":"2024-09-10T11:36:16.387273+0000","last_undegraded":"2024-09-10T11:36:16.387273+0000","last_fullsized":"2024-09-10T11:36:16.387273+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:15.822438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389923+0000","last_change":"2024-09-10T11:36:15.379624+0000","last_active":"2024-09-10T11:36:16.389923+0000","last_peered":"2024-09-10T11:36:16.389923+0000","last_clean":"2024-09-10T11:36:16.389923+0000","last_became_active":"2024-09-10T11:36:15.379160+0000","last_became_peered":"2024-09-10T11:36:15.379160+0000","last_unstale":"2024-09-10T11:36:16.389923+0000","last_undegraded":"2024-09-10T11:36:16.389923+0000","last_fullsized":"2024-09-10T11:36:16.389923+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:39:19.953466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.385214+0000","last_change":"2024-09-10T11:36:16.384892+0000","last_active":"2024-09-10T11:36:16.385214+0000","last_peered":"2024-09-10T11:36:16.385214+0000","last_clean":"2024-09-10T11:36:16.385214+0000","last_became_active":"2024-09-10T11:36:16.384641+0000","last_became_peered":"2024-09-10T11:36:16.384641+0000","last_unstale":"2024-09-10T11:36:16.385214+0000","last_undegraded":"2024-09-10T11:36:16.385214+0000","last_fullsized":"2024-09-10T11:36:16.385214+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:09.660585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390362+0000","last_change":"2024-09-10T11:36:15.377277+0000","last_active":"2024-09-10T11:36:16.390362+0000","last_peered":"2024-09-10T11:36:16.390362+0000","last_clean":"2024-09-10T11:36:16.390362+0000","last_became_active":"2024-09-10T11:36:15.375763+0000","last_became_peered":"2024-09-10T11:36:15.375763+0000","last_unstale":"2024-09-10T11:36:16.390362+0000","last_undegraded":"2024-09-10T11:36:16.390362+0000","last_fullsized":"2024-09-10T11:36:16.390362+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:53.672885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391591+0000","last_change":"2024-09-10T11:36:16.391487+0000","last_active":"2024-09-10T11:36:16.391591+0000","last_peered":"2024-09-10T11:36:16.391591+0000","last_clean":"2024-09-10T11:36:16.391591+0000","last_became_active":"2024-09-10T11:36:16.390897+0000","last_became_peered":"2024-09-10T11:36:16.390897+0000","last_unstale":"2024-09-10T11:36:16.391591+0000","last_undegraded":"2024-09-10T11:36:16.391591+0000","last_fullsized":"2024-09-10T11:36:16.391591+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:03.967204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389627+0000","last_change":"2024-09-10T11:36:15.379738+0000","last_active":"2024-09-10T11:36:16.389627+0000","last_peered":"2024-09-10T11:36:16.389627+0000","last_clean":"2024-09-10T11:36:16.389627+0000","last_became_active":"2024-09-10T11:36:15.379474+0000","last_became_peered":"2024-09-10T11:36:15.379474+0000","last_unstale":"2024-09-10T11:36:16.389627+0000","last_undegraded":"2024-09-10T11:36:16.389627+0000","last_fullsized":"2024-09-10T11:36:16.389627+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:33.251117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387337+0000","last_change":"2024-09-10T11:36:16.383504+0000","last_active":"2024-09-10T11:36:16.387337+0000","last_peered":"2024-09-10T11:36:16.387337+0000","last_clean":"2024-09-10T11:36:16.387337+0000","last_became_active":"2024-09-10T11:36:16.383280+0000","last_became_peered":"2024-09-10T11:36:16.383280+0000","last_unstale":"2024-09-10T11:36:16.387337+0000","last_undegraded":"2024-09-10T11:36:16.387337+0000","last_fullsized":"2024-09-10T11:36:16.387337+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:59.102987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384309+0000","last_change":"2024-09-10T11:36:15.376945+0000","last_active":"2024-09-10T11:36:16.384309+0000","last_peered":"2024-09-10T11:36:16.384309+0000","last_clean":"2024-09-10T11:36:16.384309+0000","last_became_active":"2024-09-10T11:36:15.376707+0000","last_became_peered":"2024-09-10T11:36:15.376707+0000","last_unstale":"2024-09-10T11:36:16.384309+0000","last_undegraded":"2024-09-10T11:36:16.384309+0000","last_fullsized":"2024-09-10T11:36:16.384309+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:32.564882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384112+0000","last_change":"2024-09-10T11:36:16.383783+0000","last_active":"2024-09-10T11:36:16.384112+0000","last_peered":"2024-09-10T11:36:16.384112+0000","last_clean":"2024-09-10T11:36:16.384112+0000","last_became_active":"2024-09-10T11:36:16.383518+0000","last_became_peered":"2024-09-10T11:36:16.383518+0000","last_unstale":"2024-09-10T11:36:16.384112+0000","last_undegraded":"2024-09-10T11:36:16.384112+0000","last_fullsized":"2024-09-10T11:36:16.384112+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:01.893256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376015+0000","last_change":"2024-09-10T11:36:15.377956+0000","last_active":"2024-09-10T11:36:16.376015+0000","last_peered":"2024-09-10T11:36:16.376015+0000","last_clean":"2024-09-10T11:36:16.376015+0000","last_became_active":"2024-09-10T11:36:15.377656+0000","last_became_peered":"2024-09-10T11:36:15.377656+0000","last_unstale":"2024-09-10T11:36:16.376015+0000","last_undegraded":"2024-09-10T11:36:16.376015+0000","last_fullsized":"2024-09-10T11:36:16.376015+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:21.054760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384045+0000","last_change":"2024-09-10T11:36:16.383929+0000","last_active":"2024-09-10T11:36:16.384045+0000","last_peered":"2024-09-10T11:36:16.384045+0000","last_clean":"2024-09-10T11:36:16.384045+0000","last_became_active":"2024-09-10T11:36:16.383315+0000","last_became_peered":"2024-09-10T11:36:16.383315+0000","last_unstale":"2024-09-10T11:36:16.384045+0000","last_undegraded":"2024-09-10T11:36:16.384045+0000","last_fullsized":"2024-09-10T11:36:16.384045+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:55.611733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.380919+0000","last_change":"2024-09-10T11:36:15.382121+0000","last_active":"2024-09-10T11:36:16.380919+0000","last_peered":"2024-09-10T11:36:16.380919+0000","last_clean":"2024-09-10T11:36:16.380919+0000","last_became_active":"2024-09-10T11:36:15.381755+0000","last_became_peered":"2024-09-10T11:36:15.381755+0000","last_unstale":"2024-09-10T11:36:16.380919+0000","last_undegraded":"2024-09-10T11:36:16.380919+0000","last_fullsized":"2024-09-10T11:36:16.380919+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:34.486042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389112+0000","last_change":"2024-09-10T11:36:16.388136+0000","last_active":"2024-09-10T11:36:16.389112+0000","last_peered":"2024-09-10T11:36:16.389112+0000","last_clean":"2024-09-10T11:36:16.389112+0000","last_became_active":"2024-09-10T11:36:16.387902+0000","last_became_peered":"2024-09-10T11:36:16.387902+0000","last_unstale":"2024-09-10T11:36:16.389112+0000","last_undegraded":"2024-09-10T11:36:16.389112+0000","last_fullsized":"2024-09-10T11:36:16.389112+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:06:03.664994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.377227+0000","last_change":"2024-09-10T11:36:15.379673+0000","last_active":"2024-09-10T11:36:16.377227+0000","last_peered":"2024-09-10T11:36:16.377227+0000","last_clean":"2024-09-10T11:36:16.377227+0000","last_became_active":"2024-09-10T11:36:15.378888+0000","last_became_peered":"2024-09-10T11:36:15.378888+0000","last_unstale":"2024-09-10T11:36:16.377227+0000","last_undegraded":"2024-09-10T11:36:16.377227+0000","last_fullsized":"2024-09-10T11:36:16.377227+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:11.589068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396586+0000","last_change":"2024-09-10T11:36:16.394841+0000","last_active":"2024-09-10T11:36:16.396586+0000","last_peered":"2024-09-10T11:36:16.396586+0000","last_clean":"2024-09-10T11:36:16.396586+0000","last_became_active":"2024-09-10T11:36:16.394391+0000","last_became_peered":"2024-09-10T11:36:16.394391+0000","last_unstale":"2024-09-10T11:36:16.396586+0000","last_undegraded":"2024-09-10T11:36:16.396586+0000","last_fullsized":"2024-09-10T11:36:16.396586+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:59:48.736908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"152'3","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388995+0000","last_change":"2024-09-10T11:36:15.381114+0000","last_active":"2024-09-10T11:36:16.388995+0000","last_peered":"2024-09-10T11:36:16.388995+0000","last_clean":"2024-09-10T11:36:16.388995+0000","last_became_active":"2024-09-10T11:36:15.380840+0000","last_became_peered":"2024-09-10T11:36:15.380840+0000","last_unstale":"2024-09-10T11:36:16.388995+0000","last_undegraded":"2024-09-10T11:36:16.388995+0000","last_fullsized":"2024-09-10T11:36:16.388995+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:28.335370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387313+0000","last_change":"2024-09-10T11:36:16.387093+0000","last_active":"2024-09-10T11:36:16.387313+0000","last_peered":"2024-09-10T11:36:16.387313+0000","last_clean":"2024-09-10T11:36:16.387313+0000","last_became_active":"2024-09-10T11:36:16.386646+0000","last_became_peered":"2024-09-10T11:36:16.386646+0000","last_unstale":"2024-09-10T11:36:16.387313+0000","last_undegraded":"2024-09-10T11:36:16.387313+0000","last_fullsized":"2024-09-10T11:36:16.387313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:41:35.682697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"152'6","reported_seq":18,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.398213+0000","last_change":"2024-09-10T11:36:15.379134+0000","last_active":"2024-09-10T11:36:16.398213+0000","last_peered":"2024-09-10T11:36:16.398213+0000","last_clean":"2024-09-10T11:36:16.398213+0000","last_became_active":"2024-09-10T11:36:15.378904+0000","last_became_peered":"2024-09-10T11:36:15.378904+0000","last_unstale":"2024-09-10T11:36:16.398213+0000","last_undegraded":"2024-09-10T11:36:16.398213+0000","last_fullsized":"2024-09-10T11:36:16.398213+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:37.970209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382404+0000","last_change":"2024-09-10T11:36:16.382254+0000","last_active":"2024-09-10T11:36:16.382404+0000","last_peered":"2024-09-10T11:36:16.382404+0000","last_clean":"2024-09-10T11:36:16.382404+0000","last_became_active":"2024-09-10T11:36:16.382007+0000","last_became_peered":"2024-09-10T11:36:16.382007+0000","last_unstale":"2024-09-10T11:36:16.382404+0000","last_undegraded":"2024-09-10T11:36:16.382404+0000","last_fullsized":"2024-09-10T11:36:16.382404+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:13.728178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376150+0000","last_change":"2024-09-10T11:36:15.383585+0000","last_active":"2024-09-10T11:36:16.376150+0000","last_peered":"2024-09-10T11:36:16.376150+0000","last_clean":"2024-09-10T11:36:16.376150+0000","last_became_active":"2024-09-10T11:36:15.383286+0000","last_became_peered":"2024-09-10T11:36:15.383286+0000","last_unstale":"2024-09-10T11:36:16.376150+0000","last_undegraded":"2024-09-10T11:36:16.376150+0000","last_fullsized":"2024-09-10T11:36:16.376150+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:40.237745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392304+0000","last_change":"2024-09-10T11:36:16.392182+0000","last_active":"2024-09-10T11:36:16.392304+0000","last_peered":"2024-09-10T11:36:16.392304+0000","last_clean":"2024-09-10T11:36:16.392304+0000","last_became_active":"2024-09-10T11:36:16.391925+0000","last_became_peered":"2024-09-10T11:36:16.391925+0000","last_unstale":"2024-09-10T11:36:16.392304+0000","last_undegraded":"2024-09-10T11:36:16.392304+0000","last_fullsized":"2024-09-10T11:36:16.392304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:43.289706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381229+0000","last_change":"2024-09-10T11:36:15.381521+0000","last_active":"2024-09-10T11:36:16.381229+0000","last_peered":"2024-09-10T11:36:16.381229+0000","last_clean":"2024-09-10T11:36:16.381229+0000","last_became_active":"2024-09-10T11:36:15.381218+0000","last_became_peered":"2024-09-10T11:36:15.381218+0000","last_unstale":"2024-09-10T11:36:16.381229+0000","last_undegraded":"2024-09-10T11:36:16.381229+0000","last_fullsized":"2024-09-10T11:36:16.381229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:11.634535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387313+0000","last_change":"2024-09-10T11:36:16.383031+0000","last_active":"2024-09-10T11:36:16.387313+0000","last_peered":"2024-09-10T11:36:16.387313+0000","last_clean":"2024-09-10T11:36:16.387313+0000","last_became_active":"2024-09-10T11:36:16.382809+0000","last_became_peered":"2024-09-10T11:36:16.382809+0000","last_unstale":"2024-09-10T11:36:16.387313+0000","last_undegraded":"2024-09-10T11:36:16.387313+0000","last_fullsized":"2024-09-10T11:36:16.387313+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:23:56.047494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"152'8","reported_seq":20,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394435+0000","last_change":"2024-09-10T11:36:15.383605+0000","last_active":"2024-09-10T11:36:16.394435+0000","last_peered":"2024-09-10T11:36:16.394435+0000","last_clean":"2024-09-10T11:36:16.394435+0000","last_became_active":"2024-09-10T11:36:15.383024+0000","last_became_peered":"2024-09-10T11:36:15.383024+0000","last_unstale":"2024-09-10T11:36:16.394435+0000","last_undegraded":"2024-09-10T11:36:16.394435+0000","last_fullsized":"2024-09-10T11:36:16.394435+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:01.603297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387051+0000","last_change":"2024-09-10T11:36:16.386191+0000","last_active":"2024-09-10T11:36:16.387051+0000","last_peered":"2024-09-10T11:36:16.387051+0000","last_clean":"2024-09-10T11:36:16.387051+0000","last_became_active":"2024-09-10T11:36:16.385962+0000","last_became_peered":"2024-09-10T11:36:16.385962+0000","last_unstale":"2024-09-10T11:36:16.387051+0000","last_undegraded":"2024-09-10T11:36:16.387051+0000","last_fullsized":"2024-09-10T11:36:16.387051+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:45.990807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396651+0000","last_change":"2024-09-10T11:36:15.382077+0000","last_active":"2024-09-10T11:36:16.396651+0000","last_peered":"2024-09-10T11:36:16.396651+0000","last_clean":"2024-09-10T11:36:16.396651+0000","last_became_active":"2024-09-10T11:36:15.381720+0000","last_became_peered":"2024-09-10T11:36:15.381720+0000","last_unstale":"2024-09-10T11:36:16.396651+0000","last_undegraded":"2024-09-10T11:36:16.396651+0000","last_fullsized":"2024-09-10T11:36:16.396651+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:38.022335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387033+0000","last_change":"2024-09-10T11:36:16.385653+0000","last_active":"2024-09-10T11:36:16.387033+0000","last_peered":"2024-09-10T11:36:16.387033+0000","last_clean":"2024-09-10T11:36:16.387033+0000","last_became_active":"2024-09-10T11:36:16.385344+0000","last_became_peered":"2024-09-10T11:36:16.385344+0000","last_unstale":"2024-09-10T11:36:16.387033+0000","last_undegraded":"2024-09-10T11:36:16.387033+0000","last_fullsized":"2024-09-10T11:36:16.387033+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:40.382030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.396058+0000","last_change":"2024-09-10T11:36:15.380319+0000","last_active":"2024-09-10T11:36:16.396058+0000","last_peered":"2024-09-10T11:36:16.396058+0000","last_clean":"2024-09-10T11:36:16.396058+0000","last_became_active":"2024-09-10T11:36:15.379678+0000","last_became_peered":"2024-09-10T11:36:15.379678+0000","last_unstale":"2024-09-10T11:36:16.396058+0000","last_undegraded":"2024-09-10T11:36:16.396058+0000","last_fullsized":"2024-09-10T11:36:16.396058+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:51.748635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.390090+0000","last_change":"2024-09-10T11:36:16.390012+0000","last_active":"2024-09-10T11:36:16.390090+0000","last_peered":"2024-09-10T11:36:16.390090+0000","last_clean":"2024-09-10T11:36:16.390090+0000","last_became_active":"2024-09-10T11:36:16.389234+0000","last_became_peered":"2024-09-10T11:36:16.389234+0000","last_unstale":"2024-09-10T11:36:16.390090+0000","last_undegraded":"2024-09-10T11:36:16.390090+0000","last_fullsized":"2024-09-10T11:36:16.390090+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:06.252213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"152'1","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.395791+0000","last_change":"2024-09-10T11:36:15.379482+0000","last_active":"2024-09-10T11:36:16.395791+0000","last_peered":"2024-09-10T11:36:16.395791+0000","last_clean":"2024-09-10T11:36:16.395791+0000","last_became_active":"2024-09-10T11:36:15.378991+0000","last_became_peered":"2024-09-10T11:36:15.378991+0000","last_unstale":"2024-09-10T11:36:16.395791+0000","last_undegraded":"2024-09-10T11:36:16.395791+0000","last_fullsized":"2024-09-10T11:36:16.395791+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:34.742364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.386981+0000","last_change":"2024-09-10T11:36:16.385779+0000","last_active":"2024-09-10T11:36:16.386981+0000","last_peered":"2024-09-10T11:36:16.386981+0000","last_clean":"2024-09-10T11:36:16.386981+0000","last_became_active":"2024-09-10T11:36:16.385545+0000","last_became_peered":"2024-09-10T11:36:16.385545+0000","last_unstale":"2024-09-10T11:36:16.386981+0000","last_undegraded":"2024-09-10T11:36:16.386981+0000","last_fullsized":"2024-09-10T11:36:16.386981+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:49.930668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.384834+0000","last_change":"2024-09-10T11:36:15.381530+0000","last_active":"2024-09-10T11:36:16.384834+0000","last_peered":"2024-09-10T11:36:16.384834+0000","last_clean":"2024-09-10T11:36:16.384834+0000","last_became_active":"2024-09-10T11:36:15.381294+0000","last_became_peered":"2024-09-10T11:36:15.381294+0000","last_unstale":"2024-09-10T11:36:16.384834+0000","last_undegraded":"2024-09-10T11:36:16.384834+0000","last_fullsized":"2024-09-10T11:36:16.384834+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:56.652537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389818+0000","last_change":"2024-09-10T11:36:16.389688+0000","last_active":"2024-09-10T11:36:16.389818+0000","last_peered":"2024-09-10T11:36:16.389818+0000","last_clean":"2024-09-10T11:36:16.389818+0000","last_became_active":"2024-09-10T11:36:16.389079+0000","last_became_peered":"2024-09-10T11:36:16.389079+0000","last_unstale":"2024-09-10T11:36:16.389818+0000","last_undegraded":"2024-09-10T11:36:16.389818+0000","last_fullsized":"2024-09-10T11:36:16.389818+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:48.448566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.397234+0000","last_change":"2024-09-10T11:36:15.379605+0000","last_active":"2024-09-10T11:36:16.397234+0000","last_peered":"2024-09-10T11:36:16.397234+0000","last_clean":"2024-09-10T11:36:16.397234+0000","last_became_active":"2024-09-10T11:36:15.379229+0000","last_became_peered":"2024-09-10T11:36:15.379229+0000","last_unstale":"2024-09-10T11:36:16.397234+0000","last_undegraded":"2024-09-10T11:36:16.397234+0000","last_fullsized":"2024-09-10T11:36:16.397234+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:31:36.089134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394641+0000","last_change":"2024-09-10T11:36:16.394466+0000","last_active":"2024-09-10T11:36:16.394641+0000","last_peered":"2024-09-10T11:36:16.394641+0000","last_clean":"2024-09-10T11:36:16.394641+0000","last_became_active":"2024-09-10T11:36:16.394203+0000","last_became_peered":"2024-09-10T11:36:16.394203+0000","last_unstale":"2024-09-10T11:36:16.394641+0000","last_undegraded":"2024-09-10T11:36:16.394641+0000","last_fullsized":"2024-09-10T11:36:16.394641+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:51.973289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379405+0000","last_change":"2024-09-10T11:36:15.382118+0000","last_active":"2024-09-10T11:36:16.379405+0000","last_peered":"2024-09-10T11:36:16.379405+0000","last_clean":"2024-09-10T11:36:16.379405+0000","last_became_active":"2024-09-10T11:36:15.381406+0000","last_became_peered":"2024-09-10T11:36:15.381406+0000","last_unstale":"2024-09-10T11:36:16.379405+0000","last_undegraded":"2024-09-10T11:36:16.379405+0000","last_fullsized":"2024-09-10T11:36:16.379405+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:13.674525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382466+0000","last_change":"2024-09-10T11:36:16.382352+0000","last_active":"2024-09-10T11:36:16.382466+0000","last_peered":"2024-09-10T11:36:16.382466+0000","last_clean":"2024-09-10T11:36:16.382466+0000","last_became_active":"2024-09-10T11:36:16.381616+0000","last_became_peered":"2024-09-10T11:36:16.381616+0000","last_unstale":"2024-09-10T11:36:16.382466+0000","last_undegraded":"2024-09-10T11:36:16.382466+0000","last_fullsized":"2024-09-10T11:36:16.382466+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:52.079847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376436+0000","last_change":"2024-09-10T11:36:15.382446+0000","last_active":"2024-09-10T11:36:16.376436+0000","last_peered":"2024-09-10T11:36:16.376436+0000","last_clean":"2024-09-10T11:36:16.376436+0000","last_became_active":"2024-09-10T11:36:15.382222+0000","last_became_peered":"2024-09-10T11:36:15.382222+0000","last_unstale":"2024-09-10T11:36:16.376436+0000","last_undegraded":"2024-09-10T11:36:16.376436+0000","last_fullsized":"2024-09-10T11:36:16.376436+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:09.660672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.387296+0000","last_change":"2024-09-10T11:36:16.383018+0000","last_active":"2024-09-10T11:36:16.387296+0000","last_peered":"2024-09-10T11:36:16.387296+0000","last_clean":"2024-09-10T11:36:16.387296+0000","last_became_active":"2024-09-10T11:36:16.382065+0000","last_became_peered":"2024-09-10T11:36:16.382065+0000","last_unstale":"2024-09-10T11:36:16.387296+0000","last_undegraded":"2024-09-10T11:36:16.387296+0000","last_fullsized":"2024-09-10T11:36:16.387296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:00.304500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382230+0000","last_change":"2024-09-10T11:36:15.380816+0000","last_active":"2024-09-10T11:36:16.382230+0000","last_peered":"2024-09-10T11:36:16.382230+0000","last_clean":"2024-09-10T11:36:16.382230+0000","last_became_active":"2024-09-10T11:36:15.380480+0000","last_became_peered":"2024-09-10T11:36:15.380480+0000","last_unstale":"2024-09-10T11:36:16.382230+0000","last_undegraded":"2024-09-10T11:36:16.382230+0000","last_fullsized":"2024-09-10T11:36:16.382230+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:51.206134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392486+0000","last_change":"2024-09-10T11:36:16.392291+0000","last_active":"2024-09-10T11:36:16.392486+0000","last_peered":"2024-09-10T11:36:16.392486+0000","last_clean":"2024-09-10T11:36:16.392486+0000","last_became_active":"2024-09-10T11:36:16.392032+0000","last_became_peered":"2024-09-10T11:36:16.392032+0000","last_unstale":"2024-09-10T11:36:16.392486+0000","last_undegraded":"2024-09-10T11:36:16.392486+0000","last_fullsized":"2024-09-10T11:36:16.392486+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:12.297982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392281+0000","last_change":"2024-09-10T11:36:15.380841+0000","last_active":"2024-09-10T11:36:16.392281+0000","last_peered":"2024-09-10T11:36:16.392281+0000","last_clean":"2024-09-10T11:36:16.392281+0000","last_became_active":"2024-09-10T11:36:15.380583+0000","last_became_peered":"2024-09-10T11:36:15.380583+0000","last_unstale":"2024-09-10T11:36:16.392281+0000","last_undegraded":"2024-09-10T11:36:16.392281+0000","last_fullsized":"2024-09-10T11:36:16.392281+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:00.766804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.389948+0000","last_change":"2024-09-10T11:36:16.389788+0000","last_active":"2024-09-10T11:36:16.389948+0000","last_peered":"2024-09-10T11:36:16.389948+0000","last_clean":"2024-09-10T11:36:16.389948+0000","last_became_active":"2024-09-10T11:36:16.388909+0000","last_became_peered":"2024-09-10T11:36:16.388909+0000","last_unstale":"2024-09-10T11:36:16.389948+0000","last_undegraded":"2024-09-10T11:36:16.389948+0000","last_fullsized":"2024-09-10T11:36:16.389948+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:09.622045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.382981+0000","last_change":"2024-09-10T11:36:15.382141+0000","last_active":"2024-09-10T11:36:16.382981+0000","last_peered":"2024-09-10T11:36:16.382981+0000","last_clean":"2024-09-10T11:36:16.382981+0000","last_became_active":"2024-09-10T11:36:15.381702+0000","last_became_peered":"2024-09-10T11:36:15.381702+0000","last_unstale":"2024-09-10T11:36:16.382981+0000","last_undegraded":"2024-09-10T11:36:16.382981+0000","last_fullsized":"2024-09-10T11:36:16.382981+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:59.793080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.378262+0000","last_change":"2024-09-10T11:36:16.378145+0000","last_active":"2024-09-10T11:36:16.378262+0000","last_peered":"2024-09-10T11:36:16.378262+0000","last_clean":"2024-09-10T11:36:16.378262+0000","last_became_active":"2024-09-10T11:36:16.377899+0000","last_became_peered":"2024-09-10T11:36:16.377899+0000","last_unstale":"2024-09-10T11:36:16.378262+0000","last_undegraded":"2024-09-10T11:36:16.378262+0000","last_fullsized":"2024-09-10T11:36:16.378262+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:00.811347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.379556+0000","last_change":"2024-09-10T11:36:15.382395+0000","last_active":"2024-09-10T11:36:16.379556+0000","last_peered":"2024-09-10T11:36:16.379556+0000","last_clean":"2024-09-10T11:36:16.379556+0000","last_became_active":"2024-09-10T11:36:15.381863+0000","last_became_peered":"2024-09-10T11:36:15.381863+0000","last_unstale":"2024-09-10T11:36:16.379556+0000","last_undegraded":"2024-09-10T11:36:16.379556+0000","last_fullsized":"2024-09-10T11:36:16.379556+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:50.290083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.388433+0000","last_change":"2024-09-10T11:36:16.387713+0000","last_active":"2024-09-10T11:36:16.388433+0000","last_peered":"2024-09-10T11:36:16.388433+0000","last_clean":"2024-09-10T11:36:16.388433+0000","last_became_active":"2024-09-10T11:36:16.386594+0000","last_became_peered":"2024-09-10T11:36:16.386594+0000","last_unstale":"2024-09-10T11:36:16.388433+0000","last_undegraded":"2024-09-10T11:36:16.388433+0000","last_fullsized":"2024-09-10T11:36:16.388433+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:08:46.750642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376611+0000","last_change":"2024-09-10T11:36:15.383769+0000","last_active":"2024-09-10T11:36:16.376611+0000","last_peered":"2024-09-10T11:36:16.376611+0000","last_clean":"2024-09-10T11:36:16.376611+0000","last_became_active":"2024-09-10T11:36:15.383513+0000","last_became_peered":"2024-09-10T11:36:15.383513+0000","last_unstale":"2024-09-10T11:36:16.376611+0000","last_undegraded":"2024-09-10T11:36:16.376611+0000","last_fullsized":"2024-09-10T11:36:16.376611+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:42.097339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391374+0000","last_change":"2024-09-10T11:36:16.391247+0000","last_active":"2024-09-10T11:36:16.391374+0000","last_peered":"2024-09-10T11:36:16.391374+0000","last_clean":"2024-09-10T11:36:16.391374+0000","last_became_active":"2024-09-10T11:36:16.390546+0000","last_became_peered":"2024-09-10T11:36:16.390546+0000","last_unstale":"2024-09-10T11:36:16.391374+0000","last_undegraded":"2024-09-10T11:36:16.391374+0000","last_fullsized":"2024-09-10T11:36:16.391374+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:26.944382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.376830+0000","last_change":"2024-09-10T11:36:15.376778+0000","last_active":"2024-09-10T11:36:16.376830+0000","last_peered":"2024-09-10T11:36:16.376830+0000","last_clean":"2024-09-10T11:36:16.376830+0000","last_became_active":"2024-09-10T11:36:15.376295+0000","last_became_peered":"2024-09-10T11:36:15.376295+0000","last_unstale":"2024-09-10T11:36:16.376830+0000","last_undegraded":"2024-09-10T11:36:16.376830+0000","last_fullsized":"2024-09-10T11:36:16.376830+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:24:08.711894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392338+0000","last_change":"2024-09-10T11:36:16.392072+0000","last_active":"2024-09-10T11:36:16.392338+0000","last_peered":"2024-09-10T11:36:16.392338+0000","last_clean":"2024-09-10T11:36:16.392338+0000","last_became_active":"2024-09-10T11:36:16.391845+0000","last_became_peered":"2024-09-10T11:36:16.391845+0000","last_unstale":"2024-09-10T11:36:16.392338+0000","last_undegraded":"2024-09-10T11:36:16.392338+0000","last_fullsized":"2024-09-10T11:36:16.392338+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:24.677935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381994+0000","last_change":"2024-09-10T11:36:15.382708+0000","last_active":"2024-09-10T11:36:16.381994+0000","last_peered":"2024-09-10T11:36:16.381994+0000","last_clean":"2024-09-10T11:36:16.381994+0000","last_became_active":"2024-09-10T11:36:15.381919+0000","last_became_peered":"2024-09-10T11:36:15.381919+0000","last_unstale":"2024-09-10T11:36:16.381994+0000","last_undegraded":"2024-09-10T11:36:16.381994+0000","last_fullsized":"2024-09-10T11:36:16.381994+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:42:33.781561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.394798+0000","last_change":"2024-09-10T11:36:16.394685+0000","last_active":"2024-09-10T11:36:16.394798+0000","last_peered":"2024-09-10T11:36:16.394798+0000","last_clean":"2024-09-10T11:36:16.394798+0000","last_became_active":"2024-09-10T11:36:16.394465+0000","last_became_peered":"2024-09-10T11:36:16.394465+0000","last_unstale":"2024-09-10T11:36:16.394798+0000","last_undegraded":"2024-09-10T11:36:16.394798+0000","last_fullsized":"2024-09-10T11:36:16.394798+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:41.872627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.381266+0000","last_change":"2024-09-10T11:36:15.382257+0000","last_active":"2024-09-10T11:36:16.381266+0000","last_peered":"2024-09-10T11:36:16.381266+0000","last_clean":"2024-09-10T11:36:16.381266+0000","last_became_active":"2024-09-10T11:36:15.381948+0000","last_became_peered":"2024-09-10T11:36:15.381948+0000","last_unstale":"2024-09-10T11:36:16.381266+0000","last_undegraded":"2024-09-10T11:36:16.381266+0000","last_fullsized":"2024-09-10T11:36:16.381266+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:48.598882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.378034+0000","last_change":"2024-09-10T11:36:16.377893+0000","last_active":"2024-09-10T11:36:16.378034+0000","last_peered":"2024-09-10T11:36:16.378034+0000","last_clean":"2024-09-10T11:36:16.378034+0000","last_became_active":"2024-09-10T11:36:16.377508+0000","last_became_peered":"2024-09-10T11:36:16.377508+0000","last_unstale":"2024-09-10T11:36:16.378034+0000","last_undegraded":"2024-09-10T11:36:16.378034+0000","last_fullsized":"2024-09-10T11:36:16.378034+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:55.733317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.375186+0000","last_change":"2024-09-10T11:36:15.384761+0000","last_active":"2024-09-10T11:36:16.375186+0000","last_peered":"2024-09-10T11:36:16.375186+0000","last_clean":"2024-09-10T11:36:16.375186+0000","last_became_active":"2024-09-10T11:36:15.384431+0000","last_became_peered":"2024-09-10T11:36:15.384431+0000","last_unstale":"2024-09-10T11:36:16.375186+0000","last_undegraded":"2024-09-10T11:36:16.375186+0000","last_fullsized":"2024-09-10T11:36:16.375186+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:26.174416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.392447+0000","last_change":"2024-09-10T11:36:16.390866+0000","last_active":"2024-09-10T11:36:16.392447+0000","last_peered":"2024-09-10T11:36:16.392447+0000","last_clean":"2024-09-10T11:36:16.392447+0000","last_became_active":"2024-09-10T11:36:16.390636+0000","last_became_peered":"2024-09-10T11:36:16.390636+0000","last_unstale":"2024-09-10T11:36:16.392447+0000","last_undegraded":"2024-09-10T11:36:16.392447+0000","last_fullsized":"2024-09-10T11:36:16.392447+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:15.363240+0000","last_clean_scrub_stamp":"2024-09-10T11:36:15.363240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.440433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"152'2","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.391096+0000","last_change":"2024-09-10T11:36:15.380862+0000","last_active":"2024-09-10T11:36:16.391096+0000","last_peered":"2024-09-10T11:36:16.391096+0000","last_clean":"2024-09-10T11:36:16.391096+0000","last_became_active":"2024-09-10T11:36:15.380644+0000","last_became_peered":"2024-09-10T11:36:15.380644+0000","last_unstale":"2024-09-10T11:36:16.391096+0000","last_undegraded":"2024-09-10T11:36:16.391096+0000","last_fullsized":"2024-09-10T11:36:16.391096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:36:14.361225+0000","last_clean_scrub_stamp":"2024-09-10T11:36:14.361225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:58:54.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":435,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T11:36:16.374090+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:36:16.374090+0000","last_peered":"2024-09-10T11:36:16.374090+0000","last_clean":"2024-09-10T11:36:16.374090+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:36:16.374090+0000","last_undegraded":"2024-09-10T11:36:16.374090+0000","last_fullsized":"2024-09-10T11:36:16.374090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154470,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574923,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1395,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193916,"statfs":{"total":200000000,"available":198570911,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574922,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574923,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":825,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199154434,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":826,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199153832,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193908,"statfs":{"total":200000000,"available":198562502,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574923,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":827,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194485,"statfs":{"total":200000000,"available":199152836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:36:19.655 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:36:19.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:36:19.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.795+0000 7fa14f082640 1 Processor -- start 2024-09-10T11:36:19.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.795+0000 7fa14f082640 1 -- start start 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14f082640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 0x7fa148160660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14f082640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa148160ba0 0x7fa1480cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14f082640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14806f2c0 con 0x7fa148146920 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14f082640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa14806f5a0 con 0x7fa148160ba0 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14f082640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa1480ced70 con 0x7fa148161270 2024-09-10T11:36:19.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14d61f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa148160ba0 0x7fa1480cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14ce1e640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa148146920 0x7fa1480718f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51312/0 (socket says 172.21.15.99:51312) 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14d61f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa148160ba0 0x7fa1480cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34490/0 (socket says 172.21.15.99:34490) 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.796+0000 7fa14ce1e640 1 -- 172.21.15.99:0/964926932 learned_addr learned my addr 172.21.15.99:0/964926932 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa14d61f640 1 -- 172.21.15.99:0/964926932 >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 msgr2=0x7fa148160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa147fff640 1 --2- 172.21.15.99:0/964926932 >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 0x7fa148160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa1477fe640 1 -- 172.21.15.99:0/964926932 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066575087 0 0) 0x7fa14806f2c0 con 0x7fa148146920 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa14d61f640 1 --2- 172.21.15.99:0/964926932 >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 0x7fa148160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa14d61f640 1 -- 172.21.15.99:0/964926932 >> v1:172.21.15.99:6789/0 conn(0x7fa148146920 legacy=0x7fa1480718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa14d61f640 1 -- 172.21.15.99:0/964926932 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1480cf050 con 0x7fa148160ba0 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa147fff640 1 --2- 172.21.15.99:0/964926932 >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 0x7fa148160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa14d61f640 1 --2- 172.21.15.99:0/964926932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148160ba0 0x7fa1480cf570 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa148072790 tx=0x7fa14002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7642f022703cd306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa1477fe640 1 -- 172.21.15.99:0/964926932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007740 con 0x7fa148160ba0 2024-09-10T11:36:19.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa1477fe640 1 -- 172.21.15.99:0/964926932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1400078e0 con 0x7fa148160ba0 2024-09-10T11:36:19.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.797+0000 7fa1477fe640 1 -- 172.21.15.99:0/964926932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140007cb0 con 0x7fa148160ba0 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 -- 172.21.15.99:0/964926932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148160ba0 msgr2=0x7fa1480cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 --2- 172.21.15.99:0/964926932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148160ba0 0x7fa1480cf570 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa148072790 tx=0x7fa14002a5f0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 -- 172.21.15.99:0/964926932 shutdown_connections 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 --2- 172.21.15.99:0/964926932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148160ba0 0x7fa1480cf570 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 --2- 172.21.15.99:0/964926932 >> v2:172.21.15.196:3300/0 conn(0x7fa148161270 0x7fa148160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 -- 172.21.15.99:0/964926932 >> 172.21.15.99:0/964926932 conn(0x7fa1480b2a80 msgr2=0x7fa1480b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 -- 172.21.15.99:0/964926932 shutdown_connections 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.798+0000 7fa14f082640 1 -- 172.21.15.99:0/964926932 wait complete. 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.799+0000 7fa14f082640 1 Processor -- start 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.799+0000 7fa14f082640 1 -- start start 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 0x7fa1481f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 0x7fa1481f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480cfc30 con 0x7fa148146920 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480cfab0 con 0x7fa148161270 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14f082640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1480cfdb0 con 0x7fa148160ba0 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34492/0 (socket says 172.21.15.99:34492) 2024-09-10T11:36:19.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 -- 172.21.15.99:0/1701820021 learned_addr learned my addr 172.21.15.99:0/1701820021 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa147fff640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 0x7fa1481f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14d61f640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 0x7fa1481f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 -- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 msgr2=0x7fa1481f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 0x7fa1481f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 -- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 msgr2=0x7fa1481f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 0x7fa1481f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14ce1e640 1 -- 172.21.15.99:0/1701820021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14809bcb0 con 0x7fa148146920 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa147fff640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 0x7fa1481f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.800+0000 7fa14d61f640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 0x7fa1481f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa14ce1e640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c4d0 tx=0x7fa13800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138018070 con 0x7fa148146920 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa138007500 con 0x7fa148146920 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138004a20 con 0x7fa148146920 2024-09-10T11:36:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1480af7b0 con 0x7fa148146920 2024-09-10T11:36:19.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.801+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14809c230 con 0x7fa148146920 2024-09-10T11:36:19.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.802+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1480cf570 con 0x7fa148146920 2024-09-10T11:36:19.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.802+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa13801b070 con 0x7fa148146920 2024-09-10T11:36:19.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.803+0000 7fa1457fa640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 0x7fa13406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.803+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7fa13808a1b0 con 0x7fa148146920 2024-09-10T11:36:19.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.803+0000 7fa147fff640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 0x7fa13406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.803+0000 7fa147fff640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 0x7fa13406e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006fd0 tx=0x7fa13c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:19.807+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa138058b60 con 0x7fa148146920 2024-09-10T11:36:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.036+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa148160690 con 0x7fa148146920 2024-09-10T11:36:20.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.039+0000 7fa1457fa640 1 -- 172.21.15.99:0/1701820021 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7fa138058460 con 0x7fa148146920 2024-09-10T11:36:20.040 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:36:20.041 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:36:20.041 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:36:20.041 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:36:20.041 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:36:20.041 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 msgr2=0x7fa13406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 0x7fa13406e2f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006fd0 tx=0x7fa13c008040 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 msgr2=0x7fa1481f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c4d0 tx=0x7fa13800c990 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 shutdown_connections 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa13406be30 0x7fa13406e2f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148161270 0x7fa1481f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa148160ba0 0x7fa1481f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 --2- 172.21.15.99:0/1701820021 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa148146920 0x7fa1481f6550 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 >> 172.21.15.99:0/1701820021 conn(0x7fa1480b2a80 msgr2=0x7fa1480b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 shutdown_connections 2024-09-10T11:36:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.041+0000 7fa14f082640 1 -- 172.21.15.99:0/1701820021 wait complete. 2024-09-10T11:36:20.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:20.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 Processor -- start 2024-09-10T11:36:20.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 -- start start 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f765c0d1200 0x7f765c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c06e5d0 0x7f765c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765c152e60 con 0x7f765c15c060 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f765c06ef50 con 0x7f765c0d1200 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.150+0000 7f7662a47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f765c0d15e0 con 0x7f765c06e5d0 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f7660fe4640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f765c15c060 0x7f765c15c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51316/0 (socket says 172.21.15.99:51316) 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f7660fe4640 1 -- 172.21.15.99:0/2674924014 learned_addr learned my addr 172.21.15.99:0/2674924014 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c06e5d0 0x7f765c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765bfff640 1 --2- 172.21.15.99:0/2674924014 >> v2:172.21.15.196:3300/0 conn(0x7f765c0d1200 0x7f765c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325266877 0 0) 0x7f765c152e60 con 0x7f765c15c060 2024-09-10T11:36:20.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7644003520 con 0x7f765c15c060 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 393204000 0 0) 0x7f7644003520 con 0x7f765c15c060 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c06e5d0 msgr2=0x7f765c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c06e5d0 0x7f765c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 >> v2:172.21.15.196:3300/0 conn(0x7f765c0d1200 msgr2=0x7f765c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 --2- 172.21.15.99:0/2674924014 >> v2:172.21.15.196:3300/0 conn(0x7f765c0d1200 0x7f765c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c072680 con 0x7f765c15c060 2024-09-10T11:36:20.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f76540019f0 con 0x7f765c15c060 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 >> v1:172.21.15.99:6789/0 conn(0x7f765c15c060 legacy=0x7f765c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76440039b0 0x7f7644003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f7644004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7644004f90 0x7f764400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f765c06ef50 con 0x7f76440043a0 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f765c0d15e0 con 0x7f7644004f90 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f764400bf30 con 0x7f76440039b0 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c06e5d0 0x7f765c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765bfff640 1 --2- 172.21.15.99:0/2674924014 >> v2:172.21.15.196:3300/0 conn(0x7f765c0d1200 0x7f765c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:20.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.151+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f7644004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f7660fe4640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7644004f90 0x7f764400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765bfff640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76440039b0 0x7f7644003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 -- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76440039b0 msgr2=0x7f7644003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76440039b0 0x7f7644003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 -- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7644004f90 msgr2=0x7f764400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7644004f90 0x7f764400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 -- 172.21.15.99:0/2674924014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7644003520 con 0x7f76440043a0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f7660fe4640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7644004f90 0x7f764400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f7644004880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f7650009b00 tx=0x7f7644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65afea2069a9ff72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.152+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650036070 con 0x7f76440043a0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7650004030 con 0x7f76440043a0 2024-09-10T11:36:20.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f765affd640 1 -- 172.21.15.99:0/2674924014 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76500097d0 con 0x7f76440043a0 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 -- 172.21.15.99:0/2674924014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 msgr2=0x7f7644004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 --2- 172.21.15.99:0/2674924014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f7644004880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f7650009b00 tx=0x7f7644005830 comp rx=0 tx=0).stop 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7660fe4640 1 -- 172.21.15.99:0/2674924014 reap_dead start 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 -- 172.21.15.99:0/2674924014 shutdown_connections 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 -- 172.21.15.99:0/2674924014 >> 172.21.15.99:0/2674924014 conn(0x7f765c06bc10 msgr2=0x7f765c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 -- 172.21.15.99:0/2674924014 shutdown_connections 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.153+0000 7f7662a47640 1 -- 172.21.15.99:0/2674924014 wait complete. 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.154+0000 7f7662a47640 1 Processor -- start 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 -- start start 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 0x7f765c150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 0x7f765c0c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f765c072f20 con 0x7f76440043a0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f765c072c40 con 0x7f765c151050 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7662a47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f765c15cd90 con 0x7f765c0afcf0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f765bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f765bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34504/0 (socket says 172.21.15.99:34504) 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f765bfff640 1 -- 172.21.15.99:0/2531637564 learned_addr learned my addr 172.21.15.99:0/2531637564 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f7660fe4640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 0x7f765c0c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.155+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 0x7f765c150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 -- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 msgr2=0x7f765c0c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 0x7f765c0c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 -- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 msgr2=0x7f765c150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 0x7f765c150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 -- 172.21.15.99:0/2531637564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765c150e60 con 0x7f76440043a0 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 0x7f765c150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:20.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7660fe4640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 0x7f765c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f765bfff640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f764c011960 tx=0x7f764c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c00fb70 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c00fd10 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765c0aec50 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c015660 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.156+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765c0af210 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.157+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765c073440 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.157+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f764c015800 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.158+0000 7f7658ff9640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 0x7f764806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.158+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f764c08b920 con 0x7f76440043a0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.158+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 0x7f764806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.158+0000 7f765b7fe640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 0x7f764806e320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7650033000 tx=0x7f7650008070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.162+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f764c019030 con 0x7f76440043a0 2024-09-10T11:36:20.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.361+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f765c06df70 con 0x7f76440043a0 2024-09-10T11:36:20.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.362+0000 7f7658ff9640 1 -- 172.21.15.99:0/2531637564 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f764c05a360 con 0x7f76440043a0 2024-09-10T11:36:20.364 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:20.364 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:36:20.364 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:36:20.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 msgr2=0x7f764806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 0x7f764806e320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7650033000 tx=0x7f7650008070 comp rx=0 tx=0).stop 2024-09-10T11:36:20.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 msgr2=0x7f765c0ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f764c011960 tx=0x7f764c011e20 comp rx=0 tx=0).stop 2024-09-10T11:36:20.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 shutdown_connections 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f764806be60 0x7f764806e320 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f765c0afcf0 0x7f765c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f765c151050 0x7f765c150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 --2- 172.21.15.99:0/2531637564 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76440043a0 0x7f765c0ae670 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 >> 172.21.15.99:0/2531637564 conn(0x7f765c06bc10 msgr2=0x7f765c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 shutdown_connections 2024-09-10T11:36:20.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.364+0000 7f7662a47640 1 -- 172.21.15.99:0/2531637564 wait complete. 2024-09-10T11:36:20.780 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:36:15.712585+0000', 'modified': '2024-09-10T11:36:17.711254+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7341}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7341': {'gid': 7341, 'name': 'a', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6837/1210854172', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6835', 'nonce': 1210854172}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 1210854172}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7341, 'qdb_cluster': [7341]} 2024-09-10T11:36:20.780 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:36:20.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:36:20.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.881+0000 7f9970f49640 1 Processor -- start 2024-09-10T11:36:20.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.881+0000 7f9970f49640 1 -- start start 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f9970f49640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f996c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f9970f49640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0ceb10 con 0x7f996c06ee30 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f9970f49640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f996c0b3a90 con 0x7f996c15c7b0 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f9970f49640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f996c06f2a0 con 0x7f996c0d1180 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996ad9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f996c06ee30 0x7f996c06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:51318/0 (socket says 172.21.15.99:51318) 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996ad9d640 1 -- 172.21.15.99:0/3771805966 learned_addr learned my addr 172.21.15.99:0/3771805966 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f9969d9b640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f996c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469416527 0 0) 0x7f9964001490 con 0x7f996c06ee30 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9958003520 con 0x7f996c06ee30 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125117581 0 0) 0x7f9958003520 con 0x7f996c06ee30 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 msgr2=0x7f996c06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f996c06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 >> v1:172.21.15.196:6789/0 conn(0x7f996c15c7b0 legacy=0x7f996c0d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c0b3d70 con 0x7f996c06ee30 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9964001bc0 con 0x7f996c06ee30 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 >> v1:172.21.15.99:6789/0 conn(0x7f996c06ee30 legacy=0x7f996c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99580039e0 0x7f9958003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f9958004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f99580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f996c06f2a0 con 0x7f9958004d40 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f995800ba20 con 0x7f99580039e0 2024-09-10T11:36:20.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f995800bc20 con 0x7f996c0d1180 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.882+0000 7f996ad9d640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f99580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996a59c640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99580039e0 0x7f9958003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f9969d9b640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f9958004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 -- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 msgr2=0x7f9958004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c0d1180 0x7f9958004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 -- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99580039e0 msgr2=0x7f9958003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99580039e0 0x7f9958003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 -- 172.21.15.99:0/3771805966 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9958003520 con 0x7f9958004d40 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.883+0000 7f996ad9d640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f99580051c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f996400aa50 tx=0x7f9958005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125c8c12aa70d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996400a670 con 0x7f9958004d40 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996402f070 con 0x7f9958004d40 2024-09-10T11:36:20.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f996959a640 1 -- 172.21.15.99:0/3771805966 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964034470 con 0x7f9958004d40 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 -- 172.21.15.99:0/3771805966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 msgr2=0x7f99580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 --2- 172.21.15.99:0/3771805966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f99580051c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f996400aa50 tx=0x7f9958005830 comp rx=0 tx=0).stop 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f996ad9d640 1 -- 172.21.15.99:0/3771805966 reap_dead start 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 -- 172.21.15.99:0/3771805966 shutdown_connections 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 -- 172.21.15.99:0/3771805966 >> 172.21.15.99:0/3771805966 conn(0x7f996c06bc10 msgr2=0x7f996c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 -- 172.21.15.99:0/3771805966 shutdown_connections 2024-09-10T11:36:20.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.884+0000 7f9970f49640 1 -- 172.21.15.99:0/3771805966 wait complete. 2024-09-10T11:36:20.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.885+0000 7f9970f49640 1 Processor -- start 2024-09-10T11:36:20.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 -- start start 2024-09-10T11:36:20.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c152ae0 0x7f996c146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996c152f60 0x7f996c1473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f996c0b4330 con 0x7f9958004d40 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f996c0b3840 con 0x7f996c152f60 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f9970f49640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f996c0b4610 con 0x7f996c152ae0 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34526/0 (socket says 172.21.15.99:34526) 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.886+0000 7f996a59c640 1 -- 172.21.15.99:0/1423667008 learned_addr learned my addr 172.21.15.99:0/1423667008 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 -- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c152ae0 msgr2=0x7f996c146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c152ae0 0x7f996c146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 -- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996c152f60 msgr2=0x7f996c1473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996c152f60 0x7f996c1473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 -- 172.21.15.99:0/1423667008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1535d0 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f996a59c640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f995c00f7c0 tx=0x7f995c00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f9962ffd640 1 -- 172.21.15.99:0/1423667008 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c0052e0 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f9962ffd640 1 -- 172.21.15.99:0/1423667008 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c018070 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f9962ffd640 1 -- 172.21.15.99:0/1423667008 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00d8f0 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1538b0 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.887+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c153e70 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.888+0000 7f9962ffd640 1 -- 172.21.15.99:0/1423667008 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f995c00da90 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.889+0000 7f9962ffd640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 0x7f994c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.889+0000 7f9962ffd640 1 -- 172.21.15.99:0/1423667008 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f995c08ad70 con 0x7f9958004d40 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.889+0000 7f9969d9b640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 0x7f994c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.889+0000 7f9969d9b640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 0x7f994c06e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f996c06eb00 tx=0x7f9954008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.890+0000 7f9973dbb740 1 Processor -- start 2024-09-10T11:36:20.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.890+0000 7f9973dbb740 1 -- start start 2024-09-10T11:36:20.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fea3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 0x55e3fe9f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e3fe958520 con 0x55e3fe9f7a50 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3fe9003e0 con 0x55e3fe9f3a80 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e3fe9544e0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34536/0 (socket says 172.21.15.99:34536) 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 -- 172.21.15.99:0/1897320994 learned_addr learned my addr 172.21.15.99:0/1897320994 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996a59c640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fea3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9969d9b640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 0x55e3fe9f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 -- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 msgr2=0x55e3fea3ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fea3ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 -- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 msgr2=0x55e3fe9f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 0x55e3fe9f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 -- 172.21.15.99:0/1897320994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fe91b2f0 con 0x55e3fe9f7a50 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f9969d9b640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 0x55e3fe9f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f996ad9d640 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f996400ab40 tx=0x7f996402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f142f1ca7f184da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f994bfff640 1 -- 172.21.15.99:0/1897320994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964001980 con 0x55e3fe9f7a50 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f994bfff640 1 -- 172.21.15.99:0/1897320994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996402f070 con 0x55e3fe9f7a50 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.891+0000 7f994bfff640 1 -- 172.21.15.99:0/1897320994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9964004b60 con 0x55e3fe9f7a50 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 -- 172.21.15.99:0/1897320994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 msgr2=0x55e3fe9f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f996400ab40 tx=0x7f996402d000 comp rx=0 tx=0).stop 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 -- 172.21.15.99:0/1897320994 shutdown_connections 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fe9f08b0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f3a80 0x55e3fe9f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 --2- 172.21.15.99:0/1897320994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fea3ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 -- 172.21.15.99:0/1897320994 >> 172.21.15.99:0/1897320994 conn(0x55e3fe9f2a80 msgr2=0x55e3fea3af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 -- 172.21.15.99:0/1897320994 shutdown_connections 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.892+0000 7f9973dbb740 1 -- 172.21.15.99:0/1897320994 wait complete. 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.893+0000 7f9973dbb740 1 -- start start 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.893+0000 7f9973dbb740 1 Processor -- start 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 0x55e3fea17000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 0x55e3fe9f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e3fe9f1f20 con 0x55e3fe9f62c0 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3fe8e1450 con 0x55e3fe9f7a50 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e3fea3c290 con 0x55e3fea17540 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 10 client.0 _ll_register_callbacks cb 0x55e3fe66aac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:36:20.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34552/0 (socket says 172.21.15.99:34552) 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 -- 172.21.15.99:0/907778894 learned_addr learned my addr 172.21.15.99:0/907778894 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9969d9b640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 0x55e3fea17000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996ad9d640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 0x55e3fe9f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 -- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 msgr2=0x55e3fe9f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 0x55e3fe9f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 -- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 msgr2=0x55e3fea17000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 0x55e3fea17000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 -- 172.21.15.99:0/907778894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fe66b2d0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9969d9b640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 0x55e3fea17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996ad9d640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 0x55e3fe9f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f996a59c640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f995c058fb0 tx=0x7f995c01a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce52375bb61b3712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f99497fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c05d3d0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f995c00dd80 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f996402f070 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f995c061c00 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 10 client.7470 fetch_fsmap learned FSMap version 61 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.894+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55e3fe7356b0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f996402f070 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f995c058470 con 0x55e3fe9f62c0 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 10 client.7470 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 10 client.7470 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:36:20.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 20 client.7470 populate_metadata read hostname 'smithi099' 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea1f6c0 0x55e3fea1fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 4 client.7470 mds_command: new command op to 7341 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 --> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3fe965c50 con 0x55e3fea1f6c0 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9969d9b640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea1f6c0 0x55e3fea1fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f99497fa640 10 client.7470 ms_handle_connect on v2:172.21.15.99:6835/1210854172 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.895+0000 7f9969d9b640 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea1f6c0 0x55e3fea1fac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e042c2dbfd065e1e server_cookie=2a51cd0c1794a22 in_seq=0 out_seq=0 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f99497fa640 1 -- 172.21.15.99:0/907778894 <== mds.0 v2:172.21.15.99:6835/1210854172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3fe965c50 con 0x55e3fea1f6c0 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f99497fa640 10 client.7470 handle_command_reply: tid=0 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 client.7470 shutdown 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 msgr2=0x55e3fea14f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f995c058fb0 tx=0x7f995c01a310 comp rx=0 tx=0).stop 2024-09-10T11:36:20.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 shutdown_connections 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea1f6c0 0x55e3fea1fac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fea17540 0x55e3fe9f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fe9f7a50 0x55e3fea17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 --2- 172.21.15.99:0/907778894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3fea14f00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f99497fa640 0 client.7470 ms_handle_reset on v2:172.21.15.99:6835/1210854172 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 >> 172.21.15.99:0/907778894 conn(0x55e3fe9f3a80 msgr2=0x55e3fea38ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 shutdown_connections 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 1 -- 172.21.15.99:0/907778894 wait complete. 2024-09-10T11:36:20.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.896+0000 7f9973dbb740 20 client.7470 trim_cache size 0 max 16384 2024-09-10T11:36:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 Processor -- start 2024-09-10T11:36:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 -- start start 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fe9f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea13fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1fe7d0 con 0x55e3fe9f7a50 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f996c15ffe0 con 0x55e3fea17540 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f996c1fe250 con 0x55e3fe9f62c0 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9969d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9969d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34560/0 (socket says 172.21.15.99:34560) 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.914+0000 7f9969d9b640 1 -- 172.21.15.99:0/2285482395 learned_addr learned my addr 172.21.15.99:0/2285482395 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 -- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 msgr2=0x55e3fe9f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f996a59c640 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fe9f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fe9f0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 -- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 msgr2=0x55e3fea13fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea13fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 -- 172.21.15.99:0/2285482395 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3fea17370 con 0x55e3fe9f7a50 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f9969d9b640 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9954011630 tx=0x7f9954016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7c76d58ccc5855 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f994b7fe640 1 -- 172.21.15.99:0/2285482395 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954011cb0 con 0x55e3fe9f7a50 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f994b7fe640 1 -- 172.21.15.99:0/2285482395 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995402f070 con 0x55e3fe9f7a50 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.915+0000 7f994b7fe640 1 -- 172.21.15.99:0/2285482395 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995402a4e0 con 0x55e3fe9f7a50 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- 172.21.15.99:0/2285482395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 msgr2=0x55e3fea390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f9954011630 tx=0x7f9954016000 comp rx=0 tx=0).stop 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- 172.21.15.99:0/2285482395 shutdown_connections 2024-09-10T11:36:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea13fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f7a50 0x55e3fea390d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 --2- 172.21.15.99:0/2285482395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f62c0 0x55e3fe9f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- 172.21.15.99:0/2285482395 >> 172.21.15.99:0/2285482395 conn(0x55e3fe9f3a80 msgr2=0x55e3fe9ec100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- 172.21.15.99:0/2285482395 shutdown_connections 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- 172.21.15.99:0/2285482395 wait complete. 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 -- start start 2024-09-10T11:36:20.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.916+0000 7f9973dbb740 1 Processor -- start 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f7a50 0x55e3feab99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea1faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e3fea15440 con 0x55e3fe9f62c0 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e3fea16230 con 0x55e3fea17540 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55e3fe9ef9e0 con 0x55e3fe9f7a50 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9973dbb740 10 client.0 _ll_register_callbacks cb 0x7f995c058e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34564/0 (socket says 172.21.15.99:34564) 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 -- 172.21.15.99:0/334607901 learned_addr learned my addr 172.21.15.99:0/334607901 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:20.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f9969d9b640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f7a50 0x55e3feab99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996ad9d640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea1faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 -- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f7a50 msgr2=0x55e3feab99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f7a50 0x55e3feab99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 -- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 msgr2=0x55e3fea1faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.917+0000 7f996a59c640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea1faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f996a59c640 1 -- 172.21.15.99:0/334607901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3fea1d0d0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f996ad9d640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea1faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f996a59c640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f995c0572d0 tx=0x7f995c089170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e842a4573ed3df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c05d3d0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f995c057ee0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995402f070 con 0x55e3fe9f62c0 2024-09-10T11:36:20.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f995c058980 con 0x55e3fe9f62c0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 10 client.7476 fetch_fsmap learned FSMap version 61 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x55e3feab8ec0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995402f070 con 0x55e3fe9f62c0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 61) ==== 3502+0+0 (secure 0 0 0) 0x7f995c0580e0 con 0x55e3fe9f62c0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 10 client.7476 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 10 client.7476 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 20 client.7476 populate_metadata read hostname 'smithi099' 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea146b0 0x55e3fea39df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 4 client.7476 mds_command: new command op to 7341 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 --> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] -- command(tid 0: {"prefix": "status"}) -- 0x55e3fe95ffa0 con 0x55e3fea146b0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.918+0000 7f9969d9b640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea146b0 0x55e3fea39df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f99497fa640 10 client.7476 ms_handle_connect on v2:172.21.15.99:6835/1210854172 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9969d9b640 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea146b0 0x55e3fea39df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74f878b2f5d60a4 server_cookie=68b87a642bf64091 in_seq=0 out_seq=0 2024-09-10T11:36:20.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f99497fa640 1 -- 172.21.15.99:0/334607901 <== mds.0 v2:172.21.15.99:6835/1210854172 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55e3fe95ffa0 con 0x55e3fea146b0 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f99497fa640 10 client.7476 handle_command_reply: tid=0 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 client.7476 shutdown 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 msgr2=0x55e3feab3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f995c0572d0 tx=0x7f995c089170 comp rx=0 tx=0).stop 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 shutdown_connections 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e3fea17540 0x55e3fea1faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:6835/1210854172,v1:172.21.15.99:6837/1210854172] conn(0x55e3fea146b0 0x55e3fea39df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e3fe9f7a50 0x55e3feab99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 --2- 172.21.15.99:0/334607901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e3fe9f62c0 0x55e3feab3060 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f99497fa640 0 client.7476 ms_handle_reset on v2:172.21.15.99:6835/1210854172 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 >> 172.21.15.99:0/334607901 conn(0x55e3fe9f3a80 msgr2=0x55e3fe9ebef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 shutdown_connections 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 1 -- 172.21.15.99:0/334607901 wait complete. 2024-09-10T11:36:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.919+0000 7f9973dbb740 20 client.7476 trim_cache size 0 max 16384 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7341, 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:36:20.928 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5475593480000001, 2024-09-10T11:36:20.929 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 61, 2024-09-10T11:36:20.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 153, 2024-09-10T11:36:20.929 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 153, 2024-09-10T11:36:20.929 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 8.9009286240000005 2024-09-10T11:36:20.929 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 msgr2=0x7f994c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 0x7f994c06e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f996c06eb00 tx=0x7f9954008040 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 msgr2=0x7f996c06e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f995c00f7c0 tx=0x7f995c00fc80 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 shutdown_connections 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f994c06be60 0x7f994c06e320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996c152f60 0x7f996c1473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f996c152ae0 0x7f996c146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 --2- 172.21.15.99:0/1423667008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9958004d40 0x7f996c06e1d0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 >> 172.21.15.99:0/1423667008 conn(0x7f996c06bc10 msgr2=0x7f996c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 shutdown_connections 2024-09-10T11:36:20.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:20.920+0000 7f9970f49640 1 -- 172.21.15.99:0/1423667008 wait complete. 2024-09-10T11:36:20.932 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:36:21.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.029+0000 7f64530d0640 1 Processor -- start 2024-09-10T11:36:21.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.029+0000 7f64530d0640 1 -- start start 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f64530d0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 0x7f644c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f64530d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f644c06e5d0 0x7f644c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f64530d0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644c152e60 con 0x7f644c15c060 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f64530d0640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f644c06ef80 con 0x7f644c06e5d0 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f64530d0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f644c0d15e0 con 0x7f644c0d1200 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f644bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f644c06e5d0 0x7f644c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f644bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f644c06e5d0 0x7f644c06ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34570/0 (socket says 172.21.15.99:34570) 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f644bfff640 1 -- 172.21.15.99:0/3360078771 learned_addr learned my addr 172.21.15.99:0/3360078771 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:21.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.030+0000 7f6450e6c640 1 --2- 172.21.15.99:0/3360078771 >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 0x7f644c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644bfff640 1 -- 172.21.15.99:0/3360078771 >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 msgr2=0x7f644c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644bfff640 1 --2- 172.21.15.99:0/3360078771 >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 0x7f644c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644bfff640 1 -- 172.21.15.99:0/3360078771 >> v1:172.21.15.196:6789/0 conn(0x7f644c15c060 legacy=0x7f644c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644bfff640 1 -- 172.21.15.99:0/3360078771 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c072680 con 0x7f644c06e5d0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f6450e6c640 1 --2- 172.21.15.99:0/3360078771 >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 0x7f644c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644bfff640 1 --2- 172.21.15.99:0/3360078771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06e5d0 0x7f644c06ea40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f644c0d1850 tx=0x7f644002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d3efd8050190a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644b7fe640 1 -- 172.21.15.99:0/3360078771 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440037070 con 0x7f644c06e5d0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644b7fe640 1 -- 172.21.15.99:0/3360078771 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6440032440 con 0x7f644c06e5d0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.031+0000 7f644b7fe640 1 -- 172.21.15.99:0/3360078771 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440032810 con 0x7f644c06e5d0 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 -- 172.21.15.99:0/3360078771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06e5d0 msgr2=0x7f644c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 --2- 172.21.15.99:0/3360078771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06e5d0 0x7f644c06ea40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f644c0d1850 tx=0x7f644002a1a0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 -- 172.21.15.99:0/3360078771 shutdown_connections 2024-09-10T11:36:21.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 --2- 172.21.15.99:0/3360078771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c06e5d0 0x7f644c06ea40 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 --2- 172.21.15.99:0/3360078771 >> v2:172.21.15.196:3300/0 conn(0x7f644c0d1200 0x7f644c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 -- 172.21.15.99:0/3360078771 >> 172.21.15.99:0/3360078771 conn(0x7f644c06bc10 msgr2=0x7f644c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:21.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 -- 172.21.15.99:0/3360078771 shutdown_connections 2024-09-10T11:36:21.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.032+0000 7f64530d0640 1 -- 172.21.15.99:0/3360078771 wait complete. 2024-09-10T11:36:21.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.033+0000 7f64530d0640 1 Processor -- start 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.033+0000 7f64530d0640 1 -- start start 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c06e5d0 0x7f644c14a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c15c060 0x7f644c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f644c072f20 con 0x7f644c0d1200 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f644c072c40 con 0x7f644c15c060 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f64530d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f644c15cd90 con 0x7f644c06e5d0 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34580/0 (socket says 172.21.15.99:34580) 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 -- 172.21.15.99:0/1731071649 learned_addr learned my addr 172.21.15.99:0/1731071649 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f6450e6c640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c06e5d0 0x7f644c14a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f645166d640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c15c060 0x7f644c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 -- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c06e5d0 msgr2=0x7f644c14a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:21.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c06e5d0 0x7f644c14a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 -- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c15c060 msgr2=0x7f644c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c15c060 0x7f644c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.034+0000 7f644bfff640 1 -- 172.21.15.99:0/1731071649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644c0b21d0 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f644bfff640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6440000c00 tx=0x7f64400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440041020 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6440030970 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440037040 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644c14ad10 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.035+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644c14b2d0 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.036+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644c06ea40 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.036+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6440044020 con 0x7f644c0d1200 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.037+0000 7f64497fa640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 0x7f643806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.037+0000 7f6450e6c640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 0x7f643806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:21.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.037+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f64400ab930 con 0x7f644c0d1200 2024-09-10T11:36:21.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.037+0000 7f6450e6c640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 0x7f643806e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f643c00a950 tx=0x7f643c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:21.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.041+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f644004b030 con 0x7f644c0d1200 2024-09-10T11:36:21.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.240+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f644c14e470 con 0x7f644c0d1200 2024-09-10T11:36:21.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.240+0000 7f64497fa640 1 -- 172.21.15.99:0/1731071649 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f644007a250 con 0x7f644c0d1200 2024-09-10T11:36:21.242 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:36:21.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 msgr2=0x7f643806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:21.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 0x7f643806e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f643c00a950 tx=0x7f643c008040 comp rx=0 tx=0).stop 2024-09-10T11:36:21.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 msgr2=0x7f644c14df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6440000c00 tx=0x7f64400046c0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 shutdown_connections 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f643806be30 0x7f643806e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f644c15c060 0x7f644c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f644c0d1200 0x7f644c14df30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 --2- 172.21.15.99:0/1731071649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f644c06e5d0 0x7f644c14a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 >> 172.21.15.99:0/1731071649 conn(0x7f644c06bc10 msgr2=0x7f644c06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 shutdown_connections 2024-09-10T11:36:21.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21.242+0000 7f64530d0640 1 -- 172.21.15.99:0/1731071649 wait complete. 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:36:21.255 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:36:21.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.256 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:36:21.271 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:21.272 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.273 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:21.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.274 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:36:21.335 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:36:21.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.337 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:36:21.393 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:36:21.393 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:36:21.393 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:36:21.393 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:36:21.393 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:36:21.393 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:36:21.393 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:36:21.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:36:21.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:36:21.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:36:21.394 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:36:21.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:36:21.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:36:21.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.519 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:36:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:36:21.534 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:36:21.535 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:36:21.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:36:21.599 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:36:21.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:36:21.599 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:21.655 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:36:21.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:36:21.655 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:21.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:36:21.775 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:36:21.831 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:36:21.831 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:36:21.832 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:36:21.833 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:36:21.834 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:36:21.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:21.835 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:36:21.890 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:36:21.891 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:36:21.933 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:36:21.958 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:36:22.015 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:36:22.016 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:36:22.017 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:36:22.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.018 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:36:22.018 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:36:22.015+0000 7f75b920e480 -1 init, newargv = 0x55d9258f0cb0 newargc=15 2024-09-10T11:36:22.019 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[58128]: starting ceph client 2024-09-10T11:36:22.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[58128]: starting fuse 2024-09-10T11:36:22.033 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:36:22.033 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:36:22.034 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:36:22.035 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:36:22.151 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.58128.asok 2024-09-10T11:36:22.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:36:22.160 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.58128.asok 2024-09-10T11:36:22.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.58128.asok status 2024-09-10T11:36:22.293 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:36:22.293 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:36:22.293 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:36:22.293 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "58128", 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:22.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7488, 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 7488 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:22.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2669503470 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2669503470 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.7488 192.168.0.1:0/2669503470", 2024-09-10T11:36:22.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2669503470", 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 61, 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 153, 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:36:22.297 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:36:22.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.302 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:36:22.302 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:22.317 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:36:22.318 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:22.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:36:22.366 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:36:22.364+0000 7f6770a3d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:36:22.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:36:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.518+0000 7fa40cb78640 1 Processor -- start 2024-09-10T11:36:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 -- start start 2024-09-10T11:36:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa40815eb80 0x7fa4080d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40815ff70 con 0x7fa4081626c0 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa4080d1620 con 0x7fa40815eb80 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.519+0000 7fa40cb78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa408162bc0 con 0x7fa4080d5010 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa406d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa406d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:52422/0 (socket says 172.21.15.99:52422) 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa406d9d640 1 -- 172.21.15.99:0/1552946703 learned_addr learned my addr 172.21.15.99:0/1552946703 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa40659c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa40815eb80 0x7fa4080d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa40659c640 1 -- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 msgr2=0x7fa4080d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa40659c640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa40659c640 1 -- 172.21.15.99:0/1552946703 >> v1:172.21.15.196:6789/0 conn(0x7fa4081626c0 legacy=0x7fa4080d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa40659c640 1 -- 172.21.15.99:0/1552946703 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa408162ea0 con 0x7fa40815eb80 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.520+0000 7fa406d9d640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.521+0000 7fa40659c640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815eb80 0x7fa4080d29b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097d0 tx=0x7fa40002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c8127a574b1477 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.521+0000 7fa40559a640 1 -- 172.21.15.99:0/1552946703 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400037070 con 0x7fa40815eb80 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.521+0000 7fa40559a640 1 -- 172.21.15.99:0/1552946703 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40002ae40 con 0x7fa40815eb80 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.521+0000 7fa40559a640 1 -- 172.21.15.99:0/1552946703 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa400032660 con 0x7fa40815eb80 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 -- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815eb80 msgr2=0x7fa4080d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815eb80 0x7fa4080d29b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4000097d0 tx=0x7fa40002a130 comp rx=0 tx=0).stop 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 -- 172.21.15.99:0/1552946703 shutdown_connections 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4080d5010 0x7fa4080d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 --2- 172.21.15.99:0/1552946703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40815eb80 0x7fa4080d29b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 -- 172.21.15.99:0/1552946703 >> 172.21.15.99:0/1552946703 conn(0x7fa40808e680 msgr2=0x7fa40809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 -- 172.21.15.99:0/1552946703 shutdown_connections 2024-09-10T11:36:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.522+0000 7fa40cb78640 1 -- 172.21.15.99:0/1552946703 wait complete. 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.523+0000 7fa40cb78640 1 Processor -- start 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.523+0000 7fa40cb78640 1 -- start start 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 0x7fa4080b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4081626c0 0x7fa40815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4080d2ef0 con 0x7fa40815eb80 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa408163460 con 0x7fa4080d5010 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40cb78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4080d3070 con 0x7fa4081626c0 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34590/0 (socket says 172.21.15.99:34590) 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 -- 172.21.15.99:0/630404709 learned_addr learned my addr 172.21.15.99:0/630404709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40659c640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 0x7fa4080b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa406d9d640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4081626c0 0x7fa40815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 -- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4081626c0 msgr2=0x7fa40815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4081626c0 0x7fa40815c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 -- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 msgr2=0x7fa4080b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 0x7fa4080b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa405d9b640 1 -- 172.21.15.99:0/630404709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40815cd90 con 0x7fa40815eb80 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.524+0000 7fa40659c640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 0x7fa4080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa405d9b640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c470 tx=0x7fa3f000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00052d0 con 0x7fa40815eb80 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005490 con 0x7fa40815eb80 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4080b1a10 con 0x7fa40815eb80 2024-09-10T11:36:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007260 con 0x7fa40815eb80 2024-09-10T11:36:22.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.525+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4080b1fd0 con 0x7fa40815eb80 2024-09-10T11:36:22.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.526+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4080d29b0 con 0x7fa40815eb80 2024-09-10T11:36:22.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.526+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3f0007400 con 0x7fa40815eb80 2024-09-10T11:36:22.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.527+0000 7fa3feffd640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 0x7fa3e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:22.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.527+0000 7fa40659c640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 0x7fa3e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:22.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.527+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7fa3f008a310 con 0x7fa40815eb80 2024-09-10T11:36:22.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.527+0000 7fa40659c640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 0x7fa3e806e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa400034000 tx=0x7fa40002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:22.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.531+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3f0058cc0 con 0x7fa40815eb80 2024-09-10T11:36:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.730+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa4080d0160 con 0x7fa40815eb80 2024-09-10T11:36:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.730+0000 7fa3feffd640 1 -- 172.21.15.99:0/630404709 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7fa3f00585c0 con 0x7fa40815eb80 2024-09-10T11:36:22.732 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 msgr2=0x7fa3e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 0x7fa3e806e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa400034000 tx=0x7fa40002e000 comp rx=0 tx=0).stop 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 msgr2=0x7fa408155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c470 tx=0x7fa3f000c930 comp rx=0 tx=0).stop 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 shutdown_connections 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa3e806be30 0x7fa3e806e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4081626c0 0x7fa40815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa40815eb80 0x7fa408155260 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 --2- 172.21.15.99:0/630404709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4080d5010 0x7fa4080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 >> 172.21.15.99:0/630404709 conn(0x7fa40808e680 msgr2=0x7fa40806ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 shutdown_connections 2024-09-10T11:36:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:22.732+0000 7fa40cb78640 1 -- 172.21.15.99:0/630404709 wait complete. 2024-09-10T11:36:22.744 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:36:22.744 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:36:22.745 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:36:22.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.745 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:36:22.765 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42482sec preferred_lft 42482sec 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.766 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:36:22.767 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:36:22.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.768 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:36:22.834 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:36:22.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:22.836 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:36:22.895 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:36:22.895 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:36:22.895 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:36:22.896 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:36:22.896 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:36:22.896 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:36:22.989 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:36:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:36:23.023 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:36:23.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.030 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:36:23.046 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:36:23.046 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:36:23.046 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:36:23.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.046 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:36:23.111 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:36:23.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:36:23.111 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:23.167 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:36:23.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:36:23.167 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:23.225 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:36:23.290 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:36:23.346 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:36:23.347 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:36:23.348 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:36:23.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.349 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:36:23.401 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:36:23.402 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:36:23.445 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:36:23.469 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:36:23.526 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:36:23.527 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:36:23.528 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:36:23.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.529 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:36:23.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:36:23.526+0000 7ff462a29480 -1 init, newargv = 0x55b3cdaadcd0 newargc=15 2024-09-10T11:36:23.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38886]: starting ceph client 2024-09-10T11:36:23.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38886]: starting fuse 2024-09-10T11:36:23.546 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:36:23.546 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:36:23.546 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:36:23.547 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:36:23.548 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:36:23.548 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:36:23.548 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:36:23.548 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:36:23.662 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.38886.asok 2024-09-10T11:36:23.665 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:36:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:36:23.673 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38886.asok 2024-09-10T11:36:23.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.674 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38886.asok status 2024-09-10T11:36:23.806 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:36:23.806 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:36:23.806 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "38886", 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 6086, 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:36:23.807 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 6086 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4131306680 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4131306680 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:36:23.808 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.6086 192.168.0.1:0/4131306680", 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/4131306680", 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 61, 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 153, 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:36:23.809 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:36:23.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.814 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:36:23.815 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:23.830 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:36:23.830 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:23.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:36:23.830 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:36:23.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:36:24.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.004+0000 7f55b0e65640 1 Processor -- start 2024-09-10T11:36:24.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.004+0000 7f55b0e65640 1 -- start start 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.005+0000 7f55b0e65640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.005+0000 7f55b0e65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.005+0000 7f55b0e65640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55ac070b70 con 0x7f55ac161160 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.005+0000 7f55b0e65640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f55ac160cd0 con 0x7f55ac15d5c0 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.005+0000 7f55b0e65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f55ac06c580 con 0x7f55ac06bc10 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aa59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aa59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:36500/0 (socket says 172.21.15.99:36500) 2024-09-10T11:36:24.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55a9d9b640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f55ac161160 0x7f55ac070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:39008/0 (socket says 172.21.15.99:39008) 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:39274/0 (socket says 172.21.15.99:39274) 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aa59c640 1 -- 172.21.15.99:0/2864187789 learned_addr learned my addr 172.21.15.99:0/2864187789 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55a959a640 1 -- 172.21.15.99:0/2864187789 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978004867 0 0) 0x7f55ac070b70 con 0x7f55ac161160 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55a959a640 1 -- 172.21.15.99:0/2864187789 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5598003520 con 0x7f55ac161160 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 -- 172.21.15.99:0/2864187789 >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 msgr2=0x7f55ac15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 --2- 172.21.15.99:0/2864187789 >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 -- 172.21.15.99:0/2864187789 >> v1:172.21.15.196:6789/0 conn(0x7f55ac161160 legacy=0x7f55ac070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aad9d640 1 -- 172.21.15.99:0/2864187789 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac070e00 con 0x7f55ac06bc10 2024-09-10T11:36:24.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.006+0000 7f55aa59c640 1 --2- 172.21.15.99:0/2864187789 >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.007+0000 7f55aad9d640 1 --2- 172.21.15.99:0/2864187789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f559c0097a0 tx=0x7f559c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be384319c6b41b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.007+0000 7f55a959a640 1 -- 172.21.15.99:0/2864187789 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c037070 con 0x7f55ac06bc10 2024-09-10T11:36:24.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.007+0000 7f55a959a640 1 -- 172.21.15.99:0/2864187789 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f559c032440 con 0x7f55ac06bc10 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.007+0000 7f55a959a640 1 -- 172.21.15.99:0/2864187789 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0327f0 con 0x7f55ac06bc10 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 -- 172.21.15.99:0/2864187789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 msgr2=0x7f55ac06c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2864187789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f559c0097a0 tx=0x7f559c02a180 comp rx=0 tx=0).stop 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 -- 172.21.15.99:0/2864187789 shutdown_connections 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2864187789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac06c040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2864187789 >> v2:172.21.15.196:3300/0 conn(0x7f55ac15d5c0 0x7f55ac15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 -- 172.21.15.99:0/2864187789 >> 172.21.15.99:0/2864187789 conn(0x7f55ac0d1900 msgr2=0x7f55ac0d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 -- 172.21.15.99:0/2864187789 shutdown_connections 2024-09-10T11:36:24.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.008+0000 7f55b0e65640 1 -- 172.21.15.99:0/2864187789 wait complete. 2024-09-10T11:36:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.009+0000 7f55b0e65640 1 Processor -- start 2024-09-10T11:36:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.009+0000 7f55b0e65640 1 -- start start 2024-09-10T11:36:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ac161160 0x7f55ac154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac06e660 con 0x7f55ac15d5c0 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f55ac160960 con 0x7f55ac161160 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55b0e65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f55ac06e7e0 con 0x7f55ac06bc10 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41866/0 (socket says 172.21.15.99:41866) 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 -- 172.21.15.99:0/2527367771 learned_addr learned my addr 172.21.15.99:0/2527367771 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55aad9d640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ac161160 0x7f55ac154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55aa59c640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 -- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 msgr2=0x7f55ac152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 -- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ac161160 msgr2=0x7f55ac154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ac161160 0x7f55ac154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.010+0000 7f55a9d9b640 1 -- 172.21.15.99:0/2527367771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55ac155550 con 0x7f55ac15d5c0 2024-09-10T11:36:24.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55a9d9b640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f559400c390 tx=0x7f559400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55940074e0 con 0x7f55ac15d5c0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55940047e0 con 0x7f55ac15d5c0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55ac155830 con 0x7f55ac15d5c0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594004bd0 con 0x7f55ac15d5c0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.011+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55ac13dea0 con 0x7f55ac15d5c0 2024-09-10T11:36:24.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.012+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55ac06c040 con 0x7f55ac15d5c0 2024-09-10T11:36:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.012+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f559401a020 con 0x7f55ac15d5c0 2024-09-10T11:36:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.013+0000 7f55a2ffd640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 0x7f558c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.013+0000 7f55aa59c640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 0x7f558c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.013+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f559408a690 con 0x7f55ac15d5c0 2024-09-10T11:36:24.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.013+0000 7f55aa59c640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 0x7f558c06e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55ac15ce80 tx=0x7f55a400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.017+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5594059100 con 0x7f55ac15d5c0 2024-09-10T11:36:24.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.216+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55ac070560 con 0x7f55ac15d5c0 2024-09-10T11:36:24.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.216+0000 7f55a2ffd640 1 -- 172.21.15.99:0/2527367771 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f5594058a00 con 0x7f55ac15d5c0 2024-09-10T11:36:24.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:36:24.218 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:36:24.219 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:36:24.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 msgr2=0x7f558c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 0x7f558c06e2f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f55ac15ce80 tx=0x7f55a400b040 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 msgr2=0x7f55ac152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f559400c390 tx=0x7f559400c850 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 shutdown_connections 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f558c06be30 0x7f558c06e2f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55ac161160 0x7f55ac154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55ac15d5c0 0x7f55ac152b70 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 --2- 172.21.15.99:0/2527367771 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55ac06bc10 0x7f55ac152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 >> 172.21.15.99:0/2527367771 conn(0x7f55ac0d1900 msgr2=0x7f55ac15db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 shutdown_connections 2024-09-10T11:36:24.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.218+0000 7f55b0e65640 1 -- 172.21.15.99:0/2527367771 wait complete. 2024-09-10T11:36:24.232 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:36:24.347 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":7341,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9656593200000003,"mdsmap_epoch":61,"osdmap_epoch":153,"osdmap_epoch_barrier":153,"uptime":12.319030857} 2024-09-10T11:36:24.348 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 7341, "mdsmap_epoch": 61, "osdmap_epoch": 153, "osdmap_epoch_barrier": 153, "rank_uptime": 7.96565932, "state": "up:active", "uptime": 12.319030857, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:36:24.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-09-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.451+0000 7f83a5111640 1 Processor -- start 2024-09-10T11:36:24.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f83a5111640 1 -- start start 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f83a5111640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83a00b59b0 0x7f83a00d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f83a5111640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a00d5b70 con 0x7f83a00d16f0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f83a5111640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a0163b30 con 0x7f83a00d5740 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f83a5111640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f83a00d0d30 con 0x7f83a00b59b0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f839ed9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f83a00d16f0 0x7f83a0162d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46412/0 (socket says 172.21.15.99:46412) 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.452+0000 7f839ed9d640 1 -- 172.21.15.99:0/1032386099 learned_addr learned my addr 172.21.15.99:0/1032386099 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430994578 0 0) 0x7f8390001110 con 0x7f83a00d16f0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8388003520 con 0x7f83a00d16f0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839e59c640 1 --2- 172.21.15.99:0/1032386099 >> v2:172.21.15.196:3300/0 conn(0x7f83a00b59b0 0x7f83a00d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3668815030 0 0) 0x7f8388003520 con 0x7f83a00d16f0 2024-09-10T11:36:24.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 >> v2:172.21.15.196:3300/0 conn(0x7f83a00b59b0 msgr2=0x7f83a00d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 --2- 172.21.15.99:0/1032386099 >> v2:172.21.15.196:3300/0 conn(0x7f83a00b59b0 0x7f83a00d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 >> v1:172.21.15.196:6789/0 conn(0x7f83a00d5740 legacy=0x7f83a0163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00d2150 con 0x7f83a00d16f0 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8390001840 con 0x7f83a00d16f0 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 >> v1:172.21.15.99:6789/0 conn(0x7f83a00d16f0 legacy=0x7f83a0162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 0x7f8388003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8388004800 0x7f8388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388005210 0x7f838800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00d0d30 con 0x7f8388003a00 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f838800c160 con 0x7f8388005210 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f838800c360 con 0x7f8388004800 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 0x7f8388003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839f59e640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388005210 0x7f838800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839e59c640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8388004800 0x7f8388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 -- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8388004800 msgr2=0x7f8388004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8388004800 0x7f8388004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 -- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388005210 msgr2=0x7f838800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388005210 0x7f838800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839ed9d640 1 -- 172.21.15.99:0/1032386099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8388003520 con 0x7f8388003a00 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.453+0000 7f839f59e640 1 -- 172.21.15.99:0/1032386099 reap_dead start 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.454+0000 7f839ed9d640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 0x7f8388003e20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8390001050 tx=0x7f8388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84a98f171d69489 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.454+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390036070 con 0x7f8388003a00 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.454+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8390005af0 con 0x7f8388003a00 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.454+0000 7f839dd9b640 1 -- 172.21.15.99:0/1032386099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390007e10 con 0x7f8388003a00 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 -- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 msgr2=0x7f8388003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 0x7f8388003e20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8390001050 tx=0x7f8388005830 comp rx=0 tx=0).stop 2024-09-10T11:36:24.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 -- 172.21.15.99:0/1032386099 shutdown_connections 2024-09-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 --2- 172.21.15.99:0/1032386099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8388003a00 0x7f8388003e20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 -- 172.21.15.99:0/1032386099 >> 172.21.15.99:0/1032386099 conn(0x7f83a00d4c20 msgr2=0x7f83a00d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 -- 172.21.15.99:0/1032386099 shutdown_connections 2024-09-10T11:36:24.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.455+0000 7f83a5111640 1 -- 172.21.15.99:0/1032386099 wait complete. 2024-09-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.456+0000 7f83a5111640 1 Processor -- start 2024-09-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.456+0000 7f83a5111640 1 -- start start 2024-09-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 0x7f83a00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83a00b2c00 0x7f83a00b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00b52f0 con 0x7f83a01570e0 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00b4d30 con 0x7f8388003a00 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f83a5111640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00d23f0 con 0x7f83a00b2c00 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41894/0 (socket says 172.21.15.99:41894) 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 -- 172.21.15.99:0/3501447701 learned_addr learned my addr 172.21.15.99:0/3501447701 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839f59e640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83a00b2c00 0x7f83a00b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839ed9d640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 0x7f83a00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 -- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83a00b2c00 msgr2=0x7f83a00b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83a00b2c00 0x7f83a00b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 -- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 msgr2=0x7f83a00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 0x7f83a00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839e59c640 1 -- 172.21.15.99:0/3501447701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a0157540 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.457+0000 7f839ed9d640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 0x7f83a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f839e59c640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f83940116d0 tx=0x7f8394011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400f9c0 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839400fb60 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394015460 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a00cccc0 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.458+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a00cd280 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.459+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00d1ad0 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.459+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8394015600 con 0x7f83a01570e0 2024-09-10T11:36:24.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.460+0000 7f83877fe640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 0x7f838c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.460+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f839408b680 con 0x7f83a01570e0 2024-09-10T11:36:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.460+0000 7f839ed9d640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 0x7f838c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.460+0000 7f839ed9d640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 0x7f838c06e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8390033000 tx=0x7f8390009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:24.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.464+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83a0157540 con 0x7f83a01570e0 2024-09-10T11:36:24.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:24.630+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f83a00ba860 con 0x7f83a01570e0 2024-09-10T11:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.421+0000 7f83877fe640 1 -- 172.21.15.99:0/3501447701 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v154) ==== 181+0+0 (secure 0 0 0) 0x7f8394019030 con 0x7f83a01570e0 2024-09-10T11:36:25.423 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 msgr2=0x7f838c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 0x7f838c06e320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8390033000 tx=0x7f8390009030 comp rx=0 tx=0).stop 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 msgr2=0x7f83a00b9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f83940116d0 tx=0x7f8394011b90 comp rx=0 tx=0).stop 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 shutdown_connections 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838c06be60 0x7f838c06e320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83a00b2c00 0x7f83a00b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:25.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f83a01570e0 0x7f83a00b9660 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 --2- 172.21.15.99:0/3501447701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8388003a00 0x7f83a00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 >> 172.21.15.99:0/3501447701 conn(0x7f83a00d4c20 msgr2=0x7f83a01610f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 shutdown_connections 2024-09-10T11:36:25.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:36:25.424+0000 7f83a5111640 1 -- 172.21.15.99:0/3501447701 wait complete. 2024-09-10T11:36:25.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:36:25.555 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:36:25.555 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:36:25.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-09-10T11:36:25.658 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2024-09-10T11:36:25.659 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-09-10T11:36:25.659 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> import time 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2024-09-10T11:36:25.659 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.660 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 0.9)): 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> print("done writing {0} bytes".format(bytes)) 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> # OK, now we should sneak in under the full condition 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> # due to the time it takes the OSDs to report to the 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> # mons, and get a successful fsync on our full-making data 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> # buffered write, add more dirty data to the buffer 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> print("starting buffered write") 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 0.2)): 2024-09-10T11:36:25.661 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> pass; 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> # Wait long enough for a background flush that should fail 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(30) 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> # ...and check that the failed background flush is reflected in fclose 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:36:25.662 DEBUG:teuthology.orchestra.run.smithi099:> except OSError: 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> print("close() returned an error as expected") 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("close() failed to raise error") 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> # The kernel cephfs client does not raise errors on fclose 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-09-10T11:36:25.663 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:36:29.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:36:29.153 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:36:59.208 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:36:59.212 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:37:29.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:37:29.364 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:37:59.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:37:59.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:38:12.213 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2024-09-10T11:38:12.213 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-09-10T11:38:12.214 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-09-10T11:38:12.215 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:done writing 29884416 bytes 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:successfully fsync'ed prior to getting full state reported 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:starting buffered write 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-09-10T11:38:12.216 INFO:teuthology.orchestra.run.smithi099.stdout:close() returned an error as expected 2024-09-10T11:38:12.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:38:12.227 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:38:12.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:38:12.275 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:38:12.318 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:38:12.319 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:38:12.319 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:38:12.319 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:12.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:38:12.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.481+0000 7f208f96d640 1 Processor -- start 2024-09-10T11:38:12.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.481+0000 7f208f96d640 1 -- start start 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208f96d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f208815eb80 0x7f20880d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208f96d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20880d5010 0x7f20880d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208f96d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f208815ff70 con 0x7f20881626c0 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208f96d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f20880d1620 con 0x7f20880d5010 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208f96d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2088162bc0 con 0x7f208815eb80 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208cf08640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f20881626c0 0x7f20880d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:43892/0 (socket says 172.21.15.99:43892) 2024-09-10T11:38:12.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208cf08640 1 -- 172.21.15.99:0/3498833080 learned_addr learned my addr 172.21.15.99:0/3498833080 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208df0a640 1 --2- 172.21.15.99:0/3498833080 >> v2:172.21.15.99:3300/0 conn(0x7f20880d5010 0x7f20880d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208d709640 1 --2- 172.21.15.99:0/3498833080 >> v2:172.21.15.196:3300/0 conn(0x7f208815eb80 0x7f20880d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f207bfff640 1 -- 172.21.15.99:0/3498833080 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692012650 0 0) 0x7f208815ff70 con 0x7f20881626c0 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f207bfff640 1 -- 172.21.15.99:0/3498833080 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2074003520 con 0x7f20881626c0 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208df0a640 1 -- 172.21.15.99:0/3498833080 >> v2:172.21.15.196:3300/0 conn(0x7f208815eb80 msgr2=0x7f20880d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208df0a640 1 --2- 172.21.15.99:0/3498833080 >> v2:172.21.15.196:3300/0 conn(0x7f208815eb80 0x7f20880d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208df0a640 1 -- 172.21.15.99:0/3498833080 >> v1:172.21.15.99:6789/0 conn(0x7f20881626c0 legacy=0x7f20880d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.482+0000 7f208df0a640 1 -- 172.21.15.99:0/3498833080 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2088162ea0 con 0x7f20880d5010 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208df0a640 1 --2- 172.21.15.99:0/3498833080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20880d5010 0x7f20880d1b20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2080009d90 tx=0x7f208002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d75164d6e69756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f207bfff640 1 -- 172.21.15.99:0/3498833080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080031600 con 0x7f20880d5010 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f207bfff640 1 -- 172.21.15.99:0/3498833080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20800317a0 con 0x7f20880d5010 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f207bfff640 1 -- 172.21.15.99:0/3498833080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080031b70 con 0x7f20880d5010 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 -- 172.21.15.99:0/3498833080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20880d5010 msgr2=0x7f20880d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 --2- 172.21.15.99:0/3498833080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20880d5010 0x7f20880d1b20 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2080009d90 tx=0x7f208002a610 comp rx=0 tx=0).stop 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 -- 172.21.15.99:0/3498833080 shutdown_connections 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 --2- 172.21.15.99:0/3498833080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20880d5010 0x7f20880d1b20 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 --2- 172.21.15.99:0/3498833080 >> v2:172.21.15.196:3300/0 conn(0x7f208815eb80 0x7f20880d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.483+0000 7f208f96d640 1 -- 172.21.15.99:0/3498833080 >> 172.21.15.99:0/3498833080 conn(0x7f208808e680 msgr2=0x7f208809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.484+0000 7f208f96d640 1 -- 172.21.15.99:0/3498833080 shutdown_connections 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.484+0000 7f208f96d640 1 -- 172.21.15.99:0/3498833080 wait complete. 2024-09-10T11:38:12.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 Processor -- start 2024-09-10T11:38:12.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 -- start start 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20880d5010 0x7f20880b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208815eb80 0x7f20881532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20880d2ef0 con 0x7f20881626c0 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2088163460 con 0x7f20880d5010 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.485+0000 7f208f96d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20880d3070 con 0x7f208815eb80 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:37282/0 (socket says 172.21.15.99:37282) 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 -- 172.21.15.99:0/3549310729 learned_addr learned my addr 172.21.15.99:0/3549310729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208d709640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20880d5010 0x7f20880b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208cf08640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208815eb80 0x7f20881532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 -- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208815eb80 msgr2=0x7f20881532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208815eb80 0x7f20881532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 -- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20880d5010 msgr2=0x7f20880b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20880d5010 0x7f20880b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20880b1e70 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208df0a640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2080009eb0 tx=0x7f2080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208003f070 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2080030b60 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20880b2150 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080009450 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.486+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20880b2710 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.487+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20880d29b0 con 0x7f20881626c0 2024-09-10T11:38:12.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.488+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f208003b410 con 0x7f20881626c0 2024-09-10T11:38:12.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.488+0000 7f2079ffb640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 0x7f206c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.489+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f20800ac3b0 con 0x7f20881626c0 2024-09-10T11:38:12.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.489+0000 7f208d709640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 0x7f206c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.489+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f206c073990 con 0x7f20881626c0 2024-09-10T11:38:12.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.489+0000 7f208d709640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 0x7f206c06e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.493+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f208007a7a0 con 0x7f20881626c0 2024-09-10T11:38:12.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.672+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f20880b3cd0 con 0x7f20881626c0 2024-09-10T11:38:12.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.672+0000 7f2079ffb640 1 -- 172.21.15.99:0/3549310729 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) ==== 80+0+0 (secure 0 0 0) 0x7f208007eaa0 con 0x7f20881626c0 2024-09-10T11:38:12.673 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:38:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 msgr2=0x7f206c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 0x7f206c06e320 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c008040 comp rx=0 tx=0).stop 2024-09-10T11:38:12.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 msgr2=0x7f20880ba1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2080009eb0 tx=0x7f2080004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 shutdown_connections 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f206c06be60 0x7f206c06e320 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20881626c0 0x7f20880ba1c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208815eb80 0x7f20881532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 --2- 172.21.15.99:0/3549310729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20880d5010 0x7f20880b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 >> 172.21.15.99:0/3549310729 conn(0x7f208808e680 msgr2=0x7f20880b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 shutdown_connections 2024-09-10T11:38:12.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.674+0000 7f208f96d640 1 -- 172.21.15.99:0/3549310729 wait complete. 2024-09-10T11:38:12.686 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:38:12.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:12.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:12.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[58128]: fuse finished with error 0 and tester_r 0 2024-09-10T11:38:12.711 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:38:12.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:12.711 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:38:12.768 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:38:12.768 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:38:12.768 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:12.788 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:38:12.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.904+0000 7f554b311640 1 Processor -- start 2024-09-10T11:38:12.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.904+0000 7f554b311640 1 -- start start 2024-09-10T11:38:12.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f554b311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f554415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f554b311640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5544152e60 con 0x7f55440d1200 2024-09-10T11:38:12.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f554b311640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55440d15e0 con 0x7f554406ece0 2024-09-10T11:38:12.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f554b311640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5544072680 con 0x7f5544073440 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f55490ad640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f55440d1200 0x7f554406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35292/0 (socket says 172.21.15.99:35292) 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f55490ad640 1 -- 172.21.15.99:0/3637734332 learned_addr learned my addr 172.21.15.99:0/3637734332 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3371469029 0 0) 0x7f5544152e60 con 0x7f55440d1200 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5530003520 con 0x7f55440d1200 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f55498ae640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f554415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 680264342 0 0) 0x7f5530003520 con 0x7f55440d1200 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 msgr2=0x7f554415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f554415c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 >> v1:172.21.15.196:6789/0 conn(0x7f554406ece0 legacy=0x7f554415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5544072960 con 0x7f55440d1200 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5540001670 con 0x7f55440d1200 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 >> v1:172.21.15.99:6789/0 conn(0x7f55440d1200 legacy=0x7f554406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55300039e0 0x7f5530003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f5530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f55300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5544072680 con 0x7f5530004d40 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f553000dd00 con 0x7f55300039e0 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f553000bac0 con 0x7f5544073440 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.905+0000 7f55498ae640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f55300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55490ad640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55300039e0 0x7f5530003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55488ac640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f5530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 -- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 msgr2=0x7f5530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:12.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5544073440 0x7f5530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 -- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55300039e0 msgr2=0x7f5530003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55300039e0 0x7f5530003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 -- 172.21.15.99:0/3637734332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530003520 con 0x7f5530004d40 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.906+0000 7f55498ae640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f55300051c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f553c009fc0 tx=0x7f5530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c506f4808323c93c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c009c10 con 0x7f5530004d40 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c036070 con 0x7f5530004d40 2024-09-10T11:38:12.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f553bfff640 1 -- 172.21.15.99:0/3637734332 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c031880 con 0x7f5530004d40 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 -- 172.21.15.99:0/3637734332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 msgr2=0x7f55300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 --2- 172.21.15.99:0/3637734332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f55300051c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f553c009fc0 tx=0x7f5530005830 comp rx=0 tx=0).stop 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f55498ae640 1 -- 172.21.15.99:0/3637734332 reap_dead start 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 -- 172.21.15.99:0/3637734332 shutdown_connections 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 -- 172.21.15.99:0/3637734332 >> 172.21.15.99:0/3637734332 conn(0x7f554406bc10 msgr2=0x7f554406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 -- 172.21.15.99:0/3637734332 shutdown_connections 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.907+0000 7f554b311640 1 -- 172.21.15.99:0/3637734332 wait complete. 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.908+0000 7f554b311640 1 Processor -- start 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.908+0000 7f554b311640 1 -- start start 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55441542b0 0x7f55441e5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f554413f090 0x7f55441551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f554415cd90 con 0x7f5530004d40 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5544072f20 con 0x7f55441542b0 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f554b311640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f554415c770 con 0x7f554413f090 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52752/0 (socket says 172.21.15.99:52752) 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 -- 172.21.15.99:0/2031415161 learned_addr learned my addr 172.21.15.99:0/2031415161 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55488ac640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55441542b0 0x7f55441e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 -- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f554413f090 msgr2=0x7f55441551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f554413f090 0x7f55441551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 -- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55441542b0 msgr2=0x7f55441e5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55441542b0 0x7f55441e5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.909+0000 7f55490ad640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5544153210 con 0x7f5530004d40 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f55490ad640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5540000fb0 tx=0x7f5540007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000f7b0 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5540018070 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55441534f0 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000fb90 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.910+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5544153ab0 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.911+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55440b42d0 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.911+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f554000fd30 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.912+0000 7f5539ffb640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 0x7f552806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.912+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f554008ae70 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.912+0000 7f55488ac640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 0x7f552806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.912+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f5528073990 con 0x7f5530004d40 2024-09-10T11:38:12.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.912+0000 7f55488ac640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 0x7f552806e320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5544069250 tx=0x7f5534009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:12.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:12.916+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5540059820 con 0x7f5530004d40 2024-09-10T11:38:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.094+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55440cceb0 con 0x7f5530004d40 2024-09-10T11:38:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.094+0000 7f5539ffb640 1 -- 172.21.15.99:0/2031415161 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v155) ==== 80+0+0 (secure 0 0 0) 0x7f5540059120 con 0x7f5530004d40 2024-09-10T11:38:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.096+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 msgr2=0x7f552806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.096+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 0x7f552806e320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5544069250 tx=0x7f5534009040 comp rx=0 tx=0).stop 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.096+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 msgr2=0x7f5544152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.096+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5540000fb0 tx=0x7f5540007720 comp rx=0 tx=0).stop 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 shutdown_connections 2024-09-10T11:38:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f552806be60 0x7f552806e320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f554413f090 0x7f55441551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55441542b0 0x7f55441e5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 --2- 172.21.15.99:0/2031415161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5530004d40 0x7f5544152c30 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 >> 172.21.15.99:0/2031415161 conn(0x7f554406bc10 msgr2=0x7f554406d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 shutdown_connections 2024-09-10T11:38:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.097+0000 7f554b311640 1 -- 172.21.15.99:0/2031415161 wait complete. 2024-09-10T11:38:13.108 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:38:13.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:13.109 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:13.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[38886]: fuse finished with error 0 and tester_r 0 2024-09-10T11:38:13.138 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:38:13.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:13.139 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:38:13.271 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:38:13.271 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:38:13.271 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:13.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:38:13.287 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:38:13.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:13.288 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:38:13.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:38:13.343 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:38:13.343 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:38:13.343 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:38:13.343 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:13.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:38:13.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:38:13.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:13.364 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:38:13.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:38:13.423 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:38:13.424 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:13.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.523+0000 7fd349c27640 1 Processor -- start 2024-09-10T11:38:13.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.523+0000 7fd349c27640 1 -- start start 2024-09-10T11:38:13.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd349c27640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd349c27640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 0x7fd34406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd349c27640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd344152e60 con 0x7fd34415c060 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd349c27640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd34406ef50 con 0x7fd34406e5d0 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd349c27640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd3440d15e0 con 0x7fd3440d1200 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd33bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 0x7fd34406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:54782/0 (socket says 172.21.15.99:54782) 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 -- 172.21.15.99:0/36795705 learned_addr learned my addr 172.21.15.99:0/36795705 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 -- 172.21.15.99:0/36795705 >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 msgr2=0x7fd34406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 --2- 172.21.15.99:0/36795705 >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 0x7fd34406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 -- 172.21.15.99:0/36795705 >> v1:172.21.15.196:6789/0 conn(0x7fd34415c060 legacy=0x7fd34415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd3437fe640 1 -- 172.21.15.99:0/36795705 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd344072680 con 0x7fd3440d1200 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd33bfff640 1 --2- 172.21.15.99:0/36795705 >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 0x7fd34406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:38:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.524+0000 7fd342ffd640 1 -- 172.21.15.99:0/36795705 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1648448678 0 0) 0x7fd344152e60 con 0x7fd34415c060 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.525+0000 7fd3437fe640 1 --2- 172.21.15.99:0/36795705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b3760523480bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.525+0000 7fd342ffd640 1 -- 172.21.15.99:0/36795705 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334037070 con 0x7fd3440d1200 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.525+0000 7fd342ffd640 1 -- 172.21.15.99:0/36795705 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd334032440 con 0x7fd3440d1200 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.525+0000 7fd342ffd640 1 -- 172.21.15.99:0/36795705 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334032810 con 0x7fd3440d1200 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 -- 172.21.15.99:0/36795705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 msgr2=0x7fd344073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 --2- 172.21.15.99:0/36795705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 -- 172.21.15.99:0/36795705 shutdown_connections 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 --2- 172.21.15.99:0/36795705 >> v2:172.21.15.196:3300/0 conn(0x7fd34406e5d0 0x7fd34406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 --2- 172.21.15.99:0/36795705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd344073440 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 -- 172.21.15.99:0/36795705 >> 172.21.15.99:0/36795705 conn(0x7fd34406bc10 msgr2=0x7fd34406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 -- 172.21.15.99:0/36795705 shutdown_connections 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.526+0000 7fd349c27640 1 -- 172.21.15.99:0/36795705 wait complete. 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.527+0000 7fd349c27640 1 Processor -- start 2024-09-10T11:38:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.527+0000 7fd349c27640 1 -- start start 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd34406e5d0 0x7fd344156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd34415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd344072f20 con 0x7fd34415c060 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd344072c40 con 0x7fd34406e5d0 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd349c27640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd34415cd90 con 0x7fd3440d1200 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52766/0 (socket says 172.21.15.99:52766) 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 -- 172.21.15.99:0/3814603279 learned_addr learned my addr 172.21.15.99:0/3814603279 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:13.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd33bfff640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd34415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd3437fe640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd34406e5d0 0x7fd344156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 -- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 msgr2=0x7fd34415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd34415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 -- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd34406e5d0 msgr2=0x7fd344156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd34406e5d0 0x7fd344156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd344154e10 con 0x7fd34415c060 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.528+0000 7fd343fff640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c790 tx=0x7fd33c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.529+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003da0 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.529+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd33c005070 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.529+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3441553d0 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.529+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005460 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.529+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd344155990 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.530+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd344155b90 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.530+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd33c005600 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.531+0000 7fd340ff9640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 0x7fd32406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.531+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7fd33c08aca0 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.531+0000 7fd3437fe640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 0x7fd32406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.531+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7fd3240739c0 con 0x7fd34415c060 2024-09-10T11:38:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.531+0000 7fd3437fe640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 0x7fd32406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd334034000 tx=0x7fd33402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.535+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd33c059000 con 0x7fd34415c060 2024-09-10T11:38:13.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.735+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3440b45f0 con 0x7fd34415c060 2024-09-10T11:38:13.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.735+0000 7fd340ff9640 1 -- 172.21.15.99:0/3814603279 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7fd33c05d300 con 0x7fd34415c060 2024-09-10T11:38:13.736 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:13.736 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:38:13.736 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 msgr2=0x7fd32406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 0x7fd32406e2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd334034000 tx=0x7fd33402e000 comp rx=0 tx=0).stop 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 msgr2=0x7fd344143d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c790 tx=0x7fd33c00cc50 comp rx=0 tx=0).stop 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 shutdown_connections 2024-09-10T11:38:13.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd32406be30 0x7fd32406e2f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd34415c060 0x7fd344143d70 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3440d1200 0x7fd34415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 --2- 172.21.15.99:0/3814603279 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd34406e5d0 0x7fd344156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 >> 172.21.15.99:0/3814603279 conn(0x7fd34406bc10 msgr2=0x7fd3440b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 shutdown_connections 2024-09-10T11:38:13.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.737+0000 7fd349c27640 1 -- 172.21.15.99:0/3814603279 wait complete. 2024-09-10T11:38:13.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:13.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.846+0000 7fdbcc78f640 1 Processor -- start 2024-09-10T11:38:13.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.846+0000 7fdbcc78f640 1 -- start start 2024-09-10T11:38:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcc78f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbc40d1200 0x7fdbc4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcc78f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc415c770 0x7fdbc406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcc78f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbc4152e60 con 0x7fdbc406e5d0 2024-09-10T11:38:13.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcc78f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdbc406ef80 con 0x7fdbc40d1200 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcc78f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc40d15e0 con 0x7fdbc415c770 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbc40d1200 0x7fdbc4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbc40d1200 0x7fdbc4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52782/0 (socket says 172.21.15.99:52782) 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 -- 172.21.15.99:0/1312790724 learned_addr learned my addr 172.21.15.99:0/1312790724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 -- 172.21.15.99:0/1312790724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc415c770 msgr2=0x7fdbc406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbcad2c640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc415c770 0x7fdbc406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc415c770 0x7fdbc406ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 -- 172.21.15.99:0/1312790724 >> v1:172.21.15.196:6789/0 conn(0x7fdbc406e5d0 legacy=0x7fdbc415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:13.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.847+0000 7fdbca52b640 1 -- 172.21.15.99:0/1312790724 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc4072680 con 0x7fdbc40d1200 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.848+0000 7fdbca52b640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc40d1200 0x7fdbc4073440 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009870 tx=0x7fdbc002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d05fafd932bfdda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.848+0000 7fdbc9529640 1 -- 172.21.15.99:0/1312790724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0037070 con 0x7fdbc40d1200 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.848+0000 7fdbc9529640 1 -- 172.21.15.99:0/1312790724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc0032440 con 0x7fdbc40d1200 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.848+0000 7fdbc9529640 1 -- 172.21.15.99:0/1312790724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0032810 con 0x7fdbc40d1200 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 -- 172.21.15.99:0/1312790724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc40d1200 msgr2=0x7fdbc4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc40d1200 0x7fdbc4073440 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009870 tx=0x7fdbc002a180 comp rx=0 tx=0).stop 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 -- 172.21.15.99:0/1312790724 shutdown_connections 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc415c770 0x7fdbc406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/1312790724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc40d1200 0x7fdbc4073440 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 -- 172.21.15.99:0/1312790724 >> 172.21.15.99:0/1312790724 conn(0x7fdbc406bc10 msgr2=0x7fdbc406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 -- 172.21.15.99:0/1312790724 shutdown_connections 2024-09-10T11:38:13.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.849+0000 7fdbcc78f640 1 -- 172.21.15.99:0/1312790724 wait complete. 2024-09-10T11:38:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.850+0000 7fdbcc78f640 1 Processor -- start 2024-09-10T11:38:13.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.850+0000 7fdbcc78f640 1 -- start start 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406e5d0 0x7fdbc4159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40d1200 0x7fdbc41571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4072f20 con 0x7fdbc415c770 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4072c40 con 0x7fdbc406e5d0 2024-09-10T11:38:13.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc415cd90 con 0x7fdbc40d1200 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52798/0 (socket says 172.21.15.99:52798) 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 -- 172.21.15.99:0/2914384069 learned_addr learned my addr 172.21.15.99:0/2914384069 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbca52b640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406e5d0 0x7fdbc4159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbc9d2a640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40d1200 0x7fdbc41571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 -- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40d1200 msgr2=0x7fdbc41571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40d1200 0x7fdbc41571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 -- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406e5d0 msgr2=0x7fdbc4159050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406e5d0 0x7fdbc4159050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc4167d10 con 0x7fdbc415c770 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcad2c640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cc30 tx=0x7fdbbc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00ce40 con 0x7fdbc415c770 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbbc003990 con 0x7fdbc415c770 2024-09-10T11:38:13.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc003d40 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc4167ff0 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.851+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc4159590 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.852+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4073440 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.853+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdbbc003ee0 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.853+0000 7fdbbaffd640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 0x7fdba806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.853+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc08b300 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.853+0000 7fdbca52b640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 0x7fdba806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.853+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7fdba80739f0 con 0x7fdbc415c770 2024-09-10T11:38:13.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.854+0000 7fdbca52b640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 0x7fdba806e320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0034000 tx=0x7fdbc002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:13.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:13.858+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdbbc014030 con 0x7fdbc415c770 2024-09-10T11:38:14.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.055+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbc4159980 con 0x7fdbc415c770 2024-09-10T11:38:14.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.056+0000 7fdbbaffd640 1 -- 172.21.15.99:0/2914384069 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7fdbbc0595d0 con 0x7fdbc415c770 2024-09-10T11:38:14.056 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:14.057 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:38:14.057 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 msgr2=0x7fdba806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 0x7fdba806e320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0034000 tx=0x7fdbc002e000 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 msgr2=0x7fdbc4151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cc30 tx=0x7fdbbc00a5c0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 shutdown_connections 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdba806be60 0x7fdba806e320 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbc415c770 0x7fdbc4151140 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbc40d1200 0x7fdbc41571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 --2- 172.21.15.99:0/2914384069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbc406e5d0 0x7fdbc4159050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 >> 172.21.15.99:0/2914384069 conn(0x7fdbc406bc10 msgr2=0x7fdbc415e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 shutdown_connections 2024-09-10T11:38:14.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.058+0000 7fdbcc78f640 1 -- 172.21.15.99:0/2914384069 wait complete. 2024-09-10T11:38:14.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:14.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.167+0000 7fd1d93d8640 1 Processor -- start 2024-09-10T11:38:14.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.167+0000 7fd1d93d8640 1 -- start start 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d93d8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd1d415eb80 0x7fd1d40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d93d8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd1d41626c0 0x7fd1d40d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d93d8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d415ff70 con 0x7fd1d40d4e40 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d93d8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d40d1620 con 0x7fd1d41626c0 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d93d8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d4162bc0 con 0x7fd1d415eb80 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd1d41626c0 0x7fd1d40d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d37fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd1d40d4e40 0x7fd1d40d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35304/0 (socket says 172.21.15.99:35304) 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d37fe640 1 -- 172.21.15.99:0/1095762683 learned_addr learned my addr 172.21.15.99:0/1095762683 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd1d41626c0 0x7fd1d40d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52806/0 (socket says 172.21.15.99:52806) 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1095762683 >> v2:172.21.15.196:3300/0 conn(0x7fd1d415eb80 msgr2=0x7fd1d40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 --2- 172.21.15.99:0/1095762683 >> v2:172.21.15.196:3300/0 conn(0x7fd1d415eb80 0x7fd1d40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1095762683 >> v1:172.21.15.99:6789/0 conn(0x7fd1d40d4e40 legacy=0x7fd1d40d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.168+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1095762683 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d4162ea0 con 0x7fd1d41626c0 2024-09-10T11:38:14.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d27fc640 1 --2- 172.21.15.99:0/1095762683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d41626c0 0x7fd1d40d4900 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097a0 tx=0x7fd1c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde7d71e281bec5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d1ffb640 1 -- 172.21.15.99:0/1095762683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8037070 con 0x7fd1d41626c0 2024-09-10T11:38:14.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d1ffb640 1 -- 172.21.15.99:0/1095762683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c802ae50 con 0x7fd1d41626c0 2024-09-10T11:38:14.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d1ffb640 1 -- 172.21.15.99:0/1095762683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8032660 con 0x7fd1d41626c0 2024-09-10T11:38:14.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1095762683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d41626c0 msgr2=0x7fd1d40d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.169+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1095762683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d41626c0 0x7fd1d40d4900 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80097a0 tx=0x7fd1c802a140 comp rx=0 tx=0).stop 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1095762683 shutdown_connections 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1095762683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d41626c0 0x7fd1d40d4900 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1095762683 >> v2:172.21.15.196:3300/0 conn(0x7fd1d415eb80 0x7fd1d40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1095762683 >> 172.21.15.99:0/1095762683 conn(0x7fd1d408e680 msgr2=0x7fd1d409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1095762683 shutdown_connections 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.170+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1095762683 wait complete. 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.171+0000 7fd1d93d8640 1 Processor -- start 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.171+0000 7fd1d93d8640 1 -- start start 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 0x7fd1d4156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1d41626c0 0x7fd1d4161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d40d5290 con 0x7fd1d415eb80 2024-09-10T11:38:14.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4163460 con 0x7fd1d41626c0 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d40d2ef0 con 0x7fd1d40d4e40 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52820/0 (socket says 172.21.15.99:52820) 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1051725746 learned_addr learned my addr 172.21.15.99:0/1051725746 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d2ffd640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 0x7fd1d4156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d37fe640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1d41626c0 0x7fd1d4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 msgr2=0x7fd1d4156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 0x7fd1d4156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1d41626c0 msgr2=0x7fd1d4161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1d41626c0 0x7fd1d4161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d41f5f10 con 0x7fd1d415eb80 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d2ffd640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 0x7fd1d4156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d27fc640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8035040 tx=0x7fd1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8042020 con 0x7fd1d415eb80 2024-09-10T11:38:14.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c803d6b0 con 0x7fd1d415eb80 2024-09-10T11:38:14.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d41f8820 con 0x7fd1d415eb80 2024-09-10T11:38:14.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8037040 con 0x7fd1d415eb80 2024-09-10T11:38:14.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.172+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41f8de0 con 0x7fd1d415eb80 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.173+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d40d29b0 con 0x7fd1d415eb80 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.174+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd1c8045020 con 0x7fd1d415eb80 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.174+0000 7fd1bb7fe640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.174+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7fd1c80ab720 con 0x7fd1d415eb80 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.174+0000 7fd1d2ffd640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.174+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7fd1c00739c0 con 0x7fd1d415eb80 2024-09-10T11:38:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.175+0000 7fd1d2ffd640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 0x7fd1c006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009890 tx=0x7fd1c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.178+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd1c804c030 con 0x7fd1d415eb80 2024-09-10T11:38:14.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.342+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1d40d0e60 con 0x7fd1d415eb80 2024-09-10T11:38:14.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.343+0000 7fd1bb7fe640 1 -- 172.21.15.99:0/1051725746 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16714 (secure 0 0 0) 0x7fd1c807a0e0 con 0x7fd1d415eb80 2024-09-10T11:38:14.344 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:14.344 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":155,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:37:25.693850+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:36:14.040327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T11:36:14.680742+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 msgr2=0x7fd1c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 0x7fd1c006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009890 tx=0x7fd1c4008040 comp rx=0 tx=0).stop 2024-09-10T11:38:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 msgr2=0x7fd1d41611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8035040 tx=0x7fd1c80046c0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 shutdown_connections 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd1c006be30 0x7fd1c006e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1d41626c0 0x7fd1d4161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd1d415eb80 0x7fd1d41611c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 --2- 172.21.15.99:0/1051725746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1d40d4e40 0x7fd1d4156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 >> 172.21.15.99:0/1051725746 conn(0x7fd1d408e680 msgr2=0x7fd1d40b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 shutdown_connections 2024-09-10T11:38:14.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.345+0000 7fd1d93d8640 1 -- 172.21.15.99:0/1051725746 wait complete. 2024-09-10T11:38:14.357 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:38:14.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:14.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.457+0000 7f24ea7c4640 1 Processor -- start 2024-09-10T11:38:14.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.457+0000 7f24ea7c4640 1 -- start start 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24ea7c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 0x7f24e40b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24ea7c4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24e40b00c0 0x7f24e4071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24ea7c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24e40b4ed0 con 0x7f24e40721f0 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24ea7c4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f24e40b5d20 con 0x7f24e40b00c0 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24ea7c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f24e40b3e30 con 0x7f24e415c950 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24e40b00c0 0x7f24e4071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f24e40b00c0 0x7f24e4071cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52828/0 (socket says 172.21.15.99:52828) 2024-09-10T11:38:14.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 -- 172.21.15.99:0/4266696070 learned_addr learned my addr 172.21.15.99:0/4266696070 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e3fff640 1 --2- 172.21.15.99:0/4266696070 >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 0x7f24e40b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 -- 172.21.15.99:0/4266696070 >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 msgr2=0x7f24e40b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 --2- 172.21.15.99:0/4266696070 >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 0x7f24e40b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 -- 172.21.15.99:0/4266696070 >> v1:172.21.15.196:6789/0 conn(0x7f24e40721f0 legacy=0x7f24e40ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e37fe640 1 -- 172.21.15.99:0/4266696070 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e40b4110 con 0x7f24e40b00c0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.458+0000 7f24e3fff640 1 --2- 172.21.15.99:0/4266696070 >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 0x7f24e40b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.459+0000 7f24e37fe640 1 --2- 172.21.15.99:0/4266696070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40b00c0 0x7f24e4071cb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f24e40d1620 tx=0x7f24d802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c3f7653240e96b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.459+0000 7f24e2ffd640 1 -- 172.21.15.99:0/4266696070 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8037070 con 0x7f24e40b00c0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.459+0000 7f24e2ffd640 1 -- 172.21.15.99:0/4266696070 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d802ae50 con 0x7f24e40b00c0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.459+0000 7f24e2ffd640 1 -- 172.21.15.99:0/4266696070 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8032860 con 0x7f24e40b00c0 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 -- 172.21.15.99:0/4266696070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40b00c0 msgr2=0x7f24e4071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/4266696070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40b00c0 0x7f24e4071cb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f24e40d1620 tx=0x7f24d802a300 comp rx=0 tx=0).stop 2024-09-10T11:38:14.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 -- 172.21.15.99:0/4266696070 shutdown_connections 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/4266696070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40b00c0 0x7f24e4071cb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/4266696070 >> v2:172.21.15.196:3300/0 conn(0x7f24e415c950 0x7f24e40b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 -- 172.21.15.99:0/4266696070 >> 172.21.15.99:0/4266696070 conn(0x7f24e406bc10 msgr2=0x7f24e406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 -- 172.21.15.99:0/4266696070 shutdown_connections 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.460+0000 7f24ea7c4640 1 -- 172.21.15.99:0/4266696070 wait complete. 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.461+0000 7f24ea7c4640 1 Processor -- start 2024-09-10T11:38:14.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.461+0000 7f24ea7c4640 1 -- start start 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 0x7f24e41579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 0x7f24e41f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40b49b0 con 0x7f24e40721f0 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f24e40b46d0 con 0x7f24e4157f10 2024-09-10T11:38:14.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24ea7c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f24e40b3a50 con 0x7f24e415c950 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52844/0 (socket says 172.21.15.99:52844) 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 -- 172.21.15.99:0/1413496840 learned_addr learned my addr 172.21.15.99:0/1413496840 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e8d61640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 0x7f24e41f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e37fe640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 0x7f24e41579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 -- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 msgr2=0x7f24e41579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 0x7f24e41579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 -- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 msgr2=0x7f24e41f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 0x7f24e41f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e41f5350 con 0x7f24e40721f0 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e37fe640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 0x7f24e41579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e8d61640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 0x7f24e41f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:14.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.462+0000 7f24e3fff640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e7e0 tx=0x7f24d400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.463+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d400f040 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.463+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24d4004030 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.463+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4004400 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.463+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e41f5630 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.463+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e4160750 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.464+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e4071cb0 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.464+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24d400c6d0 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.465+0000 7f24e0ff9640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 0x7f24d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.465+0000 7f24e37fe640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 0x7f24d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.465+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f24d408ac40 con 0x7f24e40721f0 2024-09-10T11:38:14.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.465+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f24d0073960 con 0x7f24e40721f0 2024-09-10T11:38:14.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.465+0000 7f24e37fe640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 0x7f24d006e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24d8034000 tx=0x7f24d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.469+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f24d4017020 con 0x7f24e40721f0 2024-09-10T11:38:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.666+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24e40b59e0 con 0x7f24e40721f0 2024-09-10T11:38:14.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.666+0000 7f24e0ff9640 1 -- 172.21.15.99:0/1413496840 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f24d40595f0 con 0x7f24e40721f0 2024-09-10T11:38:14.668 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:14.668 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:38:14.668 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:38:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 msgr2=0x7f24d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 0x7f24d006e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f24d8034000 tx=0x7f24d802e000 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 msgr2=0x7f24e4158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f24d400e7e0 tx=0x7f24d400eca0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 shutdown_connections 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f24d006be30 0x7f24d006e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24e4157f10 0x7f24e41f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24e415c950 0x7f24e41579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 --2- 172.21.15.99:0/1413496840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24e40721f0 0x7f24e4158ce0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.668+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 >> 172.21.15.99:0/1413496840 conn(0x7f24e406bc10 msgr2=0x7f24e40ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.669+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 shutdown_connections 2024-09-10T11:38:14.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.669+0000 7f24ea7c4640 1 -- 172.21.15.99:0/1413496840 wait complete. 2024-09-10T11:38:14.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.777+0000 7f4b192ba640 1 Processor -- start 2024-09-10T11:38:14.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.777+0000 7f4b192ba640 1 -- start start 2024-09-10T11:38:14.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b140d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 0x7f4b140d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b140d4e40 0x7f4b140d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4b140d52d0 con 0x7f4b140d4e40 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4b140d1620 con 0x7f4b141626c0 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b192ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b14162bc0 con 0x7f4b1415eb80 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b140d4e40 0x7f4b140d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b140d4e40 0x7f4b140d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52854/0 (socket says 172.21.15.99:52854) 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 -- 172.21.15.99:0/1099651478 learned_addr learned my addr 172.21.15.99:0/1099651478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b12ffd640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b140d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b0a7fc640 1 --2- 172.21.15.99:0/1099651478 >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 0x7f4b140d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 -- 172.21.15.99:0/1099651478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 msgr2=0x7f4b140d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b140d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 -- 172.21.15.99:0/1099651478 >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 msgr2=0x7f4b140d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1099651478 >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 0x7f4b140d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b137fe640 1 -- 172.21.15.99:0/1099651478 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b14162ea0 con 0x7f4b140d4e40 2024-09-10T11:38:14.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.778+0000 7f4b0a7fc640 1 --2- 172.21.15.99:0/1099651478 >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 0x7f4b140d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.779+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b140d4e40 0x7f4b140d1b20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4b1408db70 tx=0x7f4b0c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7f56d2263f9ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.779+0000 7f4b127fc640 1 -- 172.21.15.99:0/1099651478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c036070 con 0x7f4b140d4e40 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.779+0000 7f4b127fc640 1 -- 172.21.15.99:0/1099651478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c031600 con 0x7f4b140d4e40 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.779+0000 7f4b127fc640 1 -- 172.21.15.99:0/1099651478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c0319d0 con 0x7f4b140d4e40 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 -- 172.21.15.99:0/1099651478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b140d4e40 msgr2=0x7f4b140d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b140d4e40 0x7f4b140d1b20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4b1408db70 tx=0x7f4b0c02a610 comp rx=0 tx=0).stop 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 -- 172.21.15.99:0/1099651478 shutdown_connections 2024-09-10T11:38:14.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b140d4e40 0x7f4b140d1b20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1099651478 >> v2:172.21.15.196:3300/0 conn(0x7f4b141626c0 0x7f4b140d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1099651478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b140d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 -- 172.21.15.99:0/1099651478 >> 172.21.15.99:0/1099651478 conn(0x7f4b1408e680 msgr2=0x7f4b1409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 -- 172.21.15.99:0/1099651478 shutdown_connections 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.780+0000 7f4b192ba640 1 -- 172.21.15.99:0/1099651478 wait complete. 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.781+0000 7f4b192ba640 1 Processor -- start 2024-09-10T11:38:14.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.781+0000 7f4b192ba640 1 -- start start 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b140d4e40 0x7f4b140cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b1415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b140d2ef0 con 0x7f4b141626c0 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b14163460 con 0x7f4b140d4e40 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b192ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b140d3070 con 0x7f4b1415eb80 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52860/0 (socket says 172.21.15.99:52860) 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 -- 172.21.15.99:0/1627706676 learned_addr learned my addr 172.21.15.99:0/1627706676 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b0a7fc640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b1415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b12ffd640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b140d4e40 0x7f4b140cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 -- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 msgr2=0x7f4b1415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b1415d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 -- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b140d4e40 msgr2=0x7f4b140cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b140d4e40 0x7f4b140cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b137fe640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1415d9b0 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.782+0000 7f4b0a7fc640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b1415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b137fe640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034040 tx=0x7f4b0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c041020 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0077a0 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b140b48f0 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c036040 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.783+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b140b4eb0 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.784+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b140d29b0 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.784+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b0c044020 con 0x7f4b141626c0 2024-09-10T11:38:14.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.785+0000 7f4b0bfff640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 0x7f4af406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.785+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f4b0c0abad0 con 0x7f4b141626c0 2024-09-10T11:38:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.785+0000 7f4b12ffd640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 0x7f4af406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.785+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f4af4073960 con 0x7f4b141626c0 2024-09-10T11:38:14.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.785+0000 7f4b12ffd640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 0x7f4af406e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009970 tx=0x7f4b04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:14.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.789+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b0c04b030 con 0x7f4b141626c0 2024-09-10T11:38:14.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.987+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b140ceba0 con 0x7f4b141626c0 2024-09-10T11:38:14.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.987+0000 7f4b0bfff640 1 -- 172.21.15.99:0/1627706676 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f4b0c07a480 con 0x7f4b141626c0 2024-09-10T11:38:14.988 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:14.988 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":61,"btime":"2024-09-10T11:36:18:387289+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:36:15.712585+0000","modified":"2024-09-10T11:36:17.711254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7341},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7341":{"gid":7341,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/1210854172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":1210854172},{"type":"v1","addr":"172.21.15.99:6837","nonce":1210854172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7341,"qdb_cluster":[7341]},"id":9}]} 2024-09-10T11:38:14.989 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 61 2024-09-10T11:38:14.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 msgr2=0x7f4af406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 0x7f4af406e2f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009970 tx=0x7f4b04008040 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 msgr2=0x7f4b140b5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c034040 tx=0x7f4b0c004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 shutdown_connections 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4af406be30 0x7f4af406e2f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.989+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b141626c0 0x7f4b140b5990 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.990+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b1415eb80 0x7f4b1415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.990+0000 7f4b192ba640 1 --2- 172.21.15.99:0/1627706676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b140d4e40 0x7f4b140cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.990+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 >> 172.21.15.99:0/1627706676 conn(0x7f4b1408e680 msgr2=0x7f4b140d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.990+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 shutdown_connections 2024-09-10T11:38:14.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:14.990+0000 7f4b192ba640 1 -- 172.21.15.99:0/1627706676 wait complete. 2024-09-10T11:38:15.001 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.098+0000 7f5cfcaf1640 1 Processor -- start 2024-09-10T11:38:15.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.098+0000 7f5cfcaf1640 1 -- start start 2024-09-10T11:38:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cfcaf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cfcaf1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf8152e60 con 0x7f5cf815c060 2024-09-10T11:38:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cfcaf1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf80d15e0 con 0x7f5cf80d1200 2024-09-10T11:38:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cfcaf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf8072680 con 0x7f5cf806ece0 2024-09-10T11:38:15.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf6d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5cf815c060 0x7f5cf815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35308/0 (socket says 172.21.15.99:35308) 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf6d9d640 1 -- 172.21.15.99:0/3799778389 learned_addr learned my addr 172.21.15.99:0/3799778389 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf5d9b640 1 --2- 172.21.15.99:0/3799778389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3530388644 0 0) 0x7f5cf8152e60 con 0x7f5cf815c060 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ce0003520 con 0x7f5cf815c060 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3715480236 0 0) 0x7f5ce0003520 con 0x7f5cf815c060 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.099+0000 7f5cf5d9b640 1 -- 172.21.15.99:0/3799778389 >> v1:172.21.15.196:6789/0 conn(0x7f5cf80d1200 legacy=0x7f5cf806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.100+0000 7f5cf5d9b640 1 -- 172.21.15.99:0/3799778389 >> v1:172.21.15.99:6789/0 conn(0x7f5cf815c060 legacy=0x7f5cf815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.100+0000 7f5cf5d9b640 1 -- 172.21.15.99:0/3799778389 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf8072960 con 0x7f5cf806ece0 2024-09-10T11:38:15.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.100+0000 7f5cf5d9b640 1 --2- 172.21.15.99:0/3799778389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf8073440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0097d0 tx=0x7f5cec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d097d6cbfed1e16c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.100+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec037070 con 0x7f5cf806ece0 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cec02ab90 con 0x7f5cf806ece0 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cf559a640 1 -- 172.21.15.99:0/3799778389 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00019f0 con 0x7f5cf806ece0 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3799778389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 msgr2=0x7f5cf8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3799778389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf8073440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0097d0 tx=0x7f5cec02a100 comp rx=0 tx=0).stop 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3799778389 shutdown_connections 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3799778389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf8073440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3799778389 >> 172.21.15.99:0/3799778389 conn(0x7f5cf806bc10 msgr2=0x7f5cf806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3799778389 shutdown_connections 2024-09-10T11:38:15.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.101+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3799778389 wait complete. 2024-09-10T11:38:15.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.102+0000 7f5cfcaf1640 1 Processor -- start 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.102+0000 7f5cfcaf1640 1 -- start start 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf81521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf80d1200 0x7f5cf8157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf815cd90 con 0x7f5cf815c060 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8072f20 con 0x7f5cf80d1200 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cfcaf1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf8159b50 con 0x7f5cf806ece0 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cf6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52870/0 (socket says 172.21.15.99:52870) 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cf6d9d640 1 -- 172.21.15.99:0/3353761927 learned_addr learned my addr 172.21.15.99:0/3353761927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cf5d9b640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf80d1200 0x7f5cf8157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.103+0000 7f5cf659c640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf81521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 -- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 msgr2=0x7f5cf81521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf81521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 -- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf80d1200 msgr2=0x7f5cf8157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf80d1200 0x7f5cf8157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf8157940 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cf6d9d640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000cc00 tx=0x7f5cf000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0017070 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf000ce10 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf8157c20 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0012600 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.104+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81581e0 con 0x7f5cf815c060 2024-09-10T11:38:15.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.105+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8073440 con 0x7f5cf815c060 2024-09-10T11:38:15.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.105+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cf001a070 con 0x7f5cf815c060 2024-09-10T11:38:15.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.106+0000 7f5ce6ffd640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 0x7f5cd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.106+0000 7f5cf659c640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 0x7f5cd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.106+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f5cf008ac60 con 0x7f5cf815c060 2024-09-10T11:38:15.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.106+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f5cd8073960 con 0x7f5cf815c060 2024-09-10T11:38:15.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.107+0000 7f5cf659c640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 0x7f5cd806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000f60 tx=0x7f5ce8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.110+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5cf0058fc0 con 0x7f5cf815c060 2024-09-10T11:38:15.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.278+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cf806c470 con 0x7f5cf815c060 2024-09-10T11:38:15.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.279+0000 7f5ce6ffd640 1 -- 172.21.15.99:0/3353761927 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16714 (secure 0 0 0) 0x7f5cf005d2c0 con 0x7f5cf815c060 2024-09-10T11:38:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":155,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:37:25.693850+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:36:14.040327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-10T11:36:14.680742+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 msgr2=0x7f5cd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 0x7f5cd806e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8000f60 tx=0x7f5ce8009040 comp rx=0 tx=0).stop 2024-09-10T11:38:15.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 msgr2=0x7f5cf80ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000cc00 tx=0x7f5cf000a430 comp rx=0 tx=0).stop 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 shutdown_connections 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5cd806be30 0x7f5cd806e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cf815c060 0x7f5cf80ad0a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf80d1200 0x7f5cf8157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 --2- 172.21.15.99:0/3353761927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5cf806ece0 0x7f5cf81521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 >> 172.21.15.99:0/3353761927 conn(0x7f5cf806bc10 msgr2=0x7f5cf8159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 shutdown_connections 2024-09-10T11:38:15.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.281+0000 7f5cfcaf1640 1 -- 172.21.15.99:0/3353761927 wait complete. 2024-09-10T11:38:15.294 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:38:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.393+0000 7f6b570dd640 1 Processor -- start 2024-09-10T11:38:15.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.393+0000 7f6b570dd640 1 -- start start 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b570dd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b5015c7b0 0x7f6b500d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b570dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 0x7f6b5006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b570dd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b500ceb10 con 0x7f6b5006ee30 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b570dd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6b500d1420 con 0x7f6b5015c7b0 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b570dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b5006f2a0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b5567a640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6b5006ee30 0x7f6b501604b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35320/0 (socket says 172.21.15.99:35320) 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b5567a640 1 -- 172.21.15.99:0/3242399886 learned_addr learned my addr 172.21.15.99:0/3242399886 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:15.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b47fff640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 0x7f6b5006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b54e79640 1 --2- 172.21.15.99:0/3242399886 >> v2:172.21.15.196:3300/0 conn(0x7f6b5015c7b0 0x7f6b500d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b477fe640 1 -- 172.21.15.99:0/3242399886 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432269269 0 0) 0x7f6b48001490 con 0x7f6b5006ee30 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b477fe640 1 -- 172.21.15.99:0/3242399886 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b38003520 con 0x7f6b5006ee30 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b54e79640 1 -- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 msgr2=0x7f6b5006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b54e79640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 0x7f6b5006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b54e79640 1 -- 172.21.15.99:0/3242399886 >> v1:172.21.15.99:6789/0 conn(0x7f6b5006ee30 legacy=0x7f6b501604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b54e79640 1 -- 172.21.15.99:0/3242399886 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b500b3a90 con 0x7f6b5015c7b0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.394+0000 7f6b47fff640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 0x7f6b5006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.395+0000 7f6b54e79640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b500d0a70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adb9b2d43e497dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.395+0000 7f6b477fe640 1 -- 172.21.15.99:0/3242399886 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c037070 con 0x7f6b5015c7b0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.395+0000 7f6b477fe640 1 -- 172.21.15.99:0/3242399886 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4c032440 con 0x7f6b5015c7b0 2024-09-10T11:38:15.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.395+0000 7f6b477fe640 1 -- 172.21.15.99:0/3242399886 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c032810 con 0x7f6b5015c7b0 2024-09-10T11:38:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 -- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 msgr2=0x7f6b500d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b500d0a70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02a180 comp rx=0 tx=0).stop 2024-09-10T11:38:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 -- 172.21.15.99:0/3242399886 shutdown_connections 2024-09-10T11:38:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500d0fb0 0x7f6b5006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 --2- 172.21.15.99:0/3242399886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b500d0a70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 -- 172.21.15.99:0/3242399886 >> 172.21.15.99:0/3242399886 conn(0x7f6b5006bc10 msgr2=0x7f6b5006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 -- 172.21.15.99:0/3242399886 shutdown_connections 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.396+0000 7f6b570dd640 1 -- 172.21.15.99:0/3242399886 wait complete. 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.397+0000 7f6b570dd640 1 Processor -- start 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.397+0000 7f6b570dd640 1 -- start start 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5006ee30 0x7f6b50142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b50150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b500b4050 con 0x7f6b500d0fb0 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b500b3840 con 0x7f6b5015c7b0 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b570dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b500b4330 con 0x7f6b5006ee30 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52886/0 (socket says 172.21.15.99:52886) 2024-09-10T11:38:15.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 -- 172.21.15.99:0/1121158366 learned_addr learned my addr 172.21.15.99:0/1121158366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b54e79640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5006ee30 0x7f6b50142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b5567a640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b50150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 -- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5006ee30 msgr2=0x7f6b50142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5006ee30 0x7f6b50142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 -- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 msgr2=0x7f6b50150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b50150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5014ea00 con 0x7f6b500d0fb0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b5567a640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b50150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.398+0000 7f6b47fff640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000ebe0 tx=0x7f6b4000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.399+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40017070 con 0x7f6b500d0fb0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.399+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4000cc30 con 0x7f6b500d0fb0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.399+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5014ece0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.399+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b40013470 con 0x7f6b500d0fb0 2024-09-10T11:38:15.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.399+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b50152710 con 0x7f6b500d0fb0 2024-09-10T11:38:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.400+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b50152bf0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.400+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b4001a070 con 0x7f6b500d0fb0 2024-09-10T11:38:15.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.400+0000 7f6b457fa640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 0x7f6b3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.401+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5709+0+0 (secure 0 0 0) 0x7f6b4008b0f0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.401+0000 7f6b54e79640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 0x7f6b3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.401+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=156}) -- 0x7f6b3c073960 con 0x7f6b500d0fb0 2024-09-10T11:38:15.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.401+0000 7f6b54e79640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 0x7f6b3c06e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c034000 tx=0x7f6b4c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.405+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b50152bf0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.608+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6b5015cb90 con 0x7f6b500d0fb0 2024-09-10T11:38:15.610 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:38:15.609+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:38:15.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.610+0000 7f6b457fa640 1 -- 172.21.15.99:0/1121158366 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f6b400593d0 con 0x7f6b500d0fb0 2024-09-10T11:38:15.611 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:38:15.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.612+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 msgr2=0x7f6b3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.612+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 0x7f6b3c06e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c034000 tx=0x7f6b4c02e000 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.612+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 msgr2=0x7f6b501502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.612+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6b4000ebe0 tx=0x7f6b4000c520 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 shutdown_connections 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3c06be30 0x7f6b3c06e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b5015c7b0 0x7f6b50150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b500d0fb0 0x7f6b501502f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 --2- 172.21.15.99:0/1121158366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b5006ee30 0x7f6b50142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 >> 172.21.15.99:0/1121158366 conn(0x7f6b5006bc10 msgr2=0x7f6b50157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 shutdown_connections 2024-09-10T11:38:15.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.613+0000 7f6b570dd640 1 -- 172.21.15.99:0/1121158366 wait complete. 2024-09-10T11:38:15.617 INFO:tasks.ceph.mds.a.smithi099.stderr: -9353> 2024-09-10T11:36:22.364+0000 7f6770a3d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:38:15.626 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:38:15.651 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:38:15.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.727+0000 7f12c732c640 1 Processor -- start 2024-09-10T11:38:15.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.727+0000 7f12c732c640 1 -- start start 2024-09-10T11:38:15.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c732c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12c00d1200 0x7f12c0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c732c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f12c006e5d0 0x7f12c006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c732c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12c0152e60 con 0x7f12c015c060 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c732c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f12c006ef80 con 0x7f12c00d1200 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c732c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f12c00d15e0 con 0x7f12c006e5d0 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12c00d1200 0x7f12c0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12c00d1200 0x7f12c0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52894/0 (socket says 172.21.15.99:52894) 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 -- 172.21.15.99:0/1410415684 learned_addr learned my addr 172.21.15.99:0/1410415684 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c48c7640 1 --2- 172.21.15.99:0/1410415684 >> v2:172.21.15.196:3300/0 conn(0x7f12c006e5d0 0x7f12c006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 -- 172.21.15.99:0/1410415684 >> v2:172.21.15.196:3300/0 conn(0x7f12c006e5d0 msgr2=0x7f12c006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 --2- 172.21.15.99:0/1410415684 >> v2:172.21.15.196:3300/0 conn(0x7f12c006e5d0 0x7f12c006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 -- 172.21.15.99:0/1410415684 >> v1:172.21.15.196:6789/0 conn(0x7f12c015c060 legacy=0x7f12c015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.728+0000 7f12c50c8640 1 -- 172.21.15.99:0/1410415684 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c0072680 con 0x7f12c00d1200 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.729+0000 7f12c50c8640 1 --2- 172.21.15.99:0/1410415684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00d1200 0x7f12c0073440 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f12c0071bb0 tx=0x7f12bc02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7fe3537a4dca3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.729+0000 7f12b7fff640 1 -- 172.21.15.99:0/1410415684 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc032710 con 0x7f12c00d1200 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.729+0000 7f12b7fff640 1 -- 172.21.15.99:0/1410415684 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc0328b0 con 0x7f12c00d1200 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.729+0000 7f12b7fff640 1 -- 172.21.15.99:0/1410415684 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc032ca0 con 0x7f12c00d1200 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 -- 172.21.15.99:0/1410415684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00d1200 msgr2=0x7f12c0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 --2- 172.21.15.99:0/1410415684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00d1200 0x7f12c0073440 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f12c0071bb0 tx=0x7f12bc02c850 comp rx=0 tx=0).stop 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 -- 172.21.15.99:0/1410415684 shutdown_connections 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 --2- 172.21.15.99:0/1410415684 >> v2:172.21.15.196:3300/0 conn(0x7f12c006e5d0 0x7f12c006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 --2- 172.21.15.99:0/1410415684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c00d1200 0x7f12c0073440 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 -- 172.21.15.99:0/1410415684 >> 172.21.15.99:0/1410415684 conn(0x7f12c006bc10 msgr2=0x7f12c006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:15.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 -- 172.21.15.99:0/1410415684 shutdown_connections 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.730+0000 7f12c732c640 1 -- 172.21.15.99:0/1410415684 wait complete. 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.731+0000 7f12c732c640 1 Processor -- start 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.731+0000 7f12c732c640 1 -- start start 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c006e5d0 0x7f12c0158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 0x7f12c0153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0072f20 con 0x7f12c015c060 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0072c40 con 0x7f12c006e5d0 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c732c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12c015cd90 con 0x7f12c00d1200 2024-09-10T11:38:15.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52904/0 (socket says 172.21.15.99:52904) 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 -- 172.21.15.99:0/2203510700 learned_addr learned my addr 172.21.15.99:0/2203510700 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c50c8640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c006e5d0 0x7f12c0158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c48c7640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 0x7f12c0153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 -- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 msgr2=0x7f12c0153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 0x7f12c0153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 -- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c006e5d0 msgr2=0x7f12c0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c006e5d0 0x7f12c0158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c01587c0 con 0x7f12c015c060 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c48c7640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 0x7f12c0153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12c58c9640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c7a0 tx=0x7f12b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.732+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8016020 con 0x7f12c015c060 2024-09-10T11:38:15.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.733+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b800aa50 con 0x7f12c015c060 2024-09-10T11:38:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.733+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c0158aa0 con 0x7f12c015c060 2024-09-10T11:38:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.733+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8012420 con 0x7f12c015c060 2024-09-10T11:38:15.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.733+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c015a4d0 con 0x7f12c015c060 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.734+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c006ea40 con 0x7f12c015c060 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.734+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12b8019020 con 0x7f12c015c060 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.734+0000 7f12b5ffb640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 0x7f12a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.735+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5795+0+0 (secure 0 0 0) 0x7f12b808a8a0 con 0x7f12c015c060 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.735+0000 7f12c50c8640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 0x7f12a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.735+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) -- 0x7f12a4073a70 con 0x7f12c015c060 2024-09-10T11:38:15.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.735+0000 7f12c50c8640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 0x7f12a406e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f12bc037000 tx=0x7f12bc036040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:15.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.739+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12b8020020 con 0x7f12c015c060 2024-09-10T11:38:15.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:15.939+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f12c00761b0 con 0x7f12c015c060 2024-09-10T11:38:16.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.613+0000 7f12b5ffb640 1 -- 172.21.15.99:0/2203510700 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7f12b80591f0 con 0x7f12c015c060 2024-09-10T11:38:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 msgr2=0x7f12a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 0x7f12a406e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f12bc037000 tx=0x7f12bc036040 comp rx=0 tx=0).stop 2024-09-10T11:38:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 msgr2=0x7f12c01f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:16.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f12b800c7a0 tx=0x7f12b800cc60 comp rx=0 tx=0).stop 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 shutdown_connections 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12a406be30 0x7f12a406e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12c015c060 0x7f12c01f4100 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c00d1200 0x7f12c0153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 --2- 172.21.15.99:0/2203510700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c006e5d0 0x7f12c0158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 >> 172.21.15.99:0/2203510700 conn(0x7f12c006bc10 msgr2=0x7f12c006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 shutdown_connections 2024-09-10T11:38:16.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.616+0000 7f12c732c640 1 -- 172.21.15.99:0/2203510700 wait complete. 2024-09-10T11:38:16.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:38:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.777+0000 7f1fea3a8640 1 Processor -- start 2024-09-10T11:38:16.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 -- start start 2024-09-10T11:38:16.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fe4070c80 0x7f1fe40b6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe406cd00 0x7f1fe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fe4071060 con 0x7f1fe41627f0 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1fe40b6d40 con 0x7f1fe4070c80 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.778+0000 7f1fea3a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40b7000 con 0x7f1fe406cd00 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fe4070c80 0x7f1fe40b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fe4070c80 0x7f1fe40b6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52912/0 (socket says 172.21.15.99:52912) 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3986339720 learned_addr learned my addr 172.21.15.99:0/3986339720 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe8945640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe406cd00 0x7f1fe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3986339720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe406cd00 msgr2=0x7f1fe406d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe406cd00 0x7f1fe406d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3986339720 >> v1:172.21.15.196:6789/0 conn(0x7f1fe41627f0 legacy=0x7f1fe4162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.779+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3986339720 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe40b72e0 con 0x7f1fe4070c80 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fe37fe640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe40b6470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009870 tx=0x7f1fd802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37495ca3859d2dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:16.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fe2ffd640 1 -- 172.21.15.99:0/3986339720 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8037070 con 0x7f1fe4070c80 2024-09-10T11:38:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fe2ffd640 1 -- 172.21.15.99:0/3986339720 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd8032440 con 0x7f1fe4070c80 2024-09-10T11:38:16.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fe2ffd640 1 -- 172.21.15.99:0/3986339720 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8032810 con 0x7f1fe4070c80 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3986339720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 msgr2=0x7f1fe40b6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe40b6470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8009870 tx=0x7f1fd802a210 comp rx=0 tx=0).stop 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3986339720 shutdown_connections 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe406cd00 0x7f1fe406d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3986339720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe40b6470 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3986339720 >> 172.21.15.99:0/3986339720 conn(0x7f1fe409ba30 msgr2=0x7f1fe409be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3986339720 shutdown_connections 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.780+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3986339720 wait complete. 2024-09-10T11:38:16.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 Processor -- start 2024-09-10T11:38:16.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 -- start start 2024-09-10T11:38:16.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 0x7f1fe415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 0x7f1fe414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.782+0000 7f1fea3a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe40b69b0 con 0x7f1fe4070c80 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fea3a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe40b78a0 con 0x7f1fe406cd00 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fea3a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe40b22b0 con 0x7f1fe41627f0 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52918/0 (socket says 172.21.15.99:52918) 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3048940130 learned_addr learned my addr 172.21.15.99:0/3048940130 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe3fff640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 0x7f1fe415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe8945640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 0x7f1fe414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 msgr2=0x7f1fe414fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:16.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 0x7f1fe414fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 msgr2=0x7f1fe415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 0x7f1fe415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4152b90 con 0x7f1fe4070c80 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe3fff640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 0x7f1fe415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe8945640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 0x7f1fe414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe37fe640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80097c0 tx=0x7f1fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8041020 con 0x7f1fe4070c80 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd80306c0 con 0x7f1fe4070c80 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8037040 con 0x7f1fe4070c80 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe415ef70 con 0x7f1fe4070c80 2024-09-10T11:38:16.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.783+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe415f530 con 0x7f1fe4070c80 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.784+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40b6470 con 0x7f1fe4070c80 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.785+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fd8044020 con 0x7f1fe4070c80 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.785+0000 7f1fe0ff9640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 0x7f1fd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.785+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5795+0+0 (secure 0 0 0) 0x7f1fd80ab6b0 con 0x7f1fe4070c80 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.785+0000 7f1fe3fff640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 0x7f1fd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.785+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=157}) -- 0x7f1fd0073a70 con 0x7f1fe4070c80 2024-09-10T11:38:16.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.786+0000 7f1fe3fff640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 0x7f1fd006e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003eb0 tx=0x7f1fd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:16.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.789+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1fd804b030 con 0x7f1fe4070c80 2024-09-10T11:38:16.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:16.954+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1fe415e2c0 con 0x7f1fe4070c80 2024-09-10T11:38:17.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.617+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v157) ==== 181+0+0 (secure 0 0 0) 0x7f1fd807a000 con 0x7f1fe4070c80 2024-09-10T11:38:17.618 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:38:17.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.617+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(157..157 src has 1..157) ==== 296+0+0 (secure 0 0 0) 0x7f1fd80dfd50 con 0x7f1fe4070c80 2024-09-10T11:38:17.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.617+0000 7f1fe0ff9640 1 -- 172.21.15.99:0/3048940130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=158}) -- 0x7f1fd0071ea0 con 0x7f1fe4070c80 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.623+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 msgr2=0x7f1fd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.623+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 0x7f1fd006e2f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003eb0 tx=0x7f1fd4009040 comp rx=0 tx=0).stop 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.623+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 msgr2=0x7f1fe414f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.623+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80097c0 tx=0x7f1fd80046c0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 shutdown_connections 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1fd006be30 0x7f1fd006e2f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fe41627f0 0x7f1fe414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fe4070c80 0x7f1fe414f6c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 --2- 172.21.15.99:0/3048940130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fe406cd00 0x7f1fe415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 >> 172.21.15.99:0/3048940130 conn(0x7f1fe409ba30 msgr2=0x7f1fe415d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:17.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 shutdown_connections 2024-09-10T11:38:17.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.624+0000 7f1fea3a8640 1 -- 172.21.15.99:0/3048940130 wait complete. 2024-09-10T11:38:17.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:38:17.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.741+0000 7fcf6052a640 1 Processor -- start 2024-09-10T11:38:17.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.741+0000 7fcf6052a640 1 -- start start 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf6052a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcf5815ee00 0x7fcf580b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf6052a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf580b0090 0x7fcf580b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf6052a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf580b40e0 con 0x7fcf5815f1e0 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf6052a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fcf580b5ab0 con 0x7fcf5815ee00 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf6052a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf58073c50 con 0x7fcf580b0090 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf5e2c6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcf5815ee00 0x7fcf580b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:17.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf5dac5640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf5815f1e0 0x7fcf580b1560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35334/0 (socket says 172.21.15.99:35334) 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.742+0000 7fcf5dac5640 1 -- 172.21.15.99:0/1724232442 learned_addr learned my addr 172.21.15.99:0/1724232442 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5e2c6640 1 -- 172.21.15.99:0/1724232442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf580b0090 msgr2=0x7fcf580b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5e2c6640 1 --2- 172.21.15.99:0/1724232442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf580b0090 0x7fcf580b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5e2c6640 1 -- 172.21.15.99:0/1724232442 >> v1:172.21.15.99:6789/0 conn(0x7fcf5815f1e0 legacy=0x7fcf580b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5e2c6640 1 -- 172.21.15.99:0/1724232442 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf580b4380 con 0x7fcf5815ee00 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5d2c4640 1 -- 172.21.15.99:0/1724232442 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287973780 0 0) 0x7fcf580b40e0 con 0x7fcf5815f1e0 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5e2c6640 1 --2- 172.21.15.99:0/1724232442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815ee00 0x7fcf580b5f90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcf540097a0 tx=0x7fcf5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385a3fa478a85027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5d2c4640 1 -- 172.21.15.99:0/1724232442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54037070 con 0x7fcf5815ee00 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5d2c4640 1 -- 172.21.15.99:0/1724232442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf54032440 con 0x7fcf5815ee00 2024-09-10T11:38:17.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.743+0000 7fcf5d2c4640 1 -- 172.21.15.99:0/1724232442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf54032810 con 0x7fcf5815ee00 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 -- 172.21.15.99:0/1724232442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815ee00 msgr2=0x7fcf580b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 --2- 172.21.15.99:0/1724232442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815ee00 0x7fcf580b5f90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcf540097a0 tx=0x7fcf5402a180 comp rx=0 tx=0).stop 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 -- 172.21.15.99:0/1724232442 shutdown_connections 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 --2- 172.21.15.99:0/1724232442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf580b0090 0x7fcf580b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 --2- 172.21.15.99:0/1724232442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815ee00 0x7fcf580b5f90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 -- 172.21.15.99:0/1724232442 >> 172.21.15.99:0/1724232442 conn(0x7fcf580730c0 msgr2=0x7fcf580734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:17.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 -- 172.21.15.99:0/1724232442 shutdown_connections 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.744+0000 7fcf6052a640 1 -- 172.21.15.99:0/1724232442 wait complete. 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.745+0000 7fcf6052a640 1 Processor -- start 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.745+0000 7fcf6052a640 1 -- start start 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf580b0090 0x7fcf580cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf5815ee00 0x7fcf5809eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580b21b0 con 0x7fcf5815f1e0 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf580afd60 con 0x7fcf580b0090 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf6052a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fcf580b0600 con 0x7fcf5815ee00 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:17.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52932/0 (socket says 172.21.15.99:52932) 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 -- 172.21.15.99:0/736387207 learned_addr learned my addr 172.21.15.99:0/736387207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 -- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf5815ee00 msgr2=0x7fcf5809eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5dac5640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf5815ee00 0x7fcf5809eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf5815ee00 0x7fcf5809eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 -- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf580b0090 msgr2=0x7fcf580cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf580b0090 0x7fcf580cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5809c3e0 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.746+0000 7fcf5eac7640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b970 tx=0x7fcf5000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.747+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50019070 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.747+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf5000cb60 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.747+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5809b340 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.747+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50014440 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.747+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf5809b900 con 0x7fcf5815f1e0 2024-09-10T11:38:17.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.748+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf5809bf10 con 0x7fcf5815f1e0 2024-09-10T11:38:17.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.748+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf5001c070 con 0x7fcf5815f1e0 2024-09-10T11:38:17.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.748+0000 7fcf4effd640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:17.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.749+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5357+0+0 (secure 0 0 0) 0x7fcf5008a9c0 con 0x7fcf5815f1e0 2024-09-10T11:38:17.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.749+0000 7fcf5e2c6640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:17.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.749+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=158}) -- 0x7fcf3c072c00 con 0x7fcf5815f1e0 2024-09-10T11:38:17.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.749+0000 7fcf5e2c6640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf54034000 tx=0x7fcf5402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:17.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.753+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf5008ce20 con 0x7fcf5815f1e0 2024-09-10T11:38:17.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:17.917+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcf580b5f90 con 0x7fcf5815f1e0 2024-09-10T11:38:18.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.617+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v158) ==== 169+0+0 (secure 0 0 0) 0x7fcf50059560 con 0x7fcf5815f1e0 2024-09-10T11:38:18.620 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:38:18.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.618+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(158..158 src has 1..158) ==== 296+0+0 (secure 0 0 0) 0x7fcf50089160 con 0x7fcf5815f1e0 2024-09-10T11:38:18.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.618+0000 7fcf4effd640 1 -- 172.21.15.99:0/736387207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fcf3c0719f0 con 0x7fcf5815f1e0 2024-09-10T11:38:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 msgr2=0x7fcf3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcf54034000 tx=0x7fcf5402c000 comp rx=0 tx=0).stop 2024-09-10T11:38:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 msgr2=0x7fcf581f3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000b970 tx=0x7fcf5000be30 comp rx=0 tx=0).stop 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 shutdown_connections 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fcf3c06be30 0x7fcf3c06e2f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf5815f1e0 0x7fcf581f3ce0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf5815ee00 0x7fcf5809eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 --2- 172.21.15.99:0/736387207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf580b0090 0x7fcf580cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 >> 172.21.15.99:0/736387207 conn(0x7fcf580730c0 msgr2=0x7fcf5806dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:18.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.627+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 shutdown_connections 2024-09-10T11:38:18.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.628+0000 7fcf6052a640 1 -- 172.21.15.99:0/736387207 wait complete. 2024-09-10T11:38:18.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:38:18.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.748+0000 7f6e083e8640 1 Processor -- start 2024-09-10T11:38:18.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.748+0000 7f6e083e8640 1 -- start start 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e083e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015ee00 0x7f6e000b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e083e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e000b0090 0x7f6e000b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e083e8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e000b40e0 con 0x7f6e0015f1e0 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e083e8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6e000b5ab0 con 0x7f6e000b0090 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e083e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e00073c50 con 0x7f6e0015ee00 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e000b0090 0x7f6e000b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e000b0090 0x7f6e000b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52946/0 (socket says 172.21.15.99:52946) 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 -- 172.21.15.99:0/1613301052 learned_addr learned my addr 172.21.15.99:0/1613301052 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:18.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 -- 172.21.15.99:0/1613301052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015ee00 msgr2=0x7f6e000b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 --2- 172.21.15.99:0/1613301052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015ee00 0x7f6e000b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 -- 172.21.15.99:0/1613301052 >> v1:172.21.15.196:6789/0 conn(0x7f6e0015f1e0 legacy=0x7f6e000b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:18.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.749+0000 7f6e06985640 1 -- 172.21.15.99:0/1613301052 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e000b4380 con 0x7f6e000b0090 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e06985640 1 --2- 172.21.15.99:0/1613301052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e000b1c70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e50 tx=0x7f6df802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c493bf96706d22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e05182640 1 -- 172.21.15.99:0/1613301052 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8036070 con 0x7f6e000b0090 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e05182640 1 -- 172.21.15.99:0/1613301052 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df8031600 con 0x7f6e000b0090 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e05182640 1 -- 172.21.15.99:0/1613301052 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80319d0 con 0x7f6e000b0090 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e083e8640 1 -- 172.21.15.99:0/1613301052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 msgr2=0x7f6e000b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.750+0000 7f6e083e8640 1 --2- 172.21.15.99:0/1613301052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e000b1c70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009e50 tx=0x7f6df802a6c0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 -- 172.21.15.99:0/1613301052 shutdown_connections 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 --2- 172.21.15.99:0/1613301052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e000b1c70 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 --2- 172.21.15.99:0/1613301052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015ee00 0x7f6e000b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 -- 172.21.15.99:0/1613301052 >> 172.21.15.99:0/1613301052 conn(0x7f6e000730c0 msgr2=0x7f6e000734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 -- 172.21.15.99:0/1613301052 shutdown_connections 2024-09-10T11:38:18.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.751+0000 7f6e083e8640 1 -- 172.21.15.99:0/1613301052 wait complete. 2024-09-10T11:38:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 Processor -- start 2024-09-10T11:38:18.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 -- start start 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e0015ee00 0x7f6e000ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015f1e0 0x7f6e001587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e000b21b0 con 0x7f6e000b0090 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e000afd60 con 0x7f6e0015ee00 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.752+0000 7f6e083e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e000b0600 con 0x7f6e0015f1e0 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52960/0 (socket says 172.21.15.99:52960) 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 -- 172.21.15.99:0/4138695112 learned_addr learned my addr 172.21.15.99:0/4138695112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 -- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015f1e0 msgr2=0x7f6e001587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015f1e0 0x7f6e001587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 -- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e0015ee00 msgr2=0x7f6e000ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e0015ee00 0x7f6e000ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 -- 172.21.15.99:0/4138695112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e00162830 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e06184640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b4a0 tx=0x7f6dfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc014070 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc010610 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e000cbc40 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc010a00 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.753+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e000cc4e0 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.754+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e000cc030 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.755+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.755+0000 7f6df2ffd640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 0x7f6de406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.755+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7f6dfc08a4f0 con 0x7f6e000b0090 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.755+0000 7f6e05983640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 0x7f6de406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:18.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.755+0000 7f6e05983640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 0x7f6de406e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e000afbd0 tx=0x7f6df4006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:18.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.759+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6dfc059230 con 0x7f6e000b0090 2024-09-10T11:38:18.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.924+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6e000b5f90 con 0x7f6e000b0090 2024-09-10T11:38:18.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.928+0000 7f6df2ffd640 1 -- 172.21.15.99:0/4138695112 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f6dfc058b30 con 0x7f6e000b0090 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 msgr2=0x7f6de406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 0x7f6de406e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6e000afbd0 tx=0x7f6df4006ba0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 msgr2=0x7f6e001622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b4a0 tx=0x7f6dfc00b960 comp rx=0 tx=0).stop 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 shutdown_connections 2024-09-10T11:38:18.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6de406be30 0x7f6de406e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e0015f1e0 0x7f6e001587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e0015ee00 0x7f6e000ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 --2- 172.21.15.99:0/4138695112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e000b0090 0x7f6e001622f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 >> 172.21.15.99:0/4138695112 conn(0x7f6e000730c0 msgr2=0x7f6e0006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 shutdown_connections 2024-09-10T11:38:18.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:18.930+0000 7f6e083e8640 1 -- 172.21.15.99:0/4138695112 wait complete. 2024-09-10T11:38:18.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-09-10T11:38:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.040+0000 7f01622e6640 1 Processor -- start 2024-09-10T11:38:19.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.040+0000 7f01622e6640 1 -- start start 2024-09-10T11:38:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f01622e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f015c0b3960 0x7f015c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f01622e6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f015c0b3160 0x7f015c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f01622e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015c0b3620 con 0x7f015c163580 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f01622e6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f015c06d960 con 0x7f015c0b3160 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f01622e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f015c06dc40 con 0x7f015c0b3960 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f0160883640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f015c0b3160 0x7f015c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f0160883640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f015c0b3160 0x7f015c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52964/0 (socket says 172.21.15.99:52964) 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.041+0000 7f0160883640 1 -- 172.21.15.99:0/3548740647 learned_addr learned my addr 172.21.15.99:0/3548740647 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f0160883640 1 -- 172.21.15.99:0/3548740647 >> v2:172.21.15.196:3300/0 conn(0x7f015c0b3960 msgr2=0x7f015c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f0160883640 1 --2- 172.21.15.99:0/3548740647 >> v2:172.21.15.196:3300/0 conn(0x7f015c0b3960 0x7f015c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f0160883640 1 -- 172.21.15.99:0/3548740647 >> v1:172.21.15.196:6789/0 conn(0x7f015c163580 legacy=0x7f015c0d1be0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T11:38:19.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f0160883640 1 -- 172.21.15.99:0/3548740647 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c06df20 con 0x7f015c0b3160 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f0160883640 1 --2- 172.21.15.99:0/3548740647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3160 0x7f015c06cd20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f015c0c9a80 tx=0x7f015402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a14577932d9433 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f015affd640 1 -- 172.21.15.99:0/3548740647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154036070 con 0x7f015c0b3160 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.042+0000 7f015affd640 1 -- 172.21.15.99:0/3548740647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0154031600 con 0x7f015c0b3160 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f015affd640 1 -- 172.21.15.99:0/3548740647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01540319d0 con 0x7f015c0b3160 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 -- 172.21.15.99:0/3548740647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3160 msgr2=0x7f015c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 --2- 172.21.15.99:0/3548740647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3160 0x7f015c06cd20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f015c0c9a80 tx=0x7f015402a5d0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 -- 172.21.15.99:0/3548740647 shutdown_connections 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 --2- 172.21.15.99:0/3548740647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3160 0x7f015c06cd20 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 --2- 172.21.15.99:0/3548740647 >> v2:172.21.15.196:3300/0 conn(0x7f015c0b3960 0x7f015c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 -- 172.21.15.99:0/3548740647 >> 172.21.15.99:0/3548740647 conn(0x7f015c0b4570 msgr2=0x7f015c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 -- 172.21.15.99:0/3548740647 shutdown_connections 2024-09-10T11:38:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.043+0000 7f01622e6640 1 -- 172.21.15.99:0/3548740647 wait complete. 2024-09-10T11:38:19.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.044+0000 7f01622e6640 1 Processor -- start 2024-09-10T11:38:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.044+0000 7f01622e6640 1 -- start start 2024-09-10T11:38:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015c0b3160 0x7f015c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015c163580 0x7f015c0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f015c06d3e0 con 0x7f015c0b3960 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f015c06d260 con 0x7f015c0b3160 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f01622e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f015c06d560 con 0x7f015c163580 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:52974/0 (socket says 172.21.15.99:52974) 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 -- 172.21.15.99:0/3498177523 learned_addr learned my addr 172.21.15.99:0/3498177523 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 -- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015c163580 msgr2=0x7f015c0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015c163580 0x7f015c0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 -- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015c0b3160 msgr2=0x7f015c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.045+0000 7f015b7fe640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015c0b3160 0x7f015c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f015b7fe640 1 -- 172.21.15.99:0/3498177523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015c1e8df0 con 0x7f015c0b3960 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f015b7fe640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f015000b410 tx=0x7f015000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150015020 con 0x7f015c0b3960 2024-09-10T11:38:19.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150011850 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f015c1e90b0 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150011c40 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.046+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f015c0b7400 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.047+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015c06cd20 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.047+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0150018020 con 0x7f015c0b3960 2024-09-10T11:38:19.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.048+0000 7f0158ff9640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 0x7f014806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.048+0000 7f015bfff640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 0x7f014806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.048+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7f015008a970 con 0x7f015c0b3960 2024-09-10T11:38:19.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.049+0000 7f015bfff640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 0x7f014806e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f014c0096f0 tx=0x7f014c006c40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.052+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f015001f020 con 0x7f015c0b3960 2024-09-10T11:38:19.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.285+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7f015c15f7b0 con 0x7f015c0b3960 2024-09-10T11:38:19.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.629+0000 7f0158ff9640 1 -- 172.21.15.99:0/3498177523 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v384) ==== 132+0+0 (secure 0 0 0) 0x7f01500172c0 con 0x7f015c0b3960 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 msgr2=0x7f014806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 0x7f014806e2f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f014c0096f0 tx=0x7f014c006c40 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 msgr2=0x7f015c1f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f015000b410 tx=0x7f015000b8d0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 shutdown_connections 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f014806be30 0x7f014806e2f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f015c163580 0x7f015c0b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f015c0b3960 0x7f015c1f8330 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 --2- 172.21.15.99:0/3498177523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f015c0b3160 0x7f015c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 >> 172.21.15.99:0/3498177523 conn(0x7f015c0b4570 msgr2=0x7f015c15f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 shutdown_connections 2024-09-10T11:38:19.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.632+0000 7f01622e6640 1 -- 172.21.15.99:0/3498177523 wait complete. 2024-09-10T11:38:19.645 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-09-10T11:38:19.645 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-09-10T11:38:19.645 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-09-10T11:38:19.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:19.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.743+0000 7f95b0178640 1 Processor -- start 2024-09-10T11:38:19.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.743+0000 7f95b0178640 1 -- start start 2024-09-10T11:38:19.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95b0178640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95a80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95b0178640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f95a806f000 0x7f95a806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95b0178640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a80ceb10 con 0x7f95a80d0fb0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95b0178640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f95a80b3840 con 0x7f95a806f000 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95b0178640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f95a80d13f0 con 0x7f95a815c7b0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95ad713640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f95a80d0fb0 0x7f95a806e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35336/0 (socket says 172.21.15.99:35336) 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95ad713640 1 -- 172.21.15.99:0/1739178746 learned_addr learned my addr 172.21.15.99:0/1739178746 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95adf14640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95a80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95ae715640 1 --2- 172.21.15.99:0/1739178746 >> v2:172.21.15.196:3300/0 conn(0x7f95a806f000 0x7f95a806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867716204 0 0) 0x7f9598001110 con 0x7f95a80d0fb0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9594003520 con 0x7f95a80d0fb0 2024-09-10T11:38:19.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4267691357 0 0) 0x7f9594003520 con 0x7f95a80d0fb0 2024-09-10T11:38:19.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 msgr2=0x7f95a80d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:19.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95a80d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 >> v2:172.21.15.196:3300/0 conn(0x7f95a806f000 msgr2=0x7f95a806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 --2- 172.21.15.99:0/1739178746 >> v2:172.21.15.196:3300/0 conn(0x7f95a806f000 0x7f95a806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a80b3a90 con 0x7f95a80d0fb0 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9598001840 con 0x7f95a80d0fb0 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 >> v1:172.21.15.99:6789/0 conn(0x7f95a80d0fb0 legacy=0x7f95a806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f9594003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594004800 0x7f9594004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80b3840 con 0x7f95940039b0 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80d13f0 con 0x7f9594004800 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.744+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f959400b9c0 con 0x7f95a815c7b0 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f9594003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95ad713640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594004800 0x7f9594004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95ae715640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 -- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 msgr2=0x7f95940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 -- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594004800 msgr2=0x7f9594004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9594004800 0x7f9594004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95adf14640 1 -- 172.21.15.99:0/1739178746 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9594003520 con 0x7f95940039b0 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.745+0000 7f95ae715640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a815c7b0 0x7f95940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95adf14640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f9594003dd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009ac0 tx=0x7f9594005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad35711e6a7cb8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4036070 con 0x7f95940039b0 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4031400 con 0x7f95940039b0 2024-09-10T11:38:19.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95acf12640 1 -- 172.21.15.99:0/1739178746 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4009690 con 0x7f95940039b0 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 -- 172.21.15.99:0/1739178746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 msgr2=0x7f9594003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 --2- 172.21.15.99:0/1739178746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f9594003dd0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009ac0 tx=0x7f9594005830 comp rx=0 tx=0).stop 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95ae715640 1 -- 172.21.15.99:0/1739178746 reap_dead start 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 -- 172.21.15.99:0/1739178746 shutdown_connections 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 -- 172.21.15.99:0/1739178746 >> 172.21.15.99:0/1739178746 conn(0x7f95a806bc10 msgr2=0x7f95a806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 -- 172.21.15.99:0/1739178746 shutdown_connections 2024-09-10T11:38:19.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.746+0000 7f95b0178640 1 -- 172.21.15.99:0/1739178746 wait complete. 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.747+0000 7f95b0178640 1 Processor -- start 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.747+0000 7f95b0178640 1 -- start start 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 0x7f95a81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 0x7f95a8150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80b4330 con 0x7f95940039b0 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80b4050 con 0x7f95a8150690 2024-09-10T11:38:19.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95b0178640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f95a80b4610 con 0x7f95a8151340 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95adf14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95adf14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53000/0 (socket says 172.21.15.99:53000) 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95adf14640 1 -- 172.21.15.99:0/316988668 learned_addr learned my addr 172.21.15.99:0/316988668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95ad713640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 0x7f95a81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.748+0000 7f95ae715640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 0x7f95a8150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 -- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 msgr2=0x7f95a81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 0x7f95a81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 -- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 msgr2=0x7f95a8150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 0x7f95a8150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 -- 172.21.15.99:0/316988668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a8153760 con 0x7f95940039b0 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95ae715640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 0x7f95a8150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95ad713640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 0x7f95a81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95adf14640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034040 tx=0x7f95a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4040020 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a4003bc0 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a8153a40 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4036040 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.749+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a8154000 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.750+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a815cb90 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.750+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f95a4043020 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.751+0000 7f959e7fc640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 0x7f958c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.751+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7f95a40aac00 con 0x7f95940039b0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.751+0000 7f95ad713640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 0x7f958c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.751+0000 7f95ad713640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 0x7f958c06e320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95980012f0 tx=0x7f9598009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:19.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.755+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95a40b1080 con 0x7f95940039b0 2024-09-10T11:38:19.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.920+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95a80b05a0 con 0x7f95940039b0 2024-09-10T11:38:19.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.921+0000 7f959e7fc640 1 -- 172.21.15.99:0/316988668 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+13043 (secure 0 0 0) 0x7f95a40a2e20 con 0x7f95940039b0 2024-09-10T11:38:19.923 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:19.923 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":158,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:38:18.617686+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":151,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6837/1210854172":"2024-09-11T11:38:15.609339+0000","172.21.15.99:6835/1210854172":"2024-09-11T11:38:15.609339+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:19.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 msgr2=0x7f958c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 0x7f958c06e320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f95980012f0 tx=0x7f9598009040 comp rx=0 tx=0).stop 2024-09-10T11:38:19.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 msgr2=0x7f95a80aed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f95a4034040 tx=0x7f95a4004730 comp rx=0 tx=0).stop 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 shutdown_connections 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f958c06be60 0x7f958c06e320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95a8150690 0x7f95a8150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95a8151340 0x7f95a81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 --2- 172.21.15.99:0/316988668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95940039b0 0x7f95a80aed10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 >> 172.21.15.99:0/316988668 conn(0x7f95a806bc10 msgr2=0x7f95a80b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:19.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 shutdown_connections 2024-09-10T11:38:19.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:19.923+0000 7f95b0178640 1 -- 172.21.15.99:0/316988668 wait complete. 2024-09-10T11:38:19.937 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:38:19.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:38:20.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 Processor -- start 2024-09-10T11:38:20.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 -- start start 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe200cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe200ce430 con 0x7efe200d0130 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe200ce710 con 0x7efe20160030 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.037+0000 7efe266c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe200ce200 con 0x7efe200d1760 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe24c66640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efe200d0130 0x7efe200cdaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35344/0 (socket says 172.21.15.99:35344) 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe24c66640 1 -- 172.21.15.99:0/577946113 learned_addr learned my addr 172.21.15.99:0/577946113 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe200cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187538244 0 0) 0x7efe200ce430 con 0x7efe200d0130 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe08003520 con 0x7efe200d0130 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303926854 0 0) 0x7efe200ce710 con 0x7efe20160030 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe200ce430 con 0x7efe20160030 2024-09-10T11:38:20.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3031311178 0 0) 0x7efe08003520 con 0x7efe200d0130 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 msgr2=0x7efe200cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe200cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 >> v1:172.21.15.196:6789/0 conn(0x7efe20160030 legacy=0x7efe200b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200d1240 con 0x7efe200d0130 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7efe180019f0 con 0x7efe200d0130 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 >> v1:172.21.15.99:6789/0 conn(0x7efe200d0130 legacy=0x7efe200cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe08003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08004d40 0x7efe080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800ba70 con 0x7efe08003b70 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe0800bc20 con 0x7efe08004d40 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe0800be20 con 0x7efe200d1760 2024-09-10T11:38:20.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1ffff640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe08003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe24c66640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08004d40 0x7efe080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1ffff640 1 -- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 msgr2=0x7efe08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.038+0000 7efe1ffff640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe200d1760 0x7efe08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1ffff640 1 -- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08004d40 msgr2=0x7efe080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1ffff640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08004d40 0x7efe080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1ffff640 1 -- 172.21.15.99:0/577946113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe08003520 con 0x7efe08003b70 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe24c66640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe08004d40 0x7efe080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1ffff640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe08003f70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efe100018d0 tx=0x7efe08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3612d11d42aa684 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100014f0 con 0x7efe08003b70 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe100052d0 con 0x7efe08003b70 2024-09-10T11:38:20.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.039+0000 7efe1effd640 1 -- 172.21.15.99:0/577946113 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1002aca0 con 0x7efe08003b70 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 -- 172.21.15.99:0/577946113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 msgr2=0x7efe08003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 --2- 172.21.15.99:0/577946113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe08003f70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7efe100018d0 tx=0x7efe08005830 comp rx=0 tx=0).stop 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe24c66640 1 -- 172.21.15.99:0/577946113 reap_dead start 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 -- 172.21.15.99:0/577946113 shutdown_connections 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 -- 172.21.15.99:0/577946113 >> 172.21.15.99:0/577946113 conn(0x7efe200b4650 msgr2=0x7efe200b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 -- 172.21.15.99:0/577946113 shutdown_connections 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.040+0000 7efe266c9640 1 -- 172.21.15.99:0/577946113 wait complete. 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.041+0000 7efe266c9640 1 Processor -- start 2024-09-10T11:38:20.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.041+0000 7efe266c9640 1 -- start start 2024-09-10T11:38:20.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 0x7efe200c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 0x7efe200b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe200cd5b0 con 0x7efe08003b70 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe20160660 con 0x7efe2015beb0 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe266c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe200cd730 con 0x7efe2015b5a0 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53020/0 (socket says 172.21.15.99:53020) 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 -- 172.21.15.99:0/218748900 learned_addr learned my addr 172.21.15.99:0/218748900 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe24c66640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 0x7efe200b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 0x7efe200c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 -- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 msgr2=0x7efe200b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 0x7efe200b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 -- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 msgr2=0x7efe200c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 0x7efe200c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1ffff640 1 -- 172.21.15.99:0/218748900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe200c8790 con 0x7efe08003b70 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 0x7efe200c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.042+0000 7efe24c66640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 0x7efe200b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe1ffff640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7efe10005040 tx=0x7efe100012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10040020 con 0x7efe08003b70 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe10005480 con 0x7efe08003b70 2024-09-10T11:38:20.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe200b2b20 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10006d90 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.043+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe200c8cf0 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.044+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe200ce710 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.044+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe10043020 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.045+0000 7efe1cff9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 0x7efe0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.045+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7efe100aac20 con 0x7efe08003b70 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.045+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 0x7efe0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.045+0000 7efe1f7fe640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 0x7efe0c06e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe200b5410 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.049+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe1004a030 con 0x7efe08003b70 2024-09-10T11:38:20.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.217+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe200b7fa0 con 0x7efe08003b70 2024-09-10T11:38:20.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.217+0000 7efe1cff9640 1 -- 172.21.15.99:0/218748900 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7efe200c8790 con 0x7efe08003b70 2024-09-10T11:38:20.219 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 msgr2=0x7efe0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 0x7efe0c06e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efe200b5410 tx=0x7efe14008040 comp rx=0 tx=0).stop 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 msgr2=0x7efe200c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7efe10005040 tx=0x7efe100012d0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 shutdown_connections 2024-09-10T11:38:20.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe0c06be30 0x7efe0c06e2f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe2015b5a0 0x7efe200b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe2015beb0 0x7efe200c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 --2- 172.21.15.99:0/218748900 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe08003b70 0x7efe200c94f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 >> 172.21.15.99:0/218748900 conn(0x7efe200b4650 msgr2=0x7efe200b6e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 shutdown_connections 2024-09-10T11:38:20.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.219+0000 7efe266c9640 1 -- 172.21.15.99:0/218748900 wait complete. 2024-09-10T11:38:20.232 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-09-10T11:38:20.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.328+0000 7f376f152640 1 Processor -- start 2024-09-10T11:38:20.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.328+0000 7f376f152640 1 -- start start 2024-09-10T11:38:20.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376f152640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37680d1200 0x7f3768073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376f152640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 0x7f376806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376f152640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3768152e60 con 0x7f376806e5d0 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376f152640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f376806ef80 con 0x7f37680d1200 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376f152640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f37680d15e0 con 0x7f376815c770 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376d6ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 0x7f376806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376ceee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37680d1200 0x7f3768073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f3767fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f376806e5d0 0x7f376815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:44728/0 (socket says 172.21.15.99:44728) 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376ceee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37680d1200 0x7f3768073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:45108/0 (socket says 172.21.15.99:45108) 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f3767fff640 1 -- 172.21.15.99:0/133247507 learned_addr learned my addr 172.21.15.99:0/133247507 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.329+0000 7f376d6ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 0x7f376806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:55074/0 (socket says 172.21.15.99:55074) 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.330+0000 7f376ceee640 1 -- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 msgr2=0x7f376806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:20.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.330+0000 7f376ceee640 1 --2- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 0x7f376806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.330+0000 7f376ceee640 1 -- 172.21.15.99:0/133247507 >> v1:172.21.15.196:6789/0 conn(0x7f376806e5d0 legacy=0x7f376815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.330+0000 7f376ceee640 1 -- 172.21.15.99:0/133247507 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3768072680 con 0x7f37680d1200 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.330+0000 7f376ceee640 1 --2- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37680d1200 0x7f3768073440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f37580097a0 tx=0x7f375802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e4a7ec2c07fdf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f37677fe640 1 -- 172.21.15.99:0/133247507 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758037070 con 0x7f37680d1200 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f37677fe640 1 -- 172.21.15.99:0/133247507 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375802ae50 con 0x7f37680d1200 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f37677fe640 1 -- 172.21.15.99:0/133247507 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758032660 con 0x7f37680d1200 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 -- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37680d1200 msgr2=0x7f3768073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 --2- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37680d1200 0x7f3768073440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f37580097a0 tx=0x7f375802a100 comp rx=0 tx=0).stop 2024-09-10T11:38:20.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 -- 172.21.15.99:0/133247507 shutdown_connections 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 --2- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376815c770 0x7f376806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 --2- 172.21.15.99:0/133247507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37680d1200 0x7f3768073440 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 -- 172.21.15.99:0/133247507 >> 172.21.15.99:0/133247507 conn(0x7f376806bc10 msgr2=0x7f376806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 -- 172.21.15.99:0/133247507 shutdown_connections 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.331+0000 7f376f152640 1 -- 172.21.15.99:0/133247507 wait complete. 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.332+0000 7f376f152640 1 Processor -- start 2024-09-10T11:38:20.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.332+0000 7f376f152640 1 -- start start 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 0x7f376814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f376815c770 0x7f3768162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3768072f20 con 0x7f37680d1200 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3768072c40 con 0x7f376815c770 2024-09-10T11:38:20.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f376f152640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f376815cd90 con 0x7f376806e5d0 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f3767fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f3767fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53032/0 (socket says 172.21.15.99:53032) 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.333+0000 7f3767fff640 1 -- 172.21.15.99:0/3560467862 learned_addr learned my addr 172.21.15.99:0/3560467862 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f376d6ef640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f376815c770 0x7f3768162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f376ceee640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 0x7f376814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 -- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 msgr2=0x7f376814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 0x7f376814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 -- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f376815c770 msgr2=0x7f3768162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f376815c770 0x7f3768162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 -- 172.21.15.99:0/3560467862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3768150f10 con 0x7f37680d1200 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f376ceee640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 0x7f376814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f3767fff640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f375c00caa0 tx=0x7f375c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c017070 con 0x7f37680d1200 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c00ac00 con 0x7f37680d1200 2024-09-10T11:38:20.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c012400 con 0x7f37680d1200 2024-09-10T11:38:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681511f0 con 0x7f37680d1200 2024-09-10T11:38:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.334+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681517b0 con 0x7f37680d1200 2024-09-10T11:38:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.335+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37680b5780 con 0x7f37680d1200 2024-09-10T11:38:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.336+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f375c01a070 con 0x7f37680d1200 2024-09-10T11:38:20.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.336+0000 7f37657fa640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 0x7f375406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.336+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7f375c08a710 con 0x7f37680d1200 2024-09-10T11:38:20.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.336+0000 7f376ceee640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 0x7f375406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.336+0000 7f376ceee640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 0x7f375406e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3758034000 tx=0x7f375802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.340+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37680b5780 con 0x7f37680d1200 2024-09-10T11:38:20.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.568+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7f376806bff0 con 0x7f37680d1200 2024-09-10T11:38:20.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.631+0000 7f37657fa640 1 -- 172.21.15.99:0/3560467862 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v385) ==== 134+0+0 (secure 0 0 0) 0x7f375c058e00 con 0x7f37680d1200 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 msgr2=0x7f375406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 0x7f375406e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3758034000 tx=0x7f375802e000 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 msgr2=0x7f3768160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f375c00caa0 tx=0x7f375c00cf60 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 shutdown_connections 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f375406be60 0x7f375406e320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f376815c770 0x7f3768162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37680d1200 0x7f3768160880 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 --2- 172.21.15.99:0/3560467862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f376806e5d0 0x7f376814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 >> 172.21.15.99:0/3560467862 conn(0x7f376806bc10 msgr2=0x7f3768167f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 shutdown_connections 2024-09-10T11:38:20.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.634+0000 7f376f152640 1 -- 172.21.15.99:0/3560467862 wait complete. 2024-09-10T11:38:20.647 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:38:20.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.747+0000 7fa83347b640 1 Processor -- start 2024-09-10T11:38:20.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.747+0000 7fa83347b640 1 -- start start 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.748+0000 7fa83347b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.748+0000 7fa83347b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa82c163260 0x7fa82c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.748+0000 7fa83347b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa82c163760 con 0x7fa82c0b40e0 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.748+0000 7fa83347b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa82c15fca0 con 0x7fa82c163260 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.748+0000 7fa83347b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c0b65a0 con 0x7fa82c1623e0 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa831217640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa82c163260 0x7fa82c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa82c163260 0x7fa82c0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:45116/0 (socket says 172.21.15.99:45116) 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa831217640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:55086/0 (socket says 172.21.15.99:55086) 2024-09-10T11:38:20.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 -- 172.21.15.99:0/2553795164 learned_addr learned my addr 172.21.15.99:0/2553795164 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa831a18640 1 --1- 172.21.15.99:0/2553795164 >> v1:172.21.15.196:6789/0 conn(0x7fa82c0b40e0 0x7fa82c0731c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:44744/0 (socket says 172.21.15.99:44744) 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 -- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 msgr2=0x7fa82c0b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 -- 172.21.15.99:0/2553795164 >> v1:172.21.15.196:6789/0 conn(0x7fa82c0b40e0 legacy=0x7fa82c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa82bfff640 1 -- 172.21.15.99:0/2553795164 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929331670 0 0) 0x7fa82c163760 con 0x7fa82c0b40e0 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa830a16640 1 -- 172.21.15.99:0/2553795164 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c161640 con 0x7fa82c163260 2024-09-10T11:38:20.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.749+0000 7fa831217640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.750+0000 7fa830a16640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c163260 0x7fa82c0b6060 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa820009870 tx=0x7fa82002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cf3336f71c5508 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.750+0000 7fa82bfff640 1 -- 172.21.15.99:0/2553795164 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820037070 con 0x7fa82c163260 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.750+0000 7fa82bfff640 1 -- 172.21.15.99:0/2553795164 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820032440 con 0x7fa82c163260 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.750+0000 7fa82bfff640 1 -- 172.21.15.99:0/2553795164 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820032810 con 0x7fa82c163260 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 -- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c163260 msgr2=0x7fa82c0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c163260 0x7fa82c0b6060 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa820009870 tx=0x7fa82002a270 comp rx=0 tx=0).stop 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 -- 172.21.15.99:0/2553795164 shutdown_connections 2024-09-10T11:38:20.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c163260 0x7fa82c0b6060 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 --2- 172.21.15.99:0/2553795164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 -- 172.21.15.99:0/2553795164 >> 172.21.15.99:0/2553795164 conn(0x7fa82c072310 msgr2=0x7fa82c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 -- 172.21.15.99:0/2553795164 shutdown_connections 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.751+0000 7fa83347b640 1 -- 172.21.15.99:0/2553795164 wait complete. 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.752+0000 7fa83347b640 1 Processor -- start 2024-09-10T11:38:20.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.752+0000 7fa83347b640 1 -- start start 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c0b40e0 0x7fa82c157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c15f930 con 0x7fa82c163260 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c15f650 con 0x7fa82c0b40e0 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa82c160130 con 0x7fa82c1623e0 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53034/0 (socket says 172.21.15.99:53034) 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa830a16640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 -- 172.21.15.99:0/436212075 learned_addr learned my addr 172.21.15.99:0/436212075 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831217640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c0b40e0 0x7fa82c157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 -- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 msgr2=0x7fa82c0af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 -- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c0b40e0 msgr2=0x7fa82c157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c0b40e0 0x7fa82c157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 -- 172.21.15.99:0/436212075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c09b350 con 0x7fa82c163260 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa831a18640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c480 tx=0x7fa82400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824018070 con 0x7fa82c163260 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8240074e0 con 0x7fa82c163260 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c09b5c0 con 0x7fa82c163260 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.753+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824004a20 con 0x7fa82c163260 2024-09-10T11:38:20.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.754+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c09be60 con 0x7fa82c163260 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.754+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c0b3a90 con 0x7fa82c163260 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.755+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa82401b070 con 0x7fa82c163260 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.755+0000 7fa829ffb640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 0x7fa81806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.755+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7fa824089d20 con 0x7fa82c163260 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.756+0000 7fa831217640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 0x7fa81806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:20.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.756+0000 7fa831217640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 0x7fa81806e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009bd0 tx=0x7fa81c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:20.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.760+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa824058410 con 0x7fa82c163260 2024-09-10T11:38:20.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.925+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa82c075880 con 0x7fa82c163260 2024-09-10T11:38:20.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.928+0000 7fa829ffb640 1 -- 172.21.15.99:0/436212075 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fa82405c710 con 0x7fa82c163260 2024-09-10T11:38:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 msgr2=0x7fa81806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 0x7fa81806e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009bd0 tx=0x7fa81c00b040 comp rx=0 tx=0).stop 2024-09-10T11:38:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 msgr2=0x7fa82c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c480 tx=0x7fa82400c940 comp rx=0 tx=0).stop 2024-09-10T11:38:20.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 shutdown_connections 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa81806be30 0x7fa81806e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c163260 0x7fa82c15a0f0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa82c1623e0 0x7fa82c0af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 --2- 172.21.15.99:0/436212075 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa82c0b40e0 0x7fa82c157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 >> 172.21.15.99:0/436212075 conn(0x7fa82c072310 msgr2=0x7fa82c06df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 shutdown_connections 2024-09-10T11:38:20.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:20.931+0000 7fa83347b640 1 -- 172.21.15.99:0/436212075 wait complete. 2024-09-10T11:38:20.944 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:38:20.944 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:38:20.987 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:38:21.036 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:38:21.037 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:38:21.037 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:21.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:38:21.053 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:38:21.053 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:21.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:38:21.071 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:38:21.071 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:21.108 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:38:21.108 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:38:21.108 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:38:21.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:38:21.129 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:38:21.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:21.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.245+0000 7fbb8919a640 1 Processor -- start 2024-09-10T11:38:21.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.245+0000 7fbb8919a640 1 -- start start 2024-09-10T11:38:21.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8919a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbb840d1200 0x7fbb84073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8919a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb8415c770 0x7fbb8406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8919a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb84152e60 con 0x7fbb8406e5d0 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8919a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8406ef80 con 0x7fbb8415c770 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8919a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbb840d15e0 con 0x7fbb840d1200 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb8415c770 0x7fbb8406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbb8415c770 0x7fbb8406ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53042/0 (socket says 172.21.15.99:53042) 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 -- 172.21.15.99:0/1066914285 learned_addr learned my addr 172.21.15.99:0/1066914285 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 -- 172.21.15.99:0/1066914285 >> v2:172.21.15.196:3300/0 conn(0x7fbb840d1200 msgr2=0x7fbb84073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 --2- 172.21.15.99:0/1066914285 >> v2:172.21.15.196:3300/0 conn(0x7fbb840d1200 0x7fbb84073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 -- 172.21.15.99:0/1066914285 >> v1:172.21.15.196:6789/0 conn(0x7fbb8406e5d0 legacy=0x7fbb8415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.246+0000 7fbb8359e640 1 -- 172.21.15.99:0/1066914285 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb84072680 con 0x7fbb8415c770 2024-09-10T11:38:21.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.247+0000 7fbb8359e640 1 --2- 172.21.15.99:0/1066914285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb8406ea40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbb840d1850 tx=0x7fbb7c02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb00fa788e76dd57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.247+0000 7fbb8259c640 1 -- 172.21.15.99:0/1066914285 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c039070 con 0x7fbb8415c770 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.247+0000 7fbb8259c640 1 -- 172.21.15.99:0/1066914285 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7c00b2e0 con 0x7fbb8415c770 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.247+0000 7fbb8259c640 1 -- 172.21.15.99:0/1066914285 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c009820 con 0x7fbb8415c770 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 -- 172.21.15.99:0/1066914285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 msgr2=0x7fbb8406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1066914285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb8406ea40 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fbb840d1850 tx=0x7fbb7c02c860 comp rx=0 tx=0).stop 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 -- 172.21.15.99:0/1066914285 shutdown_connections 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1066914285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb8406ea40 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1066914285 >> v2:172.21.15.196:3300/0 conn(0x7fbb840d1200 0x7fbb84073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 -- 172.21.15.99:0/1066914285 >> 172.21.15.99:0/1066914285 conn(0x7fbb8406bc10 msgr2=0x7fbb8406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 -- 172.21.15.99:0/1066914285 shutdown_connections 2024-09-10T11:38:21.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.248+0000 7fbb8919a640 1 -- 172.21.15.99:0/1066914285 wait complete. 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.249+0000 7fbb8919a640 1 Processor -- start 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.249+0000 7fbb8919a640 1 -- start start 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb8406e5d0 0x7fbb841f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb840d1200 0x7fbb8414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84072f20 con 0x7fbb8415c770 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbb84072c40 con 0x7fbb8406e5d0 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8919a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8415cd90 con 0x7fbb840d1200 2024-09-10T11:38:21.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53054/0 (socket says 172.21.15.99:53054) 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 -- 172.21.15.99:0/1680773714 learned_addr learned my addr 172.21.15.99:0/1680773714 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb7a59c640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb840d1200 0x7fbb8414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb82d9d640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb8406e5d0 0x7fbb841f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 -- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb840d1200 msgr2=0x7fbb8414a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb840d1200 0x7fbb8414a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 -- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb8406e5d0 msgr2=0x7fbb841f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb8406e5d0 0x7fbb841f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 -- 172.21.15.99:0/1680773714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8414db80 con 0x7fbb8415c770 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.250+0000 7fbb8359e640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00d240 tx=0x7fbb7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.251+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c044020 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.251+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb7c0349c0 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.251+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8414b8f0 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.251+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c039040 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.251+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8414beb0 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.252+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84073440 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.252+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb7c047020 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.252+0000 7fbb7bfff640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.253+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4943+0+0 (secure 0 0 0) 0x7fbb7c0ad570 con 0x7fbb8415c770 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.253+0000 7fbb82d9d640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.253+0000 7fbb82d9d640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 0x7fbb6406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb840b3520 tx=0x7fbb74009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.257+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb7c04e030 con 0x7fbb8415c770 2024-09-10T11:38:21.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.457+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb840d0c60 con 0x7fbb8415c770 2024-09-10T11:38:21.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.457+0000 7fbb7bfff640 1 -- 172.21.15.99:0/1680773714 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+3911 (secure 0 0 0) 0x7fbb8414db80 con 0x7fbb8415c770 2024-09-10T11:38:21.459 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:21.459 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":63,"btime":"2024-09-10T11:38:16:610813+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/934839542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":934839542},{"type":"v1","addr":"172.21.15.196:6835","nonce":934839542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6715,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3993700801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3993700801},{"type":"v1","addr":"172.21.15.196:6834","nonce":3993700801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7338,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/666218850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":666218850},{"type":"v1","addr":"172.21.15.99:6836","nonce":666218850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7566,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2456945166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2456945166},{"type":"v1","addr":"172.21.15.99:6837","nonce":2456945166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-09-10T11:38:21.460 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 63 2024-09-10T11:38:21.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 msgr2=0x7fbb6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 0x7fbb6406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbb840b3520 tx=0x7fbb74009110 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 msgr2=0x7fbb84154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00d240 tx=0x7fbb7c004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 shutdown_connections 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbb6406be30 0x7fbb6406e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbb8415c770 0x7fbb84154270 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb840d1200 0x7fbb8414a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 --2- 172.21.15.99:0/1680773714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb8406e5d0 0x7fbb841f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 >> 172.21.15.99:0/1680773714 conn(0x7fbb8406bc10 msgr2=0x7fbb8415e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 shutdown_connections 2024-09-10T11:38:21.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.459+0000 7fbb8919a640 1 -- 172.21.15.99:0/1680773714 wait complete. 2024-09-10T11:38:21.472 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:38:21.473 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:38:21.473 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:38:21.473 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:38:21.473 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:38:21.473 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:38:21.473 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:38:21.473 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:38:21.474 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:38:21.474 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:38:21.474 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:38:21.474 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:38:21.474 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:38:21.474 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:38:21.475 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:38:21.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:38:21.482 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:38:21.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:38:21.483 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:38:21.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:38:21.485 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:38:21.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:38:21.486 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:38:21.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:38:21.525 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:38:21.525 DEBUG:teuthology.parallel:result is None 2024-09-10T11:38:21.526 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:38:21.526 DEBUG:teuthology.parallel:result is None 2024-09-10T11:38:21.528 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:38:21.528 DEBUG:teuthology.parallel:result is None 2024-09-10T11:38:21.529 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:38:21.529 DEBUG:teuthology.parallel:result is None 2024-09-10T11:38:21.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:38:21.599 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:38:21.600 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:38:21.609 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:38:21.610 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:38:21.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.633+0000 7fc8c2918640 1 Processor -- start 2024-09-10T11:38:21.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.633+0000 7fc8c2918640 1 -- start start 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c2918640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc06fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c2918640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c2918640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8bc06d8d0 con 0x7fc8bc15f020 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c2918640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc8bc15f400 con 0x7fc8bc0b2010 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c2918640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc06eda0 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c0eb5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bbfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc8bc15f020 0x7fc8bc06e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:44778/0 (socket says 172.21.15.99:44778) 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bbfff640 1 -- 172.21.15.99:0/3048968713 learned_addr learned my addr 172.21.15.99:0/3048968713 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c0eb5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:45210/0 (socket says 172.21.15.99:45210) 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3048968713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc06fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3048968713 >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 msgr2=0x7fc8bc0703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3048968713 >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3048968713 >> v1:172.21.15.196:6789/0 conn(0x7fc8bc15f020 legacy=0x7fc8bc06e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3048968713 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc06dbb0 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.634+0000 7fc8c0eb5640 1 --2- 172.21.15.99:0/3048968713 >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.635+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3048968713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc06fea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00097a0 tx=0x7fc8b002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211ba5ef37e7e958 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.635+0000 7fc8baffd640 1 -- 172.21.15.99:0/3048968713 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0037070 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.635+0000 7fc8baffd640 1 -- 172.21.15.99:0/3048968713 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0032440 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.635+0000 7fc8baffd640 1 -- 172.21.15.99:0/3048968713 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0032810 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 -- 172.21.15.99:0/3048968713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 msgr2=0x7fc8bc06fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3048968713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc06fea0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00097a0 tx=0x7fc8b002a180 comp rx=0 tx=0).stop 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 -- 172.21.15.99:0/3048968713 shutdown_connections 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3048968713 >> v2:172.21.15.196:3300/0 conn(0x7fc8bc0b2010 0x7fc8bc0703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3048968713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc06fea0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 -- 172.21.15.99:0/3048968713 >> 172.21.15.99:0/3048968713 conn(0x7fc8bc0b45c0 msgr2=0x7fc8bc0b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:21.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 -- 172.21.15.99:0/3048968713 shutdown_connections 2024-09-10T11:38:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.636+0000 7fc8c2918640 1 -- 172.21.15.99:0/3048968713 wait complete. 2024-09-10T11:38:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.637+0000 7fc8c2918640 1 Processor -- start 2024-09-10T11:38:21.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.637+0000 7fc8c2918640 1 -- start start 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc15f020 0x7fc8bc15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc0d1fd0 con 0x7fc8bc0b2010 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc06e200 con 0x7fc8bc15f020 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c2918640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8bc0d2150 con 0x7fc8bc06f7e0 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53098/0 (socket says 172.21.15.99:53098) 2024-09-10T11:38:21.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3086540137 learned_addr learned my addr 172.21.15.99:0/3086540137 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8c0eb5640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc15f020 0x7fc8bc15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bbfff640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 msgr2=0x7fc8bc0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc15f020 msgr2=0x7fc8bc15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc15f020 0x7fc8bc15f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 -- 172.21.15.99:0/3086540137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc158710 con 0x7fc8bc0b2010 2024-09-10T11:38:21.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.638+0000 7fc8bb7fe640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098c0 tx=0x7fc8b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.640+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0037040 con 0x7fc8bc0b2010 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.640+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8b0030440 con 0x7fc8bc0b2010 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.640+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0030810 con 0x7fc8bc0b2010 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.640+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc15fe80 con 0x7fc8bc0b2010 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.640+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc160720 con 0x7fc8bc0b2010 2024-09-10T11:38:21.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.641+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8bc0761a0 con 0x7fc8bc0b2010 2024-09-10T11:38:21.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.642+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8b0044020 con 0x7fc8bc0b2010 2024-09-10T11:38:21.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.642+0000 7fc8b8ff9640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:21.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.642+0000 7fc8bbfff640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:21.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.643+0000 7fc8bbfff640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0d0350 tx=0x7fc8ac007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.646+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 4943+0+0 (secure 0 0 0) 0x7fc8b00a95b0 con 0x7fc8bc0b2010 2024-09-10T11:38:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.651+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8b0079920 con 0x7fc8bc0b2010 2024-09-10T11:38:21.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:21.830+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc8bc0b3780 con 0x7fc8bc0b2010 2024-09-10T11:38:22.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.663+0000 7fc8b8ff9640 1 -- 172.21.15.99:0/3086540137 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v163) ==== 97+0+0 (secure 0 0 0) 0x7fc8b0079220 con 0x7fc8bc0b2010 2024-09-10T11:38:22.665 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 msgr2=0x7fc8a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc0d0350 tx=0x7fc8ac007e80 comp rx=0 tx=0).stop 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 msgr2=0x7fc8bc1581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00098c0 tx=0x7fc8b00046c0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 shutdown_connections 2024-09-10T11:38:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc15f020 0x7fc8bc15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8bc0b2010 0x7fc8bc1581d0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 --2- 172.21.15.99:0/3086540137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8bc06f7e0 0x7fc8bc0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 >> 172.21.15.99:0/3086540137 conn(0x7fc8bc0b45c0 msgr2=0x7fc8bc0b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 shutdown_connections 2024-09-10T11:38:22.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.665+0000 7fc8c2918640 1 -- 172.21.15.99:0/3086540137 wait complete. 2024-09-10T11:38:22.680 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:38:22.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.781+0000 7f2baf711640 1 Processor -- start 2024-09-10T11:38:22.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.781+0000 7f2baf711640 1 -- start start 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2baf711640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2ba8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2baf711640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ba815c060 0x7f2ba815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2baf711640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ba8152e60 con 0x7f2ba80d1200 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2baf711640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ba80d15e0 con 0x7f2ba815c060 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2baf711640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba8072680 con 0x7f2ba806ece0 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2bad4ad640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2ba80d1200 0x7f2ba806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38988/0 (socket says 172.21.15.99:38988) 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.782+0000 7f2bad4ad640 1 -- 172.21.15.99:0/3379686336 learned_addr learned my addr 172.21.15.99:0/3379686336 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 --2- 172.21.15.99:0/3379686336 >> v2:172.21.15.196:3300/0 conn(0x7f2ba815c060 0x7f2ba815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2baccac640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2ba8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4109905091 0 0) 0x7f2ba8152e60 con 0x7f2ba80d1200 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b94003520 con 0x7f2ba80d1200 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 682485067 0 0) 0x7f2b94003520 con 0x7f2ba80d1200 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 msgr2=0x7f2ba8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2ba8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 >> v2:172.21.15.196:3300/0 conn(0x7f2ba815c060 msgr2=0x7f2ba815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 --2- 172.21.15.99:0/3379686336 >> v2:172.21.15.196:3300/0 conn(0x7f2ba815c060 0x7f2ba815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba8072960 con 0x7f2ba80d1200 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f2ba4001670 con 0x7f2ba80d1200 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 >> v1:172.21.15.99:6789/0 conn(0x7f2ba80d1200 legacy=0x7f2ba806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2b940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b94003f30 0x7f2b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b94004d40 0x7f2b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba80d15e0 con 0x7f2b94004d40 2024-09-10T11:38:22.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8072680 con 0x7f2b94003f30 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9400ba10 con 0x7f2ba806ece0 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b94004d40 0x7f2b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2bad4ad640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2b940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2baccac640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b94003f30 0x7f2b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 -- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 msgr2=0x7f2b940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba806ece0 0x7f2b940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 -- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b94003f30 msgr2=0x7f2b94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b94003f30 0x7f2b94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.783+0000 7f2badcae640 1 -- 172.21.15.99:0/3379686336 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b94003520 con 0x7f2b94004d40 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.784+0000 7f2badcae640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b94004d40 0x7f2b940051c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e170 tx=0x7f2b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98120fcf3386fba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.784+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0034020 con 0x7f2b94004d40 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.784+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba0030430 con 0x7f2b94004d40 2024-09-10T11:38:22.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.784+0000 7f2b9ffff640 1 -- 172.21.15.99:0/3379686336 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0009c10 con 0x7f2b94004d40 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 -- 172.21.15.99:0/3379686336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b94004d40 msgr2=0x7f2b940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 --2- 172.21.15.99:0/3379686336 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2b94004d40 0x7f2b940051c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e170 tx=0x7f2b94005830 comp rx=0 tx=0).stop 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2badcae640 1 -- 172.21.15.99:0/3379686336 reap_dead start 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 -- 172.21.15.99:0/3379686336 shutdown_connections 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 -- 172.21.15.99:0/3379686336 >> 172.21.15.99:0/3379686336 conn(0x7f2ba806bc10 msgr2=0x7f2ba806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 -- 172.21.15.99:0/3379686336 shutdown_connections 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.785+0000 7f2baf711640 1 -- 172.21.15.99:0/3379686336 wait complete. 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.786+0000 7f2baf711640 1 Processor -- start 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.786+0000 7f2baf711640 1 -- start start 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba8145770 0x7f2ba8146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba815fcf0 0x7f2ba80d0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba815cd90 con 0x7f2ba814be00 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba8072f20 con 0x7f2ba8145770 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba815c9e0 con 0x7f2ba815fcf0 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36124/0 (socket says 172.21.15.99:36124) 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 -- 172.21.15.99:0/591369983 learned_addr learned my addr 172.21.15.99:0/591369983 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2bad4ad640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba8145770 0x7f2ba8146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2badcae640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba815fcf0 0x7f2ba80d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 -- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba815fcf0 msgr2=0x7f2ba80d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba815fcf0 0x7f2ba80d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 -- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba8145770 msgr2=0x7f2ba8146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba8145770 0x7f2ba8146260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 -- 172.21.15.99:0/591369983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba8145f80 con 0x7f2ba814be00 2024-09-10T11:38:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baccac640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2b98011910 tx=0x7f2b98011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800faf0 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b9800fc90 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98014440 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.787+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba814d6a0 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.788+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba8145b50 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.789+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba815c4a0 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.789+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2b98012040 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.789+0000 7f2b9dffb640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 0x7f2b8c06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.790+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7f2b9808b240 con 0x7f2ba814be00 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.790+0000 7f2bad4ad640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 0x7f2b8c06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.790+0000 7f2bad4ad640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 0x7f2b8c06e2c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:22.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:22.794+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2b98019030 con 0x7f2ba814be00 2024-09-10T11:38:23.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.025+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2ba814d470 con 0x7f2ba814be00 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.025+0000 7f2b9dffb640 1 -- 172.21.15.99:0/591369983 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f2b98059f40 con 0x7f2ba814be00 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.027 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:38:23.028 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.029 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.031 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.033 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:38:23.035 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:38:23.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:38:23.039 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:38:23.040 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:38:23.041 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 msgr2=0x7f2b8c06e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 0x7f2b8c06e2c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4007fd0 tx=0x7f2ba4009040 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 msgr2=0x7f2ba80cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2b98011910 tx=0x7f2b98011dd0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 shutdown_connections 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2b8c06be00 0x7f2b8c06e2c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ba815fcf0 0x7f2ba80d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ba814be00 0x7f2ba80cfdd0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 --2- 172.21.15.99:0/591369983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba8145770 0x7f2ba8146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.027+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 >> 172.21.15.99:0/591369983 conn(0x7f2ba806bc10 msgr2=0x7f2ba8150080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.028+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 shutdown_connections 2024-09-10T11:38:23.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.028+0000 7f2baf711640 1 -- 172.21.15.99:0/591369983 wait complete. 2024-09-10T11:38:23.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:38:23.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.141+0000 7fec6d58d640 1 Processor -- start 2024-09-10T11:38:23.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.141+0000 7fec6d58d640 1 -- start start 2024-09-10T11:38:23.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec6d58d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec680d5010 0x7fec680d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec6d58d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6815ff70 con 0x7fec6815eb80 2024-09-10T11:38:23.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec6d58d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec680d1620 con 0x7fec681626c0 2024-09-10T11:38:23.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec6d58d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fec680d30c0 con 0x7fec680d5010 2024-09-10T11:38:23.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec66ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fec6815eb80 0x7fec680d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38996/0 (socket says 172.21.15.99:38996) 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec66ffd640 1 -- 172.21.15.99:0/1032675596 learned_addr learned my addr 172.21.15.99:0/1032675596 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec677fe640 1 --2- 172.21.15.99:0/1032675596 >> v2:172.21.15.196:3300/0 conn(0x7fec680d5010 0x7fec680d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2758518236 0 0) 0x7fec6815ff70 con 0x7fec6815eb80 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec54003520 con 0x7fec6815eb80 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2389653126 0 0) 0x7fec54003520 con 0x7fec6815eb80 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec677fe640 1 -- 172.21.15.99:0/1032675596 >> v1:172.21.15.196:6789/0 conn(0x7fec681626c0 legacy=0x7fec680d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec677fe640 1 -- 172.21.15.99:0/1032675596 >> v1:172.21.15.99:6789/0 conn(0x7fec6815eb80 legacy=0x7fec680d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.142+0000 7fec677fe640 1 -- 172.21.15.99:0/1032675596 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec68162bc0 con 0x7fec680d5010 2024-09-10T11:38:23.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.143+0000 7fec677fe640 1 --2- 172.21.15.99:0/1032675596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec680d5010 0x7fec680d0770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec58009dc0 tx=0x7fec5802a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95d3d30cb479516 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.143+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60001670 con 0x7fec680d5010 2024-09-10T11:38:23.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.143+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58007540 con 0x7fec680d5010 2024-09-10T11:38:23.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.143+0000 7fec65ffb640 1 -- 172.21.15.99:0/1032675596 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580078f0 con 0x7fec680d5010 2024-09-10T11:38:23.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 -- 172.21.15.99:0/1032675596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec680d5010 msgr2=0x7fec680d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1032675596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec680d5010 0x7fec680d0770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fec58009dc0 tx=0x7fec5802a630 comp rx=0 tx=0).stop 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 -- 172.21.15.99:0/1032675596 shutdown_connections 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1032675596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec680d5010 0x7fec680d0770 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 -- 172.21.15.99:0/1032675596 >> 172.21.15.99:0/1032675596 conn(0x7fec6808e680 msgr2=0x7fec6809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 -- 172.21.15.99:0/1032675596 shutdown_connections 2024-09-10T11:38:23.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.144+0000 7fec6d58d640 1 -- 172.21.15.99:0/1032675596 wait complete. 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.145+0000 7fec6d58d640 1 Processor -- start 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.145+0000 7fec6d58d640 1 -- start start 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6815eb80 0x7fec680b9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec681626c0 0x7fec680ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fec68163460 con 0x7fec680d5010 2024-09-10T11:38:23.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fec68163180 con 0x7fec6815eb80 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec6d58d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fec680d0cb0 con 0x7fec681626c0 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36134/0 (socket says 172.21.15.99:36134) 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 -- 172.21.15.99:0/1418415861 learned_addr learned my addr 172.21.15.99:0/1418415861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec677fe640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec681626c0 0x7fec680ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec667fc640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6815eb80 0x7fec680b9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 -- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec681626c0 msgr2=0x7fec680ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec681626c0 0x7fec680ba300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 -- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6815eb80 msgr2=0x7fec680b9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6815eb80 0x7fec680b9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.146+0000 7fec66ffd640 1 -- 172.21.15.99:0/1418415861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec680b82a0 con 0x7fec680d5010 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec66ffd640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c580 tx=0x7fec6000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60017070 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60005d00 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec680b8580 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60007330 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.147+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec680b8860 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.148+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec680d29b0 con 0x7fec680d5010 2024-09-10T11:38:23.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.148+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec6001a070 con 0x7fec680d5010 2024-09-10T11:38:23.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.149+0000 7fec5f7fe640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 0x7fec4806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.149+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7fec60089f80 con 0x7fec680d5010 2024-09-10T11:38:23.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.149+0000 7fec667fc640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 0x7fec4806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.149+0000 7fec667fc640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 0x7fec4806e320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fec681f7940 tx=0x7fec50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.153+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec60058c90 con 0x7fec680d5010 2024-09-10T11:38:23.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.319+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fec68068440 con 0x7fec680d5010 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.320+0000 7fec5f7fe640 1 -- 172.21.15.99:0/1418415861 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v163) ==== 84+0+65 (secure 0 0 0) 0x7fec60058590 con 0x7fec680d5010 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:23.322 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 msgr2=0x7fec4806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 0x7fec4806e320 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fec681f7940 tx=0x7fec50009040 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 msgr2=0x7fec681f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c580 tx=0x7fec6000ca40 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 shutdown_connections 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec4806be60 0x7fec4806e320 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec681626c0 0x7fec680ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec6815eb80 0x7fec680b9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 --2- 172.21.15.99:0/1418415861 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fec680d5010 0x7fec681f3c30 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 >> 172.21.15.99:0/1418415861 conn(0x7fec6808e680 msgr2=0x7fec680b67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 shutdown_connections 2024-09-10T11:38:23.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.322+0000 7fec6d58d640 1 -- 172.21.15.99:0/1418415861 wait complete. 2024-09-10T11:38:23.335 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:38:23.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:38:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.436+0000 7f7981e74640 1 Processor -- start 2024-09-10T11:38:23.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.436+0000 7f7981e74640 1 -- start start 2024-09-10T11:38:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f7981e74640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f797c160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f7981e74640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f797c0b5140 con 0x7f797c06bc10 2024-09-10T11:38:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f7981e74640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f797c162b40 con 0x7f797c0b4930 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f7981e74640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f797c0b5420 con 0x7f797c163620 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797b7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f797c06bc10 0x7f797c06bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39000/0 (socket says 172.21.15.99:39000) 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797b7fe640 1 -- 172.21.15.99:0/1075058509 learned_addr learned my addr 172.21.15.99:0/1075058509 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748985958 0 0) 0x7f797c0b5140 con 0x7f797c06bc10 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7964003520 con 0x7f797c06bc10 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797affd640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f797c160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120916839 0 0) 0x7f7964003520 con 0x7f797c06bc10 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 msgr2=0x7f797c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f797c160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 >> v1:172.21.15.196:6789/0 conn(0x7f797c0b4930 legacy=0x7f797c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c162e20 con 0x7f797c06bc10 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f796c001670 con 0x7f797c06bc10 2024-09-10T11:38:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.437+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 >> v1:172.21.15.99:6789/0 conn(0x7f797c06bc10 legacy=0x7f797c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f7964003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f7964004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964004d40 0x7f79640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f797c0b5420 con 0x7f79640039e0 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f796400baf0 con 0x7f7964004d40 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f796400bcf0 con 0x7f797c163620 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f7964003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797affd640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f7964004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797bfff640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964004d40 0x7f79640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 -- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 msgr2=0x7f7964004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c163620 0x7f7964004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 -- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964004d40 msgr2=0x7f79640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964004d40 0x7f79640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797b7fe640 1 -- 172.21.15.99:0/1075058509 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7964003520 con 0x7f79640039e0 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.438+0000 7f797bfff640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7964004d40 0x7f79640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f797b7fe640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f7964003e00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c210 tx=0x7f7964005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f7b24ba5d9e84c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c007d30 con 0x7f79640039e0 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c005af0 con 0x7f79640039e0 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f797a7fc640 1 -- 172.21.15.99:0/1075058509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c007070 con 0x7f79640039e0 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 -- 172.21.15.99:0/1075058509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 msgr2=0x7f7964003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 --2- 172.21.15.99:0/1075058509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f7964003e00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c210 tx=0x7f7964005830 comp rx=0 tx=0).stop 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f797bfff640 1 -- 172.21.15.99:0/1075058509 reap_dead start 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 -- 172.21.15.99:0/1075058509 shutdown_connections 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 -- 172.21.15.99:0/1075058509 >> 172.21.15.99:0/1075058509 conn(0x7f797c0b3630 msgr2=0x7f797c0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 -- 172.21.15.99:0/1075058509 shutdown_connections 2024-09-10T11:38:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.439+0000 7f7981e74640 1 -- 172.21.15.99:0/1075058509 wait complete. 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.440+0000 7f7981e74640 1 Processor -- start 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 -- start start 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f797c15a960 0x7f797c0afa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c0aff70 0x7f797c137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f797c0b4d90 con 0x7f79640039e0 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f797c0d1560 con 0x7f797c15a960 2024-09-10T11:38:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f7981e74640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f797c135320 con 0x7f797c0aff70 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f797b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f797b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36152/0 (socket says 172.21.15.99:36152) 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.441+0000 7f797b7fe640 1 -- 172.21.15.99:0/3206155545 learned_addr learned my addr 172.21.15.99:0/3206155545 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797bfff640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c0aff70 0x7f797c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797affd640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f797c15a960 0x7f797c0afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 -- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c0aff70 msgr2=0x7f797c137600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c0aff70 0x7f797c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 -- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f797c15a960 msgr2=0x7f797c0afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f797c15a960 0x7f797c0afa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 -- 172.21.15.99:0/3206155545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c135600 con 0x7f79640039e0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f797b7fe640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c270 tx=0x7f796c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c040020 con 0x7f79640039e0 2024-09-10T11:38:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c001630 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c037040 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c1358e0 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.442+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c15a530 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.443+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c072cc0 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.443+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f796c043020 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.444+0000 7f7963fff640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 0x7f796806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.444+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 4857+0+0 (secure 0 0 0) 0x7f796c0aa800 con 0x7f79640039e0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.444+0000 7f797affd640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 0x7f796806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.444+0000 7f797affd640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 0x7f796806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f797c15ad80 tx=0x7f7970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.448+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f797c072cc0 con 0x7f79640039e0 2024-09-10T11:38:23.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.613+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f797c0b0380 con 0x7f79640039e0 2024-09-10T11:38:23.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.667+0000 7f7963fff640 1 -- 172.21.15.99:0/3206155545 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v164) ==== 138+0+0 (secure 0 0 0) 0x7f797c0b0380 con 0x7f79640039e0 2024-09-10T11:38:23.669 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 msgr2=0x7f796806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 0x7f796806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f797c15ad80 tx=0x7f7970008040 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 msgr2=0x7f797c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c270 tx=0x7f796c001a40 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 shutdown_connections 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f796806be30 0x7f796806e2f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f797c0aff70 0x7f797c137600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f797c15a960 0x7f797c0afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 --2- 172.21.15.99:0/3206155545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79640039e0 0x7f797c159ff0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.669+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 >> 172.21.15.99:0/3206155545 conn(0x7f797c0b3630 msgr2=0x7f797c0725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.670+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 shutdown_connections 2024-09-10T11:38:23.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.670+0000 7f7981e74640 1 -- 172.21.15.99:0/3206155545 wait complete. 2024-09-10T11:38:23.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:38:23.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.788+0000 7f57a9786640 1 Processor -- start 2024-09-10T11:38:23.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.788+0000 7f57a9786640 1 -- start start 2024-09-10T11:38:23.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a9786640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57a415eb80 0x7f57a40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a9786640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57a40d5010 0x7f57a40d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a9786640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a415ff70 con 0x7f57a41626c0 2024-09-10T11:38:23.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a9786640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f57a40d1620 con 0x7f57a40d5010 2024-09-10T11:38:23.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a9786640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f57a4162bc0 con 0x7f57a415eb80 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57a40d5010 0x7f57a40d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f57a40d5010 0x7f57a40d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36158/0 (socket says 172.21.15.99:36158) 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 -- 172.21.15.99:0/3029830258 learned_addr learned my addr 172.21.15.99:0/3029830258 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a2ffd640 1 --2- 172.21.15.99:0/3029830258 >> v2:172.21.15.196:3300/0 conn(0x7f57a415eb80 0x7f57a40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 -- 172.21.15.99:0/3029830258 >> v2:172.21.15.196:3300/0 conn(0x7f57a415eb80 msgr2=0x7f57a40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 --2- 172.21.15.99:0/3029830258 >> v2:172.21.15.196:3300/0 conn(0x7f57a415eb80 0x7f57a40d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 -- 172.21.15.99:0/3029830258 >> v1:172.21.15.196:6789/0 conn(0x7f57a41626c0 legacy=0x7f57a40d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:23.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.789+0000 7f57a37fe640 1 -- 172.21.15.99:0/3029830258 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a4162ea0 con 0x7f57a40d5010 2024-09-10T11:38:23.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.790+0000 7f57a37fe640 1 --2- 172.21.15.99:0/3029830258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a40d5010 0x7f57a40d1b20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e50 tx=0x7f579c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b6112b3d6d50ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.790+0000 7f57a1ffb640 1 -- 172.21.15.99:0/3029830258 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c036070 con 0x7f57a40d5010 2024-09-10T11:38:23.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.790+0000 7f57a1ffb640 1 -- 172.21.15.99:0/3029830258 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c031600 con 0x7f57a40d5010 2024-09-10T11:38:23.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.790+0000 7f57a1ffb640 1 -- 172.21.15.99:0/3029830258 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c0319d0 con 0x7f57a40d5010 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 -- 172.21.15.99:0/3029830258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a40d5010 msgr2=0x7f57a40d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 --2- 172.21.15.99:0/3029830258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a40d5010 0x7f57a40d1b20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e50 tx=0x7f579c02a6d0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 -- 172.21.15.99:0/3029830258 shutdown_connections 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 --2- 172.21.15.99:0/3029830258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a40d5010 0x7f57a40d1b20 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 --2- 172.21.15.99:0/3029830258 >> v2:172.21.15.196:3300/0 conn(0x7f57a415eb80 0x7f57a40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 -- 172.21.15.99:0/3029830258 >> 172.21.15.99:0/3029830258 conn(0x7f57a408e680 msgr2=0x7f57a409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 -- 172.21.15.99:0/3029830258 shutdown_connections 2024-09-10T11:38:23.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.791+0000 7f57a9786640 1 -- 172.21.15.99:0/3029830258 wait complete. 2024-09-10T11:38:23.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.792+0000 7f57a9786640 1 Processor -- start 2024-09-10T11:38:23.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.792+0000 7f57a9786640 1 -- start start 2024-09-10T11:38:23.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57a40d5010 0x7f57a40c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a415eb80 0x7f57a40c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40d2ef0 con 0x7f57a41626c0 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4163460 con 0x7f57a415eb80 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a9786640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f57a40d3070 con 0x7f57a40d5010 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36166/0 (socket says 172.21.15.99:36166) 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 -- 172.21.15.99:0/2202318974 learned_addr learned my addr 172.21.15.99:0/2202318974 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a27fc640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a415eb80 0x7f57a40c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a2ffd640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57a40d5010 0x7f57a40c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 -- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57a40d5010 msgr2=0x7f57a40c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57a40d5010 0x7f57a40c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 -- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a415eb80 msgr2=0x7f57a40c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:23.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a415eb80 0x7f57a40c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 -- 172.21.15.99:0/2202318974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a415bda0 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.793+0000 7f57a37fe640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e50 tx=0x7f579c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.794+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c036050 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.794+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c042070 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.794+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a415c080 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.794+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c009510 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.794+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a415c640 con 0x7f57a41626c0 2024-09-10T11:38:23.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.795+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a415e080 con 0x7f57a41626c0 2024-09-10T11:38:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.795+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f579c007bd0 con 0x7f57a41626c0 2024-09-10T11:38:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.796+0000 7f578b7fe640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 0x7f579006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.796+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5227+0+0 (secure 0 0 0) 0x7f579c0ab9c0 con 0x7f57a41626c0 2024-09-10T11:38:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.796+0000 7f57a2ffd640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 0x7f579006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:23.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.796+0000 7f57a2ffd640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 0x7f579006e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57940079a0 tx=0x7f5794008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:23.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.800+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f579c0ade20 con 0x7f57a41626c0 2024-09-10T11:38:23.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:23.967+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f57a40d29b0 con 0x7f57a41626c0 2024-09-10T11:38:24.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.676+0000 7f578b7fe640 1 -- 172.21.15.99:0/2202318974 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v165) ==== 170+0+0 (secure 0 0 0) 0x7f579c079f00 con 0x7f57a41626c0 2024-09-10T11:38:24.679 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:38:24.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 msgr2=0x7f579006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 0x7f579006e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f57940079a0 tx=0x7f5794008040 comp rx=0 tx=0).stop 2024-09-10T11:38:24.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 msgr2=0x7f57a40c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f579c009e50 tx=0x7f579c004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 shutdown_connections 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f579006be30 0x7f579006e2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f57a41626c0 0x7f57a40c9370 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57a415eb80 0x7f57a40c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 --2- 172.21.15.99:0/2202318974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57a40d5010 0x7f57a40c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 >> 172.21.15.99:0/2202318974 conn(0x7f57a408e680 msgr2=0x7f57a40d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 shutdown_connections 2024-09-10T11:38:24.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.679+0000 7f57a9786640 1 -- 172.21.15.99:0/2202318974 wait complete. 2024-09-10T11:38:24.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:38:24.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.804+0000 7f7621bd8640 1 Processor -- start 2024-09-10T11:38:24.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.804+0000 7f7621bd8640 1 -- start start 2024-09-10T11:38:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f7621bd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c0d1200 0x7f761c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f7621bd8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f761c15c770 0x7f761c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f7621bd8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c152e60 con 0x7f761c06e5d0 2024-09-10T11:38:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f7621bd8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f761c06ef50 con 0x7f761c15c770 2024-09-10T11:38:24.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f7621bd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f761c0d15e0 con 0x7f761c0d1200 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f761bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f761c15c770 0x7f761c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f761bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f761c15c770 0x7f761c06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36180/0 (socket says 172.21.15.99:36180) 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f761bfff640 1 -- 172.21.15.99:0/1860117237 learned_addr learned my addr 172.21.15.99:0/1860117237 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.805+0000 7f761b7fe640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c0d1200 0x7f761c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761bfff640 1 -- 172.21.15.99:0/1860117237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c0d1200 msgr2=0x7f761c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761bfff640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c0d1200 0x7f761c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761bfff640 1 -- 172.21.15.99:0/1860117237 >> v1:172.21.15.196:6789/0 conn(0x7f761c06e5d0 legacy=0x7f761c15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761bfff640 1 -- 172.21.15.99:0/1860117237 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c072680 con 0x7f761c15c770 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761bfff640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c15c770 0x7f761c06ea10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7614009e50 tx=0x7f761402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ebbe078e6411f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.806+0000 7f761a7fc640 1 -- 172.21.15.99:0/1860117237 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614036070 con 0x7f761c15c770 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f761a7fc640 1 -- 172.21.15.99:0/1860117237 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7614031600 con 0x7f761c15c770 2024-09-10T11:38:24.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f761a7fc640 1 -- 172.21.15.99:0/1860117237 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140319d0 con 0x7f761c15c770 2024-09-10T11:38:24.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 -- 172.21.15.99:0/1860117237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c15c770 msgr2=0x7f761c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c15c770 0x7f761c06ea10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7614009e50 tx=0x7f761402a680 comp rx=0 tx=0).stop 2024-09-10T11:38:24.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 -- 172.21.15.99:0/1860117237 shutdown_connections 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c15c770 0x7f761c06ea10 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 --2- 172.21.15.99:0/1860117237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c0d1200 0x7f761c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 -- 172.21.15.99:0/1860117237 >> 172.21.15.99:0/1860117237 conn(0x7f761c06bc10 msgr2=0x7f761c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 -- 172.21.15.99:0/1860117237 shutdown_connections 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.807+0000 7f7621bd8640 1 -- 172.21.15.99:0/1860117237 wait complete. 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.808+0000 7f7621bd8640 1 Processor -- start 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.808+0000 7f7621bd8640 1 -- start start 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c06e5d0 0x7f761c14c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761c15c770 0x7f761c0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f761c072f20 con 0x7f761c0d1200 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f761c072c40 con 0x7f761c15c770 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f7621bd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f761c15cd90 con 0x7f761c06e5d0 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f761affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f761affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36192/0 (socket says 172.21.15.99:36192) 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f761affd640 1 -- 172.21.15.99:0/4116949526 learned_addr learned my addr 172.21.15.99:0/4116949526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f761b7fe640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c06e5d0 0x7f761c14c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.809+0000 7f761bfff640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761c15c770 0x7f761c0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 -- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c06e5d0 msgr2=0x7f761c14c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c06e5d0 0x7f761c14c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 -- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761c15c770 msgr2=0x7f761c0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761c15c770 0x7f761c0ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 -- 172.21.15.99:0/4116949526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c14ce70 con 0x7f761c0d1200 2024-09-10T11:38:24.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f761affd640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f761000c440 tx=0x7f761000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:24.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100052d0 con 0x7f761c0d1200 2024-09-10T11:38:24.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7610005470 con 0x7f761c0d1200 2024-09-10T11:38:24.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c14d150 con 0x7f761c0d1200 2024-09-10T11:38:24.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610007260 con 0x7f761c0d1200 2024-09-10T11:38:24.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.810+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c14d710 con 0x7f761c0d1200 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.811+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761c06c090 con 0x7f761c0d1200 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.812+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f76100074c0 con 0x7f761c0d1200 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.812+0000 7f7603fff640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 0x7f760806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.812+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5609+0+0 (secure 0 0 0) 0x7f761008a270 con 0x7f761c0d1200 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.812+0000 7f761b7fe640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 0x7f760806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:24.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.813+0000 7f761b7fe640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 0x7f760806e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f760c0079a0 tx=0x7f760c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:24.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:24.817+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76100585a0 con 0x7f761c0d1200 2024-09-10T11:38:25.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.018+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f761c14f100 con 0x7f761c0d1200 2024-09-10T11:38:25.689 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 20 and data pool 21 2024-09-10T11:38:25.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.687+0000 7f7603fff640 1 -- 172.21.15.99:0/4116949526 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v65) ==== 174+0+0 (secure 0 0 0) 0x7f761005c8a0 con 0x7f761c0d1200 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 msgr2=0x7f760806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 0x7f760806e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f760c0079a0 tx=0x7f760c008040 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 msgr2=0x7f761c0cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f761000c440 tx=0x7f761000c900 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 shutdown_connections 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f760806be30 0x7f760806e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f761c15c770 0x7f761c0ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f761c0d1200 0x7f761c0cdb80 secure :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f761000c440 tx=0x7f761000c900 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 --2- 172.21.15.99:0/4116949526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f761c06e5d0 0x7f761c14c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.701+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 >> 172.21.15.99:0/4116949526 conn(0x7f761c06bc10 msgr2=0x7f761c15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.702+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 shutdown_connections 2024-09-10T11:38:25.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.702+0000 7f7621bd8640 1 -- 172.21.15.99:0/4116949526 wait complete. 2024-09-10T11:38:25.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:25.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.817+0000 7f5ad3942640 1 Processor -- start 2024-09-10T11:38:25.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.817+0000 7f5ad3942640 1 -- start start 2024-09-10T11:38:25.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.817+0000 7f5ad3942640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5acc15eb80 0x7f5acc0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.817+0000 7f5ad3942640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc1626c0 0x7f5acc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.817+0000 7f5ad3942640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5acc0d52f0 con 0x7f5acc0d4e40 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad3942640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5acc15ff70 con 0x7f5acc15eb80 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad3942640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc0d1620 con 0x7f5acc1626c0 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5acc15eb80 0x7f5acc0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5acc15eb80 0x7f5acc0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36194/0 (socket says 172.21.15.99:36194) 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 -- 172.21.15.99:0/2786763879 learned_addr learned my addr 172.21.15.99:0/2786763879 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad0edd640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc1626c0 0x7f5acc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 -- 172.21.15.99:0/2786763879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc1626c0 msgr2=0x7f5acc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc1626c0 0x7f5acc0d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 -- 172.21.15.99:0/2786763879 >> v1:172.21.15.196:6789/0 conn(0x7f5acc0d4e40 legacy=0x7f5acc0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.818+0000 7f5ad16de640 1 -- 172.21.15.99:0/2786763879 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc162bc0 con 0x7f5acc15eb80 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad16de640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc15eb80 0x7f5acc0d29b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009870 tx=0x7f5ac402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943d6cb34863a9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5acbfff640 1 -- 172.21.15.99:0/2786763879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037070 con 0x7f5acc15eb80 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5acbfff640 1 -- 172.21.15.99:0/2786763879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac4032440 con 0x7f5acc15eb80 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5acbfff640 1 -- 172.21.15.99:0/2786763879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4032810 con 0x7f5acc15eb80 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 -- 172.21.15.99:0/2786763879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc15eb80 msgr2=0x7f5acc0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc15eb80 0x7f5acc0d29b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009870 tx=0x7f5ac402a180 comp rx=0 tx=0).stop 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 -- 172.21.15.99:0/2786763879 shutdown_connections 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc1626c0 0x7f5acc0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 --2- 172.21.15.99:0/2786763879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc15eb80 0x7f5acc0d29b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 -- 172.21.15.99:0/2786763879 >> 172.21.15.99:0/2786763879 conn(0x7f5acc08e680 msgr2=0x7f5acc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:25.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.819+0000 7f5ad3942640 1 -- 172.21.15.99:0/2786763879 shutdown_connections 2024-09-10T11:38:25.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.820+0000 7f5ad3942640 1 -- 172.21.15.99:0/2786763879 wait complete. 2024-09-10T11:38:25.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 Processor -- start 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 -- start start 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc0d4e40 0x7f5acc159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc15eb80 0x7f5acc0b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc163460 con 0x7f5acc1626c0 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5acc163180 con 0x7f5acc0d4e40 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.821+0000 7f5ad3942640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5acc0d2ef0 con 0x7f5acc15eb80 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36208/0 (socket says 172.21.15.99:36208) 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 -- 172.21.15.99:0/824716051 learned_addr learned my addr 172.21.15.99:0/824716051 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:25.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad0edd640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc15eb80 0x7f5acc0b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad16de640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc0d4e40 0x7f5acc159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 -- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc15eb80 msgr2=0x7f5acc0b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc15eb80 0x7f5acc0b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 -- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc0d4e40 msgr2=0x7f5acc159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc0d4e40 0x7f5acc159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 -- 172.21.15.99:0/824716051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5acc1e8f90 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad1edf640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b970 tx=0x7f5abc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc019070 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5abc00cae0 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5acc1e9270 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc014440 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.822+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5acc15a3c0 con 0x7f5acc1626c0 2024-09-10T11:38:25.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.823+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5acc1e9660 con 0x7f5acc1626c0 2024-09-10T11:38:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.824+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5abc01c070 con 0x7f5acc1626c0 2024-09-10T11:38:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.824+0000 7f5ac9ffb640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.824+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5709+0+0 (secure 0 0 0) 0x7f5abc08a900 con 0x7f5acc1626c0 2024-09-10T11:38:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.824+0000 7f5ad16de640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:25.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.824+0000 7f5ad16de640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 0x7f5ab806e320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4034000 tx=0x7f5ac402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:25.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.828+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5abc058cf0 con 0x7f5acc1626c0 2024-09-10T11:38:25.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.995+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5acc0d29b0 con 0x7f5acc1626c0 2024-09-10T11:38:25.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.996+0000 7f5ac9ffb640 1 -- 172.21.15.99:0/824716051 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16753 (secure 0 0 0) 0x7f5abc05cff0 con 0x7f5acc1626c0 2024-09-10T11:38:25.998 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:25.998 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":166,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:38:25.020365+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:38:23.632388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T11:38:23.986527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:26.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 msgr2=0x7f5ab806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 0x7f5ab806e320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4034000 tx=0x7f5ac402e000 comp rx=0 tx=0).stop 2024-09-10T11:38:26.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 msgr2=0x7f5acc15bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00b970 tx=0x7f5abc00be30 comp rx=0 tx=0).stop 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 shutdown_connections 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5acc1626c0 0x7f5acc15bd00 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5acc15eb80 0x7f5acc0b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 --2- 172.21.15.99:0/824716051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5acc0d4e40 0x7f5acc159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 >> 172.21.15.99:0/824716051 conn(0x7f5acc08e680 msgr2=0x7f5acc15d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 shutdown_connections 2024-09-10T11:38:26.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:25.998+0000 7f5ad3942640 1 -- 172.21.15.99:0/824716051 wait complete. 2024-09-10T11:38:26.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:26.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.107+0000 7fe190333640 1 Processor -- start 2024-09-10T11:38:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 -- start start 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe18806f0d0 0x7fe188162d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1880b38a0 con 0x7fe188162910 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe18806dc30 con 0x7fe18806f0d0 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.108+0000 7fe190333640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe188163290 con 0x7fe188148620 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe18806f0d0 0x7fe188162d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe18806f0d0 0x7fe188162d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36222/0 (socket says 172.21.15.99:36222) 2024-09-10T11:38:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 -- 172.21.15.99:0/773603577 learned_addr learned my addr 172.21.15.99:0/773603577 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18d8ce640 1 --1- 172.21.15.99:0/773603577 >> v1:172.21.15.99:6789/0 conn(0x7fe188162910 0x7fe1880cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39006/0 (socket says 172.21.15.99:39006) 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 -- 172.21.15.99:0/773603577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 msgr2=0x7fe1880b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 --2- 172.21.15.99:0/773603577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 -- 172.21.15.99:0/773603577 >> v1:172.21.15.99:6789/0 conn(0x7fe188162910 legacy=0x7fe1880cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18e8d0640 1 -- 172.21.15.99:0/773603577 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1880d00c0 con 0x7fe18806f0d0 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.109+0000 7fe18d0cd640 1 -- 172.21.15.99:0/773603577 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1253814442 0 0) 0x7fe1880b38a0 con 0x7fe188162910 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe18e8d0640 1 --2- 172.21.15.99:0/773603577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18806f0d0 0x7fe188162d50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe180009e50 tx=0x7fe18002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f892a242e5eed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe18d0cd640 1 -- 172.21.15.99:0/773603577 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180036070 con 0x7fe18806f0d0 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe18d0cd640 1 -- 172.21.15.99:0/773603577 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe180031600 con 0x7fe18806f0d0 2024-09-10T11:38:26.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe18d0cd640 1 -- 172.21.15.99:0/773603577 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1800319d0 con 0x7fe18806f0d0 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 -- 172.21.15.99:0/773603577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18806f0d0 msgr2=0x7fe188162d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 --2- 172.21.15.99:0/773603577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18806f0d0 0x7fe188162d50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe180009e50 tx=0x7fe18002a710 comp rx=0 tx=0).stop 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 -- 172.21.15.99:0/773603577 shutdown_connections 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 --2- 172.21.15.99:0/773603577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18806f0d0 0x7fe188162d50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 --2- 172.21.15.99:0/773603577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 -- 172.21.15.99:0/773603577 >> 172.21.15.99:0/773603577 conn(0x7fe1880b2a80 msgr2=0x7fe1880b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 -- 172.21.15.99:0/773603577 shutdown_connections 2024-09-10T11:38:26.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.110+0000 7fe190333640 1 -- 172.21.15.99:0/773603577 wait complete. 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 Processor -- start 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 -- start start 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe18806f0d0 0x7fe1880cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe18806d8c0 con 0x7fe188162910 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880d0680 con 0x7fe18806f0d0 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe190333640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe1880b6000 con 0x7fe188148620 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe18e8d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe18e8d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36230/0 (socket says 172.21.15.99:36230) 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.112+0000 7fe18e8d0640 1 -- 172.21.15.99:0/3733736743 learned_addr learned my addr 172.21.15.99:0/3733736743 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 -- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 msgr2=0x7fe1880ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 -- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe18806f0d0 msgr2=0x7fe1880cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe18806f0d0 0x7fe1880cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 -- 172.21.15.99:0/3733736743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1880cb160 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe18e8d0640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe180034040 tx=0x7fe180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180041070 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe180003990 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180036040 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe188147550 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.113+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881f50e0 con 0x7fe188162910 2024-09-10T11:38:26.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.114+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1880b5600 con 0x7fe188162910 2024-09-10T11:38:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.115+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe180007800 con 0x7fe188162910 2024-09-10T11:38:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.115+0000 7fe17affd640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 0x7fe16c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.115+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5709+0+0 (secure 0 0 0) 0x7fe1800aba80 con 0x7fe188162910 2024-09-10T11:38:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.115+0000 7fe18e0cf640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 0x7fe16c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.115+0000 7fe18e0cf640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 0x7fe16c06e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe18809e7a0 tx=0x7fe184006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.119+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe1880b5600 con 0x7fe188162910 2024-09-10T11:38:26.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.285+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe188148a00 con 0x7fe188162910 2024-09-10T11:38:26.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.286+0000 7fe17affd640 1 -- 172.21.15.99:0/3733736743 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) ==== 74+0+16753 (secure 0 0 0) 0x7fe18004b020 con 0x7fe188162910 2024-09-10T11:38:26.288 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:26.289 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":166,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:38:25.020365+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:38:23.632388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T11:38:23.986527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:26.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 msgr2=0x7fe16c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 0x7fe16c06e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe18809e7a0 tx=0x7fe184006c90 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 msgr2=0x7fe188147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe180034040 tx=0x7fe180004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 shutdown_connections 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe16c06be30 0x7fe16c06e2f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe188162910 0x7fe188147010 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe188148620 0x7fe1880ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 --2- 172.21.15.99:0/3733736743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe18806f0d0 0x7fe1880cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.288+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 >> 172.21.15.99:0/3733736743 conn(0x7fe1880b2a80 msgr2=0x7fe188072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.289+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 shutdown_connections 2024-09-10T11:38:26.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.289+0000 7fe190333640 1 -- 172.21.15.99:0/3733736743 wait complete. 2024-09-10T11:38:26.301 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:38:26.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.400+0000 7fe8a60ac640 1 Processor -- start 2024-09-10T11:38:26.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.400+0000 7fe8a60ac640 1 -- start start 2024-09-10T11:38:26.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe8a60ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 0x7fe8a00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe8a60ac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8a015b2d0 0x7fe8a00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe8a60ac640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a006ee10 con 0x7fe8a00cd290 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe8a60ac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a00b0290 con 0x7fe8a015b2d0 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe8a60ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a006e570 con 0x7fe8a015aef0 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe8a00cd290 0x7fe8a00714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39020/0 (socket says 172.21.15.99:39020) 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89ffff640 1 -- 172.21.15.99:0/1883954397 learned_addr learned my addr 172.21.15.99:0/1883954397 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1883954397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 0x7fe8a00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89effd640 1 --2- 172.21.15.99:0/1883954397 >> v2:172.21.15.196:3300/0 conn(0x7fe8a015b2d0 0x7fe8a00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1528209913 0 0) 0x7fe8a006ee10 con 0x7fe8a00cd290 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe888003520 con 0x7fe8a00cd290 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89f7fe640 1 -- 172.21.15.99:0/1883954397 >> v2:172.21.15.196:3300/0 conn(0x7fe8a015b2d0 msgr2=0x7fe8a00afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:26.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1883954397 >> v2:172.21.15.196:3300/0 conn(0x7fe8a015b2d0 0x7fe8a00afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89f7fe640 1 -- 172.21.15.99:0/1883954397 >> v1:172.21.15.99:6789/0 conn(0x7fe8a00cd290 legacy=0x7fe8a00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3345284210 0 0) 0x7fe888003520 con 0x7fe8a00cd290 2024-09-10T11:38:26.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.401+0000 7fe89f7fe640 1 -- 172.21.15.99:0/1883954397 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a006e850 con 0x7fe8a015aef0 2024-09-10T11:38:26.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.402+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1883954397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 0x7fe8a00b4510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe89000a940 tx=0x7fe89002b550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab8e669575e8b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.402+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000b040 con 0x7fe8a015aef0 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.402+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe890030710 con 0x7fe8a015aef0 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.402+0000 7fe89e7fc640 1 -- 172.21.15.99:0/1883954397 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890030ac0 con 0x7fe8a015aef0 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1883954397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 msgr2=0x7fe8a00b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1883954397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 0x7fe8a00b4510 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe89000a940 tx=0x7fe89002b550 comp rx=0 tx=0).stop 2024-09-10T11:38:26.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1883954397 shutdown_connections 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1883954397 >> v2:172.21.15.196:3300/0 conn(0x7fe8a015b2d0 0x7fe8a00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1883954397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015aef0 0x7fe8a00b4510 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1883954397 >> 172.21.15.99:0/1883954397 conn(0x7fe8a006bc10 msgr2=0x7fe8a006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1883954397 shutdown_connections 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.403+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1883954397 wait complete. 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.404+0000 7fe8a60ac640 1 Processor -- start 2024-09-10T11:38:26.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.404+0000 7fe8a60ac640 1 -- start start 2024-09-10T11:38:26.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 0x7fe8a01efe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 0x7fe8a0142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0071d30 con 0x7fe8a015aef0 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0071bb0 con 0x7fe8a00cd290 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe8a60ac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a0071eb0 con 0x7fe8a015b2d0 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36246/0 (socket says 172.21.15.99:36246) 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 -- 172.21.15.99:0/1791594059 learned_addr learned my addr 172.21.15.99:0/1791594059 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89ffff640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 0x7fe8a0142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 0x7fe8a01efe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 -- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 msgr2=0x7fe8a0142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 0x7fe8a0142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 -- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 msgr2=0x7fe8a01efe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 0x7fe8a01efe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89effd640 1 -- 172.21.15.99:0/1791594059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01411f0 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89ffff640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 0x7fe8a0142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.405+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 0x7fe8a01efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe89effd640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9b0 tx=0x7fe89400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894017070 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe89400c9a0 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400cd90 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a0143620 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.406+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a0143be0 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.407+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a00ce8e0 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.407+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe89401a070 con 0x7fe8a015aef0 2024-09-10T11:38:26.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.408+0000 7fe887fff640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 0x7fe88c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.408+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5709+0+0 (secure 0 0 0) 0x7fe89408ae90 con 0x7fe8a015aef0 2024-09-10T11:38:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.408+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 0x7fe88c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.408+0000 7fe89f7fe640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 0x7fe88c06e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe890035000 tx=0x7fe89000c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.412+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe894059840 con 0x7fe8a015aef0 2024-09-10T11:38:26.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.610+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe8a015e860 con 0x7fe8a015aef0 2024-09-10T11:38:26.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.691+0000 7fe887fff640 1 -- 172.21.15.99:0/1791594059 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7fe894059140 con 0x7fe8a015aef0 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 msgr2=0x7fe88c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 0x7fe88c06e2f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe890035000 tx=0x7fe89000c040 comp rx=0 tx=0).stop 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 msgr2=0x7fe8a0140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe89400e9b0 tx=0x7fe89400ee70 comp rx=0 tx=0).stop 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 shutdown_connections 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe88c06be30 0x7fe88c06e2f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8a015b2d0 0x7fe8a0142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8a015aef0 0x7fe8a0140cb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 --2- 172.21.15.99:0/1791594059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8a00cd290 0x7fe8a01efe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 >> 172.21.15.99:0/1791594059 conn(0x7fe8a006bc10 msgr2=0x7fe8a0072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 shutdown_connections 2024-09-10T11:38:26.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.693+0000 7fe8a60ac640 1 -- 172.21.15.99:0/1791594059 wait complete. 2024-09-10T11:38:26.706 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:38:26.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:38:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.805+0000 7fb66f6f8640 1 Processor -- start 2024-09-10T11:38:26.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.806+0000 7fb66f6f8640 1 -- start start 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.806+0000 7fb66f6f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 0x7fb668163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.806+0000 7fb66f6f8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6680b5140 con 0x7fb6680b4930 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66f6f8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb668162b40 con 0x7fb668160030 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66f6f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680b5420 con 0x7fb66806bc10 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66dc95640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb6680b4930 0x7fb66806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39034/0 (socket says 172.21.15.99:39034) 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66dc95640 1 -- 172.21.15.99:0/2676787036 learned_addr learned my addr 172.21.15.99:0/2676787036 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66d494640 1 --2- 172.21.15.99:0/2676787036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 0x7fb668163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 562727647 0 0) 0x7fb6680b5140 con 0x7fb6680b4930 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003520 con 0x7fb6680b4930 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3096624094 0 0) 0x7fb668162b40 con 0x7fb668160030 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6680b5140 con 0x7fb668160030 2024-09-10T11:38:26.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66d494640 1 -- 172.21.15.99:0/2676787036 >> v1:172.21.15.196:6789/0 conn(0x7fb668160030 legacy=0x7fb66806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2077768948 0 0) 0x7fb654003520 con 0x7fb6680b4930 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66d494640 1 -- 172.21.15.99:0/2676787036 >> v1:172.21.15.99:6789/0 conn(0x7fb6680b4930 legacy=0x7fb66806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.807+0000 7fb66d494640 1 -- 172.21.15.99:0/2676787036 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb668162e20 con 0x7fb66806bc10 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.808+0000 7fb66d494640 1 --2- 172.21.15.99:0/2676787036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 0x7fb668163620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097d0 tx=0x7fb66402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed1f02236932dde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.808+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664037070 con 0x7fb66806bc10 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.808+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66402ab90 con 0x7fb66806bc10 2024-09-10T11:38:26.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.808+0000 7fb65ffff640 1 -- 172.21.15.99:0/2676787036 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600019f0 con 0x7fb66806bc10 2024-09-10T11:38:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 -- 172.21.15.99:0/2676787036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 msgr2=0x7fb668163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/2676787036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 0x7fb668163620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097d0 tx=0x7fb66402a100 comp rx=0 tx=0).stop 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 -- 172.21.15.99:0/2676787036 shutdown_connections 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/2676787036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb66806bc10 0x7fb668163620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 -- 172.21.15.99:0/2676787036 >> 172.21.15.99:0/2676787036 conn(0x7fb6680b3630 msgr2=0x7fb6680b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 -- 172.21.15.99:0/2676787036 shutdown_connections 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.809+0000 7fb66f6f8640 1 -- 172.21.15.99:0/2676787036 wait complete. 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.810+0000 7fb66f6f8640 1 Processor -- start 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.810+0000 7fb66f6f8640 1 -- start start 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6680b4930 0x7fb6681f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb668160030 0x7fb66815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680d1560 con 0x7fb66806bc10 2024-09-10T11:38:26.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb6680b4db0 con 0x7fb668160030 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb6680d16e0 con 0x7fb6680b4930 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36260/0 (socket says 172.21.15.99:36260) 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 -- 172.21.15.99:0/3162404741 learned_addr learned my addr 172.21.15.99:0/3162404741 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66dc95640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb668160030 0x7fb66815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66cc93640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6680b4930 0x7fb6681f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 -- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6680b4930 msgr2=0x7fb6681f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6680b4930 0x7fb6681f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 -- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb668160030 msgr2=0x7fb66815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb668160030 0x7fb66815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 -- 172.21.15.99:0/3162404741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66815c500 con 0x7fb66806bc10 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66d494640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097d0 tx=0x7fb6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664041070 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb66403c680 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66815c7e0 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.811+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb664037040 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.812+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66815cda0 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.812+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6680b56c0 con 0x7fb66806bc10 2024-09-10T11:38:26.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.813+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb664044070 con 0x7fb66806bc10 2024-09-10T11:38:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.814+0000 7fb65dffb640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.814+0000 7fb66cc93640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.814+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fb6640ab4d0 con 0x7fb66806bc10 2024-09-10T11:38:26.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.814+0000 7fb66cc93640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 0x7fb64c06e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6680d10b0 tx=0x7fb658009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:26.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:26.818+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6680b56c0 con 0x7fb66806bc10 2024-09-10T11:38:26.941 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:38:26.939+0000 7fa7f93ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:38:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.018+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb668162b40 con 0x7fb66806bc10 2024-09-10T11:38:27.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.699+0000 7fb65dffb640 1 -- 172.21.15.99:0/3162404741 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7fb6640a9e20 con 0x7fb66806bc10 2024-09-10T11:38:27.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 msgr2=0x7fb64c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 0x7fb64c06e2f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6680d10b0 tx=0x7fb658009040 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 msgr2=0x7fb6680d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb6640097d0 tx=0x7fb6640046c0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 shutdown_connections 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb64c06be30 0x7fb64c06e2f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb668160030 0x7fb66815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb6680b4930 0x7fb6681f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 --2- 172.21.15.99:0/3162404741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb66806bc10 0x7fb6680d0460 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 >> 172.21.15.99:0/3162404741 conn(0x7fb6680b3630 msgr2=0x7fb6680b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 shutdown_connections 2024-09-10T11:38:27.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.702+0000 7fb66f6f8640 1 -- 172.21.15.99:0/3162404741 wait complete. 2024-09-10T11:38:27.716 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:38:27.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:27.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 Processor -- start 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 -- start start 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 0x7f77a0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f77a015c060 0x7f77a015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a0152e60 con 0x7f77a00d1200 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f77a00d15e0 con 0x7f77a015c060 2024-09-10T11:38:27.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.816+0000 7f77a6cc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77a0072680 con 0x7f77a006ece0 2024-09-10T11:38:27.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f779ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 0x7f77a0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a5266640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f77a015c060 0x7f77a015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a4a65640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f77a00d1200 0x7f77a006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:41674/0 (socket says 172.21.15.99:41674) 2024-09-10T11:38:27.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f779ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 0x7f77a0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:37730/0 (socket says 172.21.15.99:37730) 2024-09-10T11:38:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a4a65640 1 -- 172.21.15.99:0/3486405437 learned_addr learned my addr 172.21.15.99:0/3486405437 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a5266640 1 -- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 msgr2=0x7f77a0073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a5266640 1 --2- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 0x7f77a0073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a5266640 1 -- 172.21.15.99:0/3486405437 >> v1:172.21.15.196:6789/0 conn(0x7f77a00d1200 legacy=0x7f77a006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.817+0000 7f77a5266640 1 -- 172.21.15.99:0/3486405437 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a0072960 con 0x7f77a015c060 2024-09-10T11:38:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.818+0000 7f77a5266640 1 --2- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a015c060 0x7f77a015c4a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7798009d90 tx=0x7f779802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a90c1da410c3c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.818+0000 7f779f7fe640 1 -- 172.21.15.99:0/3486405437 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798031600 con 0x7f77a015c060 2024-09-10T11:38:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.818+0000 7f779f7fe640 1 -- 172.21.15.99:0/3486405437 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77980317a0 con 0x7f77a015c060 2024-09-10T11:38:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.818+0000 7f779f7fe640 1 -- 172.21.15.99:0/3486405437 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798031b70 con 0x7f77a015c060 2024-09-10T11:38:27.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 -- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a015c060 msgr2=0x7f77a015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a015c060 0x7f77a015c4a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7798009d90 tx=0x7f779802a5d0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 -- 172.21.15.99:0/3486405437 shutdown_connections 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a015c060 0x7f77a015c4a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/3486405437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a006ece0 0x7f77a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 -- 172.21.15.99:0/3486405437 >> 172.21.15.99:0/3486405437 conn(0x7f77a006bc10 msgr2=0x7f77a006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 -- 172.21.15.99:0/3486405437 shutdown_connections 2024-09-10T11:38:27.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.819+0000 7f77a6cc9640 1 -- 172.21.15.99:0/3486405437 wait complete. 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.820+0000 7f77a6cc9640 1 Processor -- start 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.820+0000 7f77a6cc9640 1 -- start start 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a006ece0 0x7f77a00ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a00d1200 0x7f77a00ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77a015cd90 con 0x7f77a015c060 2024-09-10T11:38:27.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f77a0072f20 con 0x7f77a006ece0 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a6cc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f77a015c9e0 con 0x7f77a00d1200 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a5266640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a5266640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36268/0 (socket says 172.21.15.99:36268) 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a5266640 1 -- 172.21.15.99:0/4280511912 learned_addr learned my addr 172.21.15.99:0/4280511912 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f77a4a65640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a006ece0 0x7f77a00ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.821+0000 7f779ffff640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a00d1200 0x7f77a00ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 -- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a00d1200 msgr2=0x7f77a00ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a00d1200 0x7f77a00ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 -- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a006ece0 msgr2=0x7f77a00ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a006ece0 0x7f77a00ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 -- 172.21.15.99:0/4280511912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a01406f0 con 0x7f77a015c060 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a5266640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7798009eb0 tx=0x7f7798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798040070 con 0x7f77a015c060 2024-09-10T11:38:27.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7798030b60 con 0x7f77a015c060 2024-09-10T11:38:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a01409d0 con 0x7f77a015c060 2024-09-10T11:38:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798009450 con 0x7f77a015c060 2024-09-10T11:38:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.822+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a0140f90 con 0x7f77a015c060 2024-09-10T11:38:27.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.823+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a0073440 con 0x7f77a015c060 2024-09-10T11:38:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.823+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7798039040 con 0x7f77a015c060 2024-09-10T11:38:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.824+0000 7f779d7fa640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 0x7f778406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.824+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f77980ac3d0 con 0x7f77a015c060 2024-09-10T11:38:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.824+0000 7f77a4a65640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 0x7f778406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:27.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.824+0000 7f77a4a65640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 0x7f778406e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f77a00ad3f0 tx=0x7f7794007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:27.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:27.828+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f779807a730 con 0x7f77a015c060 2024-09-10T11:38:28.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.031+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77a00ccec0 con 0x7f77a015c060 2024-09-10T11:38:28.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.032+0000 7f779d7fa640 1 -- 172.21.15.99:0/4280511912 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f779807ea30 con 0x7f77a015c060 2024-09-10T11:38:28.034 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:28.034 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:38:28.034 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 msgr2=0x7f778406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 0x7f778406e2f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f77a00ad3f0 tx=0x7f7794007e80 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 msgr2=0x7f77a00b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f7798009eb0 tx=0x7f7798004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 shutdown_connections 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f778406be30 0x7f778406e2f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a015c060 0x7f77a00b0040 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77a00d1200 0x7f77a00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 --2- 172.21.15.99:0/4280511912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77a006ece0 0x7f77a00ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 >> 172.21.15.99:0/4280511912 conn(0x7f77a006bc10 msgr2=0x7f77a01e7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 shutdown_connections 2024-09-10T11:38:28.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.034+0000 7f77a6cc9640 1 -- 172.21.15.99:0/4280511912 wait complete. 2024-09-10T11:38:28.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:38:28.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.143+0000 7f4f04db7640 1 Processor -- start 2024-09-10T11:38:28.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.143+0000 7f4f04db7640 1 -- start start 2024-09-10T11:38:28.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4f04db7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f0015c7b0 0x7f4f000d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4f04db7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4f0006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4f04db7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f000ceb10 con 0x7f4f000d0fb0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4f04db7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4f000d1420 con 0x7f4f0015c7b0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4f04db7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f000b3840 con 0x7f4f0006f000 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efdd9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4f000d0fb0 0x7f4f0006e8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39038/0 (socket says 172.21.15.99:39038) 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efdd9b640 1 -- 172.21.15.99:0/373191757 learned_addr learned my addr 172.21.15.99:0/373191757 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efe59c640 1 --2- 172.21.15.99:0/373191757 >> v2:172.21.15.196:3300/0 conn(0x7f4f0015c7b0 0x7f4f000d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046163916 0 0) 0x7f4ef4001110 con 0x7f4f000d0fb0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ee8003520 con 0x7f4f000d0fb0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.144+0000 7f4efed9d640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4f0006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1440902965 0 0) 0x7f4ee8003520 con 0x7f4f000d0fb0 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 msgr2=0x7f4f0006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4f0006d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 >> v2:172.21.15.196:3300/0 conn(0x7f4f0015c7b0 msgr2=0x7f4f000d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 --2- 172.21.15.99:0/373191757 >> v2:172.21.15.196:3300/0 conn(0x7f4f0015c7b0 0x7f4f000d0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000b3a90 con 0x7f4f000d0fb0 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4ef4001840 con 0x7f4f000d0fb0 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 >> v1:172.21.15.99:6789/0 conn(0x7f4f000d0fb0 legacy=0x7f4f0006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4ee80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4ee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee8004d40 0x7f4ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000d1420 con 0x7f4ee8003f30 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b3840 con 0x7f4ee8004d40 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee800ba10 con 0x7f4f0006f000 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4ee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efed9d640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee8004d40 0x7f4ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efe59c640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4ee80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 -- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 msgr2=0x7f4ee80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f0006f000 0x7f4ee80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 -- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee8004d40 msgr2=0x7f4ee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ee8004d40 0x7f4ee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.145+0000 7f4efdd9b640 1 -- 172.21.15.99:0/373191757 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee8003520 con 0x7f4ee8003f30 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4ee8004800 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c3e0 tx=0x7f4ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5bd17e84c10707 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4032020 con 0x7f4ee8003f30 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4005ad0 con 0x7f4ee8003f30 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4efd59a640 1 -- 172.21.15.99:0/373191757 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400c0e0 con 0x7f4ee8003f30 2024-09-10T11:38:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4f04db7640 1 -- 172.21.15.99:0/373191757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 msgr2=0x7f4ee8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4f04db7640 1 --2- 172.21.15.99:0/373191757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4ee8004800 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c3e0 tx=0x7f4ee8005830 comp rx=0 tx=0).stop 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.146+0000 7f4efed9d640 1 -- 172.21.15.99:0/373191757 reap_dead start 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.147+0000 7f4f04db7640 1 -- 172.21.15.99:0/373191757 shutdown_connections 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.147+0000 7f4f04db7640 1 -- 172.21.15.99:0/373191757 >> 172.21.15.99:0/373191757 conn(0x7f4f0006bc10 msgr2=0x7f4f0006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.147+0000 7f4f04db7640 1 -- 172.21.15.99:0/373191757 shutdown_connections 2024-09-10T11:38:28.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.147+0000 7f4f04db7640 1 -- 172.21.15.99:0/373191757 wait complete. 2024-09-10T11:38:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.148+0000 7f4f04db7640 1 Processor -- start 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.148+0000 7f4f04db7640 1 -- start start 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 0x7f4f0015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f001e62f0 0x7f4f001e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b4330 con 0x7f4ee8003f30 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f000b4050 con 0x7f4f0015a590 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4f000b4610 con 0x7f4f001e62f0 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36284/0 (socket says 172.21.15.99:36284) 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 -- 172.21.15.99:0/2025382541 learned_addr learned my addr 172.21.15.99:0/2025382541 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efed9d640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f001e62f0 0x7f4f001e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 0x7f4f0015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 -- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f001e62f0 msgr2=0x7f4f001e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f001e62f0 0x7f4f001e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 -- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 msgr2=0x7f4f0015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 0x7f4f0015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 -- 172.21.15.99:0/2025382541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f001e6cb0 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 0x7f4f0015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4efe59c640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0011a70 tx=0x7f4ef0011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000fad0 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef000fc70 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f001e6f70 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0014440 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.149+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f001e7530 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.150+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0015cb90 con 0x7f4ee8003f30 2024-09-10T11:38:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.151+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ef0012040 con 0x7f4ee8003f30 2024-09-10T11:38:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.151+0000 7f4eeeffd640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 0x7f4ee006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.151+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef008b480 con 0x7f4ee8003f30 2024-09-10T11:38:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.151+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 0x7f4ee006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.152+0000 7f4efdd9b640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 0x7f4ee006e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c3b0 tx=0x7f4ef4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.155+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4ef0019030 con 0x7f4ee8003f30 2024-09-10T11:38:28.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.322+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f001582a0 con 0x7f4ee8003f30 2024-09-10T11:38:28.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.323+0000 7f4eeeffd640 1 -- 172.21.15.99:0/2025382541 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16740 (secure 0 0 0) 0x7f4ef0059e30 con 0x7f4ee8003f30 2024-09-10T11:38:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:28.325 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":167,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:38:26.679588+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:38:23.632388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T11:38:23.986527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:38:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 msgr2=0x7f4ee006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 0x7f4ee006e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400c3b0 tx=0x7f4ef4009030 comp rx=0 tx=0).stop 2024-09-10T11:38:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 msgr2=0x7f4f00143170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0011a70 tx=0x7f4ef0011f30 comp rx=0 tx=0).stop 2024-09-10T11:38:28.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 shutdown_connections 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4ee006be30 0x7f4ee006e2f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f001e62f0 0x7f4f001e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f0015a590 0x7f4f0015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 --2- 172.21.15.99:0/2025382541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee8003f30 0x7f4f00143170 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 >> 172.21.15.99:0/2025382541 conn(0x7f4f0006bc10 msgr2=0x7f4f000b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 shutdown_connections 2024-09-10T11:38:28.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.325+0000 7f4f04db7640 1 -- 172.21.15.99:0/2025382541 wait complete. 2024-09-10T11:38:28.338 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:38:28.339 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:38:28.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.435+0000 7f71ea427640 1 Processor -- start 2024-09-10T11:38:28.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.435+0000 7f71ea427640 1 -- start start 2024-09-10T11:38:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71ea427640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71e406bc10 0x7f71e4163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71ea427640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f71e4160030 0x7f71e406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71ea427640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71e406c530 con 0x7f71e40b4930 2024-09-10T11:38:28.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71ea427640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f71e40b5420 con 0x7f71e4160030 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71ea427640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f71e40b5140 con 0x7f71e406bc10 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f71e4160030 0x7f71e406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f71e4160030 0x7f71e406bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36286/0 (socket says 172.21.15.99:36286) 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 -- 172.21.15.99:0/399560361 learned_addr learned my addr 172.21.15.99:0/399560361 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e89c4640 1 --1- 172.21.15.99:0/399560361 >> v1:172.21.15.99:6789/0 conn(0x7f71e40b4930 0x7f71e406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39054/0 (socket says 172.21.15.99:39054) 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e3fff640 1 --2- 172.21.15.99:0/399560361 >> v2:172.21.15.196:3300/0 conn(0x7f71e406bc10 0x7f71e4163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 -- 172.21.15.99:0/399560361 >> v2:172.21.15.196:3300/0 conn(0x7f71e406bc10 msgr2=0x7f71e4163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 --2- 172.21.15.99:0/399560361 >> v2:172.21.15.196:3300/0 conn(0x7f71e406bc10 0x7f71e4163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 -- 172.21.15.99:0/399560361 >> v1:172.21.15.99:6789/0 conn(0x7f71e40b4930 legacy=0x7f71e406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e2ffd640 1 -- 172.21.15.99:0/399560361 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060003612 0 0) 0x7f71e406c530 con 0x7f71e40b4930 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.436+0000 7f71e37fe640 1 -- 172.21.15.99:0/399560361 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e4162b40 con 0x7f71e4160030 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.437+0000 7f71e37fe640 1 --2- 172.21.15.99:0/399560361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e4160030 0x7f71e406bff0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f71e4073870 tx=0x7f71d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34041638a08e0e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.437+0000 7f71e2ffd640 1 -- 172.21.15.99:0/399560361 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8037070 con 0x7f71e4160030 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.437+0000 7f71e2ffd640 1 -- 172.21.15.99:0/399560361 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8032440 con 0x7f71e4160030 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.437+0000 7f71e2ffd640 1 -- 172.21.15.99:0/399560361 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8032810 con 0x7f71e4160030 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 -- 172.21.15.99:0/399560361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e4160030 msgr2=0x7f71e406bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 --2- 172.21.15.99:0/399560361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e4160030 0x7f71e406bff0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f71e4073870 tx=0x7f71d802a1e0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 -- 172.21.15.99:0/399560361 shutdown_connections 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 --2- 172.21.15.99:0/399560361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e4160030 0x7f71e406bff0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 --2- 172.21.15.99:0/399560361 >> v2:172.21.15.196:3300/0 conn(0x7f71e406bc10 0x7f71e4163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 -- 172.21.15.99:0/399560361 >> 172.21.15.99:0/399560361 conn(0x7f71e40b3630 msgr2=0x7f71e40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 -- 172.21.15.99:0/399560361 shutdown_connections 2024-09-10T11:38:28.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.438+0000 7f71ea427640 1 -- 172.21.15.99:0/399560361 wait complete. 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.439+0000 7f71ea427640 1 Processor -- start 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.439+0000 7f71ea427640 1 -- start start 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71e40b4930 0x7f71e41444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71e4160030 0x7f71e4144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71e40b4dc0 con 0x7f71e406bc10 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71e4163100 con 0x7f71e40b4930 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f71e40d1560 con 0x7f71e4160030 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36296/0 (socket says 172.21.15.99:36296) 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 -- 172.21.15.99:0/1714890567 learned_addr learned my addr 172.21.15.99:0/1714890567 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e89c4640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71e4160030 0x7f71e4144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e37fe640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71e40b4930 0x7f71e41444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 -- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71e4160030 msgr2=0x7f71e4144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71e4160030 0x7f71e4144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 -- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71e40b4930 msgr2=0x7f71e41444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71e40b4930 0x7f71e41444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 -- 172.21.15.99:0/1714890567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e41450e0 con 0x7f71e406bc10 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e3fff640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f71d400d620 tx=0x7f71d400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d40125f0 con 0x7f71e406bc10 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d4019070 con 0x7f71e406bc10 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d40129d0 con 0x7f71e406bc10 2024-09-10T11:38:28.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.440+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e40afd80 con 0x7f71e406bc10 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.441+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e40b0300 con 0x7f71e406bc10 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.442+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e4163620 con 0x7f71e406bc10 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.442+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71d4012b70 con 0x7f71e406bc10 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.442+0000 7f71e0ff9640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 0x7f71d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.442+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f71d408b1c0 con 0x7f71e406bc10 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.442+0000 7f71e37fe640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 0x7f71d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.443+0000 7f71e37fe640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 0x7f71d006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f71d8034000 tx=0x7f71d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.447+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f71d4059c00 con 0x7f71e406bc10 2024-09-10T11:38:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.609+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71e4073d40 con 0x7f71d006be30 2024-09-10T11:38:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.614+0000 7f71e0ff9640 1 -- 172.21.15.99:0/1714890567 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247971 (secure 0 0 0) 0x7f71e4073d40 con 0x7f71d006be30 2024-09-10T11:38:28.619 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:38:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 msgr2=0x7f71d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 0x7f71d006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f71d8034000 tx=0x7f71d802e000 comp rx=0 tx=0).stop 2024-09-10T11:38:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 msgr2=0x7f71e40b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f71d400d620 tx=0x7f71d400dae0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 shutdown_connections 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f71d006be30 0x7f71d006e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.617+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71e4160030 0x7f71e4144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.618+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71e40b4930 0x7f71e41444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.618+0000 7f71ea427640 1 --2- 172.21.15.99:0/1714890567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e406bc10 0x7f71e40b1360 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.618+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 >> 172.21.15.99:0/1714890567 conn(0x7f71e40b3630 msgr2=0x7f71e40725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.618+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 shutdown_connections 2024-09-10T11:38:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.618+0000 7f71ea427640 1 -- 172.21.15.99:0/1714890567 wait complete. 2024-09-10T11:38:28.631 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T11:38:27.013951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554096,"statfs":{"total":1600000000,"available":1591395256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000531"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687845+0000","last_change":"2024-09-10T11:38:25.687224+0000","last_active":"2024-09-10T11:38:26.687845+0000","last_peered":"2024-09-10T11:38:26.687845+0000","last_clean":"2024-09-10T11:38:26.687845+0000","last_became_active":"2024-09-10T11:38:25.686985+0000","last_became_peered":"2024-09-10T11:38:25.686985+0000","last_unstale":"2024-09-10T11:38:26.687845+0000","last_undegraded":"2024-09-10T11:38:26.687845+0000","last_fullsized":"2024-09-10T11:38:26.687845+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:39.411934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801458+0000","last_change":"2024-09-10T11:38:25.687890+0000","last_active":"2024-09-10T11:38:26.801458+0000","last_peered":"2024-09-10T11:38:26.801458+0000","last_clean":"2024-09-10T11:38:26.801458+0000","last_became_active":"2024-09-10T11:38:25.687640+0000","last_became_peered":"2024-09-10T11:38:25.687640+0000","last_unstale":"2024-09-10T11:38:26.801458+0000","last_undegraded":"2024-09-10T11:38:26.801458+0000","last_fullsized":"2024-09-10T11:38:26.801458+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:33.234753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.693764+0000","last_change":"2024-09-10T11:38:25.690916+0000","last_active":"2024-09-10T11:38:25.693764+0000","last_peered":"2024-09-10T11:38:25.693764+0000","last_clean":"2024-09-10T11:38:25.693764+0000","last_became_active":"2024-09-10T11:38:25.690639+0000","last_became_peered":"2024-09-10T11:38:25.690639+0000","last_unstale":"2024-09-10T11:38:25.693764+0000","last_undegraded":"2024-09-10T11:38:25.693764+0000","last_fullsized":"2024-09-10T11:38:25.693764+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:29.612282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.688103+0000","last_change":"2024-09-10T11:38:25.687981+0000","last_active":"2024-09-10T11:38:25.688103+0000","last_peered":"2024-09-10T11:38:25.688103+0000","last_clean":"2024-09-10T11:38:25.688103+0000","last_became_active":"2024-09-10T11:38:25.687754+0000","last_became_peered":"2024-09-10T11:38:25.687754+0000","last_unstale":"2024-09-10T11:38:25.688103+0000","last_undegraded":"2024-09-10T11:38:25.688103+0000","last_fullsized":"2024-09-10T11:38:25.688103+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:57.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685317+0000","last_change":"2024-09-10T11:38:25.696827+0000","last_active":"2024-09-10T11:38:26.685317+0000","last_peered":"2024-09-10T11:38:26.685317+0000","last_clean":"2024-09-10T11:38:26.685317+0000","last_became_active":"2024-09-10T11:38:25.696538+0000","last_became_peered":"2024-09-10T11:38:25.696538+0000","last_unstale":"2024-09-10T11:38:26.685317+0000","last_undegraded":"2024-09-10T11:38:26.685317+0000","last_fullsized":"2024-09-10T11:38:26.685317+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.329985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682348+0000","last_change":"2024-09-10T11:38:25.689518+0000","last_active":"2024-09-10T11:38:26.682348+0000","last_peered":"2024-09-10T11:38:26.682348+0000","last_clean":"2024-09-10T11:38:26.682348+0000","last_became_active":"2024-09-10T11:38:25.689266+0000","last_became_peered":"2024-09-10T11:38:25.689266+0000","last_unstale":"2024-09-10T11:38:26.682348+0000","last_undegraded":"2024-09-10T11:38:26.682348+0000","last_fullsized":"2024-09-10T11:38:26.682348+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:17.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801693+0000","last_change":"2024-09-10T11:38:25.691690+0000","last_active":"2024-09-10T11:38:26.801693+0000","last_peered":"2024-09-10T11:38:26.801693+0000","last_clean":"2024-09-10T11:38:26.801693+0000","last_became_active":"2024-09-10T11:38:25.691438+0000","last_became_peered":"2024-09-10T11:38:25.691438+0000","last_unstale":"2024-09-10T11:38:26.801693+0000","last_undegraded":"2024-09-10T11:38:26.801693+0000","last_fullsized":"2024-09-10T11:38:26.801693+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687968+0000","last_change":"2024-09-10T11:38:25.690243+0000","last_active":"2024-09-10T11:38:26.687968+0000","last_peered":"2024-09-10T11:38:26.687968+0000","last_clean":"2024-09-10T11:38:26.687968+0000","last_became_active":"2024-09-10T11:38:25.689981+0000","last_became_peered":"2024-09-10T11:38:25.689981+0000","last_unstale":"2024-09-10T11:38:26.687968+0000","last_undegraded":"2024-09-10T11:38:26.687968+0000","last_fullsized":"2024-09-10T11:38:26.687968+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:34.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682381+0000","last_change":"2024-09-10T11:38:25.688782+0000","last_active":"2024-09-10T11:38:26.682381+0000","last_peered":"2024-09-10T11:38:26.682381+0000","last_clean":"2024-09-10T11:38:26.682381+0000","last_became_active":"2024-09-10T11:38:25.688516+0000","last_became_peered":"2024-09-10T11:38:25.688516+0000","last_unstale":"2024-09-10T11:38:26.682381+0000","last_undegraded":"2024-09-10T11:38:26.682381+0000","last_fullsized":"2024-09-10T11:38:26.682381+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.091784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.694661+0000","last_change":"2024-09-10T11:38:25.694516+0000","last_active":"2024-09-10T11:38:25.694661+0000","last_peered":"2024-09-10T11:38:25.694661+0000","last_clean":"2024-09-10T11:38:25.694661+0000","last_became_active":"2024-09-10T11:38:25.694262+0000","last_became_peered":"2024-09-10T11:38:25.694262+0000","last_unstale":"2024-09-10T11:38:25.694661+0000","last_undegraded":"2024-09-10T11:38:25.694661+0000","last_fullsized":"2024-09-10T11:38:25.694661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:27.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682466+0000","last_change":"2024-09-10T11:38:25.695919+0000","last_active":"2024-09-10T11:38:26.682466+0000","last_peered":"2024-09-10T11:38:26.682466+0000","last_clean":"2024-09-10T11:38:26.682466+0000","last_became_active":"2024-09-10T11:38:25.695302+0000","last_became_peered":"2024-09-10T11:38:25.695302+0000","last_unstale":"2024-09-10T11:38:26.682466+0000","last_undegraded":"2024-09-10T11:38:26.682466+0000","last_fullsized":"2024-09-10T11:38:26.682466+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:47.841095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800886+0000","last_change":"2024-09-10T11:38:25.690027+0000","last_active":"2024-09-10T11:38:26.800886+0000","last_peered":"2024-09-10T11:38:26.800886+0000","last_clean":"2024-09-10T11:38:26.800886+0000","last_became_active":"2024-09-10T11:38:25.689749+0000","last_became_peered":"2024-09-10T11:38:25.689749+0000","last_unstale":"2024-09-10T11:38:26.800886+0000","last_undegraded":"2024-09-10T11:38:26.800886+0000","last_fullsized":"2024-09-10T11:38:26.800886+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:00.218607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685153+0000","last_change":"2024-09-10T11:38:25.701208+0000","last_active":"2024-09-10T11:38:26.685153+0000","last_peered":"2024-09-10T11:38:26.685153+0000","last_clean":"2024-09-10T11:38:26.685153+0000","last_became_active":"2024-09-10T11:38:25.700959+0000","last_became_peered":"2024-09-10T11:38:25.700959+0000","last_unstale":"2024-09-10T11:38:26.685153+0000","last_undegraded":"2024-09-10T11:38:26.685153+0000","last_fullsized":"2024-09-10T11:38:26.685153+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:27.799154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800854+0000","last_change":"2024-09-10T11:38:25.695404+0000","last_active":"2024-09-10T11:38:26.800854+0000","last_peered":"2024-09-10T11:38:26.800854+0000","last_clean":"2024-09-10T11:38:26.800854+0000","last_became_active":"2024-09-10T11:38:25.695189+0000","last_became_peered":"2024-09-10T11:38:25.695189+0000","last_unstale":"2024-09-10T11:38:26.800854+0000","last_undegraded":"2024-09-10T11:38:26.800854+0000","last_fullsized":"2024-09-10T11:38:26.800854+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.152541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685444+0000","last_change":"2024-09-10T11:38:25.696228+0000","last_active":"2024-09-10T11:38:26.685444+0000","last_peered":"2024-09-10T11:38:26.685444+0000","last_clean":"2024-09-10T11:38:26.685444+0000","last_became_active":"2024-09-10T11:38:25.695892+0000","last_became_peered":"2024-09-10T11:38:25.695892+0000","last_unstale":"2024-09-10T11:38:26.685444+0000","last_undegraded":"2024-09-10T11:38:26.685444+0000","last_fullsized":"2024-09-10T11:38:26.685444+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:58.984346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801728+0000","last_change":"2024-09-10T11:38:25.690365+0000","last_active":"2024-09-10T11:38:26.801728+0000","last_peered":"2024-09-10T11:38:26.801728+0000","last_clean":"2024-09-10T11:38:26.801728+0000","last_became_active":"2024-09-10T11:38:25.690146+0000","last_became_peered":"2024-09-10T11:38:25.690146+0000","last_unstale":"2024-09-10T11:38:26.801728+0000","last_undegraded":"2024-09-10T11:38:26.801728+0000","last_fullsized":"2024-09-10T11:38:26.801728+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:29.762856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697699+0000","last_change":"2024-09-10T11:38:25.695403+0000","last_active":"2024-09-10T11:38:25.697699+0000","last_peered":"2024-09-10T11:38:25.697699+0000","last_clean":"2024-09-10T11:38:25.697699+0000","last_became_active":"2024-09-10T11:38:25.695046+0000","last_became_peered":"2024-09-10T11:38:25.695046+0000","last_unstale":"2024-09-10T11:38:25.697699+0000","last_undegraded":"2024-09-10T11:38:25.697699+0000","last_fullsized":"2024-09-10T11:38:25.697699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:23.262137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685255+0000","last_change":"2024-09-10T11:38:25.699619+0000","last_active":"2024-09-10T11:38:26.685255+0000","last_peered":"2024-09-10T11:38:26.685255+0000","last_clean":"2024-09-10T11:38:26.685255+0000","last_became_active":"2024-09-10T11:38:25.698673+0000","last_became_peered":"2024-09-10T11:38:25.698673+0000","last_unstale":"2024-09-10T11:38:26.685255+0000","last_undegraded":"2024-09-10T11:38:26.685255+0000","last_fullsized":"2024-09-10T11:38:26.685255+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:23.682006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687913+0000","last_change":"2024-09-10T11:38:25.684850+0000","last_active":"2024-09-10T11:38:26.687913+0000","last_peered":"2024-09-10T11:38:26.687913+0000","last_clean":"2024-09-10T11:38:26.687913+0000","last_became_active":"2024-09-10T11:38:25.684630+0000","last_became_peered":"2024-09-10T11:38:25.684630+0000","last_unstale":"2024-09-10T11:38:26.687913+0000","last_undegraded":"2024-09-10T11:38:26.687913+0000","last_fullsized":"2024-09-10T11:38:26.687913+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:15.333866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687846+0000","last_change":"2024-09-10T11:38:25.687966+0000","last_active":"2024-09-10T11:38:26.687846+0000","last_peered":"2024-09-10T11:38:26.687846+0000","last_clean":"2024-09-10T11:38:26.687846+0000","last_became_active":"2024-09-10T11:38:25.687736+0000","last_became_peered":"2024-09-10T11:38:25.687736+0000","last_unstale":"2024-09-10T11:38:26.687846+0000","last_undegraded":"2024-09-10T11:38:26.687846+0000","last_fullsized":"2024-09-10T11:38:26.687846+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:40.769460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697719+0000","last_change":"2024-09-10T11:38:25.697019+0000","last_active":"2024-09-10T11:38:25.697719+0000","last_peered":"2024-09-10T11:38:25.697719+0000","last_clean":"2024-09-10T11:38:25.697719+0000","last_became_active":"2024-09-10T11:38:25.696804+0000","last_became_peered":"2024-09-10T11:38:25.696804+0000","last_unstale":"2024-09-10T11:38:25.697719+0000","last_undegraded":"2024-09-10T11:38:25.697719+0000","last_fullsized":"2024-09-10T11:38:25.697719+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:42.245845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800974+0000","last_change":"2024-09-10T11:38:25.693388+0000","last_active":"2024-09-10T11:38:26.800974+0000","last_peered":"2024-09-10T11:38:26.800974+0000","last_clean":"2024-09-10T11:38:26.800974+0000","last_became_active":"2024-09-10T11:38:25.693137+0000","last_became_peered":"2024-09-10T11:38:25.693137+0000","last_unstale":"2024-09-10T11:38:26.800974+0000","last_undegraded":"2024-09-10T11:38:26.800974+0000","last_fullsized":"2024-09-10T11:38:26.800974+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:34.673348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.683579+0000","last_change":"2024-09-10T11:38:25.683454+0000","last_active":"2024-09-10T11:38:25.683579+0000","last_peered":"2024-09-10T11:38:25.683579+0000","last_clean":"2024-09-10T11:38:25.683579+0000","last_became_active":"2024-09-10T11:38:25.683217+0000","last_became_peered":"2024-09-10T11:38:25.683217+0000","last_unstale":"2024-09-10T11:38:25.683579+0000","last_undegraded":"2024-09-10T11:38:25.683579+0000","last_fullsized":"2024-09-10T11:38:25.683579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:04.564864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685133+0000","last_change":"2024-09-10T11:38:25.701761+0000","last_active":"2024-09-10T11:38:26.685133+0000","last_peered":"2024-09-10T11:38:26.685133+0000","last_clean":"2024-09-10T11:38:26.685133+0000","last_became_active":"2024-09-10T11:38:25.701507+0000","last_became_peered":"2024-09-10T11:38:25.701507+0000","last_unstale":"2024-09-10T11:38:26.685133+0000","last_undegraded":"2024-09-10T11:38:26.685133+0000","last_fullsized":"2024-09-10T11:38:26.685133+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:16.120928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801587+0000","last_change":"2024-09-10T11:38:25.690488+0000","last_active":"2024-09-10T11:38:26.801587+0000","last_peered":"2024-09-10T11:38:26.801587+0000","last_clean":"2024-09-10T11:38:26.801587+0000","last_became_active":"2024-09-10T11:38:25.690238+0000","last_became_peered":"2024-09-10T11:38:25.690238+0000","last_unstale":"2024-09-10T11:38:26.801587+0000","last_undegraded":"2024-09-10T11:38:26.801587+0000","last_fullsized":"2024-09-10T11:38:26.801587+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:51.627617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687703+0000","last_change":"2024-09-10T11:38:25.688753+0000","last_active":"2024-09-10T11:38:26.687703+0000","last_peered":"2024-09-10T11:38:26.687703+0000","last_clean":"2024-09-10T11:38:26.687703+0000","last_became_active":"2024-09-10T11:38:25.688473+0000","last_became_peered":"2024-09-10T11:38:25.688473+0000","last_unstale":"2024-09-10T11:38:26.687703+0000","last_undegraded":"2024-09-10T11:38:26.687703+0000","last_fullsized":"2024-09-10T11:38:26.687703+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:36.422421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801630+0000","last_change":"2024-09-10T11:38:25.692365+0000","last_active":"2024-09-10T11:38:26.801630+0000","last_peered":"2024-09-10T11:38:26.801630+0000","last_clean":"2024-09-10T11:38:26.801630+0000","last_became_active":"2024-09-10T11:38:25.691514+0000","last_became_peered":"2024-09-10T11:38:25.691514+0000","last_unstale":"2024-09-10T11:38:26.801630+0000","last_undegraded":"2024-09-10T11:38:26.801630+0000","last_fullsized":"2024-09-10T11:38:26.801630+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:53.485285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801757+0000","last_change":"2024-09-10T11:38:25.692897+0000","last_active":"2024-09-10T11:38:26.801757+0000","last_peered":"2024-09-10T11:38:26.801757+0000","last_clean":"2024-09-10T11:38:26.801757+0000","last_became_active":"2024-09-10T11:38:25.692322+0000","last_became_peered":"2024-09-10T11:38:25.692322+0000","last_unstale":"2024-09-10T11:38:26.801757+0000","last_undegraded":"2024-09-10T11:38:26.801757+0000","last_fullsized":"2024-09-10T11:38:26.801757+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688204+0000","last_change":"2024-09-10T11:38:25.692760+0000","last_active":"2024-09-10T11:38:26.688204+0000","last_peered":"2024-09-10T11:38:26.688204+0000","last_clean":"2024-09-10T11:38:26.688204+0000","last_became_active":"2024-09-10T11:38:25.692380+0000","last_became_peered":"2024-09-10T11:38:25.692380+0000","last_unstale":"2024-09-10T11:38:26.688204+0000","last_undegraded":"2024-09-10T11:38:26.688204+0000","last_fullsized":"2024-09-10T11:38:26.688204+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:49.115544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800973+0000","last_change":"2024-09-10T11:38:25.692535+0000","last_active":"2024-09-10T11:38:26.800973+0000","last_peered":"2024-09-10T11:38:26.800973+0000","last_clean":"2024-09-10T11:38:26.800973+0000","last_became_active":"2024-09-10T11:38:25.692304+0000","last_became_peered":"2024-09-10T11:38:25.692304+0000","last_unstale":"2024-09-10T11:38:26.800973+0000","last_undegraded":"2024-09-10T11:38:26.800973+0000","last_fullsized":"2024-09-10T11:38:26.800973+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:18.744856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801399+0000","last_change":"2024-09-10T11:38:25.690997+0000","last_active":"2024-09-10T11:38:26.801399+0000","last_peered":"2024-09-10T11:38:26.801399+0000","last_clean":"2024-09-10T11:38:26.801399+0000","last_became_active":"2024-09-10T11:38:25.690778+0000","last_became_peered":"2024-09-10T11:38:25.690778+0000","last_unstale":"2024-09-10T11:38:26.801399+0000","last_undegraded":"2024-09-10T11:38:26.801399+0000","last_fullsized":"2024-09-10T11:38:26.801399+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:23.280103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801786+0000","last_change":"2024-09-10T11:38:25.692932+0000","last_active":"2024-09-10T11:38:26.801786+0000","last_peered":"2024-09-10T11:38:26.801786+0000","last_clean":"2024-09-10T11:38:26.801786+0000","last_became_active":"2024-09-10T11:38:25.692699+0000","last_became_peered":"2024-09-10T11:38:25.692699+0000","last_unstale":"2024-09-10T11:38:26.801786+0000","last_undegraded":"2024-09-10T11:38:26.801786+0000","last_fullsized":"2024-09-10T11:38:26.801786+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:49.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685250+0000","last_change":"2024-09-10T11:38:25.700283+0000","last_active":"2024-09-10T11:38:26.685250+0000","last_peered":"2024-09-10T11:38:26.685250+0000","last_clean":"2024-09-10T11:38:26.685250+0000","last_became_active":"2024-09-10T11:38:25.700069+0000","last_became_peered":"2024-09-10T11:38:25.700069+0000","last_unstale":"2024-09-10T11:38:26.685250+0000","last_undegraded":"2024-09-10T11:38:26.685250+0000","last_fullsized":"2024-09-10T11:38:26.685250+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:09.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"166'4","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.704874+0000","last_change":"2024-09-10T11:38:24.694742+0000","last_active":"2024-09-10T11:38:25.704874+0000","last_peered":"2024-09-10T11:38:25.704874+0000","last_clean":"2024-09-10T11:38:25.704874+0000","last_became_active":"2024-09-10T11:38:24.694576+0000","last_became_peered":"2024-09-10T11:38:24.694576+0000","last_unstale":"2024-09-10T11:38:25.704874+0000","last_undegraded":"2024-09-10T11:38:25.704874+0000","last_fullsized":"2024-09-10T11:38:25.704874+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:25.198002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801137+0000","last_change":"2024-09-10T11:38:25.693920+0000","last_active":"2024-09-10T11:38:26.801137+0000","last_peered":"2024-09-10T11:38:26.801137+0000","last_clean":"2024-09-10T11:38:26.801137+0000","last_became_active":"2024-09-10T11:38:25.693663+0000","last_became_peered":"2024-09-10T11:38:25.693663+0000","last_unstale":"2024-09-10T11:38:26.801137+0000","last_undegraded":"2024-09-10T11:38:26.801137+0000","last_fullsized":"2024-09-10T11:38:26.801137+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:50.994377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682491+0000","last_change":"2024-09-10T11:38:24.687134+0000","last_active":"2024-09-10T11:38:26.682491+0000","last_peered":"2024-09-10T11:38:26.682491+0000","last_clean":"2024-09-10T11:38:26.682491+0000","last_became_active":"2024-09-10T11:38:24.686844+0000","last_became_peered":"2024-09-10T11:38:24.686844+0000","last_unstale":"2024-09-10T11:38:26.682491+0000","last_undegraded":"2024-09-10T11:38:26.682491+0000","last_fullsized":"2024-09-10T11:38:26.682491+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:02.555327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682558+0000","last_change":"2024-09-10T11:38:25.695767+0000","last_active":"2024-09-10T11:38:26.682558+0000","last_peered":"2024-09-10T11:38:26.682558+0000","last_clean":"2024-09-10T11:38:26.682558+0000","last_became_active":"2024-09-10T11:38:25.694847+0000","last_became_peered":"2024-09-10T11:38:25.694847+0000","last_unstale":"2024-09-10T11:38:26.682558+0000","last_undegraded":"2024-09-10T11:38:26.682558+0000","last_fullsized":"2024-09-10T11:38:26.682558+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:14.173322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.703986+0000","last_change":"2024-09-10T11:38:24.689156+0000","last_active":"2024-09-10T11:38:25.703986+0000","last_peered":"2024-09-10T11:38:25.703986+0000","last_clean":"2024-09-10T11:38:25.703986+0000","last_became_active":"2024-09-10T11:38:24.688634+0000","last_became_peered":"2024-09-10T11:38:24.688634+0000","last_unstale":"2024-09-10T11:38:25.703986+0000","last_undegraded":"2024-09-10T11:38:25.703986+0000","last_fullsized":"2024-09-10T11:38:25.703986+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:12.809145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687579+0000","last_change":"2024-09-10T11:38:25.687474+0000","last_active":"2024-09-10T11:38:25.687579+0000","last_peered":"2024-09-10T11:38:25.687579+0000","last_clean":"2024-09-10T11:38:25.687579+0000","last_became_active":"2024-09-10T11:38:25.687239+0000","last_became_peered":"2024-09-10T11:38:25.687239+0000","last_unstale":"2024-09-10T11:38:25.687579+0000","last_undegraded":"2024-09-10T11:38:25.687579+0000","last_fullsized":"2024-09-10T11:38:25.687579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:30.577654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.703709+0000","last_change":"2024-09-10T11:38:24.692321+0000","last_active":"2024-09-10T11:38:25.703709+0000","last_peered":"2024-09-10T11:38:25.703709+0000","last_clean":"2024-09-10T11:38:25.703709+0000","last_became_active":"2024-09-10T11:38:24.692052+0000","last_became_peered":"2024-09-10T11:38:24.692052+0000","last_unstale":"2024-09-10T11:38:25.703709+0000","last_undegraded":"2024-09-10T11:38:25.703709+0000","last_fullsized":"2024-09-10T11:38:25.703709+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:38.649830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801907+0000","last_change":"2024-09-10T11:38:25.693269+0000","last_active":"2024-09-10T11:38:26.801907+0000","last_peered":"2024-09-10T11:38:26.801907+0000","last_clean":"2024-09-10T11:38:26.801907+0000","last_became_active":"2024-09-10T11:38:25.692670+0000","last_became_peered":"2024-09-10T11:38:25.692670+0000","last_unstale":"2024-09-10T11:38:26.801907+0000","last_undegraded":"2024-09-10T11:38:26.801907+0000","last_fullsized":"2024-09-10T11:38:26.801907+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:48.054890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687776+0000","last_change":"2024-09-10T11:38:24.694087+0000","last_active":"2024-09-10T11:38:26.687776+0000","last_peered":"2024-09-10T11:38:26.687776+0000","last_clean":"2024-09-10T11:38:26.687776+0000","last_became_active":"2024-09-10T11:38:24.693842+0000","last_became_peered":"2024-09-10T11:38:24.693842+0000","last_unstale":"2024-09-10T11:38:26.687776+0000","last_undegraded":"2024-09-10T11:38:26.687776+0000","last_fullsized":"2024-09-10T11:38:26.687776+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:49.926387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.690415+0000","last_change":"2024-09-10T11:38:25.689130+0000","last_active":"2024-09-10T11:38:25.690415+0000","last_peered":"2024-09-10T11:38:25.690415+0000","last_clean":"2024-09-10T11:38:25.690415+0000","last_became_active":"2024-09-10T11:38:25.688724+0000","last_became_peered":"2024-09-10T11:38:25.688724+0000","last_unstale":"2024-09-10T11:38:25.690415+0000","last_undegraded":"2024-09-10T11:38:25.690415+0000","last_fullsized":"2024-09-10T11:38:25.690415+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:42.156095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685312+0000","last_change":"2024-09-10T11:38:24.691448+0000","last_active":"2024-09-10T11:38:26.685312+0000","last_peered":"2024-09-10T11:38:26.685312+0000","last_clean":"2024-09-10T11:38:26.685312+0000","last_became_active":"2024-09-10T11:38:24.691182+0000","last_became_peered":"2024-09-10T11:38:24.691182+0000","last_unstale":"2024-09-10T11:38:26.685312+0000","last_undegraded":"2024-09-10T11:38:26.685312+0000","last_fullsized":"2024-09-10T11:38:26.685312+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:35.379407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801797+0000","last_change":"2024-09-10T11:38:25.691108+0000","last_active":"2024-09-10T11:38:26.801797+0000","last_peered":"2024-09-10T11:38:26.801797+0000","last_clean":"2024-09-10T11:38:26.801797+0000","last_became_active":"2024-09-10T11:38:25.690892+0000","last_became_peered":"2024-09-10T11:38:25.690892+0000","last_unstale":"2024-09-10T11:38:26.801797+0000","last_undegraded":"2024-09-10T11:38:26.801797+0000","last_fullsized":"2024-09-10T11:38:26.801797+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:00.056160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682602+0000","last_change":"2024-09-10T11:38:24.683415+0000","last_active":"2024-09-10T11:38:26.682602+0000","last_peered":"2024-09-10T11:38:26.682602+0000","last_clean":"2024-09-10T11:38:26.682602+0000","last_became_active":"2024-09-10T11:38:24.683108+0000","last_became_peered":"2024-09-10T11:38:24.683108+0000","last_unstale":"2024-09-10T11:38:26.682602+0000","last_undegraded":"2024-09-10T11:38:26.682602+0000","last_fullsized":"2024-09-10T11:38:26.682602+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:10.496496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686477+0000","last_change":"2024-09-10T11:38:25.686190+0000","last_active":"2024-09-10T11:38:25.686477+0000","last_peered":"2024-09-10T11:38:25.686477+0000","last_clean":"2024-09-10T11:38:25.686477+0000","last_became_active":"2024-09-10T11:38:25.685606+0000","last_became_peered":"2024-09-10T11:38:25.685606+0000","last_unstale":"2024-09-10T11:38:25.686477+0000","last_undegraded":"2024-09-10T11:38:25.686477+0000","last_fullsized":"2024-09-10T11:38:25.686477+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:33.709019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686256+0000","last_change":"2024-09-10T11:38:24.686937+0000","last_active":"2024-09-10T11:38:25.686256+0000","last_peered":"2024-09-10T11:38:25.686256+0000","last_clean":"2024-09-10T11:38:25.686256+0000","last_became_active":"2024-09-10T11:38:24.686709+0000","last_became_peered":"2024-09-10T11:38:24.686709+0000","last_unstale":"2024-09-10T11:38:25.686256+0000","last_undegraded":"2024-09-10T11:38:25.686256+0000","last_fullsized":"2024-09-10T11:38:25.686256+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:15.541212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.685425+0000","last_change":"2024-09-10T11:38:25.685309+0000","last_active":"2024-09-10T11:38:25.685425+0000","last_peered":"2024-09-10T11:38:25.685425+0000","last_clean":"2024-09-10T11:38:25.685425+0000","last_became_active":"2024-09-10T11:38:25.684800+0000","last_became_peered":"2024-09-10T11:38:25.684800+0000","last_unstale":"2024-09-10T11:38:25.685425+0000","last_undegraded":"2024-09-10T11:38:25.685425+0000","last_fullsized":"2024-09-10T11:38:25.685425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:54.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682765+0000","last_change":"2024-09-10T11:38:24.691550+0000","last_active":"2024-09-10T11:38:26.682765+0000","last_peered":"2024-09-10T11:38:26.682765+0000","last_clean":"2024-09-10T11:38:26.682765+0000","last_became_active":"2024-09-10T11:38:24.691262+0000","last_became_peered":"2024-09-10T11:38:24.691262+0000","last_unstale":"2024-09-10T11:38:26.682765+0000","last_undegraded":"2024-09-10T11:38:26.682765+0000","last_fullsized":"2024-09-10T11:38:26.682765+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:48.660663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800734+0000","last_change":"2024-09-10T11:38:25.696483+0000","last_active":"2024-09-10T11:38:26.800734+0000","last_peered":"2024-09-10T11:38:26.800734+0000","last_clean":"2024-09-10T11:38:26.800734+0000","last_became_active":"2024-09-10T11:38:25.696256+0000","last_became_peered":"2024-09-10T11:38:25.696256+0000","last_unstale":"2024-09-10T11:38:26.800734+0000","last_undegraded":"2024-09-10T11:38:26.800734+0000","last_fullsized":"2024-09-10T11:38:26.800734+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:03.170739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685291+0000","last_change":"2024-09-10T11:38:24.687041+0000","last_active":"2024-09-10T11:38:26.685291+0000","last_peered":"2024-09-10T11:38:26.685291+0000","last_clean":"2024-09-10T11:38:26.685291+0000","last_became_active":"2024-09-10T11:38:24.686819+0000","last_became_peered":"2024-09-10T11:38:24.686819+0000","last_unstale":"2024-09-10T11:38:26.685291+0000","last_undegraded":"2024-09-10T11:38:26.685291+0000","last_fullsized":"2024-09-10T11:38:26.685291+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:55.204872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801320+0000","last_change":"2024-09-10T11:38:25.689829+0000","last_active":"2024-09-10T11:38:26.801320+0000","last_peered":"2024-09-10T11:38:26.801320+0000","last_clean":"2024-09-10T11:38:26.801320+0000","last_became_active":"2024-09-10T11:38:25.689571+0000","last_became_peered":"2024-09-10T11:38:25.689571+0000","last_unstale":"2024-09-10T11:38:26.801320+0000","last_undegraded":"2024-09-10T11:38:26.801320+0000","last_fullsized":"2024-09-10T11:38:26.801320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:06.017693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"166'6","reported_seq":20,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801030+0000","last_change":"2024-09-10T11:38:24.691915+0000","last_active":"2024-09-10T11:38:26.801030+0000","last_peered":"2024-09-10T11:38:26.801030+0000","last_clean":"2024-09-10T11:38:26.801030+0000","last_became_active":"2024-09-10T11:38:24.691313+0000","last_became_peered":"2024-09-10T11:38:24.691313+0000","last_unstale":"2024-09-10T11:38:26.801030+0000","last_undegraded":"2024-09-10T11:38:26.801030+0000","last_fullsized":"2024-09-10T11:38:26.801030+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:53.750660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.696903+0000","last_change":"2024-09-10T11:38:25.695422+0000","last_active":"2024-09-10T11:38:25.696903+0000","last_peered":"2024-09-10T11:38:25.696903+0000","last_clean":"2024-09-10T11:38:25.696903+0000","last_became_active":"2024-09-10T11:38:25.695160+0000","last_became_peered":"2024-09-10T11:38:25.695160+0000","last_unstale":"2024-09-10T11:38:25.696903+0000","last_undegraded":"2024-09-10T11:38:25.696903+0000","last_fullsized":"2024-09-10T11:38:25.696903+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:48.452413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.704373+0000","last_change":"2024-09-10T11:38:24.689334+0000","last_active":"2024-09-10T11:38:25.704373+0000","last_peered":"2024-09-10T11:38:25.704373+0000","last_clean":"2024-09-10T11:38:25.704373+0000","last_became_active":"2024-09-10T11:38:24.688936+0000","last_became_peered":"2024-09-10T11:38:24.688936+0000","last_unstale":"2024-09-10T11:38:25.704373+0000","last_undegraded":"2024-09-10T11:38:25.704373+0000","last_fullsized":"2024-09-10T11:38:25.704373+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:12.557337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688008+0000","last_change":"2024-09-10T11:38:25.690085+0000","last_active":"2024-09-10T11:38:26.688008+0000","last_peered":"2024-09-10T11:38:26.688008+0000","last_clean":"2024-09-10T11:38:26.688008+0000","last_became_active":"2024-09-10T11:38:25.689841+0000","last_became_peered":"2024-09-10T11:38:25.689841+0000","last_unstale":"2024-09-10T11:38:26.688008+0000","last_undegraded":"2024-09-10T11:38:26.688008+0000","last_fullsized":"2024-09-10T11:38:26.688008+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:34.727036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801089+0000","last_change":"2024-09-10T11:38:24.690075+0000","last_active":"2024-09-10T11:38:26.801089+0000","last_peered":"2024-09-10T11:38:26.801089+0000","last_clean":"2024-09-10T11:38:26.801089+0000","last_became_active":"2024-09-10T11:38:24.688979+0000","last_became_peered":"2024-09-10T11:38:24.688979+0000","last_unstale":"2024-09-10T11:38:26.801089+0000","last_undegraded":"2024-09-10T11:38:26.801089+0000","last_fullsized":"2024-09-10T11:38:26.801089+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:19.862707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687621+0000","last_change":"2024-09-10T11:38:25.687527+0000","last_active":"2024-09-10T11:38:25.687621+0000","last_peered":"2024-09-10T11:38:25.687621+0000","last_clean":"2024-09-10T11:38:25.687621+0000","last_became_active":"2024-09-10T11:38:25.687289+0000","last_became_peered":"2024-09-10T11:38:25.687289+0000","last_unstale":"2024-09-10T11:38:25.687621+0000","last_undegraded":"2024-09-10T11:38:25.687621+0000","last_fullsized":"2024-09-10T11:38:25.687621+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:07.927208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801896+0000","last_change":"2024-09-10T11:38:24.683427+0000","last_active":"2024-09-10T11:38:26.801896+0000","last_peered":"2024-09-10T11:38:26.801896+0000","last_clean":"2024-09-10T11:38:26.801896+0000","last_became_active":"2024-09-10T11:38:24.683184+0000","last_became_peered":"2024-09-10T11:38:24.683184+0000","last_unstale":"2024-09-10T11:38:26.801896+0000","last_undegraded":"2024-09-10T11:38:26.801896+0000","last_fullsized":"2024-09-10T11:38:26.801896+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:06.011224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.685844+0000","last_change":"2024-09-10T11:38:25.685553+0000","last_active":"2024-09-10T11:38:25.685844+0000","last_peered":"2024-09-10T11:38:25.685844+0000","last_clean":"2024-09-10T11:38:25.685844+0000","last_became_active":"2024-09-10T11:38:25.685093+0000","last_became_peered":"2024-09-10T11:38:25.685093+0000","last_unstale":"2024-09-10T11:38:25.685844+0000","last_undegraded":"2024-09-10T11:38:25.685844+0000","last_fullsized":"2024-09-10T11:38:25.685844+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:11.686083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"166'8","reported_seq":22,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688043+0000","last_change":"2024-09-10T11:38:24.694111+0000","last_active":"2024-09-10T11:38:26.688043+0000","last_peered":"2024-09-10T11:38:26.688043+0000","last_clean":"2024-09-10T11:38:26.688043+0000","last_became_active":"2024-09-10T11:38:24.693889+0000","last_became_peered":"2024-09-10T11:38:24.693889+0000","last_unstale":"2024-09-10T11:38:26.688043+0000","last_undegraded":"2024-09-10T11:38:26.688043+0000","last_fullsized":"2024-09-10T11:38:26.688043+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:54.359892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685571+0000","last_change":"2024-09-10T11:38:25.690497+0000","last_active":"2024-09-10T11:38:26.685571+0000","last_peered":"2024-09-10T11:38:26.685571+0000","last_clean":"2024-09-10T11:38:26.685571+0000","last_became_active":"2024-09-10T11:38:25.690242+0000","last_became_peered":"2024-09-10T11:38:25.690242+0000","last_unstale":"2024-09-10T11:38:26.685571+0000","last_undegraded":"2024-09-10T11:38:26.685571+0000","last_fullsized":"2024-09-10T11:38:26.685571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:41.853492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.700701+0000","last_change":"2024-09-10T11:38:24.692796+0000","last_active":"2024-09-10T11:38:25.700701+0000","last_peered":"2024-09-10T11:38:25.700701+0000","last_clean":"2024-09-10T11:38:25.700701+0000","last_became_active":"2024-09-10T11:38:24.692517+0000","last_became_peered":"2024-09-10T11:38:24.692517+0000","last_unstale":"2024-09-10T11:38:25.700701+0000","last_undegraded":"2024-09-10T11:38:25.700701+0000","last_fullsized":"2024-09-10T11:38:25.700701+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:49.316733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801268+0000","last_change":"2024-09-10T11:38:25.699407+0000","last_active":"2024-09-10T11:38:26.801268+0000","last_peered":"2024-09-10T11:38:26.801268+0000","last_clean":"2024-09-10T11:38:26.801268+0000","last_became_active":"2024-09-10T11:38:25.699187+0000","last_became_peered":"2024-09-10T11:38:25.699187+0000","last_unstale":"2024-09-10T11:38:26.801268+0000","last_undegraded":"2024-09-10T11:38:26.801268+0000","last_fullsized":"2024-09-10T11:38:26.801268+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:20.599279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"166'2","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688313+0000","last_change":"2024-09-10T11:38:24.694849+0000","last_active":"2024-09-10T11:38:26.688313+0000","last_peered":"2024-09-10T11:38:26.688313+0000","last_clean":"2024-09-10T11:38:26.688313+0000","last_became_active":"2024-09-10T11:38:24.694596+0000","last_became_peered":"2024-09-10T11:38:24.694596+0000","last_unstale":"2024-09-10T11:38:26.688313+0000","last_undegraded":"2024-09-10T11:38:26.688313+0000","last_fullsized":"2024-09-10T11:38:26.688313+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:15.050040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685153+0000","last_change":"2024-09-10T11:38:25.695182+0000","last_active":"2024-09-10T11:38:26.685153+0000","last_peered":"2024-09-10T11:38:26.685153+0000","last_clean":"2024-09-10T11:38:26.685153+0000","last_became_active":"2024-09-10T11:38:25.694950+0000","last_became_peered":"2024-09-10T11:38:25.694950+0000","last_unstale":"2024-09-10T11:38:26.685153+0000","last_undegraded":"2024-09-10T11:38:26.685153+0000","last_fullsized":"2024-09-10T11:38:26.685153+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:15.153296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801792+0000","last_change":"2024-09-10T11:38:24.693046+0000","last_active":"2024-09-10T11:38:26.801792+0000","last_peered":"2024-09-10T11:38:26.801792+0000","last_clean":"2024-09-10T11:38:26.801792+0000","last_became_active":"2024-09-10T11:38:24.692385+0000","last_became_peered":"2024-09-10T11:38:24.692385+0000","last_unstale":"2024-09-10T11:38:26.801792+0000","last_undegraded":"2024-09-10T11:38:26.801792+0000","last_fullsized":"2024-09-10T11:38:26.801792+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:59.501610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682662+0000","last_change":"2024-09-10T11:38:25.695556+0000","last_active":"2024-09-10T11:38:26.682662+0000","last_peered":"2024-09-10T11:38:26.682662+0000","last_clean":"2024-09-10T11:38:26.682662+0000","last_became_active":"2024-09-10T11:38:25.694492+0000","last_became_peered":"2024-09-10T11:38:25.694492+0000","last_unstale":"2024-09-10T11:38:26.682662+0000","last_undegraded":"2024-09-10T11:38:26.682662+0000","last_fullsized":"2024-09-10T11:38:26.682662+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:55.620050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801991+0000","last_change":"2024-09-10T11:38:24.689745+0000","last_active":"2024-09-10T11:38:26.801991+0000","last_peered":"2024-09-10T11:38:26.801991+0000","last_clean":"2024-09-10T11:38:26.801991+0000","last_became_active":"2024-09-10T11:38:24.689413+0000","last_became_peered":"2024-09-10T11:38:24.689413+0000","last_unstale":"2024-09-10T11:38:26.801991+0000","last_undegraded":"2024-09-10T11:38:26.801991+0000","last_fullsized":"2024-09-10T11:38:26.801991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:18.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800750+0000","last_change":"2024-09-10T11:38:25.690712+0000","last_active":"2024-09-10T11:38:26.800750+0000","last_peered":"2024-09-10T11:38:26.800750+0000","last_clean":"2024-09-10T11:38:26.800750+0000","last_became_active":"2024-09-10T11:38:25.690484+0000","last_became_peered":"2024-09-10T11:38:25.690484+0000","last_unstale":"2024-09-10T11:38:26.800750+0000","last_undegraded":"2024-09-10T11:38:26.800750+0000","last_fullsized":"2024-09-10T11:38:26.800750+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:28.667879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"166'2","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682490+0000","last_change":"2024-09-10T11:38:24.687954+0000","last_active":"2024-09-10T11:38:26.682490+0000","last_peered":"2024-09-10T11:38:26.682490+0000","last_clean":"2024-09-10T11:38:26.682490+0000","last_became_active":"2024-09-10T11:38:24.687655+0000","last_became_peered":"2024-09-10T11:38:24.687655+0000","last_unstale":"2024-09-10T11:38:26.682490+0000","last_undegraded":"2024-09-10T11:38:26.682490+0000","last_fullsized":"2024-09-10T11:38:26.682490+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:53.253371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.802055+0000","last_change":"2024-09-10T11:38:25.701825+0000","last_active":"2024-09-10T11:38:26.802055+0000","last_peered":"2024-09-10T11:38:26.802055+0000","last_clean":"2024-09-10T11:38:26.802055+0000","last_became_active":"2024-09-10T11:38:25.701587+0000","last_became_peered":"2024-09-10T11:38:25.701587+0000","last_unstale":"2024-09-10T11:38:26.802055+0000","last_undegraded":"2024-09-10T11:38:26.802055+0000","last_fullsized":"2024-09-10T11:38:26.802055+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:01.882638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.683021+0000","last_change":"2024-09-10T11:38:24.691545+0000","last_active":"2024-09-10T11:38:25.683021+0000","last_peered":"2024-09-10T11:38:25.683021+0000","last_clean":"2024-09-10T11:38:25.683021+0000","last_became_active":"2024-09-10T11:38:24.691294+0000","last_became_peered":"2024-09-10T11:38:24.691294+0000","last_unstale":"2024-09-10T11:38:25.683021+0000","last_undegraded":"2024-09-10T11:38:25.683021+0000","last_fullsized":"2024-09-10T11:38:25.683021+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:45.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800834+0000","last_change":"2024-09-10T11:38:25.694263+0000","last_active":"2024-09-10T11:38:26.800834+0000","last_peered":"2024-09-10T11:38:26.800834+0000","last_clean":"2024-09-10T11:38:26.800834+0000","last_became_active":"2024-09-10T11:38:25.693991+0000","last_became_peered":"2024-09-10T11:38:25.693991+0000","last_unstale":"2024-09-10T11:38:26.800834+0000","last_undegraded":"2024-09-10T11:38:26.800834+0000","last_fullsized":"2024-09-10T11:38:26.800834+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:57.983405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.682246+0000","last_change":"2024-09-10T11:38:24.690459+0000","last_active":"2024-09-10T11:38:25.682246+0000","last_peered":"2024-09-10T11:38:25.682246+0000","last_clean":"2024-09-10T11:38:25.682246+0000","last_became_active":"2024-09-10T11:38:24.689935+0000","last_became_peered":"2024-09-10T11:38:24.689935+0000","last_unstale":"2024-09-10T11:38:25.682246+0000","last_undegraded":"2024-09-10T11:38:25.682246+0000","last_fullsized":"2024-09-10T11:38:25.682246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:07.760102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801949+0000","last_change":"2024-09-10T11:38:25.691676+0000","last_active":"2024-09-10T11:38:26.801949+0000","last_peered":"2024-09-10T11:38:26.801949+0000","last_clean":"2024-09-10T11:38:26.801949+0000","last_became_active":"2024-09-10T11:38:25.691385+0000","last_became_peered":"2024-09-10T11:38:25.691385+0000","last_unstale":"2024-09-10T11:38:26.801949+0000","last_undegraded":"2024-09-10T11:38:26.801949+0000","last_fullsized":"2024-09-10T11:38:26.801949+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:59.549776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.682584+0000","last_change":"2024-09-10T11:38:24.691679+0000","last_active":"2024-09-10T11:38:25.682584+0000","last_peered":"2024-09-10T11:38:25.682584+0000","last_clean":"2024-09-10T11:38:25.682584+0000","last_became_active":"2024-09-10T11:38:24.691418+0000","last_became_peered":"2024-09-10T11:38:24.691418+0000","last_unstale":"2024-09-10T11:38:25.682584+0000","last_undegraded":"2024-09-10T11:38:25.682584+0000","last_fullsized":"2024-09-10T11:38:25.682584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:12.335470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.689019+0000","last_change":"2024-09-10T11:38:25.688714+0000","last_active":"2024-09-10T11:38:25.689019+0000","last_peered":"2024-09-10T11:38:25.689019+0000","last_clean":"2024-09-10T11:38:25.689019+0000","last_became_active":"2024-09-10T11:38:25.688510+0000","last_became_peered":"2024-09-10T11:38:25.688510+0000","last_unstale":"2024-09-10T11:38:25.689019+0000","last_undegraded":"2024-09-10T11:38:25.689019+0000","last_fullsized":"2024-09-10T11:38:25.689019+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:13.052391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.702304+0000","last_change":"2024-09-10T11:38:24.688503+0000","last_active":"2024-09-10T11:38:25.702304+0000","last_peered":"2024-09-10T11:38:25.702304+0000","last_clean":"2024-09-10T11:38:25.702304+0000","last_became_active":"2024-09-10T11:38:24.688230+0000","last_became_peered":"2024-09-10T11:38:24.688230+0000","last_unstale":"2024-09-10T11:38:25.702304+0000","last_undegraded":"2024-09-10T11:38:25.702304+0000","last_fullsized":"2024-09-10T11:38:25.702304+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:33.189562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.689037+0000","last_change":"2024-09-10T11:38:25.688940+0000","last_active":"2024-09-10T11:38:25.689037+0000","last_peered":"2024-09-10T11:38:25.689037+0000","last_clean":"2024-09-10T11:38:25.689037+0000","last_became_active":"2024-09-10T11:38:25.688428+0000","last_became_peered":"2024-09-10T11:38:25.688428+0000","last_unstale":"2024-09-10T11:38:25.689037+0000","last_undegraded":"2024-09-10T11:38:25.689037+0000","last_fullsized":"2024-09-10T11:38:25.689037+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:46.917036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.802149+0000","last_change":"2024-09-10T11:38:24.689759+0000","last_active":"2024-09-10T11:38:26.802149+0000","last_peered":"2024-09-10T11:38:26.802149+0000","last_clean":"2024-09-10T11:38:26.802149+0000","last_became_active":"2024-09-10T11:38:24.689481+0000","last_became_peered":"2024-09-10T11:38:24.689481+0000","last_unstale":"2024-09-10T11:38:26.802149+0000","last_undegraded":"2024-09-10T11:38:26.802149+0000","last_fullsized":"2024-09-10T11:38:26.802149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:55.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688151+0000","last_change":"2024-09-10T11:38:25.688761+0000","last_active":"2024-09-10T11:38:26.688151+0000","last_peered":"2024-09-10T11:38:26.688151+0000","last_clean":"2024-09-10T11:38:26.688151+0000","last_became_active":"2024-09-10T11:38:25.688515+0000","last_became_peered":"2024-09-10T11:38:25.688515+0000","last_unstale":"2024-09-10T11:38:26.688151+0000","last_undegraded":"2024-09-10T11:38:26.688151+0000","last_fullsized":"2024-09-10T11:38:26.688151+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:04.341101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801238+0000","last_change":"2024-09-10T11:38:24.692128+0000","last_active":"2024-09-10T11:38:26.801238+0000","last_peered":"2024-09-10T11:38:26.801238+0000","last_clean":"2024-09-10T11:38:26.801238+0000","last_became_active":"2024-09-10T11:38:24.691647+0000","last_became_peered":"2024-09-10T11:38:24.691647+0000","last_unstale":"2024-09-10T11:38:26.801238+0000","last_undegraded":"2024-09-10T11:38:26.801238+0000","last_fullsized":"2024-09-10T11:38:26.801238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:28.516731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688359+0000","last_change":"2024-09-10T11:38:25.685377+0000","last_active":"2024-09-10T11:38:26.688359+0000","last_peered":"2024-09-10T11:38:26.688359+0000","last_clean":"2024-09-10T11:38:26.688359+0000","last_became_active":"2024-09-10T11:38:25.685127+0000","last_became_peered":"2024-09-10T11:38:25.685127+0000","last_unstale":"2024-09-10T11:38:26.688359+0000","last_undegraded":"2024-09-10T11:38:26.688359+0000","last_fullsized":"2024-09-10T11:38:26.688359+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:14.626142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801403+0000","last_change":"2024-09-10T11:38:24.690814+0000","last_active":"2024-09-10T11:38:26.801403+0000","last_peered":"2024-09-10T11:38:26.801403+0000","last_clean":"2024-09-10T11:38:26.801403+0000","last_became_active":"2024-09-10T11:38:24.690564+0000","last_became_peered":"2024-09-10T11:38:24.690564+0000","last_unstale":"2024-09-10T11:38:26.801403+0000","last_undegraded":"2024-09-10T11:38:26.801403+0000","last_fullsized":"2024-09-10T11:38:26.801403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:21.576433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801850+0000","last_change":"2024-09-10T11:38:25.696483+0000","last_active":"2024-09-10T11:38:26.801850+0000","last_peered":"2024-09-10T11:38:26.801850+0000","last_clean":"2024-09-10T11:38:26.801850+0000","last_became_active":"2024-09-10T11:38:25.696320+0000","last_became_peered":"2024-09-10T11:38:25.696320+0000","last_unstale":"2024-09-10T11:38:26.801850+0000","last_undegraded":"2024-09-10T11:38:26.801850+0000","last_fullsized":"2024-09-10T11:38:26.801850+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:24.144036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801512+0000","last_change":"2024-09-10T11:38:24.685092+0000","last_active":"2024-09-10T11:38:26.801512+0000","last_peered":"2024-09-10T11:38:26.801512+0000","last_clean":"2024-09-10T11:38:26.801512+0000","last_became_active":"2024-09-10T11:38:24.684859+0000","last_became_peered":"2024-09-10T11:38:24.684859+0000","last_unstale":"2024-09-10T11:38:26.801512+0000","last_undegraded":"2024-09-10T11:38:26.801512+0000","last_fullsized":"2024-09-10T11:38:26.801512+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:15.638958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.690450+0000","last_change":"2024-09-10T11:38:25.690186+0000","last_active":"2024-09-10T11:38:25.690450+0000","last_peered":"2024-09-10T11:38:25.690450+0000","last_clean":"2024-09-10T11:38:25.690450+0000","last_became_active":"2024-09-10T11:38:25.689955+0000","last_became_peered":"2024-09-10T11:38:25.689955+0000","last_unstale":"2024-09-10T11:38:25.690450+0000","last_undegraded":"2024-09-10T11:38:25.690450+0000","last_fullsized":"2024-09-10T11:38:25.690450+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:26.863667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687368+0000","last_change":"2024-09-10T11:38:24.692994+0000","last_active":"2024-09-10T11:38:25.687368+0000","last_peered":"2024-09-10T11:38:25.687368+0000","last_clean":"2024-09-10T11:38:25.687368+0000","last_became_active":"2024-09-10T11:38:24.692755+0000","last_became_peered":"2024-09-10T11:38:24.692755+0000","last_unstale":"2024-09-10T11:38:25.687368+0000","last_undegraded":"2024-09-10T11:38:25.687368+0000","last_fullsized":"2024-09-10T11:38:25.687368+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:21.226346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686776+0000","last_change":"2024-09-10T11:38:25.686606+0000","last_active":"2024-09-10T11:38:25.686776+0000","last_peered":"2024-09-10T11:38:25.686776+0000","last_clean":"2024-09-10T11:38:25.686776+0000","last_became_active":"2024-09-10T11:38:25.685917+0000","last_became_peered":"2024-09-10T11:38:25.685917+0000","last_unstale":"2024-09-10T11:38:25.686776+0000","last_undegraded":"2024-09-10T11:38:25.686776+0000","last_fullsized":"2024-09-10T11:38:25.686776+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.244404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800740+0000","last_change":"2024-09-10T11:38:24.687939+0000","last_active":"2024-09-10T11:38:26.800740+0000","last_peered":"2024-09-10T11:38:26.800740+0000","last_clean":"2024-09-10T11:38:26.800740+0000","last_became_active":"2024-09-10T11:38:24.687712+0000","last_became_peered":"2024-09-10T11:38:24.687712+0000","last_unstale":"2024-09-10T11:38:26.800740+0000","last_undegraded":"2024-09-10T11:38:26.800740+0000","last_fullsized":"2024-09-10T11:38:26.800740+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:58.607827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697972+0000","last_change":"2024-09-10T11:38:25.697813+0000","last_active":"2024-09-10T11:38:25.697972+0000","last_peered":"2024-09-10T11:38:25.697972+0000","last_clean":"2024-09-10T11:38:25.697972+0000","last_became_active":"2024-09-10T11:38:25.697592+0000","last_became_peered":"2024-09-10T11:38:25.697592+0000","last_unstale":"2024-09-10T11:38:25.697972+0000","last_undegraded":"2024-09-10T11:38:25.697972+0000","last_fullsized":"2024-09-10T11:38:25.697972+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:52.687300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682665+0000","last_change":"2024-09-10T11:38:24.686178+0000","last_active":"2024-09-10T11:38:26.682665+0000","last_peered":"2024-09-10T11:38:26.682665+0000","last_clean":"2024-09-10T11:38:26.682665+0000","last_became_active":"2024-09-10T11:38:24.685930+0000","last_became_peered":"2024-09-10T11:38:24.685930+0000","last_unstale":"2024-09-10T11:38:26.682665+0000","last_undegraded":"2024-09-10T11:38:26.682665+0000","last_fullsized":"2024-09-10T11:38:26.682665+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:23.512009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801401+0000","last_change":"2024-09-10T11:38:25.689762+0000","last_active":"2024-09-10T11:38:26.801401+0000","last_peered":"2024-09-10T11:38:26.801401+0000","last_clean":"2024-09-10T11:38:26.801401+0000","last_became_active":"2024-09-10T11:38:25.689521+0000","last_became_peered":"2024-09-10T11:38:25.689521+0000","last_unstale":"2024-09-10T11:38:26.801401+0000","last_undegraded":"2024-09-10T11:38:26.801401+0000","last_fullsized":"2024-09-10T11:38:26.801401+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:56.040467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"166'3","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800648+0000","last_change":"2024-09-10T11:38:24.687015+0000","last_active":"2024-09-10T11:38:26.800648+0000","last_peered":"2024-09-10T11:38:26.800648+0000","last_clean":"2024-09-10T11:38:26.800648+0000","last_became_active":"2024-09-10T11:38:24.686783+0000","last_became_peered":"2024-09-10T11:38:24.686783+0000","last_unstale":"2024-09-10T11:38:26.800648+0000","last_undegraded":"2024-09-10T11:38:26.800648+0000","last_fullsized":"2024-09-10T11:38:26.800648+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:31.771346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":466,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685283+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:38:26.685283+0000","last_peered":"2024-09-10T11:38:26.685283+0000","last_clean":"2024-09-10T11:38:26.685283+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:38:26.685283+0000","last_undegraded":"2024-09-10T11:38:26.685283+0000","last_fullsized":"2024-09-10T11:38:26.685283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073389,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073389,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1483,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193829,"statfs":{"total":200000000,"available":198481215,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":907,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071173,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":905,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199072773,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073367,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1487,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193824,"statfs":{"total":200000000,"available":198476651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:38:28.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:38:28.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.769+0000 7f4b59c8d640 1 Processor -- start 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.769+0000 7f4b59c8d640 1 -- start start 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b59c8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415aef0 0x7f4b540b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b59c8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b5415b2d0 0x7f4b540afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b59c8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5406ee10 con 0x7f4b540cd290 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b59c8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4b540b0290 con 0x7f4b5415b2d0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b59c8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5406e570 con 0x7f4b5415aef0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b5415b2d0 0x7f4b540afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4b5415b2d0 0x7f4b540afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36302/0 (socket says 172.21.15.99:36302) 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b53fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4b540cd290 0x7f4b540714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39068/0 (socket says 172.21.15.99:39068) 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 -- 172.21.15.99:0/2997751198 learned_addr learned my addr 172.21.15.99:0/2997751198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b537fe640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415aef0 0x7f4b540b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 -- 172.21.15.99:0/2997751198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415aef0 msgr2=0x7f4b540b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415aef0 0x7f4b540b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 -- 172.21.15.99:0/2997751198 >> v1:172.21.15.99:6789/0 conn(0x7f4b540cd290 legacy=0x7f4b540714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b52ffd640 1 -- 172.21.15.99:0/2997751198 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3007094255 0 0) 0x7f4b5406ee10 con 0x7f4b540cd290 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.770+0000 7f4b4bfff640 1 -- 172.21.15.99:0/2997751198 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5406e850 con 0x7f4b5415b2d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b5415b2d0 0x7f4b540afbb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009870 tx=0x7f4b3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d770b9fdf5c62ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b52ffd640 1 -- 172.21.15.99:0/2997751198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c037070 con 0x7f4b5415b2d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b52ffd640 1 -- 172.21.15.99:0/2997751198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c032440 con 0x7f4b5415b2d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b52ffd640 1 -- 172.21.15.99:0/2997751198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c032810 con 0x7f4b5415b2d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 -- 172.21.15.99:0/2997751198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b5415b2d0 msgr2=0x7f4b540afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b5415b2d0 0x7f4b540afbb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009870 tx=0x7f4b3c02a270 comp rx=0 tx=0).stop 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 -- 172.21.15.99:0/2997751198 shutdown_connections 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b5415b2d0 0x7f4b540afbb0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/2997751198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415aef0 0x7f4b540b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.771+0000 7f4b59c8d640 1 -- 172.21.15.99:0/2997751198 >> 172.21.15.99:0/2997751198 conn(0x7f4b5406bc10 msgr2=0x7f4b5406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.772+0000 7f4b59c8d640 1 -- 172.21.15.99:0/2997751198 shutdown_connections 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.772+0000 7f4b59c8d640 1 -- 172.21.15.99:0/2997751198 wait complete. 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 Processor -- start 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 -- start start 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5415aef0 0x7f4b540af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415b2d0 0x7f4b54152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b54071d30 con 0x7f4b540cd290 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b54071bb0 con 0x7f4b5415aef0 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b59c8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4b54071eb0 con 0x7f4b5415b2d0 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.773+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36318/0 (socket says 172.21.15.99:36318) 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 -- 172.21.15.99:0/3298748764 learned_addr learned my addr 172.21.15.99:0/3298748764 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:28.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5415aef0 0x7f4b540af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b53fff640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415b2d0 0x7f4b54152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 -- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415b2d0 msgr2=0x7f4b54152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415b2d0 0x7f4b54152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 -- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5415aef0 msgr2=0x7f4b540af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5415aef0 0x7f4b540af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 -- 172.21.15.99:0/3298748764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b541f3430 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b537fe640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d5f0 tx=0x7f4b4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440125f0 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b44019070 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b541f2670 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440129d0 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.774+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541f2c30 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.775+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b540cd760 con 0x7f4b540cd290 2024-09-10T11:38:28.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.775+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b44012b70 con 0x7f4b540cd290 2024-09-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.776+0000 7f4b50ff9640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 0x7f4b3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.776+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f4b4408b270 con 0x7f4b540cd290 2024-09-10T11:38:28.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.776+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 0x7f4b3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.776+0000 7f4b4bfff640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 0x7f4b3406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c034000 tx=0x7f4b3c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.780+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b44059c20 con 0x7f4b540cd290 2024-09-10T11:38:28.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.941+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b540b4830 con 0x7f4b3406be30 2024-09-10T11:38:28.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.946+0000 7f4b50ff9640 1 -- 172.21.15.99:0/3298748764 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247971 (secure 0 0 0) 0x7f4b540b4830 con 0x7f4b3406be30 2024-09-10T11:38:28.950 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:28.951 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 msgr2=0x7f4b3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 0x7f4b3406e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c034000 tx=0x7f4b3c02e000 comp rx=0 tx=0).stop 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 msgr2=0x7f4b540aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400d5f0 tx=0x7f4b4400dab0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 shutdown_connections 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4b3406be30 0x7f4b3406e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b5415b2d0 0x7f4b54152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b5415aef0 0x7f4b540af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 --2- 172.21.15.99:0/3298748764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b540cd290 0x7f4b540aaea0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 >> 172.21.15.99:0/3298748764 conn(0x7f4b5406bc10 msgr2=0x7f4b54072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 shutdown_connections 2024-09-10T11:38:28.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:28.949+0000 7f4b59c8d640 1 -- 172.21.15.99:0/3298748764 wait complete. 2024-09-10T11:38:28.962 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-10T11:38:27.013951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8403,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554096,"statfs":{"total":1600000000,"available":1591395256,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000531"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687845+0000","last_change":"2024-09-10T11:38:25.687224+0000","last_active":"2024-09-10T11:38:26.687845+0000","last_peered":"2024-09-10T11:38:26.687845+0000","last_clean":"2024-09-10T11:38:26.687845+0000","last_became_active":"2024-09-10T11:38:25.686985+0000","last_became_peered":"2024-09-10T11:38:25.686985+0000","last_unstale":"2024-09-10T11:38:26.687845+0000","last_undegraded":"2024-09-10T11:38:26.687845+0000","last_fullsized":"2024-09-10T11:38:26.687845+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:39.411934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801458+0000","last_change":"2024-09-10T11:38:25.687890+0000","last_active":"2024-09-10T11:38:26.801458+0000","last_peered":"2024-09-10T11:38:26.801458+0000","last_clean":"2024-09-10T11:38:26.801458+0000","last_became_active":"2024-09-10T11:38:25.687640+0000","last_became_peered":"2024-09-10T11:38:25.687640+0000","last_unstale":"2024-09-10T11:38:26.801458+0000","last_undegraded":"2024-09-10T11:38:26.801458+0000","last_fullsized":"2024-09-10T11:38:26.801458+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:33.234753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.693764+0000","last_change":"2024-09-10T11:38:25.690916+0000","last_active":"2024-09-10T11:38:25.693764+0000","last_peered":"2024-09-10T11:38:25.693764+0000","last_clean":"2024-09-10T11:38:25.693764+0000","last_became_active":"2024-09-10T11:38:25.690639+0000","last_became_peered":"2024-09-10T11:38:25.690639+0000","last_unstale":"2024-09-10T11:38:25.693764+0000","last_undegraded":"2024-09-10T11:38:25.693764+0000","last_fullsized":"2024-09-10T11:38:25.693764+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:29.612282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.688103+0000","last_change":"2024-09-10T11:38:25.687981+0000","last_active":"2024-09-10T11:38:25.688103+0000","last_peered":"2024-09-10T11:38:25.688103+0000","last_clean":"2024-09-10T11:38:25.688103+0000","last_became_active":"2024-09-10T11:38:25.687754+0000","last_became_peered":"2024-09-10T11:38:25.687754+0000","last_unstale":"2024-09-10T11:38:25.688103+0000","last_undegraded":"2024-09-10T11:38:25.688103+0000","last_fullsized":"2024-09-10T11:38:25.688103+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:57.137610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685317+0000","last_change":"2024-09-10T11:38:25.696827+0000","last_active":"2024-09-10T11:38:26.685317+0000","last_peered":"2024-09-10T11:38:26.685317+0000","last_clean":"2024-09-10T11:38:26.685317+0000","last_became_active":"2024-09-10T11:38:25.696538+0000","last_became_peered":"2024-09-10T11:38:25.696538+0000","last_unstale":"2024-09-10T11:38:26.685317+0000","last_undegraded":"2024-09-10T11:38:26.685317+0000","last_fullsized":"2024-09-10T11:38:26.685317+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:34.329985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682348+0000","last_change":"2024-09-10T11:38:25.689518+0000","last_active":"2024-09-10T11:38:26.682348+0000","last_peered":"2024-09-10T11:38:26.682348+0000","last_clean":"2024-09-10T11:38:26.682348+0000","last_became_active":"2024-09-10T11:38:25.689266+0000","last_became_peered":"2024-09-10T11:38:25.689266+0000","last_unstale":"2024-09-10T11:38:26.682348+0000","last_undegraded":"2024-09-10T11:38:26.682348+0000","last_fullsized":"2024-09-10T11:38:26.682348+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:17.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801693+0000","last_change":"2024-09-10T11:38:25.691690+0000","last_active":"2024-09-10T11:38:26.801693+0000","last_peered":"2024-09-10T11:38:26.801693+0000","last_clean":"2024-09-10T11:38:26.801693+0000","last_became_active":"2024-09-10T11:38:25.691438+0000","last_became_peered":"2024-09-10T11:38:25.691438+0000","last_unstale":"2024-09-10T11:38:26.801693+0000","last_undegraded":"2024-09-10T11:38:26.801693+0000","last_fullsized":"2024-09-10T11:38:26.801693+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687968+0000","last_change":"2024-09-10T11:38:25.690243+0000","last_active":"2024-09-10T11:38:26.687968+0000","last_peered":"2024-09-10T11:38:26.687968+0000","last_clean":"2024-09-10T11:38:26.687968+0000","last_became_active":"2024-09-10T11:38:25.689981+0000","last_became_peered":"2024-09-10T11:38:25.689981+0000","last_unstale":"2024-09-10T11:38:26.687968+0000","last_undegraded":"2024-09-10T11:38:26.687968+0000","last_fullsized":"2024-09-10T11:38:26.687968+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:34.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682381+0000","last_change":"2024-09-10T11:38:25.688782+0000","last_active":"2024-09-10T11:38:26.682381+0000","last_peered":"2024-09-10T11:38:26.682381+0000","last_clean":"2024-09-10T11:38:26.682381+0000","last_became_active":"2024-09-10T11:38:25.688516+0000","last_became_peered":"2024-09-10T11:38:25.688516+0000","last_unstale":"2024-09-10T11:38:26.682381+0000","last_undegraded":"2024-09-10T11:38:26.682381+0000","last_fullsized":"2024-09-10T11:38:26.682381+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:31.091784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.694661+0000","last_change":"2024-09-10T11:38:25.694516+0000","last_active":"2024-09-10T11:38:25.694661+0000","last_peered":"2024-09-10T11:38:25.694661+0000","last_clean":"2024-09-10T11:38:25.694661+0000","last_became_active":"2024-09-10T11:38:25.694262+0000","last_became_peered":"2024-09-10T11:38:25.694262+0000","last_unstale":"2024-09-10T11:38:25.694661+0000","last_undegraded":"2024-09-10T11:38:25.694661+0000","last_fullsized":"2024-09-10T11:38:25.694661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:51:27.484588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682466+0000","last_change":"2024-09-10T11:38:25.695919+0000","last_active":"2024-09-10T11:38:26.682466+0000","last_peered":"2024-09-10T11:38:26.682466+0000","last_clean":"2024-09-10T11:38:26.682466+0000","last_became_active":"2024-09-10T11:38:25.695302+0000","last_became_peered":"2024-09-10T11:38:25.695302+0000","last_unstale":"2024-09-10T11:38:26.682466+0000","last_undegraded":"2024-09-10T11:38:26.682466+0000","last_fullsized":"2024-09-10T11:38:26.682466+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:47.841095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800886+0000","last_change":"2024-09-10T11:38:25.690027+0000","last_active":"2024-09-10T11:38:26.800886+0000","last_peered":"2024-09-10T11:38:26.800886+0000","last_clean":"2024-09-10T11:38:26.800886+0000","last_became_active":"2024-09-10T11:38:25.689749+0000","last_became_peered":"2024-09-10T11:38:25.689749+0000","last_unstale":"2024-09-10T11:38:26.800886+0000","last_undegraded":"2024-09-10T11:38:26.800886+0000","last_fullsized":"2024-09-10T11:38:26.800886+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:00.218607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685153+0000","last_change":"2024-09-10T11:38:25.701208+0000","last_active":"2024-09-10T11:38:26.685153+0000","last_peered":"2024-09-10T11:38:26.685153+0000","last_clean":"2024-09-10T11:38:26.685153+0000","last_became_active":"2024-09-10T11:38:25.700959+0000","last_became_peered":"2024-09-10T11:38:25.700959+0000","last_unstale":"2024-09-10T11:38:26.685153+0000","last_undegraded":"2024-09-10T11:38:26.685153+0000","last_fullsized":"2024-09-10T11:38:26.685153+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:27.799154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800854+0000","last_change":"2024-09-10T11:38:25.695404+0000","last_active":"2024-09-10T11:38:26.800854+0000","last_peered":"2024-09-10T11:38:26.800854+0000","last_clean":"2024-09-10T11:38:26.800854+0000","last_became_active":"2024-09-10T11:38:25.695189+0000","last_became_peered":"2024-09-10T11:38:25.695189+0000","last_unstale":"2024-09-10T11:38:26.800854+0000","last_undegraded":"2024-09-10T11:38:26.800854+0000","last_fullsized":"2024-09-10T11:38:26.800854+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:02:16.152541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685444+0000","last_change":"2024-09-10T11:38:25.696228+0000","last_active":"2024-09-10T11:38:26.685444+0000","last_peered":"2024-09-10T11:38:26.685444+0000","last_clean":"2024-09-10T11:38:26.685444+0000","last_became_active":"2024-09-10T11:38:25.695892+0000","last_became_peered":"2024-09-10T11:38:25.695892+0000","last_unstale":"2024-09-10T11:38:26.685444+0000","last_undegraded":"2024-09-10T11:38:26.685444+0000","last_fullsized":"2024-09-10T11:38:26.685444+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:04:58.984346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801728+0000","last_change":"2024-09-10T11:38:25.690365+0000","last_active":"2024-09-10T11:38:26.801728+0000","last_peered":"2024-09-10T11:38:26.801728+0000","last_clean":"2024-09-10T11:38:26.801728+0000","last_became_active":"2024-09-10T11:38:25.690146+0000","last_became_peered":"2024-09-10T11:38:25.690146+0000","last_unstale":"2024-09-10T11:38:26.801728+0000","last_undegraded":"2024-09-10T11:38:26.801728+0000","last_fullsized":"2024-09-10T11:38:26.801728+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:29.762856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697699+0000","last_change":"2024-09-10T11:38:25.695403+0000","last_active":"2024-09-10T11:38:25.697699+0000","last_peered":"2024-09-10T11:38:25.697699+0000","last_clean":"2024-09-10T11:38:25.697699+0000","last_became_active":"2024-09-10T11:38:25.695046+0000","last_became_peered":"2024-09-10T11:38:25.695046+0000","last_unstale":"2024-09-10T11:38:25.697699+0000","last_undegraded":"2024-09-10T11:38:25.697699+0000","last_fullsized":"2024-09-10T11:38:25.697699+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:48:23.262137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685255+0000","last_change":"2024-09-10T11:38:25.699619+0000","last_active":"2024-09-10T11:38:26.685255+0000","last_peered":"2024-09-10T11:38:26.685255+0000","last_clean":"2024-09-10T11:38:26.685255+0000","last_became_active":"2024-09-10T11:38:25.698673+0000","last_became_peered":"2024-09-10T11:38:25.698673+0000","last_unstale":"2024-09-10T11:38:26.685255+0000","last_undegraded":"2024-09-10T11:38:26.685255+0000","last_fullsized":"2024-09-10T11:38:26.685255+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:23.682006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687913+0000","last_change":"2024-09-10T11:38:25.684850+0000","last_active":"2024-09-10T11:38:26.687913+0000","last_peered":"2024-09-10T11:38:26.687913+0000","last_clean":"2024-09-10T11:38:26.687913+0000","last_became_active":"2024-09-10T11:38:25.684630+0000","last_became_peered":"2024-09-10T11:38:25.684630+0000","last_unstale":"2024-09-10T11:38:26.687913+0000","last_undegraded":"2024-09-10T11:38:26.687913+0000","last_fullsized":"2024-09-10T11:38:26.687913+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:15.333866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687846+0000","last_change":"2024-09-10T11:38:25.687966+0000","last_active":"2024-09-10T11:38:26.687846+0000","last_peered":"2024-09-10T11:38:26.687846+0000","last_clean":"2024-09-10T11:38:26.687846+0000","last_became_active":"2024-09-10T11:38:25.687736+0000","last_became_peered":"2024-09-10T11:38:25.687736+0000","last_unstale":"2024-09-10T11:38:26.687846+0000","last_undegraded":"2024-09-10T11:38:26.687846+0000","last_fullsized":"2024-09-10T11:38:26.687846+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:40.769460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697719+0000","last_change":"2024-09-10T11:38:25.697019+0000","last_active":"2024-09-10T11:38:25.697719+0000","last_peered":"2024-09-10T11:38:25.697719+0000","last_clean":"2024-09-10T11:38:25.697719+0000","last_became_active":"2024-09-10T11:38:25.696804+0000","last_became_peered":"2024-09-10T11:38:25.696804+0000","last_unstale":"2024-09-10T11:38:25.697719+0000","last_undegraded":"2024-09-10T11:38:25.697719+0000","last_fullsized":"2024-09-10T11:38:25.697719+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:42.245845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800974+0000","last_change":"2024-09-10T11:38:25.693388+0000","last_active":"2024-09-10T11:38:26.800974+0000","last_peered":"2024-09-10T11:38:26.800974+0000","last_clean":"2024-09-10T11:38:26.800974+0000","last_became_active":"2024-09-10T11:38:25.693137+0000","last_became_peered":"2024-09-10T11:38:25.693137+0000","last_unstale":"2024-09-10T11:38:26.800974+0000","last_undegraded":"2024-09-10T11:38:26.800974+0000","last_fullsized":"2024-09-10T11:38:26.800974+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:34.673348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.683579+0000","last_change":"2024-09-10T11:38:25.683454+0000","last_active":"2024-09-10T11:38:25.683579+0000","last_peered":"2024-09-10T11:38:25.683579+0000","last_clean":"2024-09-10T11:38:25.683579+0000","last_became_active":"2024-09-10T11:38:25.683217+0000","last_became_peered":"2024-09-10T11:38:25.683217+0000","last_unstale":"2024-09-10T11:38:25.683579+0000","last_undegraded":"2024-09-10T11:38:25.683579+0000","last_fullsized":"2024-09-10T11:38:25.683579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:04.564864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685133+0000","last_change":"2024-09-10T11:38:25.701761+0000","last_active":"2024-09-10T11:38:26.685133+0000","last_peered":"2024-09-10T11:38:26.685133+0000","last_clean":"2024-09-10T11:38:26.685133+0000","last_became_active":"2024-09-10T11:38:25.701507+0000","last_became_peered":"2024-09-10T11:38:25.701507+0000","last_unstale":"2024-09-10T11:38:26.685133+0000","last_undegraded":"2024-09-10T11:38:26.685133+0000","last_fullsized":"2024-09-10T11:38:26.685133+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:16.120928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801587+0000","last_change":"2024-09-10T11:38:25.690488+0000","last_active":"2024-09-10T11:38:26.801587+0000","last_peered":"2024-09-10T11:38:26.801587+0000","last_clean":"2024-09-10T11:38:26.801587+0000","last_became_active":"2024-09-10T11:38:25.690238+0000","last_became_peered":"2024-09-10T11:38:25.690238+0000","last_unstale":"2024-09-10T11:38:26.801587+0000","last_undegraded":"2024-09-10T11:38:26.801587+0000","last_fullsized":"2024-09-10T11:38:26.801587+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:51.627617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687703+0000","last_change":"2024-09-10T11:38:25.688753+0000","last_active":"2024-09-10T11:38:26.687703+0000","last_peered":"2024-09-10T11:38:26.687703+0000","last_clean":"2024-09-10T11:38:26.687703+0000","last_became_active":"2024-09-10T11:38:25.688473+0000","last_became_peered":"2024-09-10T11:38:25.688473+0000","last_unstale":"2024-09-10T11:38:26.687703+0000","last_undegraded":"2024-09-10T11:38:26.687703+0000","last_fullsized":"2024-09-10T11:38:26.687703+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:36.422421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801630+0000","last_change":"2024-09-10T11:38:25.692365+0000","last_active":"2024-09-10T11:38:26.801630+0000","last_peered":"2024-09-10T11:38:26.801630+0000","last_clean":"2024-09-10T11:38:26.801630+0000","last_became_active":"2024-09-10T11:38:25.691514+0000","last_became_peered":"2024-09-10T11:38:25.691514+0000","last_unstale":"2024-09-10T11:38:26.801630+0000","last_undegraded":"2024-09-10T11:38:26.801630+0000","last_fullsized":"2024-09-10T11:38:26.801630+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:53.485285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801757+0000","last_change":"2024-09-10T11:38:25.692897+0000","last_active":"2024-09-10T11:38:26.801757+0000","last_peered":"2024-09-10T11:38:26.801757+0000","last_clean":"2024-09-10T11:38:26.801757+0000","last_became_active":"2024-09-10T11:38:25.692322+0000","last_became_peered":"2024-09-10T11:38:25.692322+0000","last_unstale":"2024-09-10T11:38:26.801757+0000","last_undegraded":"2024-09-10T11:38:26.801757+0000","last_fullsized":"2024-09-10T11:38:26.801757+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:21.002516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688204+0000","last_change":"2024-09-10T11:38:25.692760+0000","last_active":"2024-09-10T11:38:26.688204+0000","last_peered":"2024-09-10T11:38:26.688204+0000","last_clean":"2024-09-10T11:38:26.688204+0000","last_became_active":"2024-09-10T11:38:25.692380+0000","last_became_peered":"2024-09-10T11:38:25.692380+0000","last_unstale":"2024-09-10T11:38:26.688204+0000","last_undegraded":"2024-09-10T11:38:26.688204+0000","last_fullsized":"2024-09-10T11:38:26.688204+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:49.115544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800973+0000","last_change":"2024-09-10T11:38:25.692535+0000","last_active":"2024-09-10T11:38:26.800973+0000","last_peered":"2024-09-10T11:38:26.800973+0000","last_clean":"2024-09-10T11:38:26.800973+0000","last_became_active":"2024-09-10T11:38:25.692304+0000","last_became_peered":"2024-09-10T11:38:25.692304+0000","last_unstale":"2024-09-10T11:38:26.800973+0000","last_undegraded":"2024-09-10T11:38:26.800973+0000","last_fullsized":"2024-09-10T11:38:26.800973+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:18.744856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801399+0000","last_change":"2024-09-10T11:38:25.690997+0000","last_active":"2024-09-10T11:38:26.801399+0000","last_peered":"2024-09-10T11:38:26.801399+0000","last_clean":"2024-09-10T11:38:26.801399+0000","last_became_active":"2024-09-10T11:38:25.690778+0000","last_became_peered":"2024-09-10T11:38:25.690778+0000","last_unstale":"2024-09-10T11:38:26.801399+0000","last_undegraded":"2024-09-10T11:38:26.801399+0000","last_fullsized":"2024-09-10T11:38:26.801399+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:23.280103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801786+0000","last_change":"2024-09-10T11:38:25.692932+0000","last_active":"2024-09-10T11:38:26.801786+0000","last_peered":"2024-09-10T11:38:26.801786+0000","last_clean":"2024-09-10T11:38:26.801786+0000","last_became_active":"2024-09-10T11:38:25.692699+0000","last_became_peered":"2024-09-10T11:38:25.692699+0000","last_unstale":"2024-09-10T11:38:26.801786+0000","last_undegraded":"2024-09-10T11:38:26.801786+0000","last_fullsized":"2024-09-10T11:38:26.801786+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:49.142604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685250+0000","last_change":"2024-09-10T11:38:25.700283+0000","last_active":"2024-09-10T11:38:26.685250+0000","last_peered":"2024-09-10T11:38:26.685250+0000","last_clean":"2024-09-10T11:38:26.685250+0000","last_became_active":"2024-09-10T11:38:25.700069+0000","last_became_peered":"2024-09-10T11:38:25.700069+0000","last_unstale":"2024-09-10T11:38:26.685250+0000","last_undegraded":"2024-09-10T11:38:26.685250+0000","last_fullsized":"2024-09-10T11:38:26.685250+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:50:09.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"166'4","reported_seq":16,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.704874+0000","last_change":"2024-09-10T11:38:24.694742+0000","last_active":"2024-09-10T11:38:25.704874+0000","last_peered":"2024-09-10T11:38:25.704874+0000","last_clean":"2024-09-10T11:38:25.704874+0000","last_became_active":"2024-09-10T11:38:24.694576+0000","last_became_peered":"2024-09-10T11:38:24.694576+0000","last_unstale":"2024-09-10T11:38:25.704874+0000","last_undegraded":"2024-09-10T11:38:25.704874+0000","last_fullsized":"2024-09-10T11:38:25.704874+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:10:25.198002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801137+0000","last_change":"2024-09-10T11:38:25.693920+0000","last_active":"2024-09-10T11:38:26.801137+0000","last_peered":"2024-09-10T11:38:26.801137+0000","last_clean":"2024-09-10T11:38:26.801137+0000","last_became_active":"2024-09-10T11:38:25.693663+0000","last_became_peered":"2024-09-10T11:38:25.693663+0000","last_unstale":"2024-09-10T11:38:26.801137+0000","last_undegraded":"2024-09-10T11:38:26.801137+0000","last_fullsized":"2024-09-10T11:38:26.801137+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:50.994377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682491+0000","last_change":"2024-09-10T11:38:24.687134+0000","last_active":"2024-09-10T11:38:26.682491+0000","last_peered":"2024-09-10T11:38:26.682491+0000","last_clean":"2024-09-10T11:38:26.682491+0000","last_became_active":"2024-09-10T11:38:24.686844+0000","last_became_peered":"2024-09-10T11:38:24.686844+0000","last_unstale":"2024-09-10T11:38:26.682491+0000","last_undegraded":"2024-09-10T11:38:26.682491+0000","last_fullsized":"2024-09-10T11:38:26.682491+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:02.555327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682558+0000","last_change":"2024-09-10T11:38:25.695767+0000","last_active":"2024-09-10T11:38:26.682558+0000","last_peered":"2024-09-10T11:38:26.682558+0000","last_clean":"2024-09-10T11:38:26.682558+0000","last_became_active":"2024-09-10T11:38:25.694847+0000","last_became_peered":"2024-09-10T11:38:25.694847+0000","last_unstale":"2024-09-10T11:38:26.682558+0000","last_undegraded":"2024-09-10T11:38:26.682558+0000","last_fullsized":"2024-09-10T11:38:26.682558+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:51:14.173322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.703986+0000","last_change":"2024-09-10T11:38:24.689156+0000","last_active":"2024-09-10T11:38:25.703986+0000","last_peered":"2024-09-10T11:38:25.703986+0000","last_clean":"2024-09-10T11:38:25.703986+0000","last_became_active":"2024-09-10T11:38:24.688634+0000","last_became_peered":"2024-09-10T11:38:24.688634+0000","last_unstale":"2024-09-10T11:38:25.703986+0000","last_undegraded":"2024-09-10T11:38:25.703986+0000","last_fullsized":"2024-09-10T11:38:25.703986+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:12:12.809145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687579+0000","last_change":"2024-09-10T11:38:25.687474+0000","last_active":"2024-09-10T11:38:25.687579+0000","last_peered":"2024-09-10T11:38:25.687579+0000","last_clean":"2024-09-10T11:38:25.687579+0000","last_became_active":"2024-09-10T11:38:25.687239+0000","last_became_peered":"2024-09-10T11:38:25.687239+0000","last_unstale":"2024-09-10T11:38:25.687579+0000","last_undegraded":"2024-09-10T11:38:25.687579+0000","last_fullsized":"2024-09-10T11:38:25.687579+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:30.577654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.703709+0000","last_change":"2024-09-10T11:38:24.692321+0000","last_active":"2024-09-10T11:38:25.703709+0000","last_peered":"2024-09-10T11:38:25.703709+0000","last_clean":"2024-09-10T11:38:25.703709+0000","last_became_active":"2024-09-10T11:38:24.692052+0000","last_became_peered":"2024-09-10T11:38:24.692052+0000","last_unstale":"2024-09-10T11:38:25.703709+0000","last_undegraded":"2024-09-10T11:38:25.703709+0000","last_fullsized":"2024-09-10T11:38:25.703709+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:38.649830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801907+0000","last_change":"2024-09-10T11:38:25.693269+0000","last_active":"2024-09-10T11:38:26.801907+0000","last_peered":"2024-09-10T11:38:26.801907+0000","last_clean":"2024-09-10T11:38:26.801907+0000","last_became_active":"2024-09-10T11:38:25.692670+0000","last_became_peered":"2024-09-10T11:38:25.692670+0000","last_unstale":"2024-09-10T11:38:26.801907+0000","last_undegraded":"2024-09-10T11:38:26.801907+0000","last_fullsized":"2024-09-10T11:38:26.801907+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:48.054890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.687776+0000","last_change":"2024-09-10T11:38:24.694087+0000","last_active":"2024-09-10T11:38:26.687776+0000","last_peered":"2024-09-10T11:38:26.687776+0000","last_clean":"2024-09-10T11:38:26.687776+0000","last_became_active":"2024-09-10T11:38:24.693842+0000","last_became_peered":"2024-09-10T11:38:24.693842+0000","last_unstale":"2024-09-10T11:38:26.687776+0000","last_undegraded":"2024-09-10T11:38:26.687776+0000","last_fullsized":"2024-09-10T11:38:26.687776+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:28:49.926387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.690415+0000","last_change":"2024-09-10T11:38:25.689130+0000","last_active":"2024-09-10T11:38:25.690415+0000","last_peered":"2024-09-10T11:38:25.690415+0000","last_clean":"2024-09-10T11:38:25.690415+0000","last_became_active":"2024-09-10T11:38:25.688724+0000","last_became_peered":"2024-09-10T11:38:25.688724+0000","last_unstale":"2024-09-10T11:38:25.690415+0000","last_undegraded":"2024-09-10T11:38:25.690415+0000","last_fullsized":"2024-09-10T11:38:25.690415+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:42.156095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685312+0000","last_change":"2024-09-10T11:38:24.691448+0000","last_active":"2024-09-10T11:38:26.685312+0000","last_peered":"2024-09-10T11:38:26.685312+0000","last_clean":"2024-09-10T11:38:26.685312+0000","last_became_active":"2024-09-10T11:38:24.691182+0000","last_became_peered":"2024-09-10T11:38:24.691182+0000","last_unstale":"2024-09-10T11:38:26.685312+0000","last_undegraded":"2024-09-10T11:38:26.685312+0000","last_fullsized":"2024-09-10T11:38:26.685312+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:35.379407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801797+0000","last_change":"2024-09-10T11:38:25.691108+0000","last_active":"2024-09-10T11:38:26.801797+0000","last_peered":"2024-09-10T11:38:26.801797+0000","last_clean":"2024-09-10T11:38:26.801797+0000","last_became_active":"2024-09-10T11:38:25.690892+0000","last_became_peered":"2024-09-10T11:38:25.690892+0000","last_unstale":"2024-09-10T11:38:26.801797+0000","last_undegraded":"2024-09-10T11:38:26.801797+0000","last_fullsized":"2024-09-10T11:38:26.801797+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:00.056160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682602+0000","last_change":"2024-09-10T11:38:24.683415+0000","last_active":"2024-09-10T11:38:26.682602+0000","last_peered":"2024-09-10T11:38:26.682602+0000","last_clean":"2024-09-10T11:38:26.682602+0000","last_became_active":"2024-09-10T11:38:24.683108+0000","last_became_peered":"2024-09-10T11:38:24.683108+0000","last_unstale":"2024-09-10T11:38:26.682602+0000","last_undegraded":"2024-09-10T11:38:26.682602+0000","last_fullsized":"2024-09-10T11:38:26.682602+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:10.496496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686477+0000","last_change":"2024-09-10T11:38:25.686190+0000","last_active":"2024-09-10T11:38:25.686477+0000","last_peered":"2024-09-10T11:38:25.686477+0000","last_clean":"2024-09-10T11:38:25.686477+0000","last_became_active":"2024-09-10T11:38:25.685606+0000","last_became_peered":"2024-09-10T11:38:25.685606+0000","last_unstale":"2024-09-10T11:38:25.686477+0000","last_undegraded":"2024-09-10T11:38:25.686477+0000","last_fullsized":"2024-09-10T11:38:25.686477+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:33.709019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686256+0000","last_change":"2024-09-10T11:38:24.686937+0000","last_active":"2024-09-10T11:38:25.686256+0000","last_peered":"2024-09-10T11:38:25.686256+0000","last_clean":"2024-09-10T11:38:25.686256+0000","last_became_active":"2024-09-10T11:38:24.686709+0000","last_became_peered":"2024-09-10T11:38:24.686709+0000","last_unstale":"2024-09-10T11:38:25.686256+0000","last_undegraded":"2024-09-10T11:38:25.686256+0000","last_fullsized":"2024-09-10T11:38:25.686256+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:15.541212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.685425+0000","last_change":"2024-09-10T11:38:25.685309+0000","last_active":"2024-09-10T11:38:25.685425+0000","last_peered":"2024-09-10T11:38:25.685425+0000","last_clean":"2024-09-10T11:38:25.685425+0000","last_became_active":"2024-09-10T11:38:25.684800+0000","last_became_peered":"2024-09-10T11:38:25.684800+0000","last_unstale":"2024-09-10T11:38:25.685425+0000","last_undegraded":"2024-09-10T11:38:25.685425+0000","last_fullsized":"2024-09-10T11:38:25.685425+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:54.135197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682765+0000","last_change":"2024-09-10T11:38:24.691550+0000","last_active":"2024-09-10T11:38:26.682765+0000","last_peered":"2024-09-10T11:38:26.682765+0000","last_clean":"2024-09-10T11:38:26.682765+0000","last_became_active":"2024-09-10T11:38:24.691262+0000","last_became_peered":"2024-09-10T11:38:24.691262+0000","last_unstale":"2024-09-10T11:38:26.682765+0000","last_undegraded":"2024-09-10T11:38:26.682765+0000","last_fullsized":"2024-09-10T11:38:26.682765+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:48.660663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800734+0000","last_change":"2024-09-10T11:38:25.696483+0000","last_active":"2024-09-10T11:38:26.800734+0000","last_peered":"2024-09-10T11:38:26.800734+0000","last_clean":"2024-09-10T11:38:26.800734+0000","last_became_active":"2024-09-10T11:38:25.696256+0000","last_became_peered":"2024-09-10T11:38:25.696256+0000","last_unstale":"2024-09-10T11:38:26.800734+0000","last_undegraded":"2024-09-10T11:38:26.800734+0000","last_fullsized":"2024-09-10T11:38:26.800734+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:03.170739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685291+0000","last_change":"2024-09-10T11:38:24.687041+0000","last_active":"2024-09-10T11:38:26.685291+0000","last_peered":"2024-09-10T11:38:26.685291+0000","last_clean":"2024-09-10T11:38:26.685291+0000","last_became_active":"2024-09-10T11:38:24.686819+0000","last_became_peered":"2024-09-10T11:38:24.686819+0000","last_unstale":"2024-09-10T11:38:26.685291+0000","last_undegraded":"2024-09-10T11:38:26.685291+0000","last_fullsized":"2024-09-10T11:38:26.685291+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:55.204872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801320+0000","last_change":"2024-09-10T11:38:25.689829+0000","last_active":"2024-09-10T11:38:26.801320+0000","last_peered":"2024-09-10T11:38:26.801320+0000","last_clean":"2024-09-10T11:38:26.801320+0000","last_became_active":"2024-09-10T11:38:25.689571+0000","last_became_peered":"2024-09-10T11:38:25.689571+0000","last_unstale":"2024-09-10T11:38:26.801320+0000","last_undegraded":"2024-09-10T11:38:26.801320+0000","last_fullsized":"2024-09-10T11:38:26.801320+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:05:06.017693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"166'6","reported_seq":20,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801030+0000","last_change":"2024-09-10T11:38:24.691915+0000","last_active":"2024-09-10T11:38:26.801030+0000","last_peered":"2024-09-10T11:38:26.801030+0000","last_clean":"2024-09-10T11:38:26.801030+0000","last_became_active":"2024-09-10T11:38:24.691313+0000","last_became_peered":"2024-09-10T11:38:24.691313+0000","last_unstale":"2024-09-10T11:38:26.801030+0000","last_undegraded":"2024-09-10T11:38:26.801030+0000","last_fullsized":"2024-09-10T11:38:26.801030+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:53.750660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.696903+0000","last_change":"2024-09-10T11:38:25.695422+0000","last_active":"2024-09-10T11:38:25.696903+0000","last_peered":"2024-09-10T11:38:25.696903+0000","last_clean":"2024-09-10T11:38:25.696903+0000","last_became_active":"2024-09-10T11:38:25.695160+0000","last_became_peered":"2024-09-10T11:38:25.695160+0000","last_unstale":"2024-09-10T11:38:25.696903+0000","last_undegraded":"2024-09-10T11:38:25.696903+0000","last_fullsized":"2024-09-10T11:38:25.696903+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:48.452413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.704373+0000","last_change":"2024-09-10T11:38:24.689334+0000","last_active":"2024-09-10T11:38:25.704373+0000","last_peered":"2024-09-10T11:38:25.704373+0000","last_clean":"2024-09-10T11:38:25.704373+0000","last_became_active":"2024-09-10T11:38:24.688936+0000","last_became_peered":"2024-09-10T11:38:24.688936+0000","last_unstale":"2024-09-10T11:38:25.704373+0000","last_undegraded":"2024-09-10T11:38:25.704373+0000","last_fullsized":"2024-09-10T11:38:25.704373+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:12.557337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688008+0000","last_change":"2024-09-10T11:38:25.690085+0000","last_active":"2024-09-10T11:38:26.688008+0000","last_peered":"2024-09-10T11:38:26.688008+0000","last_clean":"2024-09-10T11:38:26.688008+0000","last_became_active":"2024-09-10T11:38:25.689841+0000","last_became_peered":"2024-09-10T11:38:25.689841+0000","last_unstale":"2024-09-10T11:38:26.688008+0000","last_undegraded":"2024-09-10T11:38:26.688008+0000","last_fullsized":"2024-09-10T11:38:26.688008+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:34.727036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801089+0000","last_change":"2024-09-10T11:38:24.690075+0000","last_active":"2024-09-10T11:38:26.801089+0000","last_peered":"2024-09-10T11:38:26.801089+0000","last_clean":"2024-09-10T11:38:26.801089+0000","last_became_active":"2024-09-10T11:38:24.688979+0000","last_became_peered":"2024-09-10T11:38:24.688979+0000","last_unstale":"2024-09-10T11:38:26.801089+0000","last_undegraded":"2024-09-10T11:38:26.801089+0000","last_fullsized":"2024-09-10T11:38:26.801089+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:19.862707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687621+0000","last_change":"2024-09-10T11:38:25.687527+0000","last_active":"2024-09-10T11:38:25.687621+0000","last_peered":"2024-09-10T11:38:25.687621+0000","last_clean":"2024-09-10T11:38:25.687621+0000","last_became_active":"2024-09-10T11:38:25.687289+0000","last_became_peered":"2024-09-10T11:38:25.687289+0000","last_unstale":"2024-09-10T11:38:25.687621+0000","last_undegraded":"2024-09-10T11:38:25.687621+0000","last_fullsized":"2024-09-10T11:38:25.687621+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:09:07.927208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801896+0000","last_change":"2024-09-10T11:38:24.683427+0000","last_active":"2024-09-10T11:38:26.801896+0000","last_peered":"2024-09-10T11:38:26.801896+0000","last_clean":"2024-09-10T11:38:26.801896+0000","last_became_active":"2024-09-10T11:38:24.683184+0000","last_became_peered":"2024-09-10T11:38:24.683184+0000","last_unstale":"2024-09-10T11:38:26.801896+0000","last_undegraded":"2024-09-10T11:38:26.801896+0000","last_fullsized":"2024-09-10T11:38:26.801896+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:06.011224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.685844+0000","last_change":"2024-09-10T11:38:25.685553+0000","last_active":"2024-09-10T11:38:25.685844+0000","last_peered":"2024-09-10T11:38:25.685844+0000","last_clean":"2024-09-10T11:38:25.685844+0000","last_became_active":"2024-09-10T11:38:25.685093+0000","last_became_peered":"2024-09-10T11:38:25.685093+0000","last_unstale":"2024-09-10T11:38:25.685844+0000","last_undegraded":"2024-09-10T11:38:25.685844+0000","last_fullsized":"2024-09-10T11:38:25.685844+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:11.686083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"166'8","reported_seq":22,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688043+0000","last_change":"2024-09-10T11:38:24.694111+0000","last_active":"2024-09-10T11:38:26.688043+0000","last_peered":"2024-09-10T11:38:26.688043+0000","last_clean":"2024-09-10T11:38:26.688043+0000","last_became_active":"2024-09-10T11:38:24.693889+0000","last_became_peered":"2024-09-10T11:38:24.693889+0000","last_unstale":"2024-09-10T11:38:26.688043+0000","last_undegraded":"2024-09-10T11:38:26.688043+0000","last_fullsized":"2024-09-10T11:38:26.688043+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:54.359892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685571+0000","last_change":"2024-09-10T11:38:25.690497+0000","last_active":"2024-09-10T11:38:26.685571+0000","last_peered":"2024-09-10T11:38:26.685571+0000","last_clean":"2024-09-10T11:38:26.685571+0000","last_became_active":"2024-09-10T11:38:25.690242+0000","last_became_peered":"2024-09-10T11:38:25.690242+0000","last_unstale":"2024-09-10T11:38:26.685571+0000","last_undegraded":"2024-09-10T11:38:26.685571+0000","last_fullsized":"2024-09-10T11:38:26.685571+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:41.853492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"166'1","reported_seq":13,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.700701+0000","last_change":"2024-09-10T11:38:24.692796+0000","last_active":"2024-09-10T11:38:25.700701+0000","last_peered":"2024-09-10T11:38:25.700701+0000","last_clean":"2024-09-10T11:38:25.700701+0000","last_became_active":"2024-09-10T11:38:24.692517+0000","last_became_peered":"2024-09-10T11:38:24.692517+0000","last_unstale":"2024-09-10T11:38:25.700701+0000","last_undegraded":"2024-09-10T11:38:25.700701+0000","last_fullsized":"2024-09-10T11:38:25.700701+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:49.316733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801268+0000","last_change":"2024-09-10T11:38:25.699407+0000","last_active":"2024-09-10T11:38:26.801268+0000","last_peered":"2024-09-10T11:38:26.801268+0000","last_clean":"2024-09-10T11:38:26.801268+0000","last_became_active":"2024-09-10T11:38:25.699187+0000","last_became_peered":"2024-09-10T11:38:25.699187+0000","last_unstale":"2024-09-10T11:38:26.801268+0000","last_undegraded":"2024-09-10T11:38:26.801268+0000","last_fullsized":"2024-09-10T11:38:26.801268+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:22:20.599279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"166'2","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688313+0000","last_change":"2024-09-10T11:38:24.694849+0000","last_active":"2024-09-10T11:38:26.688313+0000","last_peered":"2024-09-10T11:38:26.688313+0000","last_clean":"2024-09-10T11:38:26.688313+0000","last_became_active":"2024-09-10T11:38:24.694596+0000","last_became_peered":"2024-09-10T11:38:24.694596+0000","last_unstale":"2024-09-10T11:38:26.688313+0000","last_undegraded":"2024-09-10T11:38:26.688313+0000","last_fullsized":"2024-09-10T11:38:26.688313+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:15.050040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685153+0000","last_change":"2024-09-10T11:38:25.695182+0000","last_active":"2024-09-10T11:38:26.685153+0000","last_peered":"2024-09-10T11:38:26.685153+0000","last_clean":"2024-09-10T11:38:26.685153+0000","last_became_active":"2024-09-10T11:38:25.694950+0000","last_became_peered":"2024-09-10T11:38:25.694950+0000","last_unstale":"2024-09-10T11:38:26.685153+0000","last_undegraded":"2024-09-10T11:38:26.685153+0000","last_fullsized":"2024-09-10T11:38:26.685153+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:15.153296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"166'1","reported_seq":15,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801792+0000","last_change":"2024-09-10T11:38:24.693046+0000","last_active":"2024-09-10T11:38:26.801792+0000","last_peered":"2024-09-10T11:38:26.801792+0000","last_clean":"2024-09-10T11:38:26.801792+0000","last_became_active":"2024-09-10T11:38:24.692385+0000","last_became_peered":"2024-09-10T11:38:24.692385+0000","last_unstale":"2024-09-10T11:38:26.801792+0000","last_undegraded":"2024-09-10T11:38:26.801792+0000","last_fullsized":"2024-09-10T11:38:26.801792+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:59.501610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682662+0000","last_change":"2024-09-10T11:38:25.695556+0000","last_active":"2024-09-10T11:38:26.682662+0000","last_peered":"2024-09-10T11:38:26.682662+0000","last_clean":"2024-09-10T11:38:26.682662+0000","last_became_active":"2024-09-10T11:38:25.694492+0000","last_became_peered":"2024-09-10T11:38:25.694492+0000","last_unstale":"2024-09-10T11:38:26.682662+0000","last_undegraded":"2024-09-10T11:38:26.682662+0000","last_fullsized":"2024-09-10T11:38:26.682662+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:55.620050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801991+0000","last_change":"2024-09-10T11:38:24.689745+0000","last_active":"2024-09-10T11:38:26.801991+0000","last_peered":"2024-09-10T11:38:26.801991+0000","last_clean":"2024-09-10T11:38:26.801991+0000","last_became_active":"2024-09-10T11:38:24.689413+0000","last_became_peered":"2024-09-10T11:38:24.689413+0000","last_unstale":"2024-09-10T11:38:26.801991+0000","last_undegraded":"2024-09-10T11:38:26.801991+0000","last_fullsized":"2024-09-10T11:38:26.801991+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:18.510404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800750+0000","last_change":"2024-09-10T11:38:25.690712+0000","last_active":"2024-09-10T11:38:26.800750+0000","last_peered":"2024-09-10T11:38:26.800750+0000","last_clean":"2024-09-10T11:38:26.800750+0000","last_became_active":"2024-09-10T11:38:25.690484+0000","last_became_peered":"2024-09-10T11:38:25.690484+0000","last_unstale":"2024-09-10T11:38:26.800750+0000","last_undegraded":"2024-09-10T11:38:26.800750+0000","last_fullsized":"2024-09-10T11:38:26.800750+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:12:28.667879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"166'2","reported_seq":16,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682490+0000","last_change":"2024-09-10T11:38:24.687954+0000","last_active":"2024-09-10T11:38:26.682490+0000","last_peered":"2024-09-10T11:38:26.682490+0000","last_clean":"2024-09-10T11:38:26.682490+0000","last_became_active":"2024-09-10T11:38:24.687655+0000","last_became_peered":"2024-09-10T11:38:24.687655+0000","last_unstale":"2024-09-10T11:38:26.682490+0000","last_undegraded":"2024-09-10T11:38:26.682490+0000","last_fullsized":"2024-09-10T11:38:26.682490+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:53.253371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.802055+0000","last_change":"2024-09-10T11:38:25.701825+0000","last_active":"2024-09-10T11:38:26.802055+0000","last_peered":"2024-09-10T11:38:26.802055+0000","last_clean":"2024-09-10T11:38:26.802055+0000","last_became_active":"2024-09-10T11:38:25.701587+0000","last_became_peered":"2024-09-10T11:38:25.701587+0000","last_unstale":"2024-09-10T11:38:26.802055+0000","last_undegraded":"2024-09-10T11:38:26.802055+0000","last_fullsized":"2024-09-10T11:38:26.802055+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:01.882638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.683021+0000","last_change":"2024-09-10T11:38:24.691545+0000","last_active":"2024-09-10T11:38:25.683021+0000","last_peered":"2024-09-10T11:38:25.683021+0000","last_clean":"2024-09-10T11:38:25.683021+0000","last_became_active":"2024-09-10T11:38:24.691294+0000","last_became_peered":"2024-09-10T11:38:24.691294+0000","last_unstale":"2024-09-10T11:38:25.683021+0000","last_undegraded":"2024-09-10T11:38:25.683021+0000","last_fullsized":"2024-09-10T11:38:25.683021+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:45.392437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800834+0000","last_change":"2024-09-10T11:38:25.694263+0000","last_active":"2024-09-10T11:38:26.800834+0000","last_peered":"2024-09-10T11:38:26.800834+0000","last_clean":"2024-09-10T11:38:26.800834+0000","last_became_active":"2024-09-10T11:38:25.693991+0000","last_became_peered":"2024-09-10T11:38:25.693991+0000","last_unstale":"2024-09-10T11:38:26.800834+0000","last_undegraded":"2024-09-10T11:38:26.800834+0000","last_fullsized":"2024-09-10T11:38:26.800834+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:57.983405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.682246+0000","last_change":"2024-09-10T11:38:24.690459+0000","last_active":"2024-09-10T11:38:25.682246+0000","last_peered":"2024-09-10T11:38:25.682246+0000","last_clean":"2024-09-10T11:38:25.682246+0000","last_became_active":"2024-09-10T11:38:24.689935+0000","last_became_peered":"2024-09-10T11:38:24.689935+0000","last_unstale":"2024-09-10T11:38:25.682246+0000","last_undegraded":"2024-09-10T11:38:25.682246+0000","last_fullsized":"2024-09-10T11:38:25.682246+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:07.760102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801949+0000","last_change":"2024-09-10T11:38:25.691676+0000","last_active":"2024-09-10T11:38:26.801949+0000","last_peered":"2024-09-10T11:38:26.801949+0000","last_clean":"2024-09-10T11:38:26.801949+0000","last_became_active":"2024-09-10T11:38:25.691385+0000","last_became_peered":"2024-09-10T11:38:25.691385+0000","last_unstale":"2024-09-10T11:38:26.801949+0000","last_undegraded":"2024-09-10T11:38:26.801949+0000","last_fullsized":"2024-09-10T11:38:26.801949+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:07:59.549776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.682584+0000","last_change":"2024-09-10T11:38:24.691679+0000","last_active":"2024-09-10T11:38:25.682584+0000","last_peered":"2024-09-10T11:38:25.682584+0000","last_clean":"2024-09-10T11:38:25.682584+0000","last_became_active":"2024-09-10T11:38:24.691418+0000","last_became_peered":"2024-09-10T11:38:24.691418+0000","last_unstale":"2024-09-10T11:38:25.682584+0000","last_undegraded":"2024-09-10T11:38:25.682584+0000","last_fullsized":"2024-09-10T11:38:25.682584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:12.335470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.689019+0000","last_change":"2024-09-10T11:38:25.688714+0000","last_active":"2024-09-10T11:38:25.689019+0000","last_peered":"2024-09-10T11:38:25.689019+0000","last_clean":"2024-09-10T11:38:25.689019+0000","last_became_active":"2024-09-10T11:38:25.688510+0000","last_became_peered":"2024-09-10T11:38:25.688510+0000","last_unstale":"2024-09-10T11:38:25.689019+0000","last_undegraded":"2024-09-10T11:38:25.689019+0000","last_fullsized":"2024-09-10T11:38:25.689019+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:13.052391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"166'2","reported_seq":14,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.702304+0000","last_change":"2024-09-10T11:38:24.688503+0000","last_active":"2024-09-10T11:38:25.702304+0000","last_peered":"2024-09-10T11:38:25.702304+0000","last_clean":"2024-09-10T11:38:25.702304+0000","last_became_active":"2024-09-10T11:38:24.688230+0000","last_became_peered":"2024-09-10T11:38:24.688230+0000","last_unstale":"2024-09-10T11:38:25.702304+0000","last_undegraded":"2024-09-10T11:38:25.702304+0000","last_fullsized":"2024-09-10T11:38:25.702304+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:33.189562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.689037+0000","last_change":"2024-09-10T11:38:25.688940+0000","last_active":"2024-09-10T11:38:25.689037+0000","last_peered":"2024-09-10T11:38:25.689037+0000","last_clean":"2024-09-10T11:38:25.689037+0000","last_became_active":"2024-09-10T11:38:25.688428+0000","last_became_peered":"2024-09-10T11:38:25.688428+0000","last_unstale":"2024-09-10T11:38:25.689037+0000","last_undegraded":"2024-09-10T11:38:25.689037+0000","last_fullsized":"2024-09-10T11:38:25.689037+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:46.917036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.802149+0000","last_change":"2024-09-10T11:38:24.689759+0000","last_active":"2024-09-10T11:38:26.802149+0000","last_peered":"2024-09-10T11:38:26.802149+0000","last_clean":"2024-09-10T11:38:26.802149+0000","last_became_active":"2024-09-10T11:38:24.689481+0000","last_became_peered":"2024-09-10T11:38:24.689481+0000","last_unstale":"2024-09-10T11:38:26.802149+0000","last_undegraded":"2024-09-10T11:38:26.802149+0000","last_fullsized":"2024-09-10T11:38:26.802149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:55.800305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688151+0000","last_change":"2024-09-10T11:38:25.688761+0000","last_active":"2024-09-10T11:38:26.688151+0000","last_peered":"2024-09-10T11:38:26.688151+0000","last_clean":"2024-09-10T11:38:26.688151+0000","last_became_active":"2024-09-10T11:38:25.688515+0000","last_became_peered":"2024-09-10T11:38:25.688515+0000","last_unstale":"2024-09-10T11:38:26.688151+0000","last_undegraded":"2024-09-10T11:38:26.688151+0000","last_fullsized":"2024-09-10T11:38:26.688151+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:04.341101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801238+0000","last_change":"2024-09-10T11:38:24.692128+0000","last_active":"2024-09-10T11:38:26.801238+0000","last_peered":"2024-09-10T11:38:26.801238+0000","last_clean":"2024-09-10T11:38:26.801238+0000","last_became_active":"2024-09-10T11:38:24.691647+0000","last_became_peered":"2024-09-10T11:38:24.691647+0000","last_unstale":"2024-09-10T11:38:26.801238+0000","last_undegraded":"2024-09-10T11:38:26.801238+0000","last_fullsized":"2024-09-10T11:38:26.801238+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:28.516731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.688359+0000","last_change":"2024-09-10T11:38:25.685377+0000","last_active":"2024-09-10T11:38:26.688359+0000","last_peered":"2024-09-10T11:38:26.688359+0000","last_clean":"2024-09-10T11:38:26.688359+0000","last_became_active":"2024-09-10T11:38:25.685127+0000","last_became_peered":"2024-09-10T11:38:25.685127+0000","last_unstale":"2024-09-10T11:38:26.688359+0000","last_undegraded":"2024-09-10T11:38:26.688359+0000","last_fullsized":"2024-09-10T11:38:26.688359+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:14.626142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801403+0000","last_change":"2024-09-10T11:38:24.690814+0000","last_active":"2024-09-10T11:38:26.801403+0000","last_peered":"2024-09-10T11:38:26.801403+0000","last_clean":"2024-09-10T11:38:26.801403+0000","last_became_active":"2024-09-10T11:38:24.690564+0000","last_became_peered":"2024-09-10T11:38:24.690564+0000","last_unstale":"2024-09-10T11:38:26.801403+0000","last_undegraded":"2024-09-10T11:38:26.801403+0000","last_fullsized":"2024-09-10T11:38:26.801403+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:19:21.576433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801850+0000","last_change":"2024-09-10T11:38:25.696483+0000","last_active":"2024-09-10T11:38:26.801850+0000","last_peered":"2024-09-10T11:38:26.801850+0000","last_clean":"2024-09-10T11:38:26.801850+0000","last_became_active":"2024-09-10T11:38:25.696320+0000","last_became_peered":"2024-09-10T11:38:25.696320+0000","last_unstale":"2024-09-10T11:38:26.801850+0000","last_undegraded":"2024-09-10T11:38:26.801850+0000","last_fullsized":"2024-09-10T11:38:26.801850+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:24.144036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801512+0000","last_change":"2024-09-10T11:38:24.685092+0000","last_active":"2024-09-10T11:38:26.801512+0000","last_peered":"2024-09-10T11:38:26.801512+0000","last_clean":"2024-09-10T11:38:26.801512+0000","last_became_active":"2024-09-10T11:38:24.684859+0000","last_became_peered":"2024-09-10T11:38:24.684859+0000","last_unstale":"2024-09-10T11:38:26.801512+0000","last_undegraded":"2024-09-10T11:38:26.801512+0000","last_fullsized":"2024-09-10T11:38:26.801512+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:15.638958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.690450+0000","last_change":"2024-09-10T11:38:25.690186+0000","last_active":"2024-09-10T11:38:25.690450+0000","last_peered":"2024-09-10T11:38:25.690450+0000","last_clean":"2024-09-10T11:38:25.690450+0000","last_became_active":"2024-09-10T11:38:25.689955+0000","last_became_peered":"2024-09-10T11:38:25.689955+0000","last_unstale":"2024-09-10T11:38:25.690450+0000","last_undegraded":"2024-09-10T11:38:25.690450+0000","last_fullsized":"2024-09-10T11:38:25.690450+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:26.863667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.687368+0000","last_change":"2024-09-10T11:38:24.692994+0000","last_active":"2024-09-10T11:38:25.687368+0000","last_peered":"2024-09-10T11:38:25.687368+0000","last_clean":"2024-09-10T11:38:25.687368+0000","last_became_active":"2024-09-10T11:38:24.692755+0000","last_became_peered":"2024-09-10T11:38:24.692755+0000","last_unstale":"2024-09-10T11:38:25.687368+0000","last_undegraded":"2024-09-10T11:38:25.687368+0000","last_fullsized":"2024-09-10T11:38:25.687368+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:21.226346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.686776+0000","last_change":"2024-09-10T11:38:25.686606+0000","last_active":"2024-09-10T11:38:25.686776+0000","last_peered":"2024-09-10T11:38:25.686776+0000","last_clean":"2024-09-10T11:38:25.686776+0000","last_became_active":"2024-09-10T11:38:25.685917+0000","last_became_peered":"2024-09-10T11:38:25.685917+0000","last_unstale":"2024-09-10T11:38:25.686776+0000","last_undegraded":"2024-09-10T11:38:25.686776+0000","last_fullsized":"2024-09-10T11:38:25.686776+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.244404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800740+0000","last_change":"2024-09-10T11:38:24.687939+0000","last_active":"2024-09-10T11:38:26.800740+0000","last_peered":"2024-09-10T11:38:26.800740+0000","last_clean":"2024-09-10T11:38:26.800740+0000","last_became_active":"2024-09-10T11:38:24.687712+0000","last_became_peered":"2024-09-10T11:38:24.687712+0000","last_unstale":"2024-09-10T11:38:26.800740+0000","last_undegraded":"2024-09-10T11:38:26.800740+0000","last_fullsized":"2024-09-10T11:38:26.800740+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:58.607827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-10T11:38:25.697972+0000","last_change":"2024-09-10T11:38:25.697813+0000","last_active":"2024-09-10T11:38:25.697972+0000","last_peered":"2024-09-10T11:38:25.697972+0000","last_clean":"2024-09-10T11:38:25.697972+0000","last_became_active":"2024-09-10T11:38:25.697592+0000","last_became_peered":"2024-09-10T11:38:25.697592+0000","last_unstale":"2024-09-10T11:38:25.697972+0000","last_undegraded":"2024-09-10T11:38:25.697972+0000","last_fullsized":"2024-09-10T11:38:25.697972+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:52.687300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.682665+0000","last_change":"2024-09-10T11:38:24.686178+0000","last_active":"2024-09-10T11:38:26.682665+0000","last_peered":"2024-09-10T11:38:26.682665+0000","last_clean":"2024-09-10T11:38:26.682665+0000","last_became_active":"2024-09-10T11:38:24.685930+0000","last_became_peered":"2024-09-10T11:38:24.685930+0000","last_unstale":"2024-09-10T11:38:26.682665+0000","last_undegraded":"2024-09-10T11:38:26.682665+0000","last_fullsized":"2024-09-10T11:38:26.682665+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:23.512009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.801401+0000","last_change":"2024-09-10T11:38:25.689762+0000","last_active":"2024-09-10T11:38:26.801401+0000","last_peered":"2024-09-10T11:38:26.801401+0000","last_clean":"2024-09-10T11:38:26.801401+0000","last_became_active":"2024-09-10T11:38:25.689521+0000","last_became_peered":"2024-09-10T11:38:25.689521+0000","last_unstale":"2024-09-10T11:38:26.801401+0000","last_undegraded":"2024-09-10T11:38:26.801401+0000","last_fullsized":"2024-09-10T11:38:26.801401+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:24.669528+0000","last_clean_scrub_stamp":"2024-09-10T11:38:24.669528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:56.040467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"166'3","reported_seq":17,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.800648+0000","last_change":"2024-09-10T11:38:24.687015+0000","last_active":"2024-09-10T11:38:26.800648+0000","last_peered":"2024-09-10T11:38:26.800648+0000","last_clean":"2024-09-10T11:38:26.800648+0000","last_became_active":"2024-09-10T11:38:24.686783+0000","last_became_peered":"2024-09-10T11:38:24.686783+0000","last_unstale":"2024-09-10T11:38:26.800648+0000","last_undegraded":"2024-09-10T11:38:26.800648+0000","last_fullsized":"2024-09-10T11:38:26.800648+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:38:23.664640+0000","last_clean_scrub_stamp":"2024-09-10T11:38:23.664640+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:31.771346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":466,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-10T11:38:26.685283+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:38:26.685283+0000","last_peered":"2024-09-10T11:38:26.685283+0000","last_clean":"2024-09-10T11:38:26.685283+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:38:26.685283+0000","last_undegraded":"2024-09-10T11:38:26.685283+0000","last_fullsized":"2024-09-10T11:38:26.685283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073389,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073389,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574949,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1483,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193829,"statfs":{"total":200000000,"available":198481215,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574948,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":907,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194405,"statfs":{"total":200000000,"available":199071173,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574949,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":905,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199072773,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574948,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073367,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1487,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193824,"statfs":{"total":200000000,"available":198476651,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":904,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194407,"statfs":{"total":200000000,"available":199073299,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:38:28.966 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:38:28.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:38:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.106+0000 7f334b92a640 1 Processor -- start 2024-09-10T11:38:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.106+0000 7f334b92a640 1 -- start start 2024-09-10T11:38:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f334415eb80 0x7f33440d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33441626c0 0x7f33440d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33440d4e40 0x7f33440d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f33440d52d0 con 0x7f334415eb80 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f33440d1620 con 0x7f33441626c0 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f334b92a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3344162bc0 con 0x7f33440d4e40 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f33496c6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f334415eb80 0x7f33440d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f33496c6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f334415eb80 0x7f33440d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36320/0 (socket says 172.21.15.99:36320) 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.107+0000 7f33496c6640 1 -- 172.21.15.99:0/1504347137 learned_addr learned my addr 172.21.15.99:0/1504347137 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f3349ec7640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33440d4e40 0x7f33440d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f3348ec5640 1 --2- 172.21.15.99:0/1504347137 >> v2:172.21.15.196:3300/0 conn(0x7f33441626c0 0x7f33440d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 -- 172.21.15.99:0/1504347137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33440d4e40 msgr2=0x7f33440d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33440d4e40 0x7f33440d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 -- 172.21.15.99:0/1504347137 >> v2:172.21.15.196:3300/0 conn(0x7f33441626c0 msgr2=0x7f33440d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 --2- 172.21.15.99:0/1504347137 >> v2:172.21.15.196:3300/0 conn(0x7f33441626c0 0x7f33440d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 -- 172.21.15.99:0/1504347137 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3344162ea0 con 0x7f334415eb80 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.108+0000 7f33496c6640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334415eb80 0x7f33440d29b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3334009e50 tx=0x7f333402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e19fe8d60cc752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f3343fff640 1 -- 172.21.15.99:0/1504347137 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334036070 con 0x7f334415eb80 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f3343fff640 1 -- 172.21.15.99:0/1504347137 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3334031600 con 0x7f334415eb80 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f3343fff640 1 -- 172.21.15.99:0/1504347137 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340319d0 con 0x7f334415eb80 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 -- 172.21.15.99:0/1504347137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334415eb80 msgr2=0x7f33440d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334415eb80 0x7f33440d29b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3334009e50 tx=0x7f333402a630 comp rx=0 tx=0).stop 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 -- 172.21.15.99:0/1504347137 shutdown_connections 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33440d4e40 0x7f33440d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 --2- 172.21.15.99:0/1504347137 >> v2:172.21.15.196:3300/0 conn(0x7f33441626c0 0x7f33440d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 --2- 172.21.15.99:0/1504347137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f334415eb80 0x7f33440d29b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 -- 172.21.15.99:0/1504347137 >> 172.21.15.99:0/1504347137 conn(0x7f334408e680 msgr2=0x7f334409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 -- 172.21.15.99:0/1504347137 shutdown_connections 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.109+0000 7f334b92a640 1 -- 172.21.15.99:0/1504347137 wait complete. 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.110+0000 7f334b92a640 1 Processor -- start 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.110+0000 7f334b92a640 1 -- start start 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f334415eb80 0x7f334415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 0x7f33440cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33440d2ef0 con 0x7f33440d4e40 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3344163460 con 0x7f33441626c0 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f334b92a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f33440d3070 con 0x7f334415eb80 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36322/0 (socket says 172.21.15.99:36322) 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 -- 172.21.15.99:0/3510813970 learned_addr learned my addr 172.21.15.99:0/3510813970 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 -- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f334415eb80 msgr2=0x7f334415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f334415eb80 0x7f334415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.111+0000 7f33496c6640 1 -- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 msgr2=0x7f33440cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f3349ec7640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 0x7f33440cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f33496c6640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 0x7f33440cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f33496c6640 1 -- 172.21.15.99:0/3510813970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3344161020 con 0x7f33440d4e40 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f3349ec7640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 0x7f33440cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f33496c6640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3334000f80 tx=0x7f3334004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334036050 con 0x7f33440d4e40 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3334042070 con 0x7f33440d4e40 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3344161300 con 0x7f33440d4e40 2024-09-10T11:38:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334009510 con 0x7f33440d4e40 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.112+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33441618c0 con 0x7f33440d4e40 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.113+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33440d29b0 con 0x7f33440d4e40 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.113+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3334007c70 con 0x7f33440d4e40 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.114+0000 7f3341ffb640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 0x7f333006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.114+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f33340abd30 con 0x7f33440d4e40 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.114+0000 7f3348ec5640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 0x7f333006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.114+0000 7f3348ec5640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 0x7f333006e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33440ce7d0 tx=0x7f3338008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.118+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f333407a770 con 0x7f33440d4e40 2024-09-10T11:38:29.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.348+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f33440cbad0 con 0x7f33440d4e40 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.350+0000 7f3341ffb640 1 -- 172.21.15.99:0/3510813970 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f333407a070 con 0x7f33440d4e40 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:38:29.352 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 msgr2=0x7f333006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 0x7f333006e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33440ce7d0 tx=0x7f3338008040 comp rx=0 tx=0).stop 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 msgr2=0x7f33441620c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3334000f80 tx=0x7f3334004a40 comp rx=0 tx=0).stop 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 shutdown_connections 2024-09-10T11:38:29.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f333006be60 0x7f333006e320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33441626c0 0x7f33440cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f334415eb80 0x7f334415a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 --2- 172.21.15.99:0/3510813970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33440d4e40 0x7f33441620c0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 >> 172.21.15.99:0/3510813970 conn(0x7f334408e680 msgr2=0x7f33440d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 shutdown_connections 2024-09-10T11:38:29.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.352+0000 7f334b92a640 1 -- 172.21.15.99:0/3510813970 wait complete. 2024-09-10T11:38:29.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:29.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 Processor -- start 2024-09-10T11:38:29.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 -- start start 2024-09-10T11:38:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6101626c0 0x7fc6100d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61015ff70 con 0x7fc6100d4e40 2024-09-10T11:38:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6100d1620 con 0x7fc61015eb80 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.463+0000 7fc616215640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc6100d30c0 con 0x7fc6101626c0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6101626c0 0x7fc6100d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6101626c0 0x7fc6100d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36330/0 (socket says 172.21.15.99:36330) 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60ffff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc6100d4e40 0x7fc6100d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39074/0 (socket says 172.21.15.99:39074) 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 -- 172.21.15.99:0/3129944791 learned_addr learned my addr 172.21.15.99:0/3129944791 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 -- 172.21.15.99:0/3129944791 >> v1:172.21.15.196:6789/0 conn(0x7fc61015eb80 legacy=0x7fc6100d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 -- 172.21.15.99:0/3129944791 >> v1:172.21.15.99:6789/0 conn(0x7fc6100d4e40 legacy=0x7fc6100d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60effd640 1 -- 172.21.15.99:0/3129944791 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc610162bc0 con 0x7fc6101626c0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.464+0000 7fc60e7fc640 1 -- 172.21.15.99:0/3129944791 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248128615 0 0) 0x7fc61015ff70 con 0x7fc6100d4e40 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc60effd640 1 --2- 172.21.15.99:0/3129944791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6101626c0 0x7fc6100d4900 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc60402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07f1de5910dd712 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc60e7fc640 1 -- 172.21.15.99:0/3129944791 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604037070 con 0x7fc6101626c0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc60e7fc640 1 -- 172.21.15.99:0/3129944791 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604032440 con 0x7fc6101626c0 2024-09-10T11:38:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc60e7fc640 1 -- 172.21.15.99:0/3129944791 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604032810 con 0x7fc6101626c0 2024-09-10T11:38:29.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 -- 172.21.15.99:0/3129944791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6101626c0 msgr2=0x7fc6100d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 --2- 172.21.15.99:0/3129944791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6101626c0 0x7fc6100d4900 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098a0 tx=0x7fc60402a290 comp rx=0 tx=0).stop 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 -- 172.21.15.99:0/3129944791 shutdown_connections 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 --2- 172.21.15.99:0/3129944791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6101626c0 0x7fc6100d4900 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 -- 172.21.15.99:0/3129944791 >> 172.21.15.99:0/3129944791 conn(0x7fc61008e680 msgr2=0x7fc61009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 -- 172.21.15.99:0/3129944791 shutdown_connections 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.465+0000 7fc616215640 1 -- 172.21.15.99:0/3129944791 wait complete. 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.466+0000 7fc616215640 1 Processor -- start 2024-09-10T11:38:29.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 -- start start 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc61015eb80 0x7fc61014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6101626c0 0x7fc6100cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc610163460 con 0x7fc6100d4e40 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc610163180 con 0x7fc6101626c0 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc616215640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc6100d5290 con 0x7fc61015eb80 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc60f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.467+0000 7fc60f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36336/0 (socket says 172.21.15.99:36336) 2024-09-10T11:38:29.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 -- 172.21.15.99:0/1980282949 learned_addr learned my addr 172.21.15.99:0/1980282949 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 -- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc61015eb80 msgr2=0x7fc61014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc61015eb80 0x7fc61014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 -- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6101626c0 msgr2=0x7fc6100cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6101626c0 0x7fc6100cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 -- 172.21.15.99:0/1980282949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6100b8030 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc60f7fe640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c390 tx=0x7fc60000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600017070 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6000052d0 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc600003830 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6100b1e40 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.468+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6100b2400 con 0x7fc6100d4e40 2024-09-10T11:38:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.469+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6100d4900 con 0x7fc6100d4e40 2024-09-10T11:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.470+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc60001a070 con 0x7fc6100d4e40 2024-09-10T11:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.470+0000 7fc5f7fff640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.470+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7fc60008a1f0 con 0x7fc6100d4e40 2024-09-10T11:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.470+0000 7fc60effd640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.471+0000 7fc60effd640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.474+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc600058ba0 con 0x7fc6100d4e40 2024-09-10T11:38:29.620 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:38:29.623 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:38:29.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.679+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6100d10f0 con 0x7fc6100d4e40 2024-09-10T11:38:29.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.679+0000 7fc5f7fff640 1 -- 172.21.15.99:0/1980282949 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7fc6000584a0 con 0x7fc6100d4e40 2024-09-10T11:38:29.681 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:29.681 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:38:29.682 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:38:29.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 msgr2=0x7fc5fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc604034000 tx=0x7fc60402c000 comp rx=0 tx=0).stop 2024-09-10T11:38:29.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 msgr2=0x7fc6100b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fc60000c390 tx=0x7fc60000c850 comp rx=0 tx=0).stop 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 shutdown_connections 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6101626c0 0x7fc6100cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc61015eb80 0x7fc61014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 --2- 172.21.15.99:0/1980282949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6100d4e40 0x7fc6100b7af0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.681+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 >> 172.21.15.99:0/1980282949 conn(0x7fc61008e680 msgr2=0x7fc6100cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.682+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 shutdown_connections 2024-09-10T11:38:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:29.682+0000 7fc616215640 1 -- 172.21.15.99:0/1980282949 wait complete. 2024-09-10T11:38:30.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:38:25.020679+0000', 'modified': '2024-09-10T11:38:27.019950+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7635}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7635': {'gid': 7635, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6835/3322393426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 3322393426}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 3322393426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7635, 'qdb_cluster': [7635]} 2024-09-10T11:38:30.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:38:30.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:38:30.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.200+0000 7effc3a80640 1 Processor -- start 2024-09-10T11:38:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.200+0000 7effc3a80640 1 -- start start 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc3a80640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effbc1610b0 0x7effbc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc3a80640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7effbc0ae020 0x7effbc15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc3a80640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effbc0d1bd0 con 0x7effbc1605c0 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc3a80640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7effbc075620 con 0x7effbc1610b0 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc3a80640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7effbc0752b0 con 0x7effbc0ae020 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effbc1610b0 0x7effbc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7effbc1610b0 0x7effbc15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36350/0 (socket says 172.21.15.99:36350) 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 -- 172.21.15.99:0/3635961494 learned_addr learned my addr 172.21.15.99:0/3635961494 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc181c640 1 --1- 172.21.15.99:0/3635961494 >> v1:172.21.15.99:6789/0 conn(0x7effbc1605c0 0x7effbc1609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39080/0 (socket says 172.21.15.99:39080) 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc201d640 1 --2- 172.21.15.99:0/3635961494 >> v2:172.21.15.196:3300/0 conn(0x7effbc0ae020 0x7effbc15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 -- 172.21.15.99:0/3635961494 >> v2:172.21.15.196:3300/0 conn(0x7effbc0ae020 msgr2=0x7effbc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 --2- 172.21.15.99:0/3635961494 >> v2:172.21.15.196:3300/0 conn(0x7effbc0ae020 0x7effbc15e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 -- 172.21.15.99:0/3635961494 >> v1:172.21.15.99:6789/0 conn(0x7effbc1605c0 legacy=0x7effbc1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc101b640 1 -- 172.21.15.99:0/3635961494 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc0d1eb0 con 0x7effbc1610b0 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.201+0000 7effc081a640 1 -- 172.21.15.99:0/3635961494 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3791791126 0 0) 0x7effbc0d1bd0 con 0x7effbc1605c0 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc101b640 1 --2- 172.21.15.99:0/3635961494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1610b0 0x7effbc15e200 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7effbc0709d0 tx=0x7effac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8f07c61c53e763 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc081a640 1 -- 172.21.15.99:0/3635961494 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac037070 con 0x7effbc1610b0 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc081a640 1 -- 172.21.15.99:0/3635961494 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac032440 con 0x7effbc1610b0 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc081a640 1 -- 172.21.15.99:0/3635961494 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac032810 con 0x7effbc1610b0 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc3a80640 1 -- 172.21.15.99:0/3635961494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1610b0 msgr2=0x7effbc15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.202+0000 7effc3a80640 1 --2- 172.21.15.99:0/3635961494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1610b0 0x7effbc15e200 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7effbc0709d0 tx=0x7effac02a270 comp rx=0 tx=0).stop 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 -- 172.21.15.99:0/3635961494 shutdown_connections 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 --2- 172.21.15.99:0/3635961494 >> v2:172.21.15.196:3300/0 conn(0x7effbc0ae020 0x7effbc15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 --2- 172.21.15.99:0/3635961494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1610b0 0x7effbc15e200 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 -- 172.21.15.99:0/3635961494 >> 172.21.15.99:0/3635961494 conn(0x7effbc0730c0 msgr2=0x7effbc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 -- 172.21.15.99:0/3635961494 shutdown_connections 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.203+0000 7effc3a80640 1 -- 172.21.15.99:0/3635961494 wait complete. 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.204+0000 7effc3a80640 1 Processor -- start 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.204+0000 7effc3a80640 1 -- start start 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.204+0000 7effc3a80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 0x7effbc1e67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.204+0000 7effc3a80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.204+0000 7effc3a80640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effbc1610b0 0x7effbc0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc3a80640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effbc073d00 con 0x7effbc1605c0 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc3a80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effbc15a9e0 con 0x7effbc0ae020 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc3a80640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7effbc073e80 con 0x7effbc1610b0 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36362/0 (socket says 172.21.15.99:36362) 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 -- 172.21.15.99:0/252891538 learned_addr learned my addr 172.21.15.99:0/252891538 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc181c640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 0x7effbc1e67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 -- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effbc1610b0 msgr2=0x7effbc0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effbc1610b0 0x7effbc0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 -- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 msgr2=0x7effbc1e67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 0x7effbc1e67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 -- 172.21.15.99:0/252891538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effbc1f8e50 con 0x7effbc1605c0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc181c640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 0x7effbc1e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc101b640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7effac0090b0 tx=0x7effac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effb27fc640 1 -- 172.21.15.99:0/252891538 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac037040 con 0x7effbc1605c0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effb27fc640 1 -- 172.21.15.99:0/252891538 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac004030 con 0x7effbc1605c0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effbc1e7000 con 0x7effbc1605c0 2024-09-10T11:38:30.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effb27fc640 1 -- 172.21.15.99:0/252891538 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac004420 con 0x7effbc1605c0 2024-09-10T11:38:30.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.205+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effbc14f070 con 0x7effbc1605c0 2024-09-10T11:38:30.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.207+0000 7effb27fc640 1 -- 172.21.15.99:0/252891538 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effac03a040 con 0x7effbc1605c0 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.207+0000 7effb27fc640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.207+0000 7effb27fc640 1 -- 172.21.15.99:0/252891538 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7effac0ab640 con 0x7effbc1605c0 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.207+0000 7effc181c640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.208+0000 7effc181c640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 0x7effa006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7effb8000f60 tx=0x7effb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.208+0000 7effc6814740 1 Processor -- start 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.208+0000 7effc6814740 1 -- start start 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fd37a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fcec8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b14fc54520 con 0x55b14fcefa80 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b14fbfc3e0 con 0x55b14fcf22c0 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b14fc504e0 con 0x55b14fcf3a50 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36374/0 (socket says 172.21.15.99:36374) 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 -- 172.21.15.99:0/1192025947 learned_addr learned my addr 172.21.15.99:0/1192025947 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc181c640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fd37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc201d640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fcec8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 -- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 msgr2=0x55b14fcec8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fcec8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 -- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 msgr2=0x55b14fd37a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fd37a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc101b640 1 -- 172.21.15.99:0/1192025947 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b14fc172f0 con 0x55b14fcefa80 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.209+0000 7effc201d640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fcec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc101b640 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7effac0a9ae0 tx=0x7effac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79d0c55d0bb5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7eff9f7fe640 1 -- 172.21.15.99:0/1192025947 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac04b030 con 0x55b14fcefa80 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7eff9f7fe640 1 -- 172.21.15.99:0/1192025947 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac079a10 con 0x55b14fcefa80 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7eff9f7fe640 1 -- 172.21.15.99:0/1192025947 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac07ddc0 con 0x55b14fcefa80 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 -- 172.21.15.99:0/1192025947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 msgr2=0x55b14fcf26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7effac0a9ae0 tx=0x7effac009040 comp rx=0 tx=0).stop 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 -- 172.21.15.99:0/1192025947 shutdown_connections 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fcec8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcefa80 0x55b14fcf26a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 --2- 172.21.15.99:0/1192025947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fd37a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 -- 172.21.15.99:0/1192025947 >> 172.21.15.99:0/1192025947 conn(0x55b14fceea80 msgr2=0x55b14fd36f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 -- 172.21.15.99:0/1192025947 shutdown_connections 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.210+0000 7effc6814740 1 -- 172.21.15.99:0/1192025947 wait complete. 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.211+0000 7effc6814740 1 -- start start 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.211+0000 7effc6814740 1 Processor -- start 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf22c0 0x55b14fd482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf3a50 0x55b14fd4b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b14fc54520 con 0x55b14fd47c00 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b14fbdd450 con 0x55b14fcf3a50 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b14fcedf20 con 0x55b14fcf22c0 2024-09-10T11:38:30.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc6814740 10 client.0 _ll_register_callbacks cb 0x55b14f966ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36386/0 (socket says 172.21.15.99:36386) 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 -- 172.21.15.99:0/3084035755 learned_addr learned my addr 172.21.15.99:0/3084035755 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc101b640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf3a50 0x55b14fd4b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc181c640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf22c0 0x55b14fd482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 -- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf22c0 msgr2=0x55b14fd482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf22c0 0x55b14fd482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 -- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf3a50 msgr2=0x55b14fd4b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf3a50 0x55b14fd4b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.212+0000 7effc201d640 1 -- 172.21.15.99:0/3084035755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b14f9672d0 con 0x55b14fd47c00 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc201d640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7effb4007a00 tx=0x7effb4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36e52fc525df7bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4010a10 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7effb4010bb0 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effac079a10 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7effb4023050 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 10 client.7767 fetch_fsmap learned FSMap version 67 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b14fa316b0 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac079a10 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7effb4026020 con 0x55b14fd47c00 2024-09-10T11:38:30.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 10 client.7767 fetch_fsmap finished waiting for FSMap version 67 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 10 client.7767 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 20 client.7767 populate_metadata read hostname 'smithi099' 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fd33c70 0x55b14fd34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 4 client.7767 mds_command: new command op to 7635 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 --> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b14fc61c50 con 0x55b14fd33c70 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.213+0000 7effc181c640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fd33c70 0x55b14fd34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7eff9cff9640 10 client.7767 ms_handle_connect on v2:172.21.15.99:6834/3322393426 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc181c640 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fd33c70 0x55b14fd34050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba9a132cc860fc5 server_cookie=c0bdb6e0cb705ce in_seq=0 out_seq=0 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7eff9cff9640 1 -- 172.21.15.99:0/3084035755 <== mds.0 v2:172.21.15.99:6834/3322393426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b14fc61c50 con 0x55b14fd33c70 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7eff9cff9640 10 client.7767 handle_command_reply: tid=0 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 client.7767 shutdown 2024-09-10T11:38:30.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 msgr2=0x55b14fceca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7effb4007a00 tx=0x7effb4007e00 comp rx=0 tx=0).stop 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 shutdown_connections 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fd47c00 0x55b14fceca10 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf3a50 0x55b14fd4b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.214+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fd33c70 0x55b14fd34050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7effc6814740 1 --2- 172.21.15.99:0/3084035755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf22c0 0x55b14fd482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7eff9cff9640 0 client.7767 ms_handle_reset on v2:172.21.15.99:6834/3322393426 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 >> 172.21.15.99:0/3084035755 conn(0x55b14fcefa80 msgr2=0x55b14fd34fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 shutdown_connections 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7effc6814740 1 -- 172.21.15.99:0/3084035755 wait complete. 2024-09-10T11:38:30.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.215+0000 7effc6814740 20 client.7767 trim_cache size 0 max 16384 2024-09-10T11:38:30.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.232+0000 7effc6814740 1 Processor -- start 2024-09-10T11:38:30.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.232+0000 7effc6814740 1 -- start start 2024-09-10T11:38:30.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fd5b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fd47c00 0x55b14fd34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0cc610 con 0x55b14fcf22c0 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7effbc0cb970 con 0x55b14fd47c00 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7effbc0cbfc0 con 0x55b14fcf3a50 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36398/0 (socket says 172.21.15.99:36398) 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 -- 172.21.15.99:0/2208773483 learned_addr learned my addr 172.21.15.99:0/2208773483 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc101b640 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fd5b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc201d640 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fd47c00 0x55b14fd34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 -- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 msgr2=0x55b14fd5b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fd5b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 -- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fd47c00 msgr2=0x55b14fd34fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fd47c00 0x55b14fd34fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.233+0000 7effc181c640 1 -- 172.21.15.99:0/2208773483 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b14fcf2a10 con 0x55b14fcf22c0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc181c640 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x55b14fd48a30 tx=0x7effb8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd1b8891c0e74e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7eff9effd640 1 -- 172.21.15.99:0/2208773483 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8005970 con 0x55b14fcf22c0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7eff9effd640 1 -- 172.21.15.99:0/2208773483 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb8005b10 con 0x55b14fcf22c0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7eff9effd640 1 -- 172.21.15.99:0/2208773483 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb800d7d0 con 0x55b14fcf22c0 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 -- 172.21.15.99:0/2208773483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 msgr2=0x55b14fd3a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x55b14fd48a30 tx=0x7effb8000ed0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 -- 172.21.15.99:0/2208773483 shutdown_connections 2024-09-10T11:38:30.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fd47c00 0x55b14fd34fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fcf3a50 0x55b14fd5b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 --2- 172.21.15.99:0/2208773483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf22c0 0x55b14fd3a160 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 -- 172.21.15.99:0/2208773483 >> 172.21.15.99:0/2208773483 conn(0x55b14fcefa80 msgr2=0x55b14fd472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 -- 172.21.15.99:0/2208773483 shutdown_connections 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.234+0000 7effc6814740 1 -- 172.21.15.99:0/2208773483 wait complete. 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.235+0000 7effc6814740 1 -- start start 2024-09-10T11:38:30.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.235+0000 7effc6814740 1 Processor -- start 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fdb4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fd47c00 0x55b14fd457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b14fcecf50 con 0x55b14fcf3a50 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b14fd48820 con 0x55b14fcf22c0 2024-09-10T11:38:30.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x55b14fd14a70 con 0x55b14fd47c00 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc6814740 10 client.0 _ll_register_callbacks cb 0x7effb40097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36408/0 (socket says 172.21.15.99:36408) 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 -- 172.21.15.99:0/1885048808 learned_addr learned my addr 172.21.15.99:0/1885048808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc181c640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fdb4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc201d640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fd47c00 0x55b14fd457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 -- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fd47c00 msgr2=0x55b14fd457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fd47c00 0x55b14fd457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 -- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 msgr2=0x55b14fdb4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fdb4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 -- 172.21.15.99:0/1885048808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b14fd34770 con 0x55b14fcf3a50 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc181c640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fdb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:30.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7effc101b640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7effac0a9bc0 tx=0x7effac009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75200e3d42b6514 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.236+0000 7eff9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac087060 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7effac0802c0 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effb8005b10 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7effac04b040 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 10 client.7773 fetch_fsmap learned FSMap version 67 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55b14fdb8f40 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb8005b10 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7effac082160 con 0x55b14fcf3a50 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 10 client.7773 fetch_fsmap finished waiting for FSMap version 67 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 10 client.7773 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 20 client.7773 populate_metadata read hostname 'smithi099' 2024-09-10T11:38:30.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fceba00 0x55b14fcebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 4 client.7773 mds_command: new command op to 7635 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 --> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] -- command(tid 0: {"prefix": "status"}) -- 0x55b14fd337e0 con 0x55b14fceba00 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc181c640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fceba00 0x55b14fcebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7eff9cff9640 10 client.7773 ms_handle_connect on v2:172.21.15.99:6834/3322393426 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.237+0000 7effc181c640 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fceba00 0x55b14fcebde0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f71c6aa2f95f80 server_cookie=8292ec716d787d24 in_seq=0 out_seq=0 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7eff9cff9640 1 -- 172.21.15.99:0/1885048808 <== mds.0 v2:172.21.15.99:6834/3322393426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b14fd337e0 con 0x55b14fceba00 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7eff9cff9640 10 client.7773 handle_command_reply: tid=0 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 client.7773 shutdown 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 msgr2=0x55b14fdb52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7effac0a9bc0 tx=0x7effac009240 comp rx=0 tx=0).stop 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 shutdown_connections 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:6834/3322393426,v1:172.21.15.99:6835/3322393426] conn(0x55b14fceba00 0x55b14fcebde0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b14fd47c00 0x55b14fd457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7eff9cff9640 0 client.7773 ms_handle_reset on v2:172.21.15.99:6834/3322393426 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b14fcf3a50 0x55b14fdb52c0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 --2- 172.21.15.99:0/1885048808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b14fcf22c0 0x55b14fdb4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 >> 172.21.15.99:0/1885048808 conn(0x55b14fcefa80 msgr2=0x55b14fd472e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 shutdown_connections 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 1 -- 172.21.15.99:0/1885048808 wait complete. 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc6814740 20 client.7773 trim_cache size 0 max 16384 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:38:30.247 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7635, 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.5496486850000002, 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 67, 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 167, 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 167, 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 8.6385918470000007 2024-09-10T11:38:30.248 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 msgr2=0x7effa006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 0x7effa006e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7effb8000f60 tx=0x7effb8009040 comp rx=0 tx=0).stop 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 msgr2=0x7effbc0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.238+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7effac0090b0 tx=0x7effac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 shutdown_connections 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7effa006be30 0x7effa006e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effbc1610b0 0x7effbc0cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effbc1605c0 0x7effbc0b4b50 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 --2- 172.21.15.99:0/252891538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effbc0ae020 0x7effbc1e67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 >> 172.21.15.99:0/252891538 conn(0x7effbc0730c0 msgr2=0x7effbc158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 shutdown_connections 2024-09-10T11:38:30.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.239+0000 7effc3a80640 1 -- 172.21.15.99:0/252891538 wait complete. 2024-09-10T11:38:30.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:38:30.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.348+0000 7eff356ba640 1 Processor -- start 2024-09-10T11:38:30.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.348+0000 7eff356ba640 1 -- start start 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff356ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 0x7eff300d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff356ba640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff300d4e40 0x7eff300d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff356ba640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff3015ff70 con 0x7eff3015eb80 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff356ba640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7eff300d30c0 con 0x7eff300d4e40 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff356ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7eff300d1620 con 0x7eff301626c0 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2effd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7eff3015eb80 0x7eff300d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39088/0 (socket says 172.21.15.99:39088) 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2f7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff300d4e40 0x7eff300d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2effd640 1 -- 172.21.15.99:0/27483404 learned_addr learned my addr 172.21.15.99:0/27483404 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff267fc640 1 --2- 172.21.15.99:0/27483404 >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 0x7eff300d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2f7fe640 1 -- 172.21.15.99:0/27483404 >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 msgr2=0x7eff300d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2e7fc640 1 -- 172.21.15.99:0/27483404 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049401516 0 0) 0x7eff3015ff70 con 0x7eff3015eb80 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2f7fe640 1 --2- 172.21.15.99:0/27483404 >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 0x7eff300d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2f7fe640 1 -- 172.21.15.99:0/27483404 >> v1:172.21.15.99:6789/0 conn(0x7eff3015eb80 legacy=0x7eff300d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff2f7fe640 1 -- 172.21.15.99:0/27483404 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff30162bc0 con 0x7eff300d4e40 2024-09-10T11:38:30.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.349+0000 7eff267fc640 1 --2- 172.21.15.99:0/27483404 >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 0x7eff300d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:30.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff2f7fe640 1 --2- 172.21.15.99:0/27483404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff300d1b20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7eff28009d90 tx=0x7eff2802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c40ce86947bf2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff2e7fc640 1 -- 172.21.15.99:0/27483404 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28031600 con 0x7eff300d4e40 2024-09-10T11:38:30.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff2e7fc640 1 -- 172.21.15.99:0/27483404 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff280317a0 con 0x7eff300d4e40 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff2e7fc640 1 -- 172.21.15.99:0/27483404 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28031b70 con 0x7eff300d4e40 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 -- 172.21.15.99:0/27483404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 msgr2=0x7eff300d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 --2- 172.21.15.99:0/27483404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff300d1b20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7eff28009d90 tx=0x7eff2802a610 comp rx=0 tx=0).stop 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 -- 172.21.15.99:0/27483404 shutdown_connections 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 --2- 172.21.15.99:0/27483404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff300d1b20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 --2- 172.21.15.99:0/27483404 >> v2:172.21.15.196:3300/0 conn(0x7eff301626c0 0x7eff300d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.350+0000 7eff356ba640 1 -- 172.21.15.99:0/27483404 >> 172.21.15.99:0/27483404 conn(0x7eff3008e680 msgr2=0x7eff3009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.351+0000 7eff356ba640 1 -- 172.21.15.99:0/27483404 shutdown_connections 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.351+0000 7eff356ba640 1 -- 172.21.15.99:0/27483404 wait complete. 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.352+0000 7eff356ba640 1 Processor -- start 2024-09-10T11:38:30.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.352+0000 7eff356ba640 1 -- start start 2024-09-10T11:38:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.352+0000 7eff356ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.352+0000 7eff356ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 0x7eff300b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.352+0000 7eff356ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 0x7eff300b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff356ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff30163460 con 0x7eff300d4e40 2024-09-10T11:38:30.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff356ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7eff30163180 con 0x7eff301626c0 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff356ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7eff300d5290 con 0x7eff3015eb80 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36414/0 (socket says 172.21.15.99:36414) 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 -- 172.21.15.99:0/3294214944 learned_addr learned my addr 172.21.15.99:0/3294214944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2f7fe640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 0x7eff300b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff267fc640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 0x7eff300b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 -- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 msgr2=0x7eff300b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 0x7eff300b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 -- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 msgr2=0x7eff300b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 0x7eff300b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 -- 172.21.15.99:0/3294214944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff301e99f0 con 0x7eff300d4e40 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff267fc640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 0x7eff300b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:30.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2f7fe640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 0x7eff300b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff2effd640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c460 tx=0x7eff2000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20018070 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff20007500 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff300b1b10 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20004a20 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.353+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff300b20d0 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.354+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff300b22d0 con 0x7eff300d4e40 2024-09-10T11:38:30.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.355+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff2001b070 con 0x7eff300d4e40 2024-09-10T11:38:30.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.355+0000 7eff27fff640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 0x7eff1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.355+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7eff2008a070 con 0x7eff300d4e40 2024-09-10T11:38:30.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.355+0000 7eff267fc640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 0x7eff1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.356+0000 7eff267fc640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 0x7eff1006e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7eff18006fd0 tx=0x7eff18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.359+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff20058ab0 con 0x7eff300d4e40 2024-09-10T11:38:30.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.559+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7eff300d4900 con 0x7eff300d4e40 2024-09-10T11:38:30.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.560+0000 7eff27fff640 1 -- 172.21.15.99:0/3294214944 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7eff200583b0 con 0x7eff300d4e40 2024-09-10T11:38:30.562 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 msgr2=0x7eff1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 0x7eff1006e320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7eff18006fd0 tx=0x7eff18008040 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 msgr2=0x7eff3015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c460 tx=0x7eff2000c920 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 shutdown_connections 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7eff1006be60 0x7eff1006e320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff301626c0 0x7eff300b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3015eb80 0x7eff300b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 --2- 172.21.15.99:0/3294214944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff300d4e40 0x7eff3015bb30 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 >> 172.21.15.99:0/3294214944 conn(0x7eff3008e680 msgr2=0x7eff3006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 shutdown_connections 2024-09-10T11:38:30.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30.562+0000 7eff356ba640 1 -- 172.21.15.99:0/3294214944 wait complete. 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:38:30.575 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:30.576 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:38:30.591 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:30.592 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42338sec preferred_lft 42338sec 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:38:30.593 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:38:30.594 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:30.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:30.595 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:38:30.655 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:38:30.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:30.656 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:38:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:38:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:38:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:38:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:38:30.713 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:38:30.714 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:38:30.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:38:30.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:38:30.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:30.838 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:38:30.853 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:38:30.853 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:38:30.854 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:38:30.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:30.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:38:30.917 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:38:30.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:38:30.918 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:30.973 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:38:30.973 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:38:30.973 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:31.029 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:38:31.094 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:38:31.151 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:38:31.152 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:38:31.153 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:38:31.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:31.154 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:38:31.206 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:38:31.207 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:38:31.249 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:38:31.274 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:38:31.330 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:38:31.331 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:38:31.332 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:38:31.333 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:38:31.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:31.334 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:38:31.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:38:31.329+0000 7feb66d93480 -1 init, newargv = 0x56485eedd210 newargc=15 2024-09-10T11:38:31.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60650]: starting ceph client 2024-09-10T11:38:31.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60650]: starting fuse 2024-09-10T11:38:31.351 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:38:31.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:38:31.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:38:31.351 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:38:31.352 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:38:31.353 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:38:31.469 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.60650.asok 2024-09-10T11:38:31.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:38:31.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60650.asok 2024-09-10T11:38:31.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:31.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60650.asok status 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "60650", 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:38:31.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7791, 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 7791 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4157895388 2024-09-10T11:38:31.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4157895388 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.7791 192.168.0.1:0/4157895388", 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/4157895388", 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:38:31.615 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 67, 2024-09-10T11:38:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 167, 2024-09-10T11:38:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:38:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:38:31.616 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:38:31.616 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:38:31.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:31.621 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:38:31.621 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:31.637 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:38:31.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:31.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:31.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:38:31.703 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:38:31.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.840+0000 7f16dc88d640 1 Processor -- start 2024-09-10T11:38:31.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.840+0000 7f16dc88d640 1 -- start start 2024-09-10T11:38:31.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16dc88d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f16d8073440 0x7f16d815c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16dc88d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d8152e60 con 0x7f16d806ece0 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16dc88d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16d80d15e0 con 0x7f16d80d1200 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16dc88d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f16d8072680 con 0x7f16d8073440 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f16d8073440 0x7f16d815c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d5d9b640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f16d806ece0 0x7f16d815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39098/0 (socket says 172.21.15.99:39098) 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d5d9b640 1 -- 172.21.15.99:0/2400519702 learned_addr learned my addr 172.21.15.99:0/2400519702 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d6d9d640 1 -- 172.21.15.99:0/2400519702 >> v1:172.21.15.196:6789/0 conn(0x7f16d80d1200 legacy=0x7f16d806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d6d9d640 1 -- 172.21.15.99:0/2400519702 >> v1:172.21.15.99:6789/0 conn(0x7f16d806ece0 legacy=0x7f16d815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d6d9d640 1 -- 172.21.15.99:0/2400519702 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d8072960 con 0x7f16d8073440 2024-09-10T11:38:31.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.841+0000 7f16d559a640 1 -- 172.21.15.99:0/2400519702 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1033481261 0 0) 0x7f16d8152e60 con 0x7f16d806ece0 2024-09-10T11:38:31.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16d6d9d640 1 --2- 172.21.15.99:0/2400519702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d8073440 0x7f16d815c770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009d90 tx=0x7f16d002c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a635a5c380826e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16d559a640 1 -- 172.21.15.99:0/2400519702 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0007740 con 0x7f16d8073440 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16d559a640 1 -- 172.21.15.99:0/2400519702 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16d00078e0 con 0x7f16d8073440 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16d559a640 1 -- 172.21.15.99:0/2400519702 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d0007cb0 con 0x7f16d8073440 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16dc88d640 1 -- 172.21.15.99:0/2400519702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d8073440 msgr2=0x7f16d815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16dc88d640 1 --2- 172.21.15.99:0/2400519702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d8073440 0x7f16d815c770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f16d0009d90 tx=0x7f16d002c820 comp rx=0 tx=0).stop 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16dc88d640 1 -- 172.21.15.99:0/2400519702 shutdown_connections 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.842+0000 7f16dc88d640 1 --2- 172.21.15.99:0/2400519702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d8073440 0x7f16d815c770 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.843+0000 7f16dc88d640 1 -- 172.21.15.99:0/2400519702 >> 172.21.15.99:0/2400519702 conn(0x7f16d806bc10 msgr2=0x7f16d806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.843+0000 7f16dc88d640 1 -- 172.21.15.99:0/2400519702 shutdown_connections 2024-09-10T11:38:31.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.843+0000 7f16dc88d640 1 -- 172.21.15.99:0/2400519702 wait complete. 2024-09-10T11:38:31.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.844+0000 7f16dc88d640 1 Processor -- start 2024-09-10T11:38:31.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.844+0000 7f16dc88d640 1 -- start start 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8073440 0x7f16d80ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80d1200 0x7f16d8158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16d815cd90 con 0x7f16d806ece0 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f16d8072f20 con 0x7f16d80d1200 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f16d8159b50 con 0x7f16d8073440 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:31.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36420/0 (socket says 172.21.15.99:36420) 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 -- 172.21.15.99:0/760261836 learned_addr learned my addr 172.21.15.99:0/760261836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d6d9d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80d1200 0x7f16d8158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d5d9b640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8073440 0x7f16d80ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 -- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8073440 msgr2=0x7f16d80ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8073440 0x7f16d80ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 -- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80d1200 msgr2=0x7f16d8158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80d1200 0x7f16d8158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 -- 172.21.15.99:0/760261836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16d80cb000 con 0x7f16d806ece0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16d659c640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f16c800b3f0 tx=0x7f16c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8015020 con 0x7f16d806ece0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16c8011830 con 0x7f16d806ece0 2024-09-10T11:38:31.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16d80ccd40 con 0x7f16d806ece0 2024-09-10T11:38:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16c8011c20 con 0x7f16d806ece0 2024-09-10T11:38:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.845+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16d81578b0 con 0x7f16d806ece0 2024-09-10T11:38:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.846+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16d8159ed0 con 0x7f16d806ece0 2024-09-10T11:38:31.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.847+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f16c8018020 con 0x7f16d806ece0 2024-09-10T11:38:31.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.847+0000 7f16ceffd640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 0x7f16b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:31.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.847+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f16c808ab00 con 0x7f16d806ece0 2024-09-10T11:38:31.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.847+0000 7f16d5d9b640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 0x7f16b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:31.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.848+0000 7f16d5d9b640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 0x7f16b806e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cd0a0 tx=0x7f16c0007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:31.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:31.852+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f16c801f030 con 0x7f16d806ece0 2024-09-10T11:38:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.053+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f16d815d0b0 con 0x7f16d806ece0 2024-09-10T11:38:32.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.053+0000 7f16ceffd640 1 -- 172.21.15.99:0/760261836 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f16c80172c0 con 0x7f16d806ece0 2024-09-10T11:38:32.055 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:38:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 msgr2=0x7f16b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 0x7f16b806e2f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f16d80cd0a0 tx=0x7f16c0007b50 comp rx=0 tx=0).stop 2024-09-10T11:38:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 msgr2=0x7f16d8152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f16c800b3f0 tx=0x7f16c800b8b0 comp rx=0 tx=0).stop 2024-09-10T11:38:32.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 shutdown_connections 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f16b806be30 0x7f16b806e2f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80d1200 0x7f16d8158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8073440 0x7f16d80ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 --2- 172.21.15.99:0/760261836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16d806ece0 0x7f16d8152ed0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 >> 172.21.15.99:0/760261836 conn(0x7f16d806bc10 msgr2=0x7f16d815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 shutdown_connections 2024-09-10T11:38:32.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:32.055+0000 7f16dc88d640 1 -- 172.21.15.99:0/760261836 wait complete. 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:38:32.068 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:38:32.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.069 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:38:32.089 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:32.090 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42352sec preferred_lft 42352sec 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:38:32.091 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:38:32.092 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:38:32.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.093 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:38:32.158 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:38:32.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.160 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:38:32.220 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:38:32.220 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:38:32.220 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:38:32.220 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:38:32.220 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:38:32.221 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:38:32.315 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:38:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:38:32.350 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:38:32.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.357 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:38:32.373 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:38:32.373 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:38:32.374 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:38:32.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.374 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:38:32.439 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:38:32.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:38:32.440 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:32.496 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:38:32.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:38:32.497 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:32.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:38:32.618 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:38:32.676 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:38:32.676 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:38:32.677 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:38:32.678 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:38:32.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.679 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:38:32.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:38:32.733 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:38:32.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:38:32.799 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:38:32.855 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:38:32.856 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:38:32.857 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:38:32.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:32.858 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:38:32.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:38:32.856+0000 7f20ef8ff480 -1 init, newargv = 0x55e2a2147f40 newargc=15 2024-09-10T11:38:32.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[39591]: starting ceph client 2024-09-10T11:38:32.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[39591]: starting fuse 2024-09-10T11:38:32.874 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:38:32.874 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:38:32.874 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:38:32.875 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:38:32.876 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:38:32.992 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.39591.asok 2024-09-10T11:38:32.995 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:38:33.002 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39591.asok 2024-09-10T11:38:33.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:33.003 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39591.asok status 2024-09-10T11:38:33.135 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:38:33.136 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:38:33.136 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:38:33.136 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "39591", 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:38:33.137 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 7060, 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 7060 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4034168664 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:38:33.138 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 4034168664 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.7060 192.168.0.1:0/4034168664", 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/4034168664", 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 67, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 167, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:38:33.139 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:38:33.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:33.144 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:38:33.144 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:33.160 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:38:33.160 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:33.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:38:33.160 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:38:33.225 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:38:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.334+0000 7f9e5eba6640 1 Processor -- start 2024-09-10T11:38:33.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.334+0000 7f9e5eba6640 1 -- start start 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 0x7f9e58073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9e5806e5d0 0x7f9e5806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 0x7f9e5815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5806ef80 con 0x7f9e5806e5d0 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9e58152e60 con 0x7f9e580d1200 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5eba6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e580d15e0 con 0x7f9e5815c060 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9e5806e5d0 0x7f9e5806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9e5806e5d0 0x7f9e5806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41456/0 (socket says 172.21.15.99:41456) 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 -- 172.21.15.99:0/699718122 learned_addr learned my addr 172.21.15.99:0/699718122 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5d143640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 0x7f9e5815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 -- 172.21.15.99:0/699718122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 msgr2=0x7f9e5815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 0x7f9e5815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5c942640 1 --2- 172.21.15.99:0/699718122 >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 0x7f9e58073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 -- 172.21.15.99:0/699718122 >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 msgr2=0x7f9e58073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 --2- 172.21.15.99:0/699718122 >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 0x7f9e58073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e57fff640 1 -- 172.21.15.99:0/699718122 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58072680 con 0x7f9e5806e5d0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5d143640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 0x7f9e5815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.335+0000 7f9e5c942640 1 --2- 172.21.15.99:0/699718122 >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 0x7f9e58073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.336+0000 7f9e57fff640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5806e5d0 0x7f9e5806ea40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9e580d1850 tx=0x7f9e4402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b1e3243aaa6a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.336+0000 7f9e577fe640 1 -- 172.21.15.99:0/699718122 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44037070 con 0x7f9e5806e5d0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.336+0000 7f9e577fe640 1 -- 172.21.15.99:0/699718122 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44032440 con 0x7f9e5806e5d0 2024-09-10T11:38:33.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.336+0000 7f9e577fe640 1 -- 172.21.15.99:0/699718122 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44032810 con 0x7f9e5806e5d0 2024-09-10T11:38:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 -- 172.21.15.99:0/699718122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5806e5d0 msgr2=0x7f9e5806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5806e5d0 0x7f9e5806ea40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f9e580d1850 tx=0x7f9e4402a140 comp rx=0 tx=0).stop 2024-09-10T11:38:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 -- 172.21.15.99:0/699718122 shutdown_connections 2024-09-10T11:38:33.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e5815c060 0x7f9e5815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/699718122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5806e5d0 0x7f9e5806ea40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/699718122 >> v2:172.21.15.196:3300/0 conn(0x7f9e580d1200 0x7f9e58073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 -- 172.21.15.99:0/699718122 >> 172.21.15.99:0/699718122 conn(0x7f9e5806bc10 msgr2=0x7f9e5806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 -- 172.21.15.99:0/699718122 shutdown_connections 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.337+0000 7f9e5eba6640 1 -- 172.21.15.99:0/699718122 wait complete. 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.338+0000 7f9e5eba6640 1 Processor -- start 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.338+0000 7f9e5eba6640 1 -- start start 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5806e5d0 0x7f9e58151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e580d1200 0x7f9e5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58072f20 con 0x7f9e5815c060 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58072c40 con 0x7f9e5806e5d0 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5eba6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5815cd90 con 0x7f9e580d1200 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5d143640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5d143640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41472/0 (socket says 172.21.15.99:41472) 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5d143640 1 -- 172.21.15.99:0/2714222302 learned_addr learned my addr 172.21.15.99:0/2714222302 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e57fff640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e580d1200 0x7f9e5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.339+0000 7f9e5c942640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5806e5d0 0x7f9e58151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 -- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e580d1200 msgr2=0x7f9e5806d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e580d1200 0x7f9e5806d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 -- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5806e5d0 msgr2=0x7f9e58151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5806e5d0 0x7f9e58151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 -- 172.21.15.99:0/2714222302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e580afa20 con 0x7f9e5815c060 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5d143640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cef0 tx=0x7f9e5000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5001a070 con 0x7f9e5815c060 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5000d860 con 0x7f9e5815c060 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000dc50 con 0x7f9e5815c060 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5806de10 con 0x7f9e5815c060 2024-09-10T11:38:33.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.340+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e580afce0 con 0x7f9e5815c060 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.341+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5806ea40 con 0x7f9e5815c060 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.341+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9e5001d070 con 0x7f9e5815c060 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.342+0000 7f9e557fa640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 0x7f9e3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.342+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f9e5008b570 con 0x7f9e5815c060 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.342+0000 7f9e5c942640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 0x7f9e3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.342+0000 7f9e5c942640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 0x7f9e3c06e320 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9e581600f0 tx=0x7f9e4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.346+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9e50059a20 con 0x7f9e5815c060 2024-09-10T11:38:33.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.544+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e580b4c60 con 0x7f9e5815c060 2024-09-10T11:38:33.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.544+0000 7f9e557fa640 1 -- 172.21.15.99:0/2714222302 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f9e5005dd20 con 0x7f9e5815c060 2024-09-10T11:38:33.547 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:38:33.547 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:38:33.547 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 msgr2=0x7f9e3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 0x7f9e3c06e320 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9e581600f0 tx=0x7f9e4c008040 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 msgr2=0x7f9e580cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000cef0 tx=0x7f9e5000a710 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 shutdown_connections 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e3c06be60 0x7f9e3c06e320 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e5815c060 0x7f9e580cfe70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e580d1200 0x7f9e5806d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 --2- 172.21.15.99:0/2714222302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5806e5d0 0x7f9e58151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 >> 172.21.15.99:0/2714222302 conn(0x7f9e5806bc10 msgr2=0x7f9e580cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 shutdown_connections 2024-09-10T11:38:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.547+0000 7f9e5eba6640 1 -- 172.21.15.99:0/2714222302 wait complete. 2024-09-10T11:38:33.560 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:38:33.676 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":7635,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.9776700299999996,"mdsmap_epoch":67,"osdmap_epoch":167,"osdmap_epoch_barrier":167,"uptime":12.066614940999999} 2024-09-10T11:38:33.677 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 7635, "mdsmap_epoch": 67, "osdmap_epoch": 167, "osdmap_epoch_barrier": 167, "rank_uptime": 7.97767003, "state": "up:active", "uptime": 12.066614941, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:38:33.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-09-10T11:38:33.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.779+0000 7f1e00e8d640 1 Processor -- start 2024-09-10T11:38:33.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.779+0000 7f1e00e8d640 1 -- start start 2024-09-10T11:38:33.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1e00e8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1e00e8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 0x7f1dfc0d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1e00e8d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dfc0d2150 con 0x7f1dfc162d10 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1e00e8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1dfc0d0d30 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1e00e8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc0b4d30 con 0x7f1dfc0b59b0 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1dfa59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1dfa59c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41476/0 (socket says 172.21.15.99:41476) 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.780+0000 7f1dfa59c640 1 -- 172.21.15.99:0/7659437 learned_addr learned my addr 172.21.15.99:0/7659437 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1df9d9b640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 0x7f1dfc0d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1dfa59c640 1 -- 172.21.15.99:0/7659437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 msgr2=0x7f1dfc0d5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1dfa59c640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 0x7f1dfc0d5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1dfa59c640 1 -- 172.21.15.99:0/7659437 >> v1:172.21.15.196:6789/0 conn(0x7f1dfc162d10 legacy=0x7f1dfc1631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1dfa59c640 1 -- 172.21.15.99:0/7659437 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc0b5780 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1df9d9b640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 0x7f1dfc0d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.781+0000 7f1dfa59c640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009870 tx=0x7f1dec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a0c277bbfeb425 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1df959a640 1 -- 172.21.15.99:0/7659437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec037070 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1df959a640 1 -- 172.21.15.99:0/7659437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dec032440 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1df959a640 1 -- 172.21.15.99:0/7659437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec032810 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 -- 172.21.15.99:0/7659437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc0d16f0 msgr2=0x7f1dfc0d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009870 tx=0x7f1dec02a210 comp rx=0 tx=0).stop 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 -- 172.21.15.99:0/7659437 shutdown_connections 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0b59b0 0x7f1dfc0d5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/7659437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc0d16f0 0x7f1dfc0d1ad0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 -- 172.21.15.99:0/7659437 >> 172.21.15.99:0/7659437 conn(0x7f1dfc0d4c20 msgr2=0x7f1dfc0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 -- 172.21.15.99:0/7659437 shutdown_connections 2024-09-10T11:38:33.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.782+0000 7f1e00e8d640 1 -- 172.21.15.99:0/7659437 wait complete. 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 Processor -- start 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 -- start start 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dfc0b59b0 0x7f1dfc0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0d16f0 0x7f1dfc0b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0b4f80 con 0x7f1dfc162d10 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc0b52f0 con 0x7f1dfc0b59b0 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1e00e8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1dfc1638b0 con 0x7f1dfc0d16f0 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.784+0000 7f1dfad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:41490/0 (socket says 172.21.15.99:41490) 2024-09-10T11:38:33.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 -- 172.21.15.99:0/1158241815 learned_addr learned my addr 172.21.15.99:0/1158241815 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfa59c640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dfc0b59b0 0x7f1dfc0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1df9d9b640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0d16f0 0x7f1dfc0b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 -- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0d16f0 msgr2=0x7f1dfc0b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0d16f0 0x7f1dfc0b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 -- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dfc0b59b0 msgr2=0x7f1dfc0c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dfc0b59b0 0x7f1dfc0c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 -- 172.21.15.99:0/1158241815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc0c88f0 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1dfad9d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b970 tx=0x7f1df400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4019070 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df400cb40 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc0b8410 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4014440 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.785+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc0c95f0 con 0x7f1dfc162d10 2024-09-10T11:38:33.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.786+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc0d1ad0 con 0x7f1dfc162d10 2024-09-10T11:38:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.787+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df401c070 con 0x7f1dfc162d10 2024-09-10T11:38:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.787+0000 7f1deaffd640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.787+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5709+0+0 (secure 0 0 0) 0x7f1df408a940 con 0x7f1dfc162d10 2024-09-10T11:38:33.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.787+0000 7f1dfa59c640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:33.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.788+0000 7f1dfa59c640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034000 tx=0x7f1dec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:33.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.791+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1df4058d30 con 0x7f1dfc162d10 2024-09-10T11:38:33.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:33.957+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f1dfc0b8930 con 0x7f1dfc162d10 2024-09-10T11:38:34.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.715+0000 7f1deaffd640 1 -- 172.21.15.99:0/1158241815 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v168) ==== 181+0+0 (secure 0 0 0) 0x7f1df405d030 con 0x7f1dfc162d10 2024-09-10T11:38:34.717 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-09-10T11:38:34.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 msgr2=0x7f1ddc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:34.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034000 tx=0x7f1dec02e000 comp rx=0 tx=0).stop 2024-09-10T11:38:34.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 msgr2=0x7f1dfc0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:34.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f1df400b970 tx=0x7f1df400be30 comp rx=0 tx=0).stop 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 shutdown_connections 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1ddc06be30 0x7f1ddc06e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dfc162d10 0x7f1dfc0b3d80 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1dfc0d16f0 0x7f1dfc0b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 --2- 172.21.15.99:0/1158241815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1dfc0b59b0 0x7f1dfc0c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 >> 172.21.15.99:0/1158241815 conn(0x7f1dfc0d4c20 msgr2=0x7f1dfc0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 shutdown_connections 2024-09-10T11:38:34.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:38:34.718+0000 7f1e00e8d640 1 -- 172.21.15.99:0/1158241815 wait complete. 2024-09-10T11:38:34.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:38:34.850 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:38:34.850 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:38:34.850 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-09-10T11:38:34.954 INFO:teuthology.orchestra.run.smithi099.stdout:{"mon_tick_interval":"5"} 2024-09-10T11:38:34.954 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-09-10T11:38:34.954 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> stdin-killer -- python3 -c ' 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> import time 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> import datetime 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> # Write some buffered data through before going full, all should be well 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> print("writing some data through which we expect to succeed") 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> bytes = 0 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-09-10T11:38:34.955 DEBUG:teuthology.orchestra.run.smithi099:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> # buffered IO, the error should always only appear from fsync and not 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> # from write 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> full = False 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> for n in range(0, int(32 * 1.1)): 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:38:34.956 DEBUG:teuthology.orchestra.run.smithi099:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> print("wrote bytes via buffered write, moving on to fsync") 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> raise 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> break 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-09-10T11:38:34.957 DEBUG:teuthology.orchestra.run.smithi099:> os.fsync(f) 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> print("fsync'"'"'ed successfully") 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> full = True 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> break 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> if n > 32 * 0.9: 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> # Be cautious in the last region where we expect to hit 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-09-10T11:38:34.958 DEBUG:teuthology.orchestra.run.smithi099:> time.sleep(15.0) 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> if not full: 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> # close() should not raise an error because we already caught it in 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> # since then because all IOs got cancelled on the full flag. 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> print("calling close") 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> os.close(f) 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> print("close() did not raise error") 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:38:34.959 DEBUG:teuthology.orchestra.run.smithi099:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-09-10T11:38:34.960 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:38:59.684 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:38:59.688 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:39:21.777 INFO:teuthology.orchestra.run.smithi099.stdout:writing some data through which we expect to succeed 2024-09-10T11:39:21.777 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 1.00 MB 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 2.00 MB 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 3.00 MB 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 4.00 MB 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.778 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 5.00 MB 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 6.00 MB 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 7.00 MB 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 8.00 MB 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.779 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 9.00 MB 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 10.00 MB 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 11.00 MB 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 12.00 MB 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.780 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 13.00 MB 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 14.00 MB 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 15.00 MB 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 16.00 MB 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.781 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 17.00 MB 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 18.00 MB 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 19.00 MB 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 20.00 MB 2024-09-10T11:39:21.782 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 21.00 MB 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 22.00 MB 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 23.00 MB 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 24.00 MB 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.783 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 25.00 MB 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 26.00 MB 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 27.00 MB 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 28.00 MB 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.784 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 29.00 MB 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 30.00 MB 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 31.00 MB 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsync'ed successfully 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:Not full yet after 32.00 MB 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-09-10T11:39:21.785 INFO:teuthology.orchestra.run.smithi099.stdout:wrote bytes via buffered write, moving on to fsync 2024-09-10T11:39:21.786 INFO:teuthology.orchestra.run.smithi099.stdout:Reached fullness after 33.00 MB 2024-09-10T11:39:21.786 INFO:teuthology.orchestra.run.smithi099.stdout:calling close 2024-09-10T11:39:21.786 INFO:teuthology.orchestra.run.smithi099.stdout:close() did not raise error 2024-09-10T11:39:21.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:39:21.790 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:39:21.790 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:39:21.835 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:39:21.894 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:39:21.894 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:39:21.895 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:21.895 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:21.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:39:22.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.055+0000 7fd7720a5640 1 Processor -- start 2024-09-10T11:39:22.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.055+0000 7fd7720a5640 1 -- start start 2024-09-10T11:39:22.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd7720a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd7720a5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd76c15f1e0 0x7fd76c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd7720a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd76c0b40e0 con 0x7fd76c0b0090 2024-09-10T11:39:22.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd7720a5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fd76c0b5ab0 con 0x7fd76c15f1e0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd7720a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c073c50 con 0x7fd76c15ee00 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd76c15f1e0 0x7fd76c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd76c15f1e0 0x7fd76c0b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47910/0 (socket says 172.21.15.99:47910) 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 -- 172.21.15.99:0/1136369837 learned_addr learned my addr 172.21.15.99:0/1136369837 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76b7fe640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 -- 172.21.15.99:0/1136369837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 msgr2=0x7fd76c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 -- 172.21.15.99:0/1136369837 >> v1:172.21.15.196:6789/0 conn(0x7fd76c0b0090 legacy=0x7fd76c0b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.056+0000 7fd76affd640 1 -- 172.21.15.99:0/1136369837 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c0b4380 con 0x7fd76c15f1e0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd76affd640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c15f1e0 0x7fd76c0b1560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd754009870 tx=0x7fd75402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914653dafb3faedd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd76a7fc640 1 -- 172.21.15.99:0/1136369837 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754037070 con 0x7fd76c15f1e0 2024-09-10T11:39:22.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd76a7fc640 1 -- 172.21.15.99:0/1136369837 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd754032440 con 0x7fd76c15f1e0 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd76a7fc640 1 -- 172.21.15.99:0/1136369837 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754032810 con 0x7fd76c15f1e0 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd7720a5640 1 -- 172.21.15.99:0/1136369837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c15f1e0 msgr2=0x7fd76c0b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.057+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c15f1e0 0x7fd76c0b1560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd754009870 tx=0x7fd75402a270 comp rx=0 tx=0).stop 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 -- 172.21.15.99:0/1136369837 shutdown_connections 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c15f1e0 0x7fd76c0b1560 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1136369837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c0b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 -- 172.21.15.99:0/1136369837 >> 172.21.15.99:0/1136369837 conn(0x7fd76c0730c0 msgr2=0x7fd76c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 -- 172.21.15.99:0/1136369837 shutdown_connections 2024-09-10T11:39:22.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.058+0000 7fd7720a5640 1 -- 172.21.15.99:0/1136369837 wait complete. 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.059+0000 7fd7720a5640 1 Processor -- start 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.059+0000 7fd7720a5640 1 -- start start 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c15f1e0 0x7fd76c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0b21b0 con 0x7fd76c0b0090 2024-09-10T11:39:22.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c0afd60 con 0x7fd76c15f1e0 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c0b0600 con 0x7fd76c15ee00 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47926/0 (socket says 172.21.15.99:47926) 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 -- 172.21.15.99:0/1160507939 learned_addr learned my addr 172.21.15.99:0/1160507939 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76affd640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76bfff640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c15f1e0 0x7fd76c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 -- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 msgr2=0x7fd76c162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 -- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c15f1e0 msgr2=0x7fd76c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c15f1e0 0x7fd76c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c15f780 con 0x7fd76c0b0090 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd76b7fe640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d830 tx=0x7fd75c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00b6c0 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd75c019070 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00ba80 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.060+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c15fa60 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.061+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c160020 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.061+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c0b5f90 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.062+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd75c01e460 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.062+0000 7fd763fff640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 0x7fd74c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.062+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7fd75c08b3e0 con 0x7fd76c0b0090 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.062+0000 7fd76affd640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 0x7fd74c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.063+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7fd74c073ab0 con 0x7fd76c0b0090 2024-09-10T11:39:22.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.063+0000 7fd76affd640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 0x7fd74c06e320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd754034000 tx=0x7fd75402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.067+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd75c059700 con 0x7fd76c0b0090 2024-09-10T11:39:22.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.242+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd76c06e730 con 0x7fd76c0b0090 2024-09-10T11:39:22.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.243+0000 7fd763fff640 1 -- 172.21.15.99:0/1160507939 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v171) ==== 80+0+0 (secure 0 0 0) 0x7fd75c05da00 con 0x7fd76c0b0090 2024-09-10T11:39:22.245 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 msgr2=0x7fd74c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 0x7fd74c06e320 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd754034000 tx=0x7fd75402e000 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 msgr2=0x7fd76c14e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00d830 tx=0x7fd75c00dcf0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 shutdown_connections 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd74c06be60 0x7fd74c06e320 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd76c15f1e0 0x7fd76c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd76c15ee00 0x7fd76c162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 --2- 172.21.15.99:0/1160507939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd76c0b0090 0x7fd76c14e490 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 >> 172.21.15.99:0/1160507939 conn(0x7fd76c0730c0 msgr2=0x7fd76c06f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 shutdown_connections 2024-09-10T11:39:22.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.245+0000 7fd7720a5640 1 -- 172.21.15.99:0/1160507939 wait complete. 2024-09-10T11:39:22.257 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:39:22.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:22.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:22.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60650]: fuse finished with error 0 and tester_r 0 2024-09-10T11:39:22.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:39:22.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:22.283 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:39:22.388 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:39:22.388 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:39:22.388 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:22.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:39:22.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.505+0000 7fb5002ba640 1 Processor -- start 2024-09-10T11:39:22.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.505+0000 7fb5002ba640 1 -- start start 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb5002ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb5002ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 0x7fb4f80b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb5002ba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f80b40e0 con 0x7fb4f815ee00 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb5002ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f80b5ab0 con 0x7fb4f80b0090 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb5002ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f8073c50 con 0x7fb4f815f1e0 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb4fe857640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 0x7fb4f80b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb4fd855640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb4fd855640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:55026/0 (socket says 172.21.15.99:55026) 2024-09-10T11:39:22.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb4fe857640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 0x7fb4f80b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:58624/0 (socket says 172.21.15.99:58624) 2024-09-10T11:39:22.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.506+0000 7fb4fd855640 1 -- 172.21.15.99:0/1607368073 learned_addr learned my addr 172.21.15.99:0/1607368073 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.507+0000 7fb4fd855640 1 -- 172.21.15.99:0/1607368073 >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 msgr2=0x7fb4f80b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.507+0000 7fb4fd855640 1 --2- 172.21.15.99:0/1607368073 >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 0x7fb4f80b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.507+0000 7fb4fd855640 1 -- 172.21.15.99:0/1607368073 >> v1:172.21.15.196:6789/0 conn(0x7fb4f815ee00 legacy=0x7fb4f80b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.507+0000 7fb4fd855640 1 -- 172.21.15.99:0/1607368073 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f80b4380 con 0x7fb4f815f1e0 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.507+0000 7fb4fd855640 1 --2- 172.21.15.99:0/1607368073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009870 tx=0x7fb4ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1eb011aaf88916 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb4fd054640 1 -- 172.21.15.99:0/1607368073 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec037070 con 0x7fb4f815f1e0 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb4fd054640 1 -- 172.21.15.99:0/1607368073 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec032440 con 0x7fb4f815f1e0 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb4fd054640 1 -- 172.21.15.99:0/1607368073 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec032810 con 0x7fb4f815f1e0 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 -- 172.21.15.99:0/1607368073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 msgr2=0x7fb4f80b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 --2- 172.21.15.99:0/1607368073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009870 tx=0x7fb4ec02a270 comp rx=0 tx=0).stop 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 -- 172.21.15.99:0/1607368073 shutdown_connections 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 --2- 172.21.15.99:0/1607368073 >> v2:172.21.15.196:3300/0 conn(0x7fb4f80b0090 0x7fb4f80b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 --2- 172.21.15.99:0/1607368073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f815f1e0 0x7fb4f80b5f90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 -- 172.21.15.99:0/1607368073 >> 172.21.15.99:0/1607368073 conn(0x7fb4f80730c0 msgr2=0x7fb4f80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 -- 172.21.15.99:0/1607368073 shutdown_connections 2024-09-10T11:39:22.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.508+0000 7fb5002ba640 1 -- 172.21.15.99:0/1607368073 wait complete. 2024-09-10T11:39:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.509+0000 7fb5002ba640 1 Processor -- start 2024-09-10T11:39:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.509+0000 7fb5002ba640 1 -- start start 2024-09-10T11:39:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f80b0090 0x7fb4f80caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4f815f1e0 0x7fb4f8158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80b21b0 con 0x7fb4f815ee00 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80afd60 con 0x7fb4f815f1e0 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb5002ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f80b0600 con 0x7fb4f80b0090 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb4fd855640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb4fd855640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47936/0 (socket says 172.21.15.99:47936) 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb4fd855640 1 -- 172.21.15.99:0/325435590 learned_addr learned my addr 172.21.15.99:0/325435590 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb4fe857640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4f815f1e0 0x7fb4f8158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.510+0000 7fb4fe056640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f80b0090 0x7fb4f80caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 -- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f80b0090 msgr2=0x7fb4f80caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f80b0090 0x7fb4f80caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 -- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4f815f1e0 msgr2=0x7fb4f8158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4f815f1e0 0x7fb4f8158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f81f5ac0 con 0x7fb4f815ee00 2024-09-10T11:39:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4fd855640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0090b0 tx=0x7fb4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec041070 con 0x7fb4f815ee00 2024-09-10T11:39:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec03c870 con 0x7fb4f815ee00 2024-09-10T11:39:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f80cc290 con 0x7fb4f815ee00 2024-09-10T11:39:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec037040 con 0x7fb4f815ee00 2024-09-10T11:39:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.511+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f80ca460 con 0x7fb4f815ee00 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.512+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f80b5f90 con 0x7fb4f815ee00 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb4ec044070 con 0x7fb4f815ee00 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4eaffd640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 0x7fb4dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7fb4ec0ab7a0 con 0x7fb4f815ee00 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4fe056640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 0x7fb4dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7fb4dc073a80 con 0x7fb4f815ee00 2024-09-10T11:39:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.513+0000 7fb4fe056640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 0x7fb4dc06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4f806f360 tx=0x7fb4f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.517+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4f81f5ac0 con 0x7fb4f815ee00 2024-09-10T11:39:22.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.696+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb4f8162680 con 0x7fb4f815ee00 2024-09-10T11:39:22.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.696+0000 7fb4eaffd640 1 -- 172.21.15.99:0/325435590 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v171) ==== 80+0+0 (secure 0 0 0) 0x7fb4ec079be0 con 0x7fb4f815ee00 2024-09-10T11:39:22.698 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 msgr2=0x7fb4dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 0x7fb4dc06e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4f806f360 tx=0x7fb4f4009040 comp rx=0 tx=0).stop 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 msgr2=0x7fb4f81f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0090b0 tx=0x7fb4ec0046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 shutdown_connections 2024-09-10T11:39:22.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb4dc06be30 0x7fb4dc06e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4f815f1e0 0x7fb4f8158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f815ee00 0x7fb4f81f5580 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 --2- 172.21.15.99:0/325435590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4f80b0090 0x7fb4f80caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 >> 172.21.15.99:0/325435590 conn(0x7fb4f80730c0 msgr2=0x7fb4f815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 shutdown_connections 2024-09-10T11:39:22.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:22.698+0000 7fb5002ba640 1 -- 172.21.15.99:0/325435590 wait complete. 2024-09-10T11:39:22.711 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:39:22.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:22.711 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:22.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[39591]: fuse finished with error 0 and tester_r 0 2024-09-10T11:39:22.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:39:22.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:22.744 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:39:22.913 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:39:22.913 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:22.913 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:22.929 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:39:22.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:39:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:22.930 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:39:22.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:39:22.985 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:39:22.986 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:39:22.986 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:39:22.986 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:23.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:39:23.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:39:23.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:23.005 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:39:23.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:39:23.062 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:39:23.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:23.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.159+0000 7f48d025e640 1 Processor -- start 2024-09-10T11:39:23.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 -- start start 2024-09-10T11:39:23.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f48c81626c0 0x7f48c80d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48c815ff70 con 0x7f48c815eb80 2024-09-10T11:39:23.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f48c80d30c0 con 0x7f48c81626c0 2024-09-10T11:39:23.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.160+0000 7f48d025e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f48c80d52d0 con 0x7f48c80d4e40 2024-09-10T11:39:23.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48cd7f9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f48c81626c0 0x7f48c80d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:33964/0 (socket says 172.21.15.99:33964) 2024-09-10T11:39:23.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48cdffa640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f48c815eb80 0x7f48c80d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:40008/0 (socket says 172.21.15.99:40008) 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 -- 172.21.15.99:0/3424039256 learned_addr learned my addr 172.21.15.99:0/3424039256 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ccff8640 1 -- 172.21.15.99:0/3424039256 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182306675 0 0) 0x7f48c815ff70 con 0x7f48c815eb80 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 -- 172.21.15.99:0/3424039256 >> v2:172.21.15.196:3300/0 conn(0x7f48c81626c0 msgr2=0x7f48c80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 --2- 172.21.15.99:0/3424039256 >> v2:172.21.15.196:3300/0 conn(0x7f48c81626c0 0x7f48c80d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 -- 172.21.15.99:0/3424039256 >> v1:172.21.15.196:6789/0 conn(0x7f48c815eb80 legacy=0x7f48c80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.161+0000 7f48ce7fb640 1 -- 172.21.15.99:0/3424039256 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c80d1620 con 0x7f48c80d4e40 2024-09-10T11:39:23.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.162+0000 7f48ce7fb640 1 --2- 172.21.15.99:0/3424039256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e80 tx=0x7f48c002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2793f4882721ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.162+0000 7f48ccff8640 1 -- 172.21.15.99:0/3424039256 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0036070 con 0x7f48c80d4e40 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.162+0000 7f48ccff8640 1 -- 172.21.15.99:0/3424039256 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c0031600 con 0x7f48c80d4e40 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.162+0000 7f48ccff8640 1 -- 172.21.15.99:0/3424039256 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00319d0 con 0x7f48c80d4e40 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 -- 172.21.15.99:0/3424039256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 msgr2=0x7f48c80d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 --2- 172.21.15.99:0/3424039256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009e80 tx=0x7f48c002a6c0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 -- 172.21.15.99:0/3424039256 shutdown_connections 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 --2- 172.21.15.99:0/3424039256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c80d1b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 --2- 172.21.15.99:0/3424039256 >> v2:172.21.15.196:3300/0 conn(0x7f48c81626c0 0x7f48c80d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 -- 172.21.15.99:0/3424039256 >> 172.21.15.99:0/3424039256 conn(0x7f48c808e680 msgr2=0x7f48c809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 -- 172.21.15.99:0/3424039256 shutdown_connections 2024-09-10T11:39:23.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.163+0000 7f48d025e640 1 -- 172.21.15.99:0/3424039256 wait complete. 2024-09-10T11:39:23.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.164+0000 7f48d025e640 1 Processor -- start 2024-09-10T11:39:23.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.164+0000 7f48d025e640 1 -- start start 2024-09-10T11:39:23.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c81626c0 0x7f48c814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8163180 con 0x7f48c815eb80 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48c8162ea0 con 0x7f48c81626c0 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48d025e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f48c8163460 con 0x7f48c80d4e40 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48552/0 (socket says 172.21.15.99:48552) 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 -- 172.21.15.99:0/3708943090 learned_addr learned my addr 172.21.15.99:0/3708943090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cdffa640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48ce7fb640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c81626c0 0x7f48c814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 -- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 msgr2=0x7f48c814e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c814e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 -- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c81626c0 msgr2=0x7f48c814b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c81626c0 0x7f48c814b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c814ba20 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.165+0000 7f48cd7f9640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48b800e870 tx=0x7f48b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8017070 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b800c940 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b800cd10 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c80ce0c0 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c80ce680 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.166+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c80d4900 con 0x7f48c815eb80 2024-09-10T11:39:23.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.167+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48b801a070 con 0x7f48c815eb80 2024-09-10T11:39:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.167+0000 7f48be7fc640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 0x7f48ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.168+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f48b808adf0 con 0x7f48c815eb80 2024-09-10T11:39:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.168+0000 7f48cdffa640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 0x7f48ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.168+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f48ac073a80 con 0x7f48c815eb80 2024-09-10T11:39:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.168+0000 7f48cdffa640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 0x7f48ac06e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f60 tx=0x7f48c4007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.172+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f48b80596e0 con 0x7f48c815eb80 2024-09-10T11:39:23.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.375+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48c8068440 con 0x7f48c815eb80 2024-09-10T11:39:23.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.375+0000 7f48be7fc640 1 -- 172.21.15.99:0/3708943090 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f48b8058fe0 con 0x7f48c815eb80 2024-09-10T11:39:23.377 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:23.377 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:39:23.378 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 msgr2=0x7f48ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 0x7f48ac06e2f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000f60 tx=0x7f48c4007c00 comp rx=0 tx=0).stop 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 msgr2=0x7f48c8151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48b800e870 tx=0x7f48b800ed30 comp rx=0 tx=0).stop 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 shutdown_connections 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f48ac06be30 0x7f48ac06e2f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48c81626c0 0x7f48c814b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48c815eb80 0x7f48c8151df0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 --2- 172.21.15.99:0/3708943090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48c80d4e40 0x7f48c814e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 >> 172.21.15.99:0/3708943090 conn(0x7f48c808e680 msgr2=0x7f48c80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 shutdown_connections 2024-09-10T11:39:23.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.378+0000 7f48d025e640 1 -- 172.21.15.99:0/3708943090 wait complete. 2024-09-10T11:39:23.391 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:23.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.487+0000 7f9a32dc4640 1 Processor -- start 2024-09-10T11:39:23.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.487+0000 7f9a32dc4640 1 -- start start 2024-09-10T11:39:23.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a32dc4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a2c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a32dc4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2c152e60 con 0x7f9a2c15c060 2024-09-10T11:39:23.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a32dc4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2c0d15e0 con 0x7f9a2c0d1200 2024-09-10T11:39:23.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a32dc4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c072680 con 0x7f9a2c06ece0 2024-09-10T11:39:23.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a31361640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9a2c15c060 0x7f9a2c15c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44322/0 (socket says 172.21.15.99:44322) 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a31361640 1 -- 172.21.15.99:0/2599468398 learned_addr learned my addr 172.21.15.99:0/2599468398 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a2c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038024000 0 0) 0x7f9a2c152e60 con 0x7f9a2c15c060 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.488+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a14003520 con 0x7f9a2c15c060 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488638504 0 0) 0x7f9a1c001110 con 0x7f9a2c0d1200 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a2c152e60 con 0x7f9a2c0d1200 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 578472367 0 0) 0x7f9a14003520 con 0x7f9a2c15c060 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 msgr2=0x7f9a2c073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a2c073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 >> v1:172.21.15.196:6789/0 conn(0x7f9a2c0d1200 legacy=0x7f9a2c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c072960 con 0x7f9a2c15c060 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f9a240019f0 con 0x7f9a2c15c060 2024-09-10T11:39:23.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a2c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 >> v1:172.21.15.99:6789/0 conn(0x7f9a2c15c060 legacy=0x7f9a2c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a14003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140040b0 0x7f9a14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c072680 con 0x7f9a14004d40 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1400bb00 con 0x7f9a140040b0 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1400bd00 con 0x7f9a2c06ece0 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a30b60640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a14003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140040b0 0x7f9a14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 -- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 msgr2=0x7f9a14003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c06ece0 0x7f9a14003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 -- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140040b0 msgr2=0x7f9a14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:23.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a140040b0 0x7f9a14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.489+0000 7f9a31361640 1 -- 172.21.15.99:0/2599468398 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a14003520 con 0x7f9a14004d40 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a31361640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a140051c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400a800 tx=0x7f9a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82753aed330ca0de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2400a430 con 0x7f9a14004d40 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a2402f070 con 0x7f9a14004d40 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a2b7fe640 1 -- 172.21.15.99:0/2599468398 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2402bc70 con 0x7f9a14004d40 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2599468398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 msgr2=0x7f9a140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.490+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2599468398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a140051c0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400a800 tx=0x7f9a14005830 comp rx=0 tx=0).stop 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.491+0000 7f9a31361640 1 -- 172.21.15.99:0/2599468398 reap_dead start 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.491+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2599468398 shutdown_connections 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.491+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2599468398 >> 172.21.15.99:0/2599468398 conn(0x7f9a2c06bc10 msgr2=0x7f9a2c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.491+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2599468398 shutdown_connections 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.491+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2599468398 wait complete. 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.492+0000 7f9a32dc4640 1 Processor -- start 2024-09-10T11:39:23.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.492+0000 7f9a32dc4640 1 -- start start 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c137d00 0x7f9a2c0c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0c5cf0 0x7f9a2c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c15cd90 con 0x7f9a14004d40 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c072f20 con 0x7f9a2c0c5cf0 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c073440 con 0x7f9a2c137d00 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48570/0 (socket says 172.21.15.99:48570) 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 -- 172.21.15.99:0/2399511503 learned_addr learned my addr 172.21.15.99:0/2399511503 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a31361640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0c5cf0 0x7f9a2c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c137d00 0x7f9a2c0c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 -- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c137d00 msgr2=0x7f9a2c0c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c137d00 0x7f9a2c0c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 -- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0c5cf0 msgr2=0x7f9a2c14ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0c5cf0 0x7f9a2c14ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c1385e0 con 0x7f9a14004d40 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a30b60640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00f7d0 tx=0x7f9a1c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0052e0 con 0x7f9a14004d40 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c018070 con 0x7f9a14004d40 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00d970 con 0x7f9a14004d40 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1388c0 con 0x7f9a14004d40 2024-09-10T11:39:23.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.493+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c138e80 con 0x7f9a14004d40 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.494+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c15c4a0 con 0x7f9a14004d40 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.495+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a1c00db10 con 0x7f9a14004d40 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.495+0000 7f9a297fa640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.496+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f9a1c08ac50 con 0x7f9a14004d40 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.496+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.496+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f9a18073a50 con 0x7f9a14004d40 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.496+0000 7f9a2bfff640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 0x7f9a1806e320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c06e1b0 tx=0x7f9a20009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.500+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a1c090020 con 0x7f9a14004d40 2024-09-10T11:39:23.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.696+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a2c0c60d0 con 0x7f9a14004d40 2024-09-10T11:39:23.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.697+0000 7f9a297fa640 1 -- 172.21.15.99:0/2399511503 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f9a2c0d15e0 con 0x7f9a14004d40 2024-09-10T11:39:23.699 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:23.699 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:39:23.699 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 msgr2=0x7f9a1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 0x7f9a1806e320 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c06e1b0 tx=0x7f9a20009190 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 msgr2=0x7f9a2c14ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00f7d0 tx=0x7f9a1c00fc90 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 shutdown_connections 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a2c0c5cf0 0x7f9a2c14ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a2c137d00 0x7f9a2c0c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 --2- 172.21.15.99:0/2399511503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a14004d40 0x7f9a2c14ca90 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 >> 172.21.15.99:0/2399511503 conn(0x7f9a2c06bc10 msgr2=0x7f9a2c0ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 shutdown_connections 2024-09-10T11:39:23.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.699+0000 7f9a32dc4640 1 -- 172.21.15.99:0/2399511503 wait complete. 2024-09-10T11:39:23.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.808+0000 7f0dc7c47640 1 Processor -- start 2024-09-10T11:39:23.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.808+0000 7f0dc7c47640 1 -- start start 2024-09-10T11:39:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc7c47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc7c47640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0dc00afbb0 0x7f0dc00714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc7c47640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc006ee10 con 0x7f0dc015aef0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc7c47640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc00afff0 con 0x7f0dc00afbb0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc7c47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00cd9a0 con 0x7f0dc015b2d0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc59e3640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0dc015aef0 0x7f0dc00cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44324/0 (socket says 172.21.15.99:44324) 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc59e3640 1 -- 172.21.15.99:0/1283590209 learned_addr learned my addr 172.21.15.99:0/1283590209 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc51e2640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc61e4640 1 --2- 172.21.15.99:0/1283590209 >> v2:172.21.15.196:3300/0 conn(0x7f0dc00afbb0 0x7f0dc00714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc49e1640 1 -- 172.21.15.99:0/1283590209 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897443325 0 0) 0x7f0dc006ee10 con 0x7f0dc015aef0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.809+0000 7f0dc49e1640 1 -- 172.21.15.99:0/1283590209 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dac003520 con 0x7f0dc015aef0 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc61e4640 1 -- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 msgr2=0x7f0dc00b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc61e4640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc00b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc61e4640 1 -- 172.21.15.99:0/1283590209 >> v1:172.21.15.99:6789/0 conn(0x7f0dc015aef0 legacy=0x7f0dc00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc61e4640 1 -- 172.21.15.99:0/1283590209 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00b0290 con 0x7f0dc00afbb0 2024-09-10T11:39:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc51e2640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc00b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.811+0000 7f0dc61e4640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc00714a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0db800aea0 tx=0x7f0db802b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5782aa0ca8cb2f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc49e1640 1 -- 172.21.15.99:0/1283590209 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8037070 con 0x7f0dc00afbb0 2024-09-10T11:39:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc49e1640 1 -- 172.21.15.99:0/1283590209 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db8008640 con 0x7f0dc00afbb0 2024-09-10T11:39:23.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc49e1640 1 -- 172.21.15.99:0/1283590209 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8008a10 con 0x7f0dc00afbb0 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 -- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 msgr2=0x7f0dc00714a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc00714a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0db800aea0 tx=0x7f0db802b7b0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 -- 172.21.15.99:0/1283590209 shutdown_connections 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc00714a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/1283590209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc00b4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 -- 172.21.15.99:0/1283590209 >> 172.21.15.99:0/1283590209 conn(0x7f0dc006bc10 msgr2=0x7f0dc006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 -- 172.21.15.99:0/1283590209 shutdown_connections 2024-09-10T11:39:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.813+0000 7f0dc7c47640 1 -- 172.21.15.99:0/1283590209 wait complete. 2024-09-10T11:39:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 Processor -- start 2024-09-10T11:39:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 -- start start 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc0154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc0150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc006eb30 con 0x7f0dc015aef0 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc006e850 con 0x7f0dc00afbb0 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.815+0000 7f0dc7c47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc00719e0 con 0x7f0dc015b2d0 2024-09-10T11:39:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48572/0 (socket says 172.21.15.99:48572) 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 -- 172.21.15.99:0/3468112770 learned_addr learned my addr 172.21.15.99:0/3468112770 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc61e4640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc0150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc59e3640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc0154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 -- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 msgr2=0x7f0dc0150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc0150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 -- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 msgr2=0x7f0dc0154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc0154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc014e530 con 0x7f0dc015aef0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc51e2640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e990 tx=0x7f0db000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0017070 con 0x7f0dc015aef0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db000c9e0 con 0x7f0dc015aef0 2024-09-10T11:39:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc014e810 con 0x7f0dc015aef0 2024-09-10T11:39:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db000cdb0 con 0x7f0dc015aef0 2024-09-10T11:39:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.816+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc014edd0 con 0x7f0dc015aef0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.817+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc00730b0 con 0x7f0dc015aef0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.817+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0db001a070 con 0x7f0dc015aef0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.818+0000 7f0db67fc640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 0x7f0da406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.818+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f0db008afb0 con 0x7f0dc015aef0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.818+0000 7f0dc59e3640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 0x7f0da406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.818+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f0da4073a20 con 0x7f0dc015aef0 2024-09-10T11:39:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.818+0000 7f0dc59e3640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 0x7f0da406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc005880 tx=0x7f0dbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:23.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.822+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0dc00730b0 con 0x7f0dc015aef0 2024-09-10T11:39:23.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.989+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dc00b4140 con 0x7f0dc015aef0 2024-09-10T11:39:23.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.990+0000 7f0db67fc640 1 -- 172.21.15.99:0/3468112770 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) ==== 74+0+16863 (secure 0 0 0) 0x7f0db0059360 con 0x7f0dc015aef0 2024-09-10T11:39:23.991 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:23.991 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":171,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:16.793503+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:38:23.632388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T11:38:23.986527+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.0","mappings":[{"from":2,"to":5}]},{"pgid":"21.10","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:23.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 msgr2=0x7f0da406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 0x7f0da406e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc005880 tx=0x7f0dbc009040 comp rx=0 tx=0).stop 2024-09-10T11:39:23.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 msgr2=0x7f0dc0150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:23.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e990 tx=0x7f0db000ee50 comp rx=0 tx=0).stop 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 shutdown_connections 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0da406be30 0x7f0da406e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dc015b2d0 0x7f0dc0150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dc015aef0 0x7f0dc0150120 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 --2- 172.21.15.99:0/3468112770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dc00afbb0 0x7f0dc0154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 >> 172.21.15.99:0/3468112770 conn(0x7f0dc006bc10 msgr2=0x7f0dc006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 shutdown_connections 2024-09-10T11:39:23.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:23.992+0000 7f0dc7c47640 1 -- 172.21.15.99:0/3468112770 wait complete. 2024-09-10T11:39:24.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:39:24.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:24.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.100+0000 7f4ddf3b7640 1 Processor -- start 2024-09-10T11:39:24.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.101+0000 7f4ddf3b7640 1 -- start start 2024-09-10T11:39:24.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.101+0000 7f4ddf3b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4dd80d1200 0x7f4dd8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddf3b7640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4dd806e5d0 0x7f4dd806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddf3b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dd8152e60 con 0x7f4dd815c060 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddf3b7640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd806ef80 con 0x7f4dd806e5d0 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddf3b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4dd80d15e0 con 0x7f4dd80d1200 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4dd806e5d0 0x7f4dd806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4dd806e5d0 0x7f4dd806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48576/0 (socket says 172.21.15.99:48576) 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 -- 172.21.15.99:0/660639387 learned_addr learned my addr 172.21.15.99:0/660639387 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddd153640 1 --2- 172.21.15.99:0/660639387 >> v2:172.21.15.196:3300/0 conn(0x7f4dd80d1200 0x7f4dd8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 -- 172.21.15.99:0/660639387 >> v2:172.21.15.196:3300/0 conn(0x7f4dd80d1200 msgr2=0x7f4dd8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 --2- 172.21.15.99:0/660639387 >> v2:172.21.15.196:3300/0 conn(0x7f4dd80d1200 0x7f4dd8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 -- 172.21.15.99:0/660639387 >> v1:172.21.15.196:6789/0 conn(0x7f4dd815c060 legacy=0x7f4dd815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.102+0000 7f4ddc952640 1 -- 172.21.15.99:0/660639387 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd8072680 con 0x7f4dd806e5d0 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddc952640 1 --2- 172.21.15.99:0/660639387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd806e5d0 0x7f4dd806ea40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80d1850 tx=0x7f4dd402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604b8c297e9129de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4dcbfff640 1 -- 172.21.15.99:0/660639387 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4037070 con 0x7f4dd806e5d0 2024-09-10T11:39:24.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4dcbfff640 1 -- 172.21.15.99:0/660639387 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd4032440 con 0x7f4dd806e5d0 2024-09-10T11:39:24.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4dcbfff640 1 -- 172.21.15.99:0/660639387 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4032810 con 0x7f4dd806e5d0 2024-09-10T11:39:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/660639387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd806e5d0 msgr2=0x7f4dd806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/660639387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd806e5d0 0x7f4dd806ea40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80d1850 tx=0x7f4dd402a1e0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/660639387 shutdown_connections 2024-09-10T11:39:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/660639387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd806e5d0 0x7f4dd806ea40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/660639387 >> v2:172.21.15.196:3300/0 conn(0x7f4dd80d1200 0x7f4dd8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.103+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/660639387 >> 172.21.15.99:0/660639387 conn(0x7f4dd806bc10 msgr2=0x7f4dd806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.104+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/660639387 shutdown_connections 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.104+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/660639387 wait complete. 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 Processor -- start 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 -- start start 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806e5d0 0x7f4dd8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 0x7f4dd81f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8072f20 con 0x7f4dd815c060 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8072c40 con 0x7f4dd80d1200 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddf3b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd815cd90 con 0x7f4dd806e5d0 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddd954640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddd954640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48592/0 (socket says 172.21.15.99:48592) 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.105+0000 7f4ddd954640 1 -- 172.21.15.99:0/146922986 learned_addr learned my addr 172.21.15.99:0/146922986 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd153640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806e5d0 0x7f4dd8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddc952640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 0x7f4dd81f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 -- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806e5d0 msgr2=0x7f4dd8158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806e5d0 0x7f4dd8158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 -- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 msgr2=0x7f4dd81f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 0x7f4dd81f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd81591c0 con 0x7f4dd815c060 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddc952640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 0x7f4dd81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddd954640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c7a0 tx=0x7f4dd000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0003b70 con 0x7f4dd815c060 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dd0003d30 con 0x7f4dd815c060 2024-09-10T11:39:24.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd000aa90 con 0x7f4dd815c060 2024-09-10T11:39:24.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd8156670 con 0x7f4dd815c060 2024-09-10T11:39:24.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.106+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd8156c30 con 0x7f4dd815c060 2024-09-10T11:39:24.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.107+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd806ea40 con 0x7f4dd815c060 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.107+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4dd000ac30 con 0x7f4dd815c060 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.108+0000 7f4dc9ffb640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 0x7f4dbc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.108+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f4dd008ae50 con 0x7f4dd815c060 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.108+0000 7f4ddd153640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 0x7f4dbc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.108+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f4dbc073a20 con 0x7f4dd815c060 2024-09-10T11:39:24.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.108+0000 7f4ddd153640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 0x7f4dbc06e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006fd0 tx=0x7f4dcc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.112+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4dd00597c0 con 0x7f4dd815c060 2024-09-10T11:39:24.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.316+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dd80761b0 con 0x7f4dd815c060 2024-09-10T11:39:24.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.317+0000 7f4dc9ffb640 1 -- 172.21.15.99:0/146922986 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f4dd00590c0 con 0x7f4dd815c060 2024-09-10T11:39:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:24.318 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:39:24.318 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 msgr2=0x7f4dbc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 0x7f4dbc06e2f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc006fd0 tx=0x7f4dcc008040 comp rx=0 tx=0).stop 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 msgr2=0x7f4dd81ee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000c7a0 tx=0x7f4dd000cc60 comp rx=0 tx=0).stop 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 shutdown_connections 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4dbc06be30 0x7f4dbc06e2f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dd815c060 0x7f4dd81ee850 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80d1200 0x7f4dd81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 --2- 172.21.15.99:0/146922986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806e5d0 0x7f4dd8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 >> 172.21.15.99:0/146922986 conn(0x7f4dd806bc10 msgr2=0x7f4dd815c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 shutdown_connections 2024-09-10T11:39:24.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.319+0000 7f4ddf3b7640 1 -- 172.21.15.99:0/146922986 wait complete. 2024-09-10T11:39:24.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:24.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.428+0000 7f2ae1053640 1 Processor -- start 2024-09-10T11:39:24.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 -- start start 2024-09-10T11:39:24.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2adc0d1410 0x7f2adc0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2adc06df80 con 0x7f2adc0cfb90 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2adc0d2120 con 0x7f2adc0d1410 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2ae1053640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc06e260 con 0x7f2adc0b3660 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.429+0000 7f2adb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2adc0d1410 0x7f2adc0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2adc0d1410 0x7f2adc0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48598/0 (socket says 172.21.15.99:48598) 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 -- 172.21.15.99:0/2498924591 learned_addr learned my addr 172.21.15.99:0/2498924591 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2ada59c640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 -- 172.21.15.99:0/2498924591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 msgr2=0x7f2adc0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:24.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 -- 172.21.15.99:0/2498924591 >> v1:172.21.15.196:6789/0 conn(0x7f2adc0cfb90 legacy=0x7f2adc0d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 -- 172.21.15.99:0/2498924591 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc0d1ad0 con 0x7f2adc0d1410 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.430+0000 7f2adb59e640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0cde00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009870 tx=0x7f2acc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5301dc92726a1597 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ad9d9b640 1 -- 172.21.15.99:0/2498924591 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc037070 con 0x7f2adc0d1410 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ad9d9b640 1 -- 172.21.15.99:0/2498924591 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc032440 con 0x7f2adc0d1410 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ad9d9b640 1 -- 172.21.15.99:0/2498924591 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc032810 con 0x7f2adc0d1410 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 -- 172.21.15.99:0/2498924591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 msgr2=0x7f2adc0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0cde00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009870 tx=0x7f2acc02a180 comp rx=0 tx=0).stop 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 -- 172.21.15.99:0/2498924591 shutdown_connections 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0cde00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 --2- 172.21.15.99:0/2498924591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 -- 172.21.15.99:0/2498924591 >> 172.21.15.99:0/2498924591 conn(0x7f2adc0b4650 msgr2=0x7f2adc0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 -- 172.21.15.99:0/2498924591 shutdown_connections 2024-09-10T11:39:24.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.431+0000 7f2ae1053640 1 -- 172.21.15.99:0/2498924591 wait complete. 2024-09-10T11:39:24.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 Processor -- start 2024-09-10T11:39:24.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 -- start start 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 0x7f2adc162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0cff70 con 0x7f2adc0d1410 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc06dd30 con 0x7f2adc0cfb90 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.433+0000 7f2ae1053640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2adc0d00f0 con 0x7f2adc0b3660 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48614/0 (socket says 172.21.15.99:48614) 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 -- 172.21.15.99:0/3128520032 learned_addr learned my addr 172.21.15.99:0/3128520032 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:24.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adad9d640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ada59c640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 0x7f2adc162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 -- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 msgr2=0x7f2adc0b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 -- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 msgr2=0x7f2adc162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 0x7f2adc162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2adc162670 con 0x7f2adc0d1410 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ada59c640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 0x7f2adc162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2adb59e640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009870 tx=0x7f2acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc041070 con 0x7f2adc0d1410 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc03c950 con 0x7f2adc0d1410 2024-09-10T11:39:24.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc037040 con 0x7f2adc0d1410 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2adc0afa60 con 0x7f2adc0d1410 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.434+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2adc0b0020 con 0x7f2adc0d1410 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.435+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2adc0b4a30 con 0x7f2adc0d1410 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.436+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2acc044070 con 0x7f2adc0d1410 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.436+0000 7f2ad37fe640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 0x7f2abc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.436+0000 7f2adad9d640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 0x7f2abc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.436+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f2acc0ab880 con 0x7f2adc0d1410 2024-09-10T11:39:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.436+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f2abc073a80 con 0x7f2adc0d1410 2024-09-10T11:39:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.437+0000 7f2adad9d640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 0x7f2abc06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.440+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2adc0b4a30 con 0x7f2adc0d1410 2024-09-10T11:39:24.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.639+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2adc0b3420 con 0x7f2adc0d1410 2024-09-10T11:39:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.640+0000 7f2ad37fe640 1 -- 172.21.15.99:0/3128520032 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5224 (secure 0 0 0) 0x7f2acc079d50 con 0x7f2adc0d1410 2024-09-10T11:39:24.641 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:24.641 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":67,"btime":"2024-09-10T11:38:27:696435+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:38:25.020679+0000","modified":"2024-09-10T11:38:27.019950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7635},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7635":{"gid":7635,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6835/3322393426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3322393426},{"type":"v1","addr":"172.21.15.99:6835","nonce":3322393426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7635,"qdb_cluster":[7635]},"id":10}]} 2024-09-10T11:39:24.641 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 67 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.641+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 msgr2=0x7f2abc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.641+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 0x7f2abc06e2f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ad4006fd0 tx=0x7f2ad4009060 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.641+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 msgr2=0x7f2adc0b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.641+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009870 tx=0x7f2acc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 shutdown_connections 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2abc06be30 0x7f2abc06e2f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2adc0d1410 0x7f2adc0b76a0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2adc0cfb90 0x7f2adc162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 --2- 172.21.15.99:0/3128520032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2adc0b3660 0x7f2adc0b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 >> 172.21.15.99:0/3128520032 conn(0x7f2adc0b4650 msgr2=0x7f2adc0ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 shutdown_connections 2024-09-10T11:39:24.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.642+0000 7f2ae1053640 1 -- 172.21.15.99:0/3128520032 wait complete. 2024-09-10T11:39:24.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:24.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.751+0000 7f642737f640 1 Processor -- start 2024-09-10T11:39:24.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.751+0000 7f642737f640 1 -- start start 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642737f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6420148400 0x7f642006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642737f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f642006c150 0x7f6420070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642737f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6420162760 con 0x7f6420070ab0 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642737f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6420162a40 con 0x7f6420148400 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642737f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6420162d20 con 0x7f642006c150 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642591c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6420070ab0 0x7f6420072620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44332/0 (socket says 172.21.15.99:44332) 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642591c640 1 -- 172.21.15.99:0/1121580849 learned_addr learned my addr 172.21.15.99:0/1121580849 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642511b640 1 --2- 172.21.15.99:0/1121580849 >> v2:172.21.15.196:3300/0 conn(0x7f6420148400 0x7f642006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.752+0000 7f642491a640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f642006c150 0x7f6420070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f6417fff640 1 -- 172.21.15.99:0/1121580849 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583147833 0 0) 0x7f6420162760 con 0x7f6420070ab0 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f6417fff640 1 -- 172.21.15.99:0/1121580849 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f640c003520 con 0x7f6420070ab0 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f642511b640 1 -- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f642006c150 msgr2=0x7f6420070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f642511b640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f642006c150 0x7f6420070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f642511b640 1 -- 172.21.15.99:0/1121580849 >> v1:172.21.15.99:6789/0 conn(0x7f6420070ab0 legacy=0x7f6420072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.753+0000 7f642511b640 1 -- 172.21.15.99:0/1121580849 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6420163000 con 0x7f6420148400 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642511b640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6420148400 0x7f642006bc10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f641c0097a0 tx=0x7f641c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b1f0554b6b9d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f6417fff640 1 -- 172.21.15.99:0/1121580849 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c037070 con 0x7f6420148400 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f6417fff640 1 -- 172.21.15.99:0/1121580849 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641c032440 con 0x7f6420148400 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f6417fff640 1 -- 172.21.15.99:0/1121580849 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c0327f0 con 0x7f6420148400 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 -- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6420148400 msgr2=0x7f642006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6420148400 0x7f642006bc10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f641c0097a0 tx=0x7f641c02a180 comp rx=0 tx=0).stop 2024-09-10T11:39:24.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 -- 172.21.15.99:0/1121580849 shutdown_connections 2024-09-10T11:39:24.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f642006c150 0x7f6420070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 --2- 172.21.15.99:0/1121580849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6420148400 0x7f642006bc10 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 -- 172.21.15.99:0/1121580849 >> 172.21.15.99:0/1121580849 conn(0x7f64200d1900 msgr2=0x7f64200d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 -- 172.21.15.99:0/1121580849 shutdown_connections 2024-09-10T11:39:24.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.754+0000 7f642737f640 1 -- 172.21.15.99:0/1121580849 wait complete. 2024-09-10T11:39:24.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 Processor -- start 2024-09-10T11:39:24.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 -- start start 2024-09-10T11:39:24.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f642006c150 0x7f64200caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 0x7f6420146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f64200d0700 con 0x7f6420148400 2024-09-10T11:39:24.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f642006c570 con 0x7f642006c150 2024-09-10T11:39:24.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642737f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f64200d0880 con 0x7f6420070ab0 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642591c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642591c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48620/0 (socket says 172.21.15.99:48620) 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.756+0000 7f642591c640 1 -- 172.21.15.99:0/3145559682 learned_addr learned my addr 172.21.15.99:0/3145559682 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642511b640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f642006c150 0x7f64200caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642491a640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 0x7f6420146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 -- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 msgr2=0x7f6420146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 0x7f6420146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 -- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f642006c150 msgr2=0x7f64200caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f642006c150 0x7f64200caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64200c7c70 con 0x7f6420148400 2024-09-10T11:39:24.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642491a640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 0x7f6420146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642591c640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f641800cb40 tx=0x7f641800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418017070 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641800aca0 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642015ff80 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418012620 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.757+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6420160540 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.758+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6420070810 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.759+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f641801a070 con 0x7f6420148400 2024-09-10T11:39:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.759+0000 7f6415ffb640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 0x7f640406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.759+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f641808ac60 con 0x7f6420148400 2024-09-10T11:39:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.759+0000 7f642511b640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 0x7f640406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.759+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f6404073a20 con 0x7f6420148400 2024-09-10T11:39:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.760+0000 7f642511b640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 0x7f640406e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.763+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6420070810 con 0x7f6420148400 2024-09-10T11:39:24.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.931+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6420068dc0 con 0x7f6420148400 2024-09-10T11:39:24.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.932+0000 7f6415ffb640 1 -- 172.21.15.99:0/3145559682 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) ==== 74+0+16863 (secure 0 0 0) 0x7f6420068dc0 con 0x7f6420148400 2024-09-10T11:39:24.933 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:24.933 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":171,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:16.793503+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:38:23.632388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-10T11:38:23.986527+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.0","mappings":[{"from":2,"to":5}]},{"pgid":"21.10","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 msgr2=0x7f640406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 0x7f640406e2f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f641c034000 tx=0x7f641c02e000 comp rx=0 tx=0).stop 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 msgr2=0x7f64200c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f641800cb40 tx=0x7f641800a060 comp rx=0 tx=0).stop 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 shutdown_connections 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f640406be30 0x7f640406e2f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6420148400 0x7f64200c7280 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6420070ab0 0x7f6420146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 --2- 172.21.15.99:0/3145559682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f642006c150 0x7f64200caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:24.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 >> 172.21.15.99:0/3145559682 conn(0x7f64200d1900 msgr2=0x7f64200ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:24.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 shutdown_connections 2024-09-10T11:39:24.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:24.934+0000 7f642737f640 1 -- 172.21.15.99:0/3145559682 wait complete. 2024-09-10T11:39:24.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:39:25.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.045+0000 7f86831fd640 1 Processor -- start 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.045+0000 7f86831fd640 1 -- start start 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.045+0000 7f86831fd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f867c15eb80 0x7f867c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.045+0000 7f86831fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c0d5010 0x7f867c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f86831fd640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c15ff70 con 0x7f867c1626c0 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f86831fd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f867c0d1620 con 0x7f867c15eb80 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f86831fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f867c162bc0 con 0x7f867c0d5010 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f867c15eb80 0x7f867c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f867bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f867c1626c0 0x7f867c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44340/0 (socket says 172.21.15.99:44340) 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f867c15eb80 0x7f867c0d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48624/0 (socket says 172.21.15.99:48624) 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f867bfff640 1 -- 172.21.15.99:0/521748417 learned_addr learned my addr 172.21.15.99:0/521748417 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:25.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 -- 172.21.15.99:0/521748417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c0d5010 msgr2=0x7f867c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 --2- 172.21.15.99:0/521748417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c0d5010 0x7f867c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 -- 172.21.15.99:0/521748417 >> v1:172.21.15.99:6789/0 conn(0x7f867c1626c0 legacy=0x7f867c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f867b7fe640 1 -- 172.21.15.99:0/521748417 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564702261 0 0) 0x7f867c15ff70 con 0x7f867c1626c0 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.046+0000 7f8680f99640 1 -- 172.21.15.99:0/521748417 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c162ea0 con 0x7f867c15eb80 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f8680f99640 1 --2- 172.21.15.99:0/521748417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0d29b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f86740097d0 tx=0x7f867402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1214e44a6adc713c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f867b7fe640 1 -- 172.21.15.99:0/521748417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674037070 con 0x7f867c15eb80 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f867b7fe640 1 -- 172.21.15.99:0/521748417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867402ae40 con 0x7f867c15eb80 2024-09-10T11:39:25.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f867b7fe640 1 -- 172.21.15.99:0/521748417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674032660 con 0x7f867c15eb80 2024-09-10T11:39:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 -- 172.21.15.99:0/521748417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 msgr2=0x7f867c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 --2- 172.21.15.99:0/521748417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0d29b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f86740097d0 tx=0x7f867402a130 comp rx=0 tx=0).stop 2024-09-10T11:39:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 -- 172.21.15.99:0/521748417 shutdown_connections 2024-09-10T11:39:25.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 --2- 172.21.15.99:0/521748417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c0d5010 0x7f867c0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 --2- 172.21.15.99:0/521748417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0d29b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 -- 172.21.15.99:0/521748417 >> 172.21.15.99:0/521748417 conn(0x7f867c08e680 msgr2=0x7f867c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 -- 172.21.15.99:0/521748417 shutdown_connections 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.047+0000 7f86831fd640 1 -- 172.21.15.99:0/521748417 wait complete. 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.048+0000 7f86831fd640 1 Processor -- start 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 -- start start 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 0x7f867c1f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 0x7f867c15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0d2ef0 con 0x7f867c15eb80 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f867c163460 con 0x7f867c0d5010 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f86831fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f867c0d3070 con 0x7f867c1626c0 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.049+0000 7f867bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48628/0 (socket says 172.21.15.99:48628) 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 -- 172.21.15.99:0/2771373067 learned_addr learned my addr 172.21.15.99:0/2771373067 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f8680f99640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 0x7f867c1f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f868179a640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 0x7f867c15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 -- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 msgr2=0x7f867c15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 0x7f867c15bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 -- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 msgr2=0x7f867c1f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 0x7f867c1f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c1f9340 con 0x7f867c15eb80 2024-09-10T11:39:25.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f8680f99640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 0x7f867c1f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f868179a640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 0x7f867c15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f867bfff640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c780 tx=0x7f866c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c018070 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c00a640 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c0b3fe0 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c00aa30 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.050+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c0b2ce0 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.051+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c0d29b0 con 0x7f867c15eb80 2024-09-10T11:39:25.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.052+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f866c01b070 con 0x7f867c15eb80 2024-09-10T11:39:25.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.052+0000 7f86797fa640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 0x7f865c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.052+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5767+0+0 (secure 0 0 0) 0x7f866c08a440 con 0x7f867c15eb80 2024-09-10T11:39:25.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.052+0000 7f8680f99640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 0x7f865c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.052+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=172}) -- 0x7f865c073ab0 con 0x7f867c15eb80 2024-09-10T11:39:25.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.053+0000 7f8680f99640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 0x7f865c06e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f867402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.056+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f866c058db0 con 0x7f867c15eb80 2024-09-10T11:39:25.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.256+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f867c0b5a60 con 0x7f867c15eb80 2024-09-10T11:39:25.257 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:39:25.257+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:39:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.258+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f866c0586b0 con 0x7f867c15eb80 2024-09-10T11:39:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:39:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.259+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(172..172 src has 1..172) ==== 374+0+0 (secure 0 0 0) 0x7f866c0bed50 con 0x7f867c15eb80 2024-09-10T11:39:25.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.259+0000 7f86797fa640 1 -- 172.21.15.99:0/2771373067 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) -- 0x7f865c0743b0 con 0x7f867c15eb80 2024-09-10T11:39:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 msgr2=0x7f865c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 0x7f865c06e320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8674034000 tx=0x7f867402e000 comp rx=0 tx=0).stop 2024-09-10T11:39:25.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 msgr2=0x7f867c0b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f866c00c780 tx=0x7f866c00cc40 comp rx=0 tx=0).stop 2024-09-10T11:39:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 shutdown_connections 2024-09-10T11:39:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f865c06be60 0x7f865c06e320 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f867c1626c0 0x7f867c15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f867c15eb80 0x7f867c0b3aa0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 --2- 172.21.15.99:0/2771373067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f867c0d5010 0x7f867c1f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.262+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 >> 172.21.15.99:0/2771373067 conn(0x7f867c08e680 msgr2=0x7f867c0d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.263+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 shutdown_connections 2024-09-10T11:39:25.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.263+0000 7f86831fd640 1 -- 172.21.15.99:0/2771373067 wait complete. 2024-09-10T11:39:25.265 INFO:tasks.ceph.mds.a.smithi099.stderr: -8340> 2024-09-10T11:38:26.939+0000 7fa7f93ff640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:39:25.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:39:25.297 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:39:25.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.376+0000 7f0d1f6e4640 1 Processor -- start 2024-09-10T11:39:25.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.376+0000 7f0d1f6e4640 1 -- start start 2024-09-10T11:39:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1f6e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d180d1200 0x7f0d18073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1f6e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d1806e5d0 0x7f0d1806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1f6e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d18152e60 con 0x7f0d1815c060 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1f6e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1806ef50 con 0x7f0d1806e5d0 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1f6e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d180d15e0 con 0x7f0d180d1200 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d1806e5d0 0x7f0d1806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0d1806e5d0 0x7f0d1806ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48654/0 (socket says 172.21.15.99:48654) 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1dc81640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0d1815c060 0x7f0d1815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44370/0 (socket says 172.21.15.99:44370) 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/1533623686 learned_addr learned my addr 172.21.15.99:0/1533623686 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1d480640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d180d1200 0x7f0d18073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/1533623686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d180d1200 msgr2=0x7f0d18073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d180d1200 0x7f0d18073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/1533623686 >> v1:172.21.15.99:6789/0 conn(0x7f0d1815c060 legacy=0x7f0d1815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d17fff640 1 -- 172.21.15.99:0/1533623686 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3898719246 0 0) 0x7f0d18152e60 con 0x7f0d1815c060 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.377+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/1533623686 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d18072680 con 0x7f0d1806e5d0 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d1cc7f640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1806e5d0 0x7f0d1806ea10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009870 tx=0x7f0d0c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3882564407d278eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d17fff640 1 -- 172.21.15.99:0/1533623686 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c037070 con 0x7f0d1806e5d0 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d17fff640 1 -- 172.21.15.99:0/1533623686 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c032440 con 0x7f0d1806e5d0 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d17fff640 1 -- 172.21.15.99:0/1533623686 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c032810 con 0x7f0d1806e5d0 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/1533623686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1806e5d0 msgr2=0x7f0d1806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.378+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1806e5d0 0x7f0d1806ea10 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009870 tx=0x7f0d0c02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/1533623686 shutdown_connections 2024-09-10T11:39:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d1806e5d0 0x7f0d1806ea10 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/1533623686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d180d1200 0x7f0d18073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/1533623686 >> 172.21.15.99:0/1533623686 conn(0x7f0d1806bc10 msgr2=0x7f0d1806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/1533623686 shutdown_connections 2024-09-10T11:39:25.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.379+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/1533623686 wait complete. 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 Processor -- start 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 -- start start 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d1806e5d0 0x7f0d18151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 0x7f0d180cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18072f20 con 0x7f0d180d1200 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0d18072c40 con 0x7f0d1806e5d0 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.380+0000 7f0d1f6e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1815cd90 con 0x7f0d1815c060 2024-09-10T11:39:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48660/0 (socket says 172.21.15.99:48660) 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/761398030 learned_addr learned my addr 172.21.15.99:0/761398030 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1d480640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d1806e5d0 0x7f0d18151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1dc81640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 0x7f0d180cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 msgr2=0x7f0d180cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 0x7f0d180cfa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d1806e5d0 msgr2=0x7f0d18151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d1806e5d0 0x7f0d18151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d18167c00 con 0x7f0d180d1200 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1dc81640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 0x7f0d180cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1cc7f640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c037040 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c042070 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d18167ee0 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c030bd0 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.381+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d18152480 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.382+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1806ea10 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.383+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d0c02ad10 con 0x7f0d180d1200 2024-09-10T11:39:25.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.383+0000 7f0d15ffb640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 0x7f0d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.383+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5853+0+0 (secure 0 0 0) 0x7f0d0c0ab7f0 con 0x7f0d180d1200 2024-09-10T11:39:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.383+0000 7f0d1d480640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 0x7f0d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.383+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) -- 0x7f0d04073b30 con 0x7f0d180d1200 2024-09-10T11:39:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.384+0000 7f0d1d480640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 0x7f0d0406e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0d1815bb20 tx=0x7f0d08007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.388+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d0c045360 con 0x7f0d180d1200 2024-09-10T11:39:25.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:25.586+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0d180b45b0 con 0x7f0d180d1200 2024-09-10T11:39:26.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.260+0000 7f0d15ffb640 1 -- 172.21.15.99:0/761398030 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f0d0c07a1a0 con 0x7f0d180d1200 2024-09-10T11:39:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 msgr2=0x7f0d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 0x7f0d0406e2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0d1815bb20 tx=0x7f0d08007900 comp rx=0 tx=0).stop 2024-09-10T11:39:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 msgr2=0x7f0d180cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 shutdown_connections 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0d0406be30 0x7f0d0406e2f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d1815c060 0x7f0d180cfa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d180d1200 0x7f0d180cf4f0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.262+0000 7f0d1f6e4640 1 --2- 172.21.15.99:0/761398030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d1806e5d0 0x7f0d18151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.263+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 >> 172.21.15.99:0/761398030 conn(0x7f0d1806bc10 msgr2=0x7f0d1815dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.263+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 shutdown_connections 2024-09-10T11:39:26.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.263+0000 7f0d1f6e4640 1 -- 172.21.15.99:0/761398030 wait complete. 2024-09-10T11:39:26.276 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:39:26.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.421+0000 7f7b78b11640 1 Processor -- start 2024-09-10T11:39:26.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.422+0000 7f7b78b11640 1 -- start start 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.422+0000 7f7b78b11640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b7415cb70 0x7f7b74072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.422+0000 7f7b78b11640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7b740b4150 0x7f7b740b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b78b11640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b74072650 con 0x7f7b740b0df0 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b78b11640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f7b740b1240 con 0x7f7b7415cb70 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b78b11640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7b7415db70 con 0x7f7b740b4150 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b7415cb70 0x7f7b74072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7b7415cb70 0x7f7b74072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48668/0 (socket says 172.21.15.99:48668) 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 -- 172.21.15.99:0/2343618047 learned_addr learned my addr 172.21.15.99:0/2343618047 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:26.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b71d9b640 1 --2- 172.21.15.99:0/2343618047 >> v2:172.21.15.196:3300/0 conn(0x7f7b740b4150 0x7f7b740b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 -- 172.21.15.99:0/2343618047 >> v2:172.21.15.196:3300/0 conn(0x7f7b740b4150 msgr2=0x7f7b740b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:26.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 --2- 172.21.15.99:0/2343618047 >> v2:172.21.15.196:3300/0 conn(0x7f7b740b4150 0x7f7b740b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 -- 172.21.15.99:0/2343618047 >> v1:172.21.15.196:6789/0 conn(0x7f7b740b0df0 legacy=0x7f7b7415cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:26.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.423+0000 7f7b7259c640 1 -- 172.21.15.99:0/2343618047 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7415de50 con 0x7f7b7415cb70 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b7259c640 1 --2- 172.21.15.99:0/2343618047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b7415cb70 0x7f7b74072de0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7b7406e080 tx=0x7f7b6c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8096828ab050eafa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b7159a640 1 -- 172.21.15.99:0/2343618047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c036070 con 0x7f7b7415cb70 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b7159a640 1 -- 172.21.15.99:0/2343618047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c031600 con 0x7f7b7415cb70 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b7159a640 1 -- 172.21.15.99:0/2343618047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0319d0 con 0x7f7b7415cb70 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 -- 172.21.15.99:0/2343618047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b7415cb70 msgr2=0x7f7b74072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 --2- 172.21.15.99:0/2343618047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b7415cb70 0x7f7b74072de0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7b7406e080 tx=0x7f7b6c02a6f0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 -- 172.21.15.99:0/2343618047 shutdown_connections 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 --2- 172.21.15.99:0/2343618047 >> v2:172.21.15.196:3300/0 conn(0x7f7b740b4150 0x7f7b740b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 --2- 172.21.15.99:0/2343618047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b7415cb70 0x7f7b74072de0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.424+0000 7f7b78b11640 1 -- 172.21.15.99:0/2343618047 >> 172.21.15.99:0/2343618047 conn(0x7f7b740b3630 msgr2=0x7f7b740b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:26.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.425+0000 7f7b78b11640 1 -- 172.21.15.99:0/2343618047 shutdown_connections 2024-09-10T11:39:26.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.425+0000 7f7b78b11640 1 -- 172.21.15.99:0/2343618047 wait complete. 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.426+0000 7f7b78b11640 1 Processor -- start 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.426+0000 7f7b78b11640 1 -- start start 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.426+0000 7f7b78b11640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740b4150 0x7f7b74160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 0x7f7b74160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74073320 con 0x7f7b740b0df0 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7415e130 con 0x7f7b740b4150 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7b740734a0 con 0x7f7b7415cb70 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48670/0 (socket says 172.21.15.99:48670) 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 -- 172.21.15.99:0/512633476 learned_addr learned my addr 172.21.15.99:0/512633476 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b71d9b640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740b4150 0x7f7b74160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b72d9d640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 0x7f7b74160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 -- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 msgr2=0x7f7b74160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 0x7f7b74160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 -- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740b4150 msgr2=0x7f7b74160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740b4150 0x7f7b74160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7415a9f0 con 0x7f7b740b0df0 2024-09-10T11:39:26.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b72d9d640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 0x7f7b74160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b7259c640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c034040 tx=0x7f7b6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c036050 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c042070 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7415ac90 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c009510 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.427+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7415b250 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.428+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b740b08b0 con 0x7f7b740b0df0 2024-09-10T11:39:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.429+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b6c007c50 con 0x7f7b740b0df0 2024-09-10T11:39:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.429+0000 7f7b6affd640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.429+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5853+0+0 (secure 0 0 0) 0x7f7b6c0abd80 con 0x7f7b740b0df0 2024-09-10T11:39:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.429+0000 7f7b71d9b640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.429+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=173}) -- 0x7f7b54073b30 con 0x7f7b740b0df0 2024-09-10T11:39:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.430+0000 7f7b71d9b640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 0x7f7b5406e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b74073050 tx=0x7f7b5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:26.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.434+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b6c07a6a0 con 0x7f7b740b0df0 2024-09-10T11:39:26.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:26.597+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b7406c9f0 con 0x7f7b740b0df0 2024-09-10T11:39:27.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.261+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v173) ==== 181+0+0 (secure 0 0 0) 0x7f7b6c079fa0 con 0x7f7b740b0df0 2024-09-10T11:39:27.262 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:39:27.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.262+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(173..173 src has 1..173) ==== 296+0+0 (secure 0 0 0) 0x7f7b6c0aa330 con 0x7f7b740b0df0 2024-09-10T11:39:27.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.262+0000 7f7b6affd640 1 -- 172.21.15.99:0/512633476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7f7b54071f60 con 0x7f7b740b0df0 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 msgr2=0x7f7b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 0x7f7b5406e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7b74073050 tx=0x7f7b5c008040 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 msgr2=0x7f7b7414e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c034040 tx=0x7f7b6c004a40 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 shutdown_connections 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b5406be30 0x7f7b5406e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b7415cb70 0x7f7b74160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740b4150 0x7f7b74160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 --2- 172.21.15.99:0/512633476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b740b0df0 0x7f7b7414e0f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 >> 172.21.15.99:0/512633476 conn(0x7f7b740b3630 msgr2=0x7f7b7406f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 shutdown_connections 2024-09-10T11:39:27.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.264+0000 7f7b78b11640 1 -- 172.21.15.99:0/512633476 wait complete. 2024-09-10T11:39:27.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:39:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.386+0000 7fa1f873b640 1 Processor -- start 2024-09-10T11:39:27.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.386+0000 7fa1f873b640 1 -- start start 2024-09-10T11:39:27.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f873b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1f015e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f873b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00d1bd0 con 0x7fa1f01610b0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f873b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00752b0 con 0x7fa1f01605c0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f873b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0075620 con 0x7fa1f00ae020 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f5cd6640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa1f01610b0 0x7fa1f015e200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44372/0 (socket says 172.21.15.99:44372) 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f5cd6640 1 -- 172.21.15.99:0/4257484175 learned_addr learned my addr 172.21.15.99:0/4257484175 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1f015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108607274 0 0) 0x7fa1f00d1bd0 con 0x7fa1f01610b0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1dc003520 con 0x7fa1f01610b0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1607676180 0 0) 0x7fa1dc003520 con 0x7fa1f01610b0 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 msgr2=0x7fa1f015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1f015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 >> v1:172.21.15.196:6789/0 conn(0x7fa1f01605c0 legacy=0x7fa1f01609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.387+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00d1eb0 con 0x7fa1f01610b0 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa1e0001670 con 0x7fa1f01610b0 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 >> v1:172.21.15.99:6789/0 conn(0x7fa1f01610b0 legacy=0x7fa1f015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1dc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1dc003f20 0x7fa1dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1dc004d40 0x7fa1dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0075620 con 0x7fa1dc004d40 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc00ba20 con 0x7fa1dc003f20 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc00bc20 con 0x7fa1f00ae020 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1dc004d40 0x7fa1dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f5cd6640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1dc003f20 0x7fa1dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f64d7640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1dc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 msgr2=0x7fa1dc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1f00ae020 0x7fa1dc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1dc003f20 msgr2=0x7fa1dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1dc003f20 0x7fa1dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.388+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/4257484175 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc003520 con 0x7fa1dc004d40 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1dc004d40 0x7fa1dc0051c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009fc0 tx=0x7fa1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673cf07886acde79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8009c10 con 0x7fa1dc004d40 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8036070 con 0x7fa1dc004d40 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f54d5640 1 -- 172.21.15.99:0/4257484175 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8031880 con 0x7fa1dc004d40 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f873b640 1 -- 172.21.15.99:0/4257484175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1dc004d40 msgr2=0x7fa1dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f873b640 1 --2- 172.21.15.99:0/4257484175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1dc004d40 0x7fa1dc0051c0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009fc0 tx=0x7fa1dc005830 comp rx=0 tx=0).stop 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/4257484175 reap_dead start 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f873b640 1 -- 172.21.15.99:0/4257484175 shutdown_connections 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.389+0000 7fa1f873b640 1 -- 172.21.15.99:0/4257484175 >> 172.21.15.99:0/4257484175 conn(0x7fa1f00730c0 msgr2=0x7fa1f00734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.390+0000 7fa1f873b640 1 -- 172.21.15.99:0/4257484175 shutdown_connections 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.390+0000 7fa1f873b640 1 -- 172.21.15.99:0/4257484175 wait complete. 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 Processor -- start 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 -- start start 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1dc004d40 0x7fa1f00cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 0x7fa1f0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0073d00 con 0x7fa1f0152480 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f015a9e0 con 0x7fa1f01f5060 2024-09-10T11:39:27.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f873b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0073e80 con 0x7fa1dc004d40 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f6cd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.391+0000 7fa1f6cd8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48684/0 (socket says 172.21.15.99:48684) 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/3177141263 learned_addr learned my addr 172.21.15.99:0/3177141263 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f64d7640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1dc004d40 0x7fa1f00cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f5cd6640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 0x7fa1f0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1dc004d40 msgr2=0x7fa1f00cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1dc004d40 0x7fa1f00cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 msgr2=0x7fa1f0151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 0x7fa1f0151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00ce380 con 0x7fa1f0152480 2024-09-10T11:39:27.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f5cd6640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 0x7fa1f0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f6cd8640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8033040 tx=0x7fa1e8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8009c10 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8031d30 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00cbad0 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8003bd0 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.392+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f00ce660 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.393+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f00d1ac0 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.394+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa1e8005d10 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.394+0000 7fa1e6ffd640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 0x7fa1d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.394+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 5415+0+0 (secure 0 0 0) 0x7fa1e80ac350 con 0x7fa1f0152480 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.394+0000 7fa1f64d7640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 0x7fa1d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:27.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.394+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=174}) -- 0x7fa1d4072cc0 con 0x7fa1f0152480 2024-09-10T11:39:27.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.395+0000 7fa1f64d7640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 0x7fa1d406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0156580 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:27.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.398+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1e804a060 con 0x7fa1f0152480 2024-09-10T11:39:27.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:27.566+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1f00d0e00 con 0x7fa1f0152480 2024-09-10T11:39:28.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.262+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v174) ==== 169+0+0 (secure 0 0 0) 0x7fa1e807afd0 con 0x7fa1f0152480 2024-09-10T11:39:28.263 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:39:28.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 <== mon.0 v2:172.21.15.99:3300/0 8 ==== osd_map(174..174 src has 1..174) ==== 330+0+0 (secure 0 0 0) 0x7fa1e80e0d50 con 0x7fa1f0152480 2024-09-10T11:39:28.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1e6ffd640 1 -- 172.21.15.99:0/3177141263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=175}) -- 0x7fa1d4071a50 con 0x7fa1f0152480 2024-09-10T11:39:28.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 msgr2=0x7fa1d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 0x7fa1d406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0156580 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2024-09-10T11:39:28.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 msgr2=0x7fa1f00cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.266+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8033040 tx=0x7fa1e8009750 comp rx=0 tx=0).stop 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 shutdown_connections 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa1d406be30 0x7fa1d406e2f0 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0156580 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1f0152480 0x7fa1f00cff20 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1f01f5060 0x7fa1f0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 --2- 172.21.15.99:0/3177141263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1dc004d40 0x7fa1f00cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.268+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 >> 172.21.15.99:0/3177141263 conn(0x7fa1f00730c0 msgr2=0x7fa1f00afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:28.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.271+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 shutdown_connections 2024-09-10T11:39:28.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.272+0000 7fa1f873b640 1 -- 172.21.15.99:0/3177141263 wait complete. 2024-09-10T11:39:28.287 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:39:28.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.388+0000 7f733bfc6640 1 Processor -- start 2024-09-10T11:39:28.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.388+0000 7f733bfc6640 1 -- start start 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f733bfc6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73340d1200 0x7f7334073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f733bfc6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f733415c770 0x7f733406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f733bfc6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7334152e60 con 0x7f733406e5d0 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f733bfc6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f733406ef80 con 0x7f73340d1200 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f733bfc6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f73340d15e0 con 0x7f733415c770 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73340d1200 0x7f7334073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73340d1200 0x7f7334073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48688/0 (socket says 172.21.15.99:48688) 2024-09-10T11:39:28.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 -- 172.21.15.99:0/3275836580 learned_addr learned my addr 172.21.15.99:0/3275836580 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 -- 172.21.15.99:0/3275836580 >> v2:172.21.15.196:3300/0 conn(0x7f733415c770 msgr2=0x7f733406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 --2- 172.21.15.99:0/3275836580 >> v2:172.21.15.196:3300/0 conn(0x7f733415c770 0x7f733406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 -- 172.21.15.99:0/3275836580 >> v1:172.21.15.196:6789/0 conn(0x7f733406e5d0 legacy=0x7f733415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.389+0000 7f7339d62640 1 -- 172.21.15.99:0/3275836580 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334072680 con 0x7f73340d1200 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.390+0000 7f7339d62640 1 --2- 172.21.15.99:0/3275836580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73340d1200 0x7f7334073440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f73340720c0 tx=0x7f733002c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c1890c09152599 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.390+0000 7f7338d60640 1 -- 172.21.15.99:0/3275836580 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330032710 con 0x7f73340d1200 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.390+0000 7f7338d60640 1 -- 172.21.15.99:0/3275836580 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73300328b0 con 0x7f73340d1200 2024-09-10T11:39:28.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.390+0000 7f7338d60640 1 -- 172.21.15.99:0/3275836580 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330032ca0 con 0x7f73340d1200 2024-09-10T11:39:28.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 -- 172.21.15.99:0/3275836580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73340d1200 msgr2=0x7f7334073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 --2- 172.21.15.99:0/3275836580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73340d1200 0x7f7334073440 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f73340720c0 tx=0x7f733002c850 comp rx=0 tx=0).stop 2024-09-10T11:39:28.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 -- 172.21.15.99:0/3275836580 shutdown_connections 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 --2- 172.21.15.99:0/3275836580 >> v2:172.21.15.196:3300/0 conn(0x7f733415c770 0x7f733406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 --2- 172.21.15.99:0/3275836580 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73340d1200 0x7f7334073440 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 -- 172.21.15.99:0/3275836580 >> 172.21.15.99:0/3275836580 conn(0x7f733406bc10 msgr2=0x7f733406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 -- 172.21.15.99:0/3275836580 shutdown_connections 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.391+0000 7f733bfc6640 1 -- 172.21.15.99:0/3275836580 wait complete. 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.392+0000 7f733bfc6640 1 Processor -- start 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.392+0000 7f733bfc6640 1 -- start start 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73340d1200 0x7f73340cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f733415c770 0x7f73340c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7334072f20 con 0x7f733406e5d0 2024-09-10T11:39:28.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7334072c40 con 0x7f73340d1200 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733bfc6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f733415cd90 con 0x7f733415c770 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48694/0 (socket says 172.21.15.99:48694) 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 -- 172.21.15.99:0/2624997638 learned_addr learned my addr 172.21.15.99:0/2624997638 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339561640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73340d1200 0x7f73340cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f733a563640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f733415c770 0x7f73340c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 -- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f733415c770 msgr2=0x7f73340c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f733415c770 0x7f73340c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 -- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73340d1200 msgr2=0x7f73340cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73340d1200 0x7f73340cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 -- 172.21.15.99:0/2624997638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334159a90 con 0x7f733406e5d0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.393+0000 7f7339d62640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f733002c890 tx=0x7f73300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.394+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330044070 con 0x7f733406e5d0 2024-09-10T11:39:28.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.394+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7330003810 con 0x7f733406e5d0 2024-09-10T11:39:28.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.394+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7334159d70 con 0x7f733406e5d0 2024-09-10T11:39:28.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.394+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330003c00 con 0x7f733406e5d0 2024-09-10T11:39:28.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.394+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733415a330 con 0x7f733406e5d0 2024-09-10T11:39:28.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.395+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7334073440 con 0x7f733406e5d0 2024-09-10T11:39:28.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.395+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7330047070 con 0x7f733406e5d0 2024-09-10T11:39:28.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.396+0000 7f732a7fc640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 0x7f731806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.396+0000 7f7339561640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 0x7f731806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.396+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7f73300ade70 con 0x7f733406e5d0 2024-09-10T11:39:28.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.396+0000 7f7339561640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 0x7f731806e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7324006fd0 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.400+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f733007ccd0 con 0x7f733406e5d0 2024-09-10T11:39:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.566+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f733406c190 con 0x7f733406e5d0 2024-09-10T11:39:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.569+0000 7f732a7fc640 1 -- 172.21.15.99:0/2624997638 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f733007c5d0 con 0x7f733406e5d0 2024-09-10T11:39:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 msgr2=0x7f731806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 0x7f731806e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7324006fd0 tx=0x7f7324008040 comp rx=0 tx=0).stop 2024-09-10T11:39:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 msgr2=0x7f73341f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f733002c890 tx=0x7f73300046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 shutdown_connections 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f731806be30 0x7f731806e2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f733415c770 0x7f73340c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73340d1200 0x7f73340cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 --2- 172.21.15.99:0/2624997638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f733406e5d0 0x7f73341f06e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 >> 172.21.15.99:0/2624997638 conn(0x7f733406bc10 msgr2=0x7f733415f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 shutdown_connections 2024-09-10T11:39:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.572+0000 7f733bfc6640 1 -- 172.21.15.99:0/2624997638 wait complete. 2024-09-10T11:39:28.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-09-10T11:39:28.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.683+0000 7f8e6ad9d640 1 Processor -- start 2024-09-10T11:39:28.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.683+0000 7f8e6ad9d640 1 -- start start 2024-09-10T11:39:28.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6ad9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 0x7f8e640cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6ad9d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640d0ba0 con 0x7f8e6415eee0 2024-09-10T11:39:28.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6ad9d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8e640d0e80 con 0x7f8e640d1570 2024-09-10T11:39:28.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6ad9d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e640d1160 con 0x7f8e640b3020 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e69d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 0x7f8e640cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e69d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:49452/0 (socket says 172.21.15.99:49452) 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.684+0000 7f8e69d9b640 1 -- 172.21.15.99:0/511860744 learned_addr learned my addr 172.21.15.99:0/511860744 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e6a59c640 1 -- 172.21.15.99:0/511860744 >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 msgr2=0x7f8e640d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e6a59c640 1 --2- 172.21.15.99:0/511860744 >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e6a59c640 1 -- 172.21.15.99:0/511860744 >> v1:172.21.15.196:6789/0 conn(0x7f8e6415eee0 legacy=0x7f8e640d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e6a59c640 1 -- 172.21.15.99:0/511860744 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e640b1810 con 0x7f8e640b3020 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e69d9b640 1 --2- 172.21.15.99:0/511860744 >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:28.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.685+0000 7f8e68d99640 1 -- 172.21.15.99:0/511860744 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016353837 0 0) 0x7f8e640d0ba0 con 0x7f8e6415eee0 2024-09-10T11:39:28.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6a59c640 1 --2- 172.21.15.99:0/511860744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 0x7f8e640cf590 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009d90 tx=0x7f8e5c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2874ee0f87a3449 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e68d99640 1 -- 172.21.15.99:0/511860744 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c007740 con 0x7f8e640b3020 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e68d99640 1 -- 172.21.15.99:0/511860744 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e5c0078e0 con 0x7f8e640b3020 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e68d99640 1 -- 172.21.15.99:0/511860744 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c007cb0 con 0x7f8e640b3020 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/511860744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 msgr2=0x7f8e640cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/511860744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 0x7f8e640cf590 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009d90 tx=0x7f8e5c02a660 comp rx=0 tx=0).stop 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/511860744 shutdown_connections 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/511860744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640b3020 0x7f8e640cf590 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/511860744 >> v2:172.21.15.196:3300/0 conn(0x7f8e640d1570 0x7f8e640d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/511860744 >> 172.21.15.99:0/511860744 conn(0x7f8e6409de40 msgr2=0x7f8e640d4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/511860744 shutdown_connections 2024-09-10T11:39:28.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.686+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/511860744 wait complete. 2024-09-10T11:39:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 Processor -- start 2024-09-10T11:39:28.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 -- start start 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 0x7f8e641f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 0x7f8e640cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640cfad0 con 0x7f8e640b3020 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640b3e80 con 0x7f8e6415eee0 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e6ad9d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e640cfc50 con 0x7f8e640d1570 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e69d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e69d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48706/0 (socket says 172.21.15.99:48706) 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.688+0000 7f8e69d9b640 1 -- 172.21.15.99:0/1837123918 learned_addr learned my addr 172.21.15.99:0/1837123918 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6a59c640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 0x7f8e640cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6959a640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 0x7f8e641f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 -- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 msgr2=0x7f8e641f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 0x7f8e641f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 -- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 msgr2=0x7f8e640cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 0x7f8e640cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 -- 172.21.15.99:0/1837123918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641f5ba0 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6959a640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 0x7f8e641f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6a59c640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 0x7f8e640cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e69d9b640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000e9d0 tx=0x7f8e6000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e60017070 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e6000ca60 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e640cc270 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6000ce10 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.689+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6415f8c0 con 0x7f8e640b3020 2024-09-10T11:39:28.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.690+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e640cf590 con 0x7f8e640b3020 2024-09-10T11:39:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.691+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e6001a070 con 0x7f8e640b3020 2024-09-10T11:39:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.691+0000 7f8e5a7fc640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 0x7f8e4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.691+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7f8e6008aa70 con 0x7f8e640b3020 2024-09-10T11:39:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.691+0000 7f8e6959a640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 0x7f8e4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:28.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.692+0000 7f8e6959a640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 0x7f8e4806e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8e641f8b50 tx=0x7f8e54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:28.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.695+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e600597b0 con 0x7f8e640b3020 2024-09-10T11:39:28.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:28.924+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7f8e640b5f10 con 0x7f8e640b3020 2024-09-10T11:39:29.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.274+0000 7f8e5a7fc640 1 -- 172.21.15.99:0/1837123918 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v431) ==== 131+0+0 (secure 0 0 0) 0x7f8e600590b0 con 0x7f8e640b3020 2024-09-10T11:39:29.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 msgr2=0x7f8e4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 0x7f8e4806e2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8e641f8b50 tx=0x7f8e54009040 comp rx=0 tx=0).stop 2024-09-10T11:39:29.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 msgr2=0x7f8e640ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000e9d0 tx=0x7f8e6000ee90 comp rx=0 tx=0).stop 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 shutdown_connections 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e4806be30 0x7f8e4806e2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e6415eee0 0x7f8e640cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e640d1570 0x7f8e641f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 --2- 172.21.15.99:0/1837123918 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e640b3020 0x7f8e640ccee0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 >> 172.21.15.99:0/1837123918 conn(0x7f8e6409de40 msgr2=0x7f8e6406db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 shutdown_connections 2024-09-10T11:39:29.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.276+0000 7f8e6ad9d640 1 -- 172.21.15.99:0/1837123918 wait complete. 2024-09-10T11:39:29.289 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-09-10T11:39:29.289 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-09-10T11:39:29.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:29.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.387+0000 7f8f973e4640 1 Processor -- start 2024-09-10T11:39:29.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.387+0000 7f8f973e4640 1 -- start start 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f973e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f9006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f973e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f90162760 con 0x7f8f9006c150 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f973e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f90162a40 con 0x7f8f90070c80 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f973e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f90162d20 con 0x7f8f90148400 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f9497f640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8f9006c150 0x7f8f90070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44384/0 (socket says 172.21.15.99:44384) 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.388+0000 7f8f9497f640 1 -- 172.21.15.99:0/1124396306 learned_addr learned my addr 172.21.15.99:0/1124396306 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f95180640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f9006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1325565602 0 0) 0x7f8f90162760 con 0x7f8f9006c150 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f7c003520 con 0x7f8f9006c150 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4084299004 0 0) 0x7f8f7c003520 con 0x7f8f9006c150 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 msgr2=0x7f8f9006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f9006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 >> v1:172.21.15.196:6789/0 conn(0x7f8f90070c80 legacy=0x7f8f900cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f90163000 con 0x7f8f9006c150 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8f80001670 con 0x7f8f9006c150 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 >> v1:172.21.15.99:6789/0 conn(0x7f8f9006c150 legacy=0x7f8f90070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f7c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f7c004d40 0x7f8f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90162d20 con 0x7f8f7c003f20 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c00ba20 con 0x7f8f7c004d40 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c00bc20 con 0x7f8f90148400 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f95981640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f7c004d40 0x7f8f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f95180640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f7c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 -- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 msgr2=0x7f8f7c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f90148400 0x7f8f7c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 -- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f7c004d40 msgr2=0x7f8f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f7c004d40 0x7f8f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.389+0000 7f8f9497f640 1 -- 172.21.15.99:0/1124396306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c003520 con 0x7f8f7c003f20 2024-09-10T11:39:29.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.390+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f7c004800 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8f80007fd0 tx=0x7f8f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f91e03e05ec98ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.390+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80007d50 con 0x7f8f7c003f20 2024-09-10T11:39:29.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.390+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f80005af0 con 0x7f8f7c003f20 2024-09-10T11:39:29.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.390+0000 7f8f87fff640 1 -- 172.21.15.99:0/1124396306 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80007070 con 0x7f8f7c003f20 2024-09-10T11:39:29.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 -- 172.21.15.99:0/1124396306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 msgr2=0x7f8f7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1124396306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f7c004800 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8f80007fd0 tx=0x7f8f7c005830 comp rx=0 tx=0).stop 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f95981640 1 -- 172.21.15.99:0/1124396306 reap_dead start 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 -- 172.21.15.99:0/1124396306 shutdown_connections 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 -- 172.21.15.99:0/1124396306 >> 172.21.15.99:0/1124396306 conn(0x7f8f900d1900 msgr2=0x7f8f900d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 -- 172.21.15.99:0/1124396306 shutdown_connections 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.391+0000 7f8f973e4640 1 -- 172.21.15.99:0/1124396306 wait complete. 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.392+0000 7f8f973e4640 1 Processor -- start 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.392+0000 7f8f973e4640 1 -- start start 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f90133fd0 0x7f8f90135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f900cdb20 0x7f8f900b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9006bd90 con 0x7f8f7c003f20 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9006bc10 con 0x7f8f90133fd0 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f973e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9006bf10 con 0x7f8f900cdb20 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48722/0 (socket says 172.21.15.99:48722) 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 -- 172.21.15.99:0/1151917199 learned_addr learned my addr 172.21.15.99:0/1151917199 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95981640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f900cdb20 0x7f8f900b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f90133fd0 0x7f8f90135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 -- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f900cdb20 msgr2=0x7f8f900b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f900cdb20 0x7f8f900b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 -- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f90133fd0 msgr2=0x7f8f90135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f90133fd0 0x7f8f90135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 -- 172.21.15.99:0/1151917199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f901347b0 con 0x7f8f7c003f20 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f95180640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0119a0 tx=0x7f8f8c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00faf0 con 0x7f8f7c003f20 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.393+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00fc90 con 0x7f8f7c003f20 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.394+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c014440 con 0x7f8f7c003f20 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.394+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f90134a90 con 0x7f8f7c003f20 2024-09-10T11:39:29.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.394+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f90135050 con 0x7f8f7c003f20 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.395+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f90070570 con 0x7f8f7c003f20 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.395+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f8c012040 con 0x7f8f7c003f20 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.396+0000 7f8f85ffb640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 0x7f8f7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.396+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 0x7f8f7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.396+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7f8f8c08b050 con 0x7f8f7c003f20 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.396+0000 7f8f9497f640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 0x7f8f7406e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f80032000 tx=0x7f8f8002d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.400+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f8c019030 con 0x7f8f7c003f20 2024-09-10T11:39:29.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.568+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f900cfc90 con 0x7f8f7c003f20 2024-09-10T11:39:29.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.569+0000 7f8f85ffb640 1 -- 172.21.15.99:0/1151917199 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v174) ==== 74+0+13043 (secure 0 0 0) 0x7f8f8c059740 con 0x7f8f7c003f20 2024-09-10T11:39:29.569 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:29.570 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":174,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:28.261887+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":170,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6835/3322393426":"2024-09-11T11:39:25.257343+0000","172.21.15.99:6834/3322393426":"2024-09-11T11:39:25.257343+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 msgr2=0x7f8f7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 0x7f8f7406e320 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8f80032000 tx=0x7f8f8002d000 comp rx=0 tx=0).stop 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 msgr2=0x7f8f900b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0119a0 tx=0x7f8f8c011e60 comp rx=0 tx=0).stop 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 shutdown_connections 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8f7406be60 0x7f8f7406e320 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f900cdb20 0x7f8f900b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f90133fd0 0x7f8f90135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 --2- 172.21.15.99:0/1151917199 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f7c003f20 0x7f8f900b2140 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 >> 172.21.15.99:0/1151917199 conn(0x7f8f900d1900 msgr2=0x7f8f900b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 shutdown_connections 2024-09-10T11:39:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.571+0000 7f8f973e4640 1 -- 172.21.15.99:0/1151917199 wait complete. 2024-09-10T11:39:29.583 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:30:16.935517+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:39:29.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T11:39:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.683+0000 7f8e90111640 1 Processor -- start 2024-09-10T11:39:29.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 -- start start 2024-09-10T11:39:29.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e88073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e8806e5d0 0x7f8e8806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e8815c060 0x7f8e8815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8806ef50 con 0x7f8e8815c060 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8e88152e60 con 0x7f8e8806e5d0 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.684+0000 7f8e90111640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e880d15e0 con 0x7f8e880d1200 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e8815c060 0x7f8e8815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8e8815c060 0x7f8e8815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48730/0 (socket says 172.21.15.99:48730) 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 -- 172.21.15.99:0/893981056 learned_addr learned my addr 172.21.15.99:0/893981056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8d6ac640 1 --2- 172.21.15.99:0/893981056 >> v2:172.21.15.196:3300/0 conn(0x7f8e8806e5d0 0x7f8e8806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8dead640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e88073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 -- 172.21.15.99:0/893981056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 msgr2=0x7f8e88073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e88073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 -- 172.21.15.99:0/893981056 >> v2:172.21.15.196:3300/0 conn(0x7f8e8806e5d0 msgr2=0x7f8e8806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 --2- 172.21.15.99:0/893981056 >> v2:172.21.15.196:3300/0 conn(0x7f8e8806e5d0 0x7f8e8806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.685+0000 7f8e8e6ae640 1 -- 172.21.15.99:0/893981056 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e88072680 con 0x7f8e8815c060 2024-09-10T11:39:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e8e6ae640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8815c060 0x7f8e8815c4a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806f8e0 tx=0x7f8e8002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa081583695c551e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e8ceab640 1 -- 172.21.15.99:0/893981056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80036070 con 0x7f8e8815c060 2024-09-10T11:39:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e8ceab640 1 -- 172.21.15.99:0/893981056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80031600 con 0x7f8e8815c060 2024-09-10T11:39:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e8ceab640 1 -- 172.21.15.99:0/893981056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e800319d0 con 0x7f8e8815c060 2024-09-10T11:39:29.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 -- 172.21.15.99:0/893981056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8815c060 msgr2=0x7f8e8815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8815c060 0x7f8e8815c4a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8e8806f8e0 tx=0x7f8e8002a610 comp rx=0 tx=0).stop 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 -- 172.21.15.99:0/893981056 shutdown_connections 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8815c060 0x7f8e8815c4a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 --2- 172.21.15.99:0/893981056 >> v2:172.21.15.196:3300/0 conn(0x7f8e8806e5d0 0x7f8e8806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 --2- 172.21.15.99:0/893981056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e88073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 -- 172.21.15.99:0/893981056 >> 172.21.15.99:0/893981056 conn(0x7f8e8806bc10 msgr2=0x7f8e8806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 -- 172.21.15.99:0/893981056 shutdown_connections 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.686+0000 7f8e90111640 1 -- 172.21.15.99:0/893981056 wait complete. 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 Processor -- start 2024-09-10T11:39:29.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 -- start start 2024-09-10T11:39:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e881565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e8815c060 0x7f8e8815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e88072f20 con 0x7f8e8806e5d0 2024-09-10T11:39:29.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e88072c40 con 0x7f8e8815c060 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e90111640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8815cd90 con 0x7f8e880d1200 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.688+0000 7f8e8dead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48732/0 (socket says 172.21.15.99:48732) 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 -- 172.21.15.99:0/2789759116 learned_addr learned my addr 172.21.15.99:0/2789759116 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8d6ac640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e881565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8e6ae640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e8815c060 0x7f8e8815e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 -- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 msgr2=0x7f8e881565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e881565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 -- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e8815c060 msgr2=0x7f8e8815e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e8815c060 0x7f8e8815e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 -- 172.21.15.99:0/2789759116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e881575e0 con 0x7f8e8806e5d0 2024-09-10T11:39:29.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8d6ac640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e881565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e8dead640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400e870 tx=0x7f8e8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84017070 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e8400c940 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e88157020 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8400cd30 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.689+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8815a690 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.690+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e8806ea10 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.691+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e8401a070 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.691+0000 7f8e7e7fc640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.691+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7f8e8408a920 con 0x7f8e8806e5d0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.691+0000 7f8e8d6ac640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.692+0000 7f8e8d6ac640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e8815ee00 tx=0x7f8e78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.695+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e84059660 con 0x7f8e8806e5d0 2024-09-10T11:39:29.815 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:39:29.818 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:39:29.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.861+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8e880b4bb0 con 0x7f8e8806e5d0 2024-09-10T11:39:29.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.862+0000 7f8e7e7fc640 1 -- 172.21.15.99:0/2789759116 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v174) ==== 93+0+10 (secure 0 0 0) 0x7f8e84058f60 con 0x7f8e8806e5d0 2024-09-10T11:39:29.863 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 msgr2=0x7f8e6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e8815ee00 tx=0x7f8e78008040 comp rx=0 tx=0).stop 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 msgr2=0x7f8e88151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400e870 tx=0x7f8e8400ed30 comp rx=0 tx=0).stop 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 shutdown_connections 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8e6c06be30 0x7f8e6c06e2f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e8815c060 0x7f8e8815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e880d1200 0x7f8e881565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 --2- 172.21.15.99:0/2789759116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e8806e5d0 0x7f8e88151b50 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 >> 172.21.15.99:0/2789759116 conn(0x7f8e8806bc10 msgr2=0x7f8e880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 shutdown_connections 2024-09-10T11:39:29.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.864+0000 7f8e90111640 1 -- 172.21.15.99:0/2789759116 wait complete. 2024-09-10T11:39:29.875 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-09-10T11:39:29.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.973+0000 7fecb5e1e640 1 Processor -- start 2024-09-10T11:39:29.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.973+0000 7fecb5e1e640 1 -- start start 2024-09-10T11:39:29.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecb5e1e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fecb00d1200 0x7fecb0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecb5e1e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb015c770 0x7fecb006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecb5e1e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecb0152e60 con 0x7fecb006e5d0 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecb5e1e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fecb006ef80 con 0x7fecb00d1200 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecb5e1e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fecb00d15e0 con 0x7fecb015c770 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fecb00d1200 0x7fecb0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaeffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fecb006e5d0 0x7fecb015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44398/0 (socket says 172.21.15.99:44398) 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fecb00d1200 0x7fecb0073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48748/0 (socket says 172.21.15.99:48748) 2024-09-10T11:39:29.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaeffd640 1 -- 172.21.15.99:0/2423498126 learned_addr learned my addr 172.21.15.99:0/2423498126 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaffff640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb015c770 0x7fecb006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 -- 172.21.15.99:0/2423498126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb015c770 msgr2=0x7fecb006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb015c770 0x7fecb006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 -- 172.21.15.99:0/2423498126 >> v1:172.21.15.99:6789/0 conn(0x7fecb006e5d0 legacy=0x7fecb015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecae7fc640 1 -- 172.21.15.99:0/2423498126 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2706837737 0 0) 0x7fecb0152e60 con 0x7fecb006e5d0 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.974+0000 7fecaf7fe640 1 -- 172.21.15.99:0/2423498126 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb0072680 con 0x7fecb00d1200 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.975+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb00d1200 0x7fecb0073440 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feca00097a0 tx=0x7feca002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50220ab4958af78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.975+0000 7fecae7fc640 1 -- 172.21.15.99:0/2423498126 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0037070 con 0x7fecb00d1200 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.975+0000 7fecae7fc640 1 -- 172.21.15.99:0/2423498126 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca002ae50 con 0x7fecb00d1200 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.975+0000 7fecae7fc640 1 -- 172.21.15.99:0/2423498126 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca0032660 con 0x7fecb00d1200 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 -- 172.21.15.99:0/2423498126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb00d1200 msgr2=0x7fecb0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb00d1200 0x7fecb0073440 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feca00097a0 tx=0x7feca002a100 comp rx=0 tx=0).stop 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 -- 172.21.15.99:0/2423498126 shutdown_connections 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb015c770 0x7fecb006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/2423498126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb00d1200 0x7fecb0073440 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 -- 172.21.15.99:0/2423498126 >> 172.21.15.99:0/2423498126 conn(0x7fecb006bc10 msgr2=0x7fecb006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 -- 172.21.15.99:0/2423498126 shutdown_connections 2024-09-10T11:39:29.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.976+0000 7fecb5e1e640 1 -- 172.21.15.99:0/2423498126 wait complete. 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.977+0000 7fecb5e1e640 1 Processor -- start 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.977+0000 7fecb5e1e640 1 -- start start 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 0x7fecb015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 0x7fecb0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0072f20 con 0x7fecb015c770 2024-09-10T11:39:29.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecb0072c40 con 0x7fecb006e5d0 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fecb015cd90 con 0x7fecb00d1200 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48754/0 (socket says 172.21.15.99:48754) 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 -- 172.21.15.99:0/3820880249 learned_addr learned my addr 172.21.15.99:0/3820880249 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaeffd640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 0x7fecb0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 0x7fecb015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 -- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 msgr2=0x7fecb0150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 0x7fecb0150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 -- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 msgr2=0x7fecb015a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 0x7fecb015a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 -- 172.21.15.99:0/3820880249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb01523a0 con 0x7fecb015c770 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaeffd640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 0x7fecb0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 0x7fecb015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecaffff640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feca8009dd0 tx=0x7feca800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8018020 con 0x7fecb015c770 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8014480 con 0x7fecb015c770 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8014870 con 0x7fecb015c770 2024-09-10T11:39:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.978+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb0139d30 con 0x7fecb015c770 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.979+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb013a2f0 con 0x7fecb015c770 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.979+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb013a4f0 con 0x7fecb015c770 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.980+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feca801b020 con 0x7fecb015c770 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.980+0000 7fec97fff640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 0x7fec9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.980+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7feca808ba50 con 0x7fecb015c770 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.981+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 0x7fec9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.981+0000 7fecaf7fe640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 0x7fec9c06e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feca0034000 tx=0x7feca002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:29.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:29.985+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fecb013a4f0 con 0x7fecb015c770 2024-09-10T11:39:30.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.215+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7fecb0073440 con 0x7fecb015c770 2024-09-10T11:39:30.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.275+0000 7fec97fff640 1 -- 172.21.15.99:0/3820880249 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v432) ==== 138+0+0 (secure 0 0 0) 0x7feca8022020 con 0x7fecb015c770 2024-09-10T11:39:30.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.277+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 msgr2=0x7fec9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.277+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 0x7fec9c06e2f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7feca0034000 tx=0x7feca002e000 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.277+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 msgr2=0x7fecb0151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.277+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feca8009dd0 tx=0x7feca800c490 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 shutdown_connections 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fec9c06be30 0x7fec9c06e2f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecb015c770 0x7fecb0151b70 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecb00d1200 0x7fecb0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 --2- 172.21.15.99:0/3820880249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecb006e5d0 0x7fecb015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 >> 172.21.15.99:0/3820880249 conn(0x7fecb006bc10 msgr2=0x7fecb006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 shutdown_connections 2024-09-10T11:39:30.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.278+0000 7fecb5e1e640 1 -- 172.21.15.99:0/3820880249 wait complete. 2024-09-10T11:39:30.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T11:39:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.390+0000 7fe4912a5640 1 Processor -- start 2024-09-10T11:39:30.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.390+0000 7fe4912a5640 1 -- start start 2024-09-10T11:39:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4912a5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe48c1626c0 0x7fe48c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4912a5640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe48c0d4e40 0x7fe48c0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4912a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe48c15ff70 con 0x7fe48c15eb80 2024-09-10T11:39:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4912a5640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe48c0d30c0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4912a5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe48c0d1620 con 0x7fe48c1626c0 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe48c0d4e40 0x7fe48c0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe48c0d4e40 0x7fe48c0d1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48760/0 (socket says 172.21.15.99:48760) 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 -- 172.21.15.99:0/2884745376 learned_addr learned my addr 172.21.15.99:0/2884745376 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe4827fc640 1 --2- 172.21.15.99:0/2884745376 >> v2:172.21.15.196:3300/0 conn(0x7fe48c1626c0 0x7fe48c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 -- 172.21.15.99:0/2884745376 >> v2:172.21.15.196:3300/0 conn(0x7fe48c1626c0 msgr2=0x7fe48c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 --2- 172.21.15.99:0/2884745376 >> v2:172.21.15.196:3300/0 conn(0x7fe48c1626c0 0x7fe48c0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 -- 172.21.15.99:0/2884745376 >> v1:172.21.15.196:6789/0 conn(0x7fe48c15eb80 legacy=0x7fe48c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.391+0000 7fe48b7fe640 1 -- 172.21.15.99:0/2884745376 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c162bc0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.392+0000 7fe48b7fe640 1 --2- 172.21.15.99:0/2884745376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0d1b20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe484009e50 tx=0x7fe48402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e61705ec4ac9048 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.392+0000 7fe48a7fc640 1 -- 172.21.15.99:0/2884745376 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484036070 con 0x7fe48c0d4e40 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.392+0000 7fe48a7fc640 1 -- 172.21.15.99:0/2884745376 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe484031600 con 0x7fe48c0d4e40 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.392+0000 7fe48a7fc640 1 -- 172.21.15.99:0/2884745376 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840319d0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 -- 172.21.15.99:0/2884745376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 msgr2=0x7fe48c0d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 --2- 172.21.15.99:0/2884745376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0d1b20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe484009e50 tx=0x7fe48402a6d0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 -- 172.21.15.99:0/2884745376 shutdown_connections 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 --2- 172.21.15.99:0/2884745376 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0d1b20 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 --2- 172.21.15.99:0/2884745376 >> v2:172.21.15.196:3300/0 conn(0x7fe48c1626c0 0x7fe48c0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 -- 172.21.15.99:0/2884745376 >> 172.21.15.99:0/2884745376 conn(0x7fe48c08e680 msgr2=0x7fe48c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 -- 172.21.15.99:0/2884745376 shutdown_connections 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.393+0000 7fe4912a5640 1 -- 172.21.15.99:0/2884745376 wait complete. 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.394+0000 7fe4912a5640 1 Processor -- start 2024-09-10T11:39:30.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.394+0000 7fe4912a5640 1 -- start start 2024-09-10T11:39:30.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 0x7fe48c1f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c1626c0 0x7fe48c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c163460 con 0x7fe48c0d4e40 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c163180 con 0x7fe48c1626c0 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4912a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe48c0d5290 con 0x7fe48c15eb80 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48762/0 (socket says 172.21.15.99:48762) 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 -- 172.21.15.99:0/4233664657 learned_addr learned my addr 172.21.15.99:0/4233664657 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48b7fe640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c1626c0 0x7fe48c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4827fc640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 0x7fe48c1f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 -- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 msgr2=0x7fe48c1f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 0x7fe48c1f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 -- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c1626c0 msgr2=0x7fe48c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c1626c0 0x7fe48c15bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 -- 172.21.15.99:0/4233664657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c1f5630 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe4827fc640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 0x7fe48c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe48affd640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c3c0 tx=0x7fe47c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.395+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0052d0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.396+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47c005490 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.396+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c007260 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.396+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c0b52e0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.396+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c1f5bb0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.397+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48c0d4900 con 0x7fe48c0d4e40 2024-09-10T11:39:30.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.397+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe47c007400 con 0x7fe48c0d4e40 2024-09-10T11:39:30.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.397+0000 7fe483fff640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 0x7fe46c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.397+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7fe47c089f70 con 0x7fe48c0d4e40 2024-09-10T11:39:30.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.397+0000 7fe4827fc640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 0x7fe46c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.398+0000 7fe4827fc640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 0x7fe46c06e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe474006fd0 tx=0x7fe474008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.402+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe48c1f5630 con 0x7fe48c0d4e40 2024-09-10T11:39:30.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.569+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe48c068440 con 0x7fe48c0d4e40 2024-09-10T11:39:30.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.572+0000 7fe483fff640 1 -- 172.21.15.99:0/4233664657 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fe47c0585d0 con 0x7fe48c0d4e40 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 msgr2=0x7fe46c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 0x7fe46c06e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe474006fd0 tx=0x7fe474008040 comp rx=0 tx=0).stop 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 msgr2=0x7fe48c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c3c0 tx=0x7fe47c00c880 comp rx=0 tx=0).stop 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 shutdown_connections 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe46c06be30 0x7fe46c06e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe48c1626c0 0x7fe48c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe48c15eb80 0x7fe48c1f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 --2- 172.21.15.99:0/4233664657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe48c0d4e40 0x7fe48c0b4d00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 >> 172.21.15.99:0/4233664657 conn(0x7fe48c08e680 msgr2=0x7fe48c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 shutdown_connections 2024-09-10T11:39:30.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.574+0000 7fe4912a5640 1 -- 172.21.15.99:0/4233664657 wait complete. 2024-09-10T11:39:30.586 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:39:30.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:39:30.629 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:39:30.669 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:39:30.669 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:30.669 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:30.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:39:30.685 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:39:30.685 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:30.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:39:30.726 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:30.726 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:30.742 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:39:30.742 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:39:30.742 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:30.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:39:30.792 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T11:39:30.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.890+0000 7fc595e3c640 1 Processor -- start 2024-09-10T11:39:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.890+0000 7fc595e3c640 1 -- start start 2024-09-10T11:39:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5900d1200 0x7fc590073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc59006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc59015c060 0x7fc59015c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fc59006ef50 con 0x7fc59015c060 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc590152e60 con 0x7fc5900d1200 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc595e3c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc5900d15e0 con 0x7fc59006e5d0 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc59015c060 0x7fc59015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc59015c060 0x7fc59015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48770/0 (socket says 172.21.15.99:48770) 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58ffff640 1 -- 172.21.15.99:0/342813079 learned_addr learned my addr 172.21.15.99:0/342813079 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58f7fe640 1 --2- 172.21.15.99:0/342813079 >> v2:172.21.15.196:3300/0 conn(0x7fc5900d1200 0x7fc590073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58effd640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc59006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58ffff640 1 -- 172.21.15.99:0/342813079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 msgr2=0x7fc59006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.891+0000 7fc58ffff640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc59006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58ffff640 1 -- 172.21.15.99:0/342813079 >> v2:172.21.15.196:3300/0 conn(0x7fc5900d1200 msgr2=0x7fc590073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58ffff640 1 --2- 172.21.15.99:0/342813079 >> v2:172.21.15.196:3300/0 conn(0x7fc5900d1200 0x7fc590073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58ffff640 1 -- 172.21.15.99:0/342813079 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc590072680 con 0x7fc59015c060 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58effd640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc59006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58ffff640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59015c060 0x7fc59015c480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc5900d1850 tx=0x7fc58802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f402276d22cff8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58e7fc640 1 -- 172.21.15.99:0/342813079 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588036070 con 0x7fc59015c060 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58e7fc640 1 -- 172.21.15.99:0/342813079 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc588031600 con 0x7fc59015c060 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.892+0000 7fc58e7fc640 1 -- 172.21.15.99:0/342813079 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880319d0 con 0x7fc59015c060 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 -- 172.21.15.99:0/342813079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59015c060 msgr2=0x7fc59015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59015c060 0x7fc59015c480 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc5900d1850 tx=0x7fc58802a610 comp rx=0 tx=0).stop 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 -- 172.21.15.99:0/342813079 shutdown_connections 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc59015c060 0x7fc59015c480 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 --2- 172.21.15.99:0/342813079 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc59006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 --2- 172.21.15.99:0/342813079 >> v2:172.21.15.196:3300/0 conn(0x7fc5900d1200 0x7fc590073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 -- 172.21.15.99:0/342813079 >> 172.21.15.99:0/342813079 conn(0x7fc59006bc10 msgr2=0x7fc59006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 -- 172.21.15.99:0/342813079 shutdown_connections 2024-09-10T11:39:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.893+0000 7fc595e3c640 1 -- 172.21.15.99:0/342813079 wait complete. 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.894+0000 7fc595e3c640 1 Processor -- start 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.894+0000 7fc595e3c640 1 -- start start 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc590143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59015c060 0x7fc5901587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc590072f20 con 0x7fc5900d1200 2024-09-10T11:39:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc590072c40 con 0x7fc59015c060 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc595e3c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc59015cd90 con 0x7fc59006e5d0 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48776/0 (socket says 172.21.15.99:48776) 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 -- 172.21.15.99:0/3744460749 learned_addr learned my addr 172.21.15.99:0/3744460749 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58f7fe640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc590143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58ffff640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59015c060 0x7fc5901587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 -- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 msgr2=0x7fc590143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc590143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 -- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59015c060 msgr2=0x7fc5901587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59015c060 0x7fc5901587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.895+0000 7fc58effd640 1 -- 172.21.15.99:0/3744460749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc590143740 con 0x7fc5900d1200 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc58effd640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d7d0 tx=0x7fc58400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc584004030 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5840041d0 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59015aa80 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58400b6c0 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.896+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59015b040 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.897+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59015e900 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.897+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc58400b860 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.898+0000 7fc577fff640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.898+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(174..174 src has 1..174) ==== 4943+0+0 (secure 0 0 0) 0x7fc58408aca0 con 0x7fc5900d1200 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.898+0000 7fc58f7fe640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.898+0000 7fc58f7fe640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 0x7fc57c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc580009a20 tx=0x7fc580009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:30.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:30.902+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc59015e900 con 0x7fc5900d1200 2024-09-10T11:39:31.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.101+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc590071c50 con 0x7fc5900d1200 2024-09-10T11:39:31.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.101+0000 7fc577fff640 1 -- 172.21.15.99:0/3744460749 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3908 (secure 0 0 0) 0x7fc590071c50 con 0x7fc5900d1200 2024-09-10T11:39:31.102 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:31.102 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":69,"btime":"2024-09-10T11:39:26:259142+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6940,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2309874660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2309874660},{"type":"v1","addr":"172.21.15.99:6837","nonce":2309874660}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6955,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/555772882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":555772882},{"type":"v1","addr":"172.21.15.196:6834","nonce":555772882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6958,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/72953920","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":72953920},{"type":"v1","addr":"172.21.15.196:6835","nonce":72953920}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7875,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1225490647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1225490647},{"type":"v1","addr":"172.21.15.99:6835","nonce":1225490647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-10T11:39:31.102 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 69 2024-09-10T11:39:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 msgr2=0x7fc57c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 0x7fc57c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc580009a20 tx=0x7fc580009190 comp rx=0 tx=0).stop 2024-09-10T11:39:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 msgr2=0x7fc5901582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d7d0 tx=0x7fc58400dc90 comp rx=0 tx=0).stop 2024-09-10T11:39:31.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 shutdown_connections 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc57c06be30 0x7fc57c06e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc59015c060 0x7fc5901587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5900d1200 0x7fc5901582b0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 --2- 172.21.15.99:0/3744460749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc59006e5d0 0x7fc590143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 >> 172.21.15.99:0/3744460749 conn(0x7fc59006bc10 msgr2=0x7fc59015d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 shutdown_connections 2024-09-10T11:39:31.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.103+0000 7fc595e3c640 1 -- 172.21.15.99:0/3744460749 wait complete. 2024-09-10T11:39:31.115 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T11:39:31.115 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T11:39:31.116 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:39:31.116 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:39:31.116 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T11:39:31.116 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T11:39:31.116 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:39:31.116 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:39:31.116 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T11:39:31.117 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T11:39:31.117 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:39:31.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:39:31.117 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T11:39:31.117 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T11:39:31.117 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:39:31.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:39:31.125 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:39:31.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T11:39:31.126 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:39:31.126 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T11:39:31.127 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:39:31.127 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T11:39:31.128 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:39:31.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T11:39:31.168 INFO:tasks.ceph.mds.a:Started 2024-09-10T11:39:31.168 DEBUG:teuthology.parallel:result is None 2024-09-10T11:39:31.169 INFO:tasks.ceph.mds.c:Started 2024-09-10T11:39:31.169 DEBUG:teuthology.parallel:result is None 2024-09-10T11:39:31.170 INFO:tasks.ceph.mds.b:Started 2024-09-10T11:39:31.170 DEBUG:teuthology.parallel:result is None 2024-09-10T11:39:31.171 INFO:tasks.ceph.mds.d:Started 2024-09-10T11:39:31.171 DEBUG:teuthology.parallel:result is None 2024-09-10T11:39:31.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T11:39:31.243 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-09-10T11:39:31.244 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:39:31.245 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-09-10T11:39:31.245 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-09-10T11:39:31.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.274+0000 7fa588e3c640 1 Processor -- start 2024-09-10T11:39:31.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.274+0000 7fa588e3c640 1 -- start start 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.275+0000 7fa588e3c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 0x7fa5840cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.275+0000 7fa588e3c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 0x7fa5840d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.275+0000 7fa588e3c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa58415f290 con 0x7fa58415eeb0 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.275+0000 7fa588e3c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa5840d2800 con 0x7fa5840d2160 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.275+0000 7fa588e3c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840d0e60 con 0x7fa5840d3530 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa582d9d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 0x7fa5840d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa58259c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa58415eeb0 0x7fa5840b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:40112/0 (socket says 172.21.15.99:40112) 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa581d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 0x7fa5840cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa58259c640 1 -- 172.21.15.99:0/3646610631 learned_addr learned my addr 172.21.15.99:0/3646610631 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa582d9d640 1 -- 172.21.15.99:0/3646610631 >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 msgr2=0x7fa5840cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:31.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa582d9d640 1 --2- 172.21.15.99:0/3646610631 >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 0x7fa5840cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa582d9d640 1 -- 172.21.15.99:0/3646610631 >> v1:172.21.15.196:6789/0 conn(0x7fa58415eeb0 legacy=0x7fa5840b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa582d9d640 1 -- 172.21.15.99:0/3646610631 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840d2a50 con 0x7fa5840d3530 2024-09-10T11:39:31.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.276+0000 7fa581d9b640 1 --2- 172.21.15.99:0/3646610631 >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 0x7fa5840cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:31.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.277+0000 7fa582d9d640 1 --2- 172.21.15.99:0/3646610631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 0x7fa5840d39c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009dc0 tx=0x7fa57c02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc8dcd23111a5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:31.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.280+0000 7fa58159a640 1 -- 172.21.15.99:0/3646610631 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c007740 con 0x7fa5840d3530 2024-09-10T11:39:31.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.280+0000 7fa58159a640 1 -- 172.21.15.99:0/3646610631 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57c0078e0 con 0x7fa5840d3530 2024-09-10T11:39:31.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.280+0000 7fa58159a640 1 -- 172.21.15.99:0/3646610631 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c007cb0 con 0x7fa5840d3530 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 -- 172.21.15.99:0/3646610631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 msgr2=0x7fa5840d39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 --2- 172.21.15.99:0/3646610631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 0x7fa5840d39c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009dc0 tx=0x7fa57c02a560 comp rx=0 tx=0).stop 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 -- 172.21.15.99:0/3646610631 shutdown_connections 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 --2- 172.21.15.99:0/3646610631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d3530 0x7fa5840d39c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 --2- 172.21.15.99:0/3646610631 >> v2:172.21.15.196:3300/0 conn(0x7fa5840d2160 0x7fa5840cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 -- 172.21.15.99:0/3646610631 >> 172.21.15.99:0/3646610631 conn(0x7fa5840d4c30 msgr2=0x7fa5840d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 -- 172.21.15.99:0/3646610631 shutdown_connections 2024-09-10T11:39:31.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.281+0000 7fa588e3c640 1 -- 172.21.15.99:0/3646610631 wait complete. 2024-09-10T11:39:31.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.283+0000 7fa588e3c640 1 Processor -- start 2024-09-10T11:39:31.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.283+0000 7fa588e3c640 1 -- start start 2024-09-10T11:39:31.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d2160 0x7fa58415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 0x7fa58415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840b4e00 con 0x7fa5840d3530 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840b4c80 con 0x7fa58415eeb0 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa588e3c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa5840d0a10 con 0x7fa5840d2160 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa58259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d2160 0x7fa58415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48828/0 (socket says 172.21.15.99:48828) 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 -- 172.21.15.99:0/864384422 learned_addr learned my addr 172.21.15.99:0/864384422 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa582d9d640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 0x7fa58415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 -- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d2160 msgr2=0x7fa58415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d2160 0x7fa58415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 -- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 msgr2=0x7fa58415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 0x7fa58415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:31.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa581d9b640 1 -- 172.21.15.99:0/864384422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5840cdc80 con 0x7fa5840d3530 2024-09-10T11:39:31.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.284+0000 7fa582d9d640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 0x7fa58415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:31.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.285+0000 7fa581d9b640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e930 tx=0x7fa57800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:31.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.286+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578017070 con 0x7fa5840d3530 2024-09-10T11:39:31.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.287+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57800ca00 con 0x7fa5840d3530 2024-09-10T11:39:31.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.287+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840cdf60 con 0x7fa5840d3530 2024-09-10T11:39:31.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.287+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57800cdf0 con 0x7fa5840d3530 2024-09-10T11:39:31.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.287+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5840ce520 con 0x7fa5840d3530 2024-09-10T11:39:31.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.288+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5840cfa20 con 0x7fa5840d3530 2024-09-10T11:39:31.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.288+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa57801a070 con 0x7fa5840d3530 2024-09-10T11:39:31.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.289+0000 7fa572ffd640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 0x7fa56406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:31.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.289+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 4943+0+0 (secure 0 0 0) 0x7fa57808a9e0 con 0x7fa5840d3530 2024-09-10T11:39:31.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.289+0000 7fa58259c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 0x7fa56406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:31.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.289+0000 7fa58259c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 0x7fa56406e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa574005880 tx=0x7fa574009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:31.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.295+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa578059030 con 0x7fa5840d3530 2024-09-10T11:39:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:31.471+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa5840babb0 con 0x7fa5840d3530 2024-09-10T11:39:32.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.289+0000 7fa572ffd640 1 -- 172.21.15.99:0/864384422 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v179) ==== 97+0+0 (secure 0 0 0) 0x7fa57805d330 con 0x7fa5840d3530 2024-09-10T11:39:32.290 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-09-10T11:39:32.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 msgr2=0x7fa56406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 0x7fa56406e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa574005880 tx=0x7fa574009040 comp rx=0 tx=0).stop 2024-09-10T11:39:32.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 msgr2=0x7fa5840b9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fa57800e930 tx=0x7fa57800edf0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 shutdown_connections 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa56406be30 0x7fa56406e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa58415eeb0 0x7fa58415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5840d3530 0x7fa5840b9490 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 --2- 172.21.15.99:0/864384422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa5840d2160 0x7fa58415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.292+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 >> 172.21.15.99:0/864384422 conn(0x7fa5840d4c30 msgr2=0x7fa5840b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.293+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 shutdown_connections 2024-09-10T11:39:32.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.293+0000 7fa588e3c640 1 -- 172.21.15.99:0/864384422 wait complete. 2024-09-10T11:39:32.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T11:39:32.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.410+0000 7fc8cca53640 1 Processor -- start 2024-09-10T11:39:32.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.410+0000 7fc8cca53640 1 -- start start 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.411+0000 7fc8cca53640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8c80d0130 0x7fc8c80cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.411+0000 7fc8cca53640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80ce430 con 0x7fc8c8160030 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.411+0000 7fc8cca53640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80ce710 con 0x7fc8c80d1760 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.411+0000 7fc8cca53640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c80ce200 con 0x7fc8c80d0130 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.411+0000 7fc8c659c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc8c8160030 0x7fc8c80b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:44412/0 (socket says 172.21.15.99:44412) 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c659c640 1 -- 172.21.15.99:0/852566216 learned_addr learned my addr 172.21.15.99:0/852566216 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/852566216 >> v2:172.21.15.196:3300/0 conn(0x7fc8c80d0130 0x7fc8c80cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2980534773 0 0) 0x7fc8c80ce430 con 0x7fc8c8160030 2024-09-10T11:39:32.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8b4003520 con 0x7fc8c8160030 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2726793139 0 0) 0x7fc8b4003520 con 0x7fc8c8160030 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 >> v2:172.21.15.196:3300/0 conn(0x7fc8c80d0130 msgr2=0x7fc8c80cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 --2- 172.21.15.99:0/852566216 >> v2:172.21.15.196:3300/0 conn(0x7fc8c80d0130 0x7fc8c80cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 >> v1:172.21.15.196:6789/0 conn(0x7fc8c80d1760 legacy=0x7fc8c80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80d1240 con 0x7fc8c8160030 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fc8b80019f0 con 0x7fc8c8160030 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 >> v1:172.21.15.99:6789/0 conn(0x7fc8c8160030 legacy=0x7fc8c80b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40039b0 0x7fc8b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b4004800 0x7fc8b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 0x7fc8b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b400c030 con 0x7fc8b40051c0 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b400c1e0 con 0x7fc8b4004800 2024-09-10T11:39:32.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b400c3e0 con 0x7fc8b40039b0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 0x7fc8b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c659c640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40039b0 0x7fc8b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.412+0000 7fc8c5d9b640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b4004800 0x7fc8b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40039b0 msgr2=0x7fc8b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40039b0 0x7fc8b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b4004800 msgr2=0x7fc8b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8b4004800 0x7fc8b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/852566216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4003520 con 0x7fc8b40051c0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/852566216 reap_dead start 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 0x7fc8b400b9c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ac0 tx=0x7fc8b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233a86c2a1e12328 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0009690 con 0x7fc8b40051c0 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0004020 con 0x7fc8b40051c0 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.413+0000 7fc8c559a640 1 -- 172.21.15.99:0/852566216 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0004410 con 0x7fc8b40051c0 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 -- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 msgr2=0x7fc8b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 0x7fc8b400b9c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ac0 tx=0x7fc8b4005830 comp rx=0 tx=0).stop 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 -- 172.21.15.99:0/852566216 shutdown_connections 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 --2- 172.21.15.99:0/852566216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8b40051c0 0x7fc8b400b9c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 -- 172.21.15.99:0/852566216 >> 172.21.15.99:0/852566216 conn(0x7fc8c80b4650 msgr2=0x7fc8c80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 -- 172.21.15.99:0/852566216 shutdown_connections 2024-09-10T11:39:32.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.414+0000 7fc8cca53640 1 -- 172.21.15.99:0/852566216 wait complete. 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.415+0000 7fc8cca53640 1 Processor -- start 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.415+0000 7fc8cca53640 1 -- start start 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40051c0 0x7fc8c80b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80b11c0 0x7fc8c815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c80b38f0 con 0x7fc8c80b0850 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c8160620 con 0x7fc8c80b11c0 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8cca53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c80b3a70 con 0x7fc8b40051c0 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8c6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8c6d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48848/0 (socket says 172.21.15.99:48848) 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/3315139816 learned_addr learned my addr 172.21.15.99:0/3315139816 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8c659c640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40051c0 0x7fc8c80b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.416+0000 7fc8c5d9b640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80b11c0 0x7fc8c815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40051c0 msgr2=0x7fc8c80b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40051c0 0x7fc8c80b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80b11c0 msgr2=0x7fc8c815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80b11c0 0x7fc8c815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 -- 172.21.15.99:0/3315139816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c80afc60 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8c6d9d640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc8c002abe0 tx=0x7fc8c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0003e20 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c0040070 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c80b1650 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c003a470 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.417+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c80b1c10 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.418+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c80b3e00 con 0x7fc8c80b0850 2024-09-10T11:39:32.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.418+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8c003a610 con 0x7fc8c80b0850 2024-09-10T11:39:32.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.419+0000 7fc8beffd640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.419+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4857+0+0 (secure 0 0 0) 0x7fc8c00aab80 con 0x7fc8c80b0850 2024-09-10T11:39:32.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.419+0000 7fc8c659c640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.419+0000 7fc8c659c640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c5fd0 tx=0x7fc8b8006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.423+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8c80afc60 con 0x7fc8c80b0850 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.649+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc8c8070750 con 0x7fc8c80b0850 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.649+0000 7fc8beffd640 1 -- 172.21.15.99:0/3315139816 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fc8c0079230 con 0x7fc8c80b0850 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-09-10T11:39:32.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBmh6KaOhAA7ctf9q/OXzV2K873iGy0TA==", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmNI3ZEhAAi2BygDEVDZeXKL6jTBljkQ==", 2024-09-10T11:39:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmSGQwBhAAOg2V+zOSnkmQ11vCiLxH1A==", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:39:32.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmnz4gGhAAW4PlbWYsMjNz9j46k9dKeQ==", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC7LeBm9bZAOBAAT+2kBsLrMgR91osF8J77wA==", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmz/qSAxAA4obiBKsZit/eD5DTKoU7eg==", 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.654 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKsJuChAA88FIWpZDDeIpEUluzvIR4w==", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC8LeBmKZRXERAAz/YDECXxt+a7t7S5E1gwVQ==", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmmdPPIRAAEGZ3g4UqJOQ7vsaPJOnaWw==", 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.656 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmA/PUKBAAtZg8jlZ82Y7xsGugvaLpmg==", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmorS+LxAAAamGMTIdd24ksTx8VHq7fw==", 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-09-10T11:39:32.658 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC/LeBmZOapNhAAN3EhwZc/fp+Uhsv4dKjSBw==", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g==", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T11:39:32.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmYdE7JxAATuLd3FBO6xRT3A4dnGqJzA==", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-09-10T11:39:32.660 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBm0Z0LJBAAS5mh07Ldtn2DDLrBLLPimw==", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC4LeBmfy3pLBAAiWFGR04Sxoh2YvKVdPoUlA==", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-09-10T11:39:32.661 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4jD2HhAAydKGa+7u1XfTnzmcEyM/Ow==", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm6KpnNxAARupymLXHrI1TVc2oLLAZ7w==", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQC3LeBm4uGRMxAACcpk+jz0xWM3df9XVh8ZFg==", 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-09-10T11:39:32.663 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:32.664 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 msgr2=0x7fc8a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80c5fd0 tx=0x7fc8b8006040 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 msgr2=0x7fc8c80cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fc8c002abe0 tx=0x7fc8c0004730 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 shutdown_connections 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fc8a806be30 0x7fc8a806e2f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc8c80b0850 0x7fc8c80cbdd0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8c80b11c0 0x7fc8c815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 --2- 172.21.15.99:0/3315139816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8b40051c0 0x7fc8c80b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 >> 172.21.15.99:0/3315139816 conn(0x7fc8c80b4650 msgr2=0x7fc8c80b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 shutdown_connections 2024-09-10T11:39:32.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.651+0000 7fc8cca53640 1 -- 172.21.15.99:0/3315139816 wait complete. 2024-09-10T11:39:32.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T11:39:32.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.765+0000 7f2d2175f640 1 Processor -- start 2024-09-10T11:39:32.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.765+0000 7f2d2175f640 1 -- start start 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d2175f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d2175f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0d2990 con 0x7f2d1c0d3a30 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d2175f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1c0d2c70 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d2175f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1c0d2f50 con 0x7f2d1c15efc0 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1a7fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2d1c0d3a30 0x7f2d1c0d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40172/0 (socket says 172.21.15.99:40172) 2024-09-10T11:39:32.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1a7fc640 1 -- 172.21.15.99:0/1542967975 learned_addr learned my addr 172.21.15.99:0/1542967975 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1affd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58252/0 (socket says 172.21.15.99:58252) 2024-09-10T11:39:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1affd640 1 -- 172.21.15.99:0/1542967975 >> v1:172.21.15.196:6789/0 conn(0x7f2d1c0d1cb0 legacy=0x7f2d1c0b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1affd640 1 -- 172.21.15.99:0/1542967975 >> v1:172.21.15.99:6789/0 conn(0x7f2d1c0d3a30 legacy=0x7f2d1c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d1affd640 1 -- 172.21.15.99:0/1542967975 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c0d3230 con 0x7f2d1c15efc0 2024-09-10T11:39:32.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.766+0000 7f2d19ffb640 1 -- 172.21.15.99:0/1542967975 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122814321 0 0) 0x7f2d1c0d2990 con 0x7f2d1c0d3a30 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.767+0000 7f2d1affd640 1 --2- 172.21.15.99:0/1542967975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ce47240808b4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.767+0000 7f2d19ffb640 1 -- 172.21.15.99:0/1542967975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037070 con 0x7f2d1c15efc0 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.767+0000 7f2d19ffb640 1 -- 172.21.15.99:0/1542967975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c032440 con 0x7f2d1c15efc0 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.767+0000 7f2d19ffb640 1 -- 172.21.15.99:0/1542967975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c032810 con 0x7f2d1c15efc0 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 -- 172.21.15.99:0/1542967975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c15efc0 msgr2=0x7f2d1c0d03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 --2- 172.21.15.99:0/1542967975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0097a0 tx=0x7f2d0c02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 -- 172.21.15.99:0/1542967975 shutdown_connections 2024-09-10T11:39:32.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 --2- 172.21.15.99:0/1542967975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c15efc0 0x7f2d1c0d03f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 -- 172.21.15.99:0/1542967975 >> 172.21.15.99:0/1542967975 conn(0x7f2d1c0d4c20 msgr2=0x7f2d1c0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:32.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 -- 172.21.15.99:0/1542967975 shutdown_connections 2024-09-10T11:39:32.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.768+0000 7f2d2175f640 1 -- 172.21.15.99:0/1542967975 wait complete. 2024-09-10T11:39:32.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.769+0000 7f2d2175f640 1 Processor -- start 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.769+0000 7f2d2175f640 1 -- start start 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d1c0d3a30 0x7f2d1c158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d1c15efc0 0x7f2d1c0b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0d0cc0 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c0d0b40 con 0x7f2d1c0d3a30 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1c0b4c70 con 0x7f2d1c15efc0 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58262/0 (socket says 172.21.15.99:58262) 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 -- 172.21.15.99:0/259906933 learned_addr learned my addr 172.21.15.99:0/259906933 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 -- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d1c15efc0 msgr2=0x7f2d1c0b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d1c15efc0 0x7f2d1c0b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1a7fc640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d1c0d3a30 0x7f2d1c158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 -- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d1c0d3a30 msgr2=0x7f2d1c158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d1c0d3a30 0x7f2d1c158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 -- 172.21.15.99:0/259906933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c1e6cb0 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d1affd640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c037040 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c004230 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c1e6f90 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c032e40 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.770+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1e45f0 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.771+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c0c8b60 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.772+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d0c030770 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.772+0000 7f2d037fe640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 0x7f2d0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.773+0000 7f2d1a7fc640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 0x7f2d0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.773+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4857+0+0 (secure 0 0 0) 0x7f2d0c0abf70 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.773+0000 7f2d1a7fc640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 0x7f2d0806e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d10007fd0 tx=0x7f2d10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:32.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.777+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d0c07ad90 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.943+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2d1c0d03f0 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.943+0000 7f2d037fe640 1 -- 172.21.15.99:0/259906933 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v179) ==== 84+0+65 (secure 0 0 0) 0x7f2d0c07a690 con 0x7f2d1c0d1cb0 2024-09-10T11:39:32.944 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:32.945 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-09-10T11:39:32.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 msgr2=0x7f2d0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 0x7f2d0806e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d10007fd0 tx=0x7f2d10009040 comp rx=0 tx=0).stop 2024-09-10T11:39:32.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 msgr2=0x7f2d1c1e40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:32.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c000c00 tx=0x7f2d0c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 shutdown_connections 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d0806be30 0x7f2d0806e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d1c15efc0 0x7f2d1c0b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d1c0d3a30 0x7f2d1c158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 --2- 172.21.15.99:0/259906933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d1c0d1cb0 0x7f2d1c1e40b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 >> 172.21.15.99:0/259906933 conn(0x7f2d1c0d4c20 msgr2=0x7f2d1c0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 shutdown_connections 2024-09-10T11:39:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:32.945+0000 7f2d2175f640 1 -- 172.21.15.99:0/259906933 wait complete. 2024-09-10T11:39:32.957 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T11:39:32.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T11:39:33.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.059+0000 7f75789fd640 1 Processor -- start 2024-09-10T11:39:33.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.059+0000 7f75789fd640 1 -- start start 2024-09-10T11:39:33.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f75789fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f75740af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f75789fd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7574162ef0 0x7f75740b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f75789fd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75741635c0 con 0x7f75740b6fe0 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f75789fd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f757406d550 con 0x7f7574162ef0 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f75789fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f75740b7b50 con 0x7f757415ee10 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7574162ef0 0x7f75740b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7574162ef0 0x7f75740b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58274/0 (socket says 172.21.15.99:58274) 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 -- 172.21.15.99:0/3820056905 learned_addr learned my addr 172.21.15.99:0/3820056905 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 -- 172.21.15.99:0/3820056905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 msgr2=0x7f75740af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f757259c640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f75740af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f75740af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 -- 172.21.15.99:0/3820056905 >> v1:172.21.15.196:6789/0 conn(0x7f75740b6fe0 legacy=0x7f75741627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f7572d9d640 1 -- 172.21.15.99:0/3820056905 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757406cf00 con 0x7f7574162ef0 2024-09-10T11:39:33.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.060+0000 7f757259c640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f75740af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:33.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f7572d9d640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7574162ef0 0x7f75740b7440 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f756c009e50 tx=0x7f756c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c00edcdb22cfa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f757159a640 1 -- 172.21.15.99:0/3820056905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c036070 con 0x7f7574162ef0 2024-09-10T11:39:33.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f757159a640 1 -- 172.21.15.99:0/3820056905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c031600 con 0x7f7574162ef0 2024-09-10T11:39:33.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f757159a640 1 -- 172.21.15.99:0/3820056905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0319d0 con 0x7f7574162ef0 2024-09-10T11:39:33.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f75789fd640 1 -- 172.21.15.99:0/3820056905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7574162ef0 msgr2=0x7f75740b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f75789fd640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7574162ef0 0x7f75740b7440 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f756c009e50 tx=0x7f756c02a6c0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f75789fd640 1 -- 172.21.15.99:0/3820056905 shutdown_connections 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f75789fd640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7574162ef0 0x7f75740b7440 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.061+0000 7f75789fd640 1 --2- 172.21.15.99:0/3820056905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f75740af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.062+0000 7f75789fd640 1 -- 172.21.15.99:0/3820056905 >> 172.21.15.99:0/3820056905 conn(0x7f7574073c60 msgr2=0x7f757409ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.062+0000 7f75789fd640 1 -- 172.21.15.99:0/3820056905 shutdown_connections 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.062+0000 7f75789fd640 1 -- 172.21.15.99:0/3820056905 wait complete. 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.063+0000 7f75789fd640 1 Processor -- start 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.063+0000 7f75789fd640 1 -- start start 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f7574151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7574162ef0 0x7f7574153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f757406db60 con 0x7f75740b6fe0 2024-09-10T11:39:33.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f757406d9e0 con 0x7f7574162ef0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f757406dce0 con 0x7f757415ee10 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58282/0 (socket says 172.21.15.99:58282) 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 -- 172.21.15.99:0/2660883581 learned_addr learned my addr 172.21.15.99:0/2660883581 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f7572d9d640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7574162ef0 0x7f7574153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f7571d9b640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f7574151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 -- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 msgr2=0x7f7574151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f7574151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 -- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7574162ef0 msgr2=0x7f7574153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7574162ef0 0x7f7574153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 -- 172.21.15.99:0/2660883581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574152940 con 0x7f75740b6fe0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f757259c640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f756400c7f0 tx=0x7f756400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564015070 con 0x7f75740b6fe0 2024-09-10T11:39:33.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7564007a30 con 0x7f75740b6fe0 2024-09-10T11:39:33.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7574152c20 con 0x7f75740b6fe0 2024-09-10T11:39:33.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564007e00 con 0x7f75740b6fe0 2024-09-10T11:39:33.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.064+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75741531e0 con 0x7f75740b6fe0 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.065+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75740af930 con 0x7f75740b6fe0 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.066+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7564018070 con 0x7f75740b6fe0 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.066+0000 7f756affd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 0x7f755406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.066+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4857+0+0 (secure 0 0 0) 0x7f756408aa50 con 0x7f75740b6fe0 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.066+0000 7f7571d9b640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 0x7f755406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.067+0000 7f7571d9b640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 0x7f755406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f75740b5c10 tx=0x7f755c007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.070+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7564059100 con 0x7f75740b6fe0 2024-09-10T11:39:33.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.236+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f75740d3790 con 0x7f75740b6fe0 2024-09-10T11:39:33.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.297+0000 7f756affd640 1 -- 172.21.15.99:0/2660883581 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v180) ==== 138+0+0 (secure 0 0 0) 0x7f756405d400 con 0x7f75740b6fe0 2024-09-10T11:39:33.298 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.299+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 msgr2=0x7f755406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 0x7f755406e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f75740b5c10 tx=0x7f755c007e30 comp rx=0 tx=0).stop 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 msgr2=0x7f75741500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f756400c7f0 tx=0x7f756400cbf0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 shutdown_connections 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f755406be30 0x7f755406e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7574162ef0 0x7f7574153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f757415ee10 0x7f7574151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 --2- 172.21.15.99:0/2660883581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75740b6fe0 0x7f75741500e0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 >> 172.21.15.99:0/2660883581 conn(0x7f7574073c60 msgr2=0x7f75740b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:33.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 shutdown_connections 2024-09-10T11:39:33.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.300+0000 7f75789fd640 1 -- 172.21.15.99:0/2660883581 wait complete. 2024-09-10T11:39:33.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T11:39:33.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.418+0000 7f9d46d5b640 1 Processor -- start 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.418+0000 7f9d46d5b640 1 -- start start 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d46d5b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d46d5b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d46d5b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d400ceb10 con 0x7f9d4015c7b0 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d46d5b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9d4006f2a0 con 0x7f9d4006ee30 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d46d5b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d400b3840 con 0x7f9d400d1180 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d452f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d452f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:36360/0 (socket says 172.21.15.99:36360) 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:45504/0 (socket says 172.21.15.99:45504) 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d452f8640 1 -- 172.21.15.99:0/782941221 learned_addr learned my addr 172.21.15.99:0/782941221 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:33.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.419+0000 7f9d44af7640 1 --1- 172.21.15.99:0/782941221 >> v1:172.21.15.196:6789/0 conn(0x7f9d4015c7b0 0x7f9d400d0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37274/0 (socket says 172.21.15.99:37274) 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.420+0000 7f9d3ffff640 1 -- 172.21.15.99:0/782941221 >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 msgr2=0x7f9d4006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.420+0000 7f9d3ffff640 1 --2- 172.21.15.99:0/782941221 >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.420+0000 7f9d3ffff640 1 -- 172.21.15.99:0/782941221 >> v1:172.21.15.196:6789/0 conn(0x7f9d4015c7b0 legacy=0x7f9d400d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.420+0000 7f9d3ffff640 1 -- 172.21.15.99:0/782941221 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d400b3a90 con 0x7f9d400d1180 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.420+0000 7f9d452f8640 1 --2- 172.21.15.99:0/782941221 >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d3ffff640 1 --2- 172.21.15.99:0/782941221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=830205f067a1d94c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d3f7fe640 1 -- 172.21.15.99:0/782941221 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007740 con 0x7f9d400d1180 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d3f7fe640 1 -- 172.21.15.99:0/782941221 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d300078e0 con 0x7f9d400d1180 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d3f7fe640 1 -- 172.21.15.99:0/782941221 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30007cb0 con 0x7f9d400d1180 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 -- 172.21.15.99:0/782941221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 msgr2=0x7f9d4006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/782941221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a660 comp rx=0 tx=0).stop 2024-09-10T11:39:33.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 -- 172.21.15.99:0/782941221 shutdown_connections 2024-09-10T11:39:33.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/782941221 >> v2:172.21.15.196:3300/0 conn(0x7f9d4006ee30 0x7f9d4006d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/782941221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4006e8f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 -- 172.21.15.99:0/782941221 >> 172.21.15.99:0/782941221 conn(0x7f9d4006bc10 msgr2=0x7f9d4006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:33.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 -- 172.21.15.99:0/782941221 shutdown_connections 2024-09-10T11:39:33.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.421+0000 7f9d46d5b640 1 -- 172.21.15.99:0/782941221 wait complete. 2024-09-10T11:39:33.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 Processor -- start 2024-09-10T11:39:33.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 -- start start 2024-09-10T11:39:33.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4014f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4015c7b0 0x7f9d4014cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9d400b4330 con 0x7f9d4006ee30 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9d400b4050 con 0x7f9d4015c7b0 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.423+0000 7f9d46d5b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9d400b4610 con 0x7f9d400d1180 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58296/0 (socket says 172.21.15.99:58296) 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 -- 172.21.15.99:0/2666786369 learned_addr learned my addr 172.21.15.99:0/2666786369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d452f8640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4015c7b0 0x7f9d4014cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d3ffff640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4014f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 -- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 msgr2=0x7f9d4014f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4014f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 -- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4015c7b0 msgr2=0x7f9d4014cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4015c7b0 0x7f9d4014cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 -- 172.21.15.99:0/2666786369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d40154780 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d44af7640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3c0 tx=0x7f9d3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380074e0 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d38007680 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38004a00 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d40154a60 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.424+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d40154430 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.425+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d4015ccb0 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.426+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d3801a020 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.426+0000 7f9d3d7fa640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 0x7f9d2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.426+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(180..180 src has 1..180) ==== 5227+0+0 (secure 0 0 0) 0x7f9d3808a370 con 0x7f9d4006ee30 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.426+0000 7f9d3ffff640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 0x7f9d2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.427+0000 7f9d3ffff640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 0x7f9d2006e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9d30034000 tx=0x7f9d3002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.431+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d400ceb10 con 0x7f9d4006ee30 2024-09-10T11:39:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:33.596+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9d400b06b0 con 0x7f9d4006ee30 2024-09-10T11:39:34.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.299+0000 7f9d3d7fa640 1 -- 172.21.15.99:0/2666786369 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v181) ==== 170+0+0 (secure 0 0 0) 0x7f9d38058f90 con 0x7f9d4006ee30 2024-09-10T11:39:34.300 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-09-10T11:39:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.302+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 msgr2=0x7f9d2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.302+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 0x7f9d2006e2f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9d30034000 tx=0x7f9d3002f000 comp rx=0 tx=0).stop 2024-09-10T11:39:34.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.302+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 msgr2=0x7f9d40151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.302+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3c0 tx=0x7f9d3800c880 comp rx=0 tx=0).stop 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 shutdown_connections 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9d2006be30 0x7f9d2006e2f0 secure :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9d30034000 tx=0x7f9d3002f000 comp rx=0 tx=0).stop 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4015c7b0 0x7f9d4014cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d400d1180 0x7f9d4014f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 --2- 172.21.15.99:0/2666786369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d4006ee30 0x7f9d40151c90 secure :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3c0 tx=0x7f9d3800c880 comp rx=0 tx=0).stop 2024-09-10T11:39:34.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.303+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 >> 172.21.15.99:0/2666786369 conn(0x7f9d4006bc10 msgr2=0x7f9d400d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.306+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 shutdown_connections 2024-09-10T11:39:34.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.307+0000 7f9d46d5b640 1 -- 172.21.15.99:0/2666786369 wait complete. 2024-09-10T11:39:34.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T11:39:34.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.431+0000 7efe3bc65640 1 Processor -- start 2024-09-10T11:39:34.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.431+0000 7efe3bc65640 1 -- start start 2024-09-10T11:39:34.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe3bc65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe340d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe3bc65640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3415ff70 con 0x7efe341626c0 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe3bc65640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe340d1620 con 0x7efe3415eb80 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe3bc65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe340d30c0 con 0x7efe340d5010 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe39200640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efe341626c0 0x7efe340d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40186/0 (socket says 172.21.15.99:40186) 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe39200640 1 -- 172.21.15.99:0/3240199662 learned_addr learned my addr 172.21.15.99:0/3240199662 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe3a202640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe340d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781006791 0 0) 0x7efe3415ff70 con 0x7efe341626c0 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.432+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe1c003520 con 0x7efe341626c0 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908180307 0 0) 0x7efe340d1620 con 0x7efe3415eb80 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe3415ff70 con 0x7efe3415eb80 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 925246047 0 0) 0x7efe1c003520 con 0x7efe341626c0 2024-09-10T11:39:34.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 msgr2=0x7efe340d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe340d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 >> v1:172.21.15.196:6789/0 conn(0x7efe3415eb80 legacy=0x7efe340d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe34162bc0 con 0x7efe341626c0 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7efe280019f0 con 0x7efe341626c0 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe3a202640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe340d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 >> v1:172.21.15.99:6789/0 conn(0x7efe341626c0 legacy=0x7efe340d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe1c003b70 0x7efe1c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe340d30c0 con 0x7efe1c004800 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c00baa0 con 0x7efe1c003b70 2024-09-10T11:39:34.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c00bca0 con 0x7efe340d5010 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39a01640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe1c003b70 0x7efe1c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe3a202640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 -- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 msgr2=0x7efe1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe340d5010 0x7efe1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 -- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe1c003b70 msgr2=0x7efe1c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe1c003b70 0x7efe1c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.433+0000 7efe39200640 1 -- 172.21.15.99:0/3240199662 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c003520 con 0x7efe1c004800 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.434+0000 7efe39200640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efe34162af0 tx=0x7efe1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12228fcaed92c5ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.434+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2800a450 con 0x7efe1c004800 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.434+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2802f070 con 0x7efe1c004800 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.434+0000 7efe389ff640 1 -- 172.21.15.99:0/3240199662 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2802bc00 con 0x7efe1c004800 2024-09-10T11:39:34.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 -- 172.21.15.99:0/3240199662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 msgr2=0x7efe1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 --2- 172.21.15.99:0/3240199662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7efe34162af0 tx=0x7efe1c005830 comp rx=0 tx=0).stop 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3a202640 1 -- 172.21.15.99:0/3240199662 reap_dead start 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 -- 172.21.15.99:0/3240199662 shutdown_connections 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 -- 172.21.15.99:0/3240199662 >> 172.21.15.99:0/3240199662 conn(0x7efe3408e680 msgr2=0x7efe3409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 -- 172.21.15.99:0/3240199662 shutdown_connections 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.435+0000 7efe3bc65640 1 -- 172.21.15.99:0/3240199662 wait complete. 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.436+0000 7efe3bc65640 1 Processor -- start 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.436+0000 7efe3bc65640 1 -- start start 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 0x7efe34160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe34160890 0x7efe34160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe34163460 con 0x7efe1c004800 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efe34163180 con 0x7efe3415abb0 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3bc65640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7efe340d0770 con 0x7efe34160890 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58322/0 (socket says 172.21.15.99:58322) 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 -- 172.21.15.99:0/676310892 learned_addr learned my addr 172.21.15.99:0/676310892 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39200640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 0x7efe34160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe3a202640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe34160890 0x7efe34160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 -- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe34160890 msgr2=0x7efe34160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe34160890 0x7efe34160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 -- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 msgr2=0x7efe34160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 0x7efe34160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 -- 172.21.15.99:0/676310892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3415c340 con 0x7efe1c004800 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39200640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 0x7efe34160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe39a01640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00f7d0 tx=0x7efe2c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.437+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00fe80 con 0x7efe1c004800 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.438+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c018070 con 0x7efe1c004800 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.438+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c00d730 con 0x7efe1c004800 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.438+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe34158350 con 0x7efe1c004800 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.438+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe340c8b40 con 0x7efe1c004800 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.439+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34158140 con 0x7efe1c004800 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.439+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe2c00d8d0 con 0x7efe1c004800 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.440+0000 7efe327fc640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 0x7efe2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.440+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(181..181 src has 1..181) ==== 5609+0+0 (secure 0 0 0) 0x7efe2c08ad50 con 0x7efe1c004800 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.440+0000 7efe39200640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 0x7efe2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.440+0000 7efe39200640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 0x7efe2006e320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efe28033000 tx=0x7efe2802d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.444+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe2c08e030 con 0x7efe1c004800 2024-09-10T11:39:34.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:34.640+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7efe340d0aa0 con 0x7efe1c004800 2024-09-10T11:39:35.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.323+0000 7efe327fc640 1 -- 172.21.15.99:0/676310892 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v71) ==== 174+0+0 (secure 0 0 0) 0x7efe2c01a310 con 0x7efe1c004800 2024-09-10T11:39:35.326 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 22 and data pool 23 2024-09-10T11:39:35.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.331+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 msgr2=0x7efe2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.331+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 0x7efe2006e320 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7efe28033000 tx=0x7efe2802d000 comp rx=0 tx=0).stop 2024-09-10T11:39:35.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.331+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 msgr2=0x7efe3415b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.331+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00f7d0 tx=0x7efe2c00fc90 comp rx=0 tx=0).stop 2024-09-10T11:39:35.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 shutdown_connections 2024-09-10T11:39:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7efe2006be60 0x7efe2006e320 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe34160890 0x7efe34160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe3415abb0 0x7efe34160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 --2- 172.21.15.99:0/676310892 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe1c004800 0x7efe3415b9b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.334+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 >> 172.21.15.99:0/676310892 conn(0x7efe3408e680 msgr2=0x7efe3409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.336+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 shutdown_connections 2024-09-10T11:39:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.337+0000 7efe3bc65640 1 -- 172.21.15.99:0/676310892 wait complete. 2024-09-10T11:39:35.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:35.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.450+0000 7f1e6720a640 1 Processor -- start 2024-09-10T11:39:35.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.450+0000 7f1e6720a640 1 -- start start 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e600d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d0fb0 0x7f1e6006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e6006ee30 0x7f1e601604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6006f280 con 0x7f1e600d0fb0 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1e600b3840 con 0x7f1e6006ee30 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e6720a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e600d13f0 con 0x7f1e6015c7b0 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d0fb0 0x7f1e6006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d0fb0 0x7f1e6006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58334/0 (socket says 172.21.15.99:58334) 2024-09-10T11:39:35.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 -- 172.21.15.99:0/1468050519 learned_addr learned my addr 172.21.15.99:0/1468050519 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 -- 172.21.15.99:0/1468050519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 msgr2=0x7f1e600d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/1468050519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e600d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 -- 172.21.15.99:0/1468050519 >> v2:172.21.15.196:3300/0 conn(0x7f1e6006ee30 msgr2=0x7f1e601604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/1468050519 >> v2:172.21.15.196:3300/0 conn(0x7f1e6006ee30 0x7f1e601604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.451+0000 7f1e5ffff640 1 -- 172.21.15.99:0/1468050519 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e600ceb10 con 0x7f1e600d0fb0 2024-09-10T11:39:35.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.452+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/1468050519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d0fb0 0x7f1e6006e8f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1e600afa30 tx=0x7f1e4c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba219d55013cc5af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.452+0000 7f1e5f7fe640 1 -- 172.21.15.99:0/1468050519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c037070 con 0x7f1e600d0fb0 2024-09-10T11:39:35.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.452+0000 7f1e5f7fe640 1 -- 172.21.15.99:0/1468050519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c032440 con 0x7f1e600d0fb0 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.452+0000 7f1e5f7fe640 1 -- 172.21.15.99:0/1468050519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c032810 con 0x7f1e600d0fb0 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 -- 172.21.15.99:0/1468050519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d0fb0 msgr2=0x7f1e6006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 --2- 172.21.15.99:0/1468050519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d0fb0 0x7f1e6006e8f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1e600afa30 tx=0x7f1e4c02a1e0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 -- 172.21.15.99:0/1468050519 shutdown_connections 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 --2- 172.21.15.99:0/1468050519 >> v2:172.21.15.196:3300/0 conn(0x7f1e6006ee30 0x7f1e601604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 --2- 172.21.15.99:0/1468050519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d0fb0 0x7f1e6006e8f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 --2- 172.21.15.99:0/1468050519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e600d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 -- 172.21.15.99:0/1468050519 >> 172.21.15.99:0/1468050519 conn(0x7f1e6006bc10 msgr2=0x7f1e6006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 -- 172.21.15.99:0/1468050519 shutdown_connections 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.453+0000 7f1e6720a640 1 -- 172.21.15.99:0/1468050519 wait complete. 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.454+0000 7f1e6720a640 1 Processor -- start 2024-09-10T11:39:35.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.454+0000 7f1e6720a640 1 -- start start 2024-09-10T11:39:35.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e600d0fb0 0x7f1e60151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e60142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600b4050 con 0x7f1e6006ee30 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1e600b3d70 con 0x7f1e600d0fb0 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1e600b4330 con 0x7f1e6015c7b0 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58340/0 (socket says 172.21.15.99:58340) 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 -- 172.21.15.99:0/302290077 learned_addr learned my addr 172.21.15.99:0/302290077 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e600d0fb0 0x7f1e60151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e657a7640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e60142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 -- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 msgr2=0x7f1e60142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e60142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 -- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e600d0fb0 msgr2=0x7f1e60151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e600d0fb0 0x7f1e60151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 -- 172.21.15.99:0/302290077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6014e6b0 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e64fa6640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b440 tx=0x7f1e5400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54015020 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e54011830 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6014e950 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e54011c20 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.455+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6014ef10 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.456+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6015cb90 con 0x7f1e6006ee30 2024-09-10T11:39:35.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.457+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e54018020 con 0x7f1e6006ee30 2024-09-10T11:39:35.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.457+0000 7f1e5d7fa640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 0x7f1e4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.457+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5408aca0 con 0x7f1e6006ee30 2024-09-10T11:39:35.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.457+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 0x7f1e4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.458+0000 7f1e5ffff640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 0x7f1e4406e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c034000 tx=0x7f1e4c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.462+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e5401f020 con 0x7f1e6006ee30 2024-09-10T11:39:35.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.629+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e6006d710 con 0x7f1e6006ee30 2024-09-10T11:39:35.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.630+0000 7f1e5d7fa640 1 -- 172.21.15.99:0/302290077 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) ==== 74+0+16753 (secure 0 0 0) 0x7f1e54058f70 con 0x7f1e6006ee30 2024-09-10T11:39:35.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:35.631 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":182,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:34.641686+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:39:33.255122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T11:39:33.614490+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 msgr2=0x7f1e4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 0x7f1e4406e2f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c034000 tx=0x7f1e4c02e000 comp rx=0 tx=0).stop 2024-09-10T11:39:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 msgr2=0x7f1e60154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400b440 tx=0x7f1e5400b900 comp rx=0 tx=0).stop 2024-09-10T11:39:35.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 shutdown_connections 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1e4406be30 0x7f1e4406e2f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e6015c7b0 0x7f1e60142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e600d0fb0 0x7f1e60151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 --2- 172.21.15.99:0/302290077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e6006ee30 0x7f1e60154740 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 >> 172.21.15.99:0/302290077 conn(0x7f1e6006bc10 msgr2=0x7f1e6006dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 shutdown_connections 2024-09-10T11:39:35.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.632+0000 7f1e6720a640 1 -- 172.21.15.99:0/302290077 wait complete. 2024-09-10T11:39:35.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:35.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.743+0000 7fa48059f640 1 Processor -- start 2024-09-10T11:39:35.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.743+0000 7fa48059f640 1 -- start start 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa48059f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa4780cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa48059f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780cea00 con 0x7fa4780d1000 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa48059f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780d1770 con 0x7fa4780cf410 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa48059f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780cd810 con 0x7fa47815cc30 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47eb3c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa4780d1000 0x7fa4780ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40192/0 (socket says 172.21.15.99:40192) 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4132616287 learned_addr learned my addr 172.21.15.99:0/4132616287 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:35.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47db3a640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa4780cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539263109 0 0) 0x7fa4780cea00 con 0x7fa4780d1000 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa464003520 con 0x7fa4780d1000 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 519896234 0 0) 0x7fa464003520 con 0x7fa4780d1000 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 msgr2=0x7fa4780cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa4780cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 >> v1:172.21.15.196:6789/0 conn(0x7fa4780cf410 legacy=0x7fa4780cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780cdaf0 con 0x7fa4780d1000 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa4700019f0 con 0x7fa4780d1000 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 >> v1:172.21.15.99:6789/0 conn(0x7fa4780d1000 legacy=0x7fa4780ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.744+0000 7fa47d339640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4640039e0 0x7fa464003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47d339640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa464004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47d339640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa464004d40 0x7fa4640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780cd810 con 0x7fa464004d40 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa46400bbb0 con 0x7fa4640039e0 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa46400bdb0 con 0x7fa47815cc30 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa464004d40 0x7fa4640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47db3a640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa464004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4640039e0 0x7fa464003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 msgr2=0x7fa464004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa47815cc30 0x7fa464004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4640039e0 msgr2=0x7fa464003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4640039e0 0x7fa464003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4132616287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa464003520 con 0x7fa464004d40 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.745+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa4640039e0 0x7fa464003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:35.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa464004d40 0x7fa4640051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa47000a850 tx=0x7fa464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa11b9743b0edac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000a4a0 con 0x7fa464004d40 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa47002f070 con 0x7fa464004d40 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa47d339640 1 -- 172.21.15.99:0/4132616287 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47002bce0 con 0x7fa464004d40 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 -- 172.21.15.99:0/4132616287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa464004d40 msgr2=0x7fa4640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 --2- 172.21.15.99:0/4132616287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa464004d40 0x7fa4640051c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa47000a850 tx=0x7fa464005830 comp rx=0 tx=0).stop 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4132616287 reap_dead start 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 -- 172.21.15.99:0/4132616287 shutdown_connections 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 -- 172.21.15.99:0/4132616287 >> 172.21.15.99:0/4132616287 conn(0x7fa4780b4570 msgr2=0x7fa4780b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 -- 172.21.15.99:0/4132616287 shutdown_connections 2024-09-10T11:39:35.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.746+0000 7fa48059f640 1 -- 172.21.15.99:0/4132616287 wait complete. 2024-09-10T11:39:35.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 Processor -- start 2024-09-10T11:39:35.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 -- start start 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 0x7fa478155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 0x7fa47815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780cee30 con 0x7fa47815d550 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780cd4c0 con 0x7fa464004d40 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa48059f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780cefb0 con 0x7fa4780cc4e0 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa47eb3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa47eb3c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58354/0 (socket says 172.21.15.99:58354) 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.748+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4263292901 learned_addr learned my addr 172.21.15.99:0/4263292901 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47db3a640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 0x7fa47815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 0x7fa478155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 msgr2=0x7fa47815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 0x7fa47815d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 msgr2=0x7fa478155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 0x7fa478155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 -- 172.21.15.99:0/4263292901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780ca5a0 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47db3a640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 0x7fa47815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 0x7fa478155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa47eb3c640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa470001cf0 tx=0x7fa470001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000a480 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470044070 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4780ca880 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700346d0 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.749+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4780cae40 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.750+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4780cb040 con 0x7fa47815d550 2024-09-10T11:39:35.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.751+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa470034870 con 0x7fa47815d550 2024-09-10T11:39:35.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.751+0000 7fa46effd640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 0x7fa45c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.751+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7fa4700ab530 con 0x7fa47815d550 2024-09-10T11:39:35.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.751+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 0x7fa45c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.751+0000 7fa47e33b640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 0x7fa45c06e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa4740016b0 tx=0x7fa474009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.755+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4780cb040 con 0x7fa47815d550 2024-09-10T11:39:35.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.919+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4780cf7f0 con 0x7fa47815d550 2024-09-10T11:39:35.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.920+0000 7fa46effd640 1 -- 172.21.15.99:0/4263292901 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) ==== 74+0+16753 (secure 0 0 0) 0x7fa470079e50 con 0x7fa47815d550 2024-09-10T11:39:35.922 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:35.922 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":182,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:34.641686+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:39:33.255122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T11:39:33.614490+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:35.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 msgr2=0x7fa45c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 0x7fa45c06e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa4740016b0 tx=0x7fa474009040 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 msgr2=0x7fa47815d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fa470001cf0 tx=0x7fa470001dc0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 shutdown_connections 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa45c06be60 0x7fa45c06e320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa47815d550 0x7fa47815d9d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa4780cc4e0 0x7fa47815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 --2- 172.21.15.99:0/4263292901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa464004d40 0x7fa478155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.922+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 >> 172.21.15.99:0/4263292901 conn(0x7fa4780b4570 msgr2=0x7fa4780b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.923+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 shutdown_connections 2024-09-10T11:39:35.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:35.923+0000 7fa48059f640 1 -- 172.21.15.99:0/4263292901 wait complete. 2024-09-10T11:39:35.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T11:39:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.044+0000 7f129225e640 1 Processor -- start 2024-09-10T11:39:36.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.044+0000 7f129225e640 1 -- start start 2024-09-10T11:39:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f129225e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f128c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f129225e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f128c0ae020 0x7f128c160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f129225e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f128c0d1bd0 con 0x7f128c15cfd0 2024-09-10T11:39:36.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f129225e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f128c075620 con 0x7f128c0ae020 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f129225e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f128c0752b0 con 0x7f128c1605c0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128bfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f128c15cfd0 0x7f128c15d480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40194/0 (socket says 172.21.15.99:40194) 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128bfff640 1 -- 172.21.15.99:0/3237780452 learned_addr learned my addr 172.21.15.99:0/3237780452 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537048583 0 0) 0x7f128c0d1bd0 con 0x7f128c15cfd0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1274003520 con 0x7f128c15cfd0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128affd640 1 --2- 172.21.15.99:0/3237780452 >> v2:172.21.15.196:3300/0 conn(0x7f128c0ae020 0x7f128c160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.045+0000 7f128b7fe640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f128c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344678917 0 0) 0x7f1274003520 con 0x7f128c15cfd0 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 msgr2=0x7f128c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f128c1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 >> v2:172.21.15.196:3300/0 conn(0x7f128c0ae020 msgr2=0x7f128c160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 --2- 172.21.15.99:0/3237780452 >> v2:172.21.15.196:3300/0 conn(0x7f128c0ae020 0x7f128c160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0d1eb0 con 0x7f128c15cfd0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f1284001670 con 0x7f128c15cfd0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 >> v1:172.21.15.99:6789/0 conn(0x7f128c15cfd0 legacy=0x7f128c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12740039b0 0x7f1274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f1274004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1274004d40 0x7f12740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f128c075620 con 0x7f12740039b0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f128c0752b0 con 0x7f1274004d40 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f127400b9c0 con 0x7f128c1605c0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12740039b0 0x7f1274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128affd640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f1274004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128bfff640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1274004d40 0x7f12740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 -- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 msgr2=0x7f1274004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f1274004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 -- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1274004d40 msgr2=0x7f12740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1274004d40 0x7f12740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128b7fe640 1 -- 172.21.15.99:0/3237780452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274003520 con 0x7f12740039b0 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.046+0000 7f128affd640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128c1605c0 0x7f1274004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.047+0000 7f128b7fe640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12740039b0 0x7f1274003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f127c009fc0 tx=0x7f1274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32c0df0b5e1d42d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.047+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c009c10 con 0x7f12740039b0 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.047+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c036070 con 0x7f12740039b0 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.047+0000 7f128a7fc640 1 -- 172.21.15.99:0/3237780452 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c031880 con 0x7f12740039b0 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 -- 172.21.15.99:0/3237780452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12740039b0 msgr2=0x7f1274003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 --2- 172.21.15.99:0/3237780452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12740039b0 0x7f1274003dd0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f127c009fc0 tx=0x7f1274005830 comp rx=0 tx=0).stop 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f128bfff640 1 -- 172.21.15.99:0/3237780452 reap_dead start 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 -- 172.21.15.99:0/3237780452 shutdown_connections 2024-09-10T11:39:36.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 -- 172.21.15.99:0/3237780452 >> 172.21.15.99:0/3237780452 conn(0x7f128c0730c0 msgr2=0x7f128c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:36.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 -- 172.21.15.99:0/3237780452 shutdown_connections 2024-09-10T11:39:36.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.048+0000 7f129225e640 1 -- 172.21.15.99:0/3237780452 wait complete. 2024-09-10T11:39:36.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.049+0000 7f129225e640 1 Processor -- start 2024-09-10T11:39:36.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.049+0000 7f129225e640 1 -- start start 2024-09-10T11:39:36.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12740039b0 0x7f128c14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128c1e70c0 0x7f128c1e7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f128c073d00 con 0x7f128c1e63e0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f128c15a9e0 con 0x7f128c1e70c0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f128c073e80 con 0x7f12740039b0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58368/0 (socket says 172.21.15.99:58368) 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 -- 172.21.15.99:0/4077480805 learned_addr learned my addr 172.21.15.99:0/4077480805 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128affd640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128c1e70c0 0x7f128c1e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128b7fe640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12740039b0 0x7f128c14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 -- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12740039b0 msgr2=0x7f128c14e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12740039b0 0x7f128c14e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 -- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128c1e70c0 msgr2=0x7f128c1e7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128c1e70c0 0x7f128c1e7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 -- 172.21.15.99:0/4077480805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f128c0b5430 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f128bfff640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1284001eb0 tx=0x7f128400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128400c5b0 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284018070 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128400c9b0 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f128c0b5710 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.050+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f128c0ca590 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.051+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f128c1609a0 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.052+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f128400cb50 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.052+0000 7f1273fff640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 0x7f127806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.052+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 5709+0+0 (secure 0 0 0) 0x7f128408a480 con 0x7f128c1e63e0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.052+0000 7f128b7fe640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 0x7f127806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.053+0000 7f128b7fe640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 0x7f127806e320 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f127c035000 tx=0x7f127c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.056+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12840587e0 con 0x7f128c1e63e0 2024-09-10T11:39:36.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.255+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f128c15b520 con 0x7f128c1e63e0 2024-09-10T11:39:36.580 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-09-10T11:39:36.580+0000 7fc46b1d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:39:36.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.720+0000 7f1273fff640 1 -- 172.21.15.99:0/4077480805 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f128405cae0 con 0x7f128c1e63e0 2024-09-10T11:39:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 msgr2=0x7f127806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 0x7f127806e320 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f127c035000 tx=0x7f127c02f000 comp rx=0 tx=0).stop 2024-09-10T11:39:36.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 msgr2=0x7f128c1e6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1284001eb0 tx=0x7f128400eb90 comp rx=0 tx=0).stop 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 shutdown_connections 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f127806be60 0x7f127806e320 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f128c1e63e0 0x7f128c1e6860 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f128c1e70c0 0x7f128c1e7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 --2- 172.21.15.99:0/4077480805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12740039b0 0x7f128c14e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.722+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 >> 172.21.15.99:0/4077480805 conn(0x7f128c0730c0 msgr2=0x7f128c15d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.723+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 shutdown_connections 2024-09-10T11:39:36.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.723+0000 7f129225e640 1 -- 172.21.15.99:0/4077480805 wait complete. 2024-09-10T11:39:36.736 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T11:39:36.736 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T11:39:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.838+0000 7f85a696f640 1 Processor -- start 2024-09-10T11:39:36.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.838+0000 7f85a696f640 1 -- start start 2024-09-10T11:39:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f85a00d1200 0x7f85a0073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a006e5d0 0x7f85a006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f85a015c060 0x7f85a015c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f85a006ef50 con 0x7f85a015c060 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f85a0152e60 con 0x7f85a00d1200 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a696f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85a00d15e0 con 0x7f85a006e5d0 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f85a015c060 0x7f85a015c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f85a015c060 0x7f85a015c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58382/0 (socket says 172.21.15.99:58382) 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 -- 172.21.15.99:0/2060879868 learned_addr learned my addr 172.21.15.99:0/2060879868 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 -- 172.21.15.99:0/2060879868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a006e5d0 msgr2=0x7f85a006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/2060879868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a006e5d0 0x7f85a006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 -- 172.21.15.99:0/2060879868 >> v2:172.21.15.196:3300/0 conn(0x7f85a00d1200 msgr2=0x7f85a0073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/2060879868 >> v2:172.21.15.196:3300/0 conn(0x7f85a00d1200 0x7f85a0073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.839+0000 7f85a4f0c640 1 -- 172.21.15.99:0/2060879868 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a0072680 con 0x7f85a015c060 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.840+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/2060879868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a015c480 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85a00d1850 tx=0x7f859802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9fbbfe6d62525a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.840+0000 7f859effd640 1 -- 172.21.15.99:0/2060879868 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598036070 con 0x7f85a015c060 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.840+0000 7f859effd640 1 -- 172.21.15.99:0/2060879868 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8598031600 con 0x7f85a015c060 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.840+0000 7f859effd640 1 -- 172.21.15.99:0/2060879868 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980319d0 con 0x7f85a015c060 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 -- 172.21.15.99:0/2060879868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 msgr2=0x7f85a015c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 --2- 172.21.15.99:0/2060879868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a015c480 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f85a00d1850 tx=0x7f859802a610 comp rx=0 tx=0).stop 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 -- 172.21.15.99:0/2060879868 shutdown_connections 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 --2- 172.21.15.99:0/2060879868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a015c480 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 --2- 172.21.15.99:0/2060879868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a006e5d0 0x7f85a006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 --2- 172.21.15.99:0/2060879868 >> v2:172.21.15.196:3300/0 conn(0x7f85a00d1200 0x7f85a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 -- 172.21.15.99:0/2060879868 >> 172.21.15.99:0/2060879868 conn(0x7f85a006bc10 msgr2=0x7f85a006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 -- 172.21.15.99:0/2060879868 shutdown_connections 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.841+0000 7f85a696f640 1 -- 172.21.15.99:0/2060879868 wait complete. 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.842+0000 7f85a696f640 1 Processor -- start 2024-09-10T11:39:36.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.842+0000 7f85a696f640 1 -- start start 2024-09-10T11:39:36.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 0x7f85a0153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 0x7f85a0154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0072f20 con 0x7f85a015c060 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0072c40 con 0x7f85a006e5d0 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a696f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f85a015cd90 con 0x7f85a00d1200 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58386/0 (socket says 172.21.15.99:58386) 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 -- 172.21.15.99:0/3403855198 learned_addr learned my addr 172.21.15.99:0/3403855198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f859f7fe640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 0x7f85a0154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f859ffff640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 0x7f85a0153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 -- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 msgr2=0x7f85a0154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 0x7f85a0154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 -- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 msgr2=0x7f85a0153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 0x7f85a0153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 -- 172.21.15.99:0/3403855198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a0153570 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f859f7fe640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 0x7f85a0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f859ffff640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 0x7f85a0153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.843+0000 7f85a4f0c640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8598009f40 tx=0x7f8598004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.844+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598041020 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.844+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85980077a0 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.844+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598036040 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.844+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a0154870 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.844+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a013da60 con 0x7f85a015c060 2024-09-10T11:39:36.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.845+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a006ea10 con 0x7f85a015c060 2024-09-10T11:39:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.845+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8598044020 con 0x7f85a015c060 2024-09-10T11:39:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.846+0000 7f859cff9640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 0x7f858c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.846+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f85980abd40 con 0x7f85a015c060 2024-09-10T11:39:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.846+0000 7f859ffff640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 0x7f858c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:36.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.846+0000 7f859ffff640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 0x7f858c06e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8590009890 tx=0x7f8590008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:36.850+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f859804b030 con 0x7f85a015c060 2024-09-10T11:39:37.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.051+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f85a00761b0 con 0x7f85a015c060 2024-09-10T11:39:37.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.726+0000 7f859cff9640 1 -- 172.21.15.99:0/3403855198 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7f859807a660 con 0x7f85a015c060 2024-09-10T11:39:37.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 msgr2=0x7f858c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:37.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 0x7f858c06e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8590009890 tx=0x7f8590008040 comp rx=0 tx=0).stop 2024-09-10T11:39:37.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 msgr2=0x7f85a006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8598009f40 tx=0x7f8598004a40 comp rx=0 tx=0).stop 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 shutdown_connections 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f858c06be60 0x7f858c06e320 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85a015c060 0x7f85a006d4c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85a00d1200 0x7f85a0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 --2- 172.21.15.99:0/3403855198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85a006e5d0 0x7f85a0153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.728+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 >> 172.21.15.99:0/3403855198 conn(0x7f85a006bc10 msgr2=0x7f85a0151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.729+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 shutdown_connections 2024-09-10T11:39:37.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.729+0000 7f85a696f640 1 -- 172.21.15.99:0/3403855198 wait complete. 2024-09-10T11:39:37.743 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T11:39:37.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:37.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.845+0000 7fe7e59e4640 1 Processor -- start 2024-09-10T11:39:37.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.845+0000 7fe7e59e4640 1 -- start start 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7e59e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7e015eb80 0x7fe7e00d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7e59e4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe7e00d5010 0x7fe7e00d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7e59e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e015ff70 con 0x7fe7e01626c0 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7e59e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e00d1620 con 0x7fe7e015eb80 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7e59e4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e0162bc0 con 0x7fe7e00d5010 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7e015eb80 0x7fe7e00d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe7e015eb80 0x7fe7e00d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58400/0 (socket says 172.21.15.99:58400) 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 -- 172.21.15.99:0/3074991011 learned_addr learned my addr 172.21.15.99:0/3074991011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7df7fe640 1 --2- 172.21.15.99:0/3074991011 >> v2:172.21.15.196:3300/0 conn(0x7fe7e00d5010 0x7fe7e00d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 -- 172.21.15.99:0/3074991011 >> v2:172.21.15.196:3300/0 conn(0x7fe7e00d5010 msgr2=0x7fe7e00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 --2- 172.21.15.99:0/3074991011 >> v2:172.21.15.196:3300/0 conn(0x7fe7e00d5010 0x7fe7e00d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 -- 172.21.15.99:0/3074991011 >> v1:172.21.15.196:6789/0 conn(0x7fe7e01626c0 legacy=0x7fe7e00d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.846+0000 7fe7deffd640 1 -- 172.21.15.99:0/3074991011 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e0162ea0 con 0x7fe7e015eb80 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.847+0000 7fe7deffd640 1 --2- 172.21.15.99:0/3074991011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e00d29b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009870 tx=0x7fe7d002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0267ad1dc02010 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.847+0000 7fe7ddffb640 1 -- 172.21.15.99:0/3074991011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0037070 con 0x7fe7e015eb80 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.847+0000 7fe7ddffb640 1 -- 172.21.15.99:0/3074991011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0032440 con 0x7fe7e015eb80 2024-09-10T11:39:37.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.847+0000 7fe7ddffb640 1 -- 172.21.15.99:0/3074991011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0032810 con 0x7fe7e015eb80 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 -- 172.21.15.99:0/3074991011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 msgr2=0x7fe7e00d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/3074991011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e00d29b0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0009870 tx=0x7fe7d002a1c0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 -- 172.21.15.99:0/3074991011 shutdown_connections 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/3074991011 >> v2:172.21.15.196:3300/0 conn(0x7fe7e00d5010 0x7fe7e00d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/3074991011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e00d29b0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 -- 172.21.15.99:0/3074991011 >> 172.21.15.99:0/3074991011 conn(0x7fe7e008e680 msgr2=0x7fe7e009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 -- 172.21.15.99:0/3074991011 shutdown_connections 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.848+0000 7fe7e59e4640 1 -- 172.21.15.99:0/3074991011 wait complete. 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.849+0000 7fe7e59e4640 1 Processor -- start 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.849+0000 7fe7e59e4640 1 -- start start 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7e00d5010 0x7fe7e015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7e01626c0 0x7fe7e00b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e00d2ef0 con 0x7fe7e015eb80 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e0163460 con 0x7fe7e01626c0 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7e59e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e00d3070 con 0x7fe7e00d5010 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58410/0 (socket says 172.21.15.99:58410) 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 -- 172.21.15.99:0/2795653103 learned_addr learned my addr 172.21.15.99:0/2795653103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7df7fe640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7e01626c0 0x7fe7e00b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7deffd640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7e00d5010 0x7fe7e015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 -- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7e00d5010 msgr2=0x7fe7e015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7e00d5010 0x7fe7e015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 -- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7e01626c0 msgr2=0x7fe7e00b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7e01626c0 0x7fe7e00b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 -- 172.21.15.99:0/2795653103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e00b25e0 con 0x7fe7e015eb80 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.850+0000 7fe7de7fc640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3c0 tx=0x7fe7d400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.851+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4011830 con 0x7fe7e015eb80 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.851+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d4016070 con 0x7fe7e015eb80 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.851+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4011c50 con 0x7fe7e015eb80 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.851+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e00b28c0 con 0x7fe7e015eb80 2024-09-10T11:39:37.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.851+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e00b2e80 con 0x7fe7e015eb80 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.852+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e00d29b0 con 0x7fe7e015eb80 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.852+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7d401b460 con 0x7fe7e015eb80 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.853+0000 7fe7c7fff640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 0x7fe7cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.853+0000 7fe7deffd640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 0x7fe7cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.853+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7fe7d408a810 con 0x7fe7e015eb80 2024-09-10T11:39:37.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.853+0000 7fe7deffd640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 0x7fe7cc06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0034000 tx=0x7fe7d002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:37.857+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7d4020020 con 0x7fe7e015eb80 2024-09-10T11:39:38.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.058+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7e00d0020 con 0x7fe7e015eb80 2024-09-10T11:39:38.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.058+0000 7fe7c7fff640 1 -- 172.21.15.99:0/2795653103 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7fe7d4058b70 con 0x7fe7e015eb80 2024-09-10T11:39:38.059 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:38.059 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:39:38.060 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 msgr2=0x7fe7cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 0x7fe7cc06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0034000 tx=0x7fe7d002e000 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 msgr2=0x7fe7e0150110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400b3c0 tx=0x7fe7d400b880 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 shutdown_connections 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fe7cc06be30 0x7fe7cc06e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7e01626c0 0x7fe7e00b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7e015eb80 0x7fe7e0150110 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 --2- 172.21.15.99:0/2795653103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7e00d5010 0x7fe7e015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 >> 172.21.15.99:0/2795653103 conn(0x7fe7e008e680 msgr2=0x7fe7e006c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 shutdown_connections 2024-09-10T11:39:38.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.061+0000 7fe7e59e4640 1 -- 172.21.15.99:0/2795653103 wait complete. 2024-09-10T11:39:38.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:39:38.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.171+0000 7f9f4259f640 1 Processor -- start 2024-09-10T11:39:38.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.171+0000 7f9f4259f640 1 -- start start 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.171+0000 7f9f4259f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4259f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9f3c0d5790 0x7f9f3c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4259f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c0cf9f0 con 0x7f9f3c0d1a90 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4259f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c0d0a90 con 0x7f9f3c15ec30 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4259f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f9f3c0d1ee0 con 0x7f9f3c0d5790 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f3c15ec30 0x7f9f3c0d5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58426/0 (socket says 172.21.15.99:58426) 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 -- 172.21.15.99:0/3300693955 learned_addr learned my addr 172.21.15.99:0/3300693955 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f40d9c640 1 --2- 172.21.15.99:0/3300693955 >> v2:172.21.15.196:3300/0 conn(0x7f9f3c0d5790 0x7f9f3c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 -- 172.21.15.99:0/3300693955 >> v2:172.21.15.196:3300/0 conn(0x7f9f3c0d5790 msgr2=0x7f9f3c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 --2- 172.21.15.99:0/3300693955 >> v2:172.21.15.196:3300/0 conn(0x7f9f3c0d5790 0x7f9f3c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 -- 172.21.15.99:0/3300693955 >> v1:172.21.15.196:6789/0 conn(0x7f9f3c0d1a90 legacy=0x7f9f3c0d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.172+0000 7f9f4159d640 1 -- 172.21.15.99:0/3300693955 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0cfcd0 con 0x7f9f3c15ec30 2024-09-10T11:39:38.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4159d640 1 --2- 172.21.15.99:0/3300693955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0d5250 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0b6600 tx=0x7f9f3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf4f69a2f032626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f3bfff640 1 -- 172.21.15.99:0/3300693955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34037070 con 0x7f9f3c15ec30 2024-09-10T11:39:38.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f3bfff640 1 -- 172.21.15.99:0/3300693955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f34032440 con 0x7f9f3c15ec30 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f3bfff640 1 -- 172.21.15.99:0/3300693955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34032810 con 0x7f9f3c15ec30 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 -- 172.21.15.99:0/3300693955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c15ec30 msgr2=0x7f9f3c0d5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 --2- 172.21.15.99:0/3300693955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0d5250 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0b6600 tx=0x7f9f3402a270 comp rx=0 tx=0).stop 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 -- 172.21.15.99:0/3300693955 shutdown_connections 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 --2- 172.21.15.99:0/3300693955 >> v2:172.21.15.196:3300/0 conn(0x7f9f3c0d5790 0x7f9f3c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 --2- 172.21.15.99:0/3300693955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0d5250 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.173+0000 7f9f4259f640 1 -- 172.21.15.99:0/3300693955 >> 172.21.15.99:0/3300693955 conn(0x7f9f3c09de10 msgr2=0x7f9f3c0d4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.174+0000 7f9f4259f640 1 -- 172.21.15.99:0/3300693955 shutdown_connections 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.174+0000 7f9f4259f640 1 -- 172.21.15.99:0/3300693955 wait complete. 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 Processor -- start 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 -- start start 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f3c0d1a90 0x7f9f3c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0d0290 con 0x7f9f3c0d5790 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0d0600 con 0x7f9f3c15ec30 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f4259f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c15f010 con 0x7f9f3c0d1a90 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f40d9c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f40d9c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58440/0 (socket says 172.21.15.99:58440) 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.175+0000 7f9f40d9c640 1 -- 172.21.15.99:0/1919150 learned_addr learned my addr 172.21.15.99:0/1919150 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f4159d640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f3c0d1a90 0x7f9f3c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f41d9e640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 -- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f3c0d1a90 msgr2=0x7f9f3c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f3c0d1a90 0x7f9f3c15cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 -- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 msgr2=0x7f9f3c0a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 -- 172.21.15.99:0/1919150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c0a1110 con 0x7f9f3c0d5790 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f41d9e640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f40d9c640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d830 tx=0x7f9f3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000b6c0 con 0x7f9f3c0d5790 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f30019070 con 0x7f9f3c0d5790 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c0a1380 con 0x7f9f3c0d5790 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000bac0 con 0x7f9f3c0d5790 2024-09-10T11:39:38.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.176+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c0cda80 con 0x7f9f3c0d5790 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.177+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c0a1b50 con 0x7f9f3c0d5790 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.178+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f3001e460 con 0x7f9f3c0d5790 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.178+0000 7f9f39ffb640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 0x7f9f2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.178+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f9f3008b600 con 0x7f9f3c0d5790 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.178+0000 7f9f4159d640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 0x7f9f2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.179+0000 7f9f4159d640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 0x7f9f2806e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f34034000 tx=0x7f9f3402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.182+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f30059f20 con 0x7f9f3c0d5790 2024-09-10T11:39:38.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.346+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f3c0cf650 con 0x7f9f3c0d5790 2024-09-10T11:39:38.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.347+0000 7f9f39ffb640 1 -- 172.21.15.99:0/1919150 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+16740 (secure 0 0 0) 0x7f9f30059820 con 0x7f9f3c0d5790 2024-09-10T11:39:38.348 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:38.348 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":183,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:39:36.314361+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:39:33.255122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T11:39:33.614490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:39:38.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 msgr2=0x7f9f2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 0x7f9f2806e2f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9f34034000 tx=0x7f9f3402e000 comp rx=0 tx=0).stop 2024-09-10T11:39:38.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 msgr2=0x7f9f3c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d830 tx=0x7f9f3000dcf0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 shutdown_connections 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9f2806be30 0x7f9f2806e2f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9f3c15ec30 0x7f9f3c0a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f3c0d5790 0x7f9f3c0cd0c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 --2- 172.21.15.99:0/1919150 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9f3c0d1a90 0x7f9f3c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 >> 172.21.15.99:0/1919150 conn(0x7f9f3c09de10 msgr2=0x7f9f3c15dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 shutdown_connections 2024-09-10T11:39:38.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.349+0000 7f9f4259f640 1 -- 172.21.15.99:0/1919150 wait complete. 2024-09-10T11:39:38.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:39:38.361 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:39:38.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.459+0000 7f15a12f1640 1 Processor -- start 2024-09-10T11:39:38.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.459+0000 7f15a12f1640 1 -- start start 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.459+0000 7f15a12f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f159c1626c0 0x7f159c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.459+0000 7f15a12f1640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c15ff70 con 0x7f159c0d4e40 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15a12f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159c0d1620 con 0x7f159c15eb80 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15a12f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f159c0d30c0 con 0x7f159c1626c0 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f159b7fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f159c0d4e40 0x7f159c0d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40208/0 (socket says 172.21.15.99:40208) 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f159b7fe640 1 -- 172.21.15.99:0/2997292267 learned_addr learned my addr 172.21.15.99:0/2997292267 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15927fc640 1 --2- 172.21.15.99:0/2997292267 >> v2:172.21.15.196:3300/0 conn(0x7f159c1626c0 0x7f159c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f159a7fc640 1 -- 172.21.15.99:0/2997292267 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028771518 0 0) 0x7f159c15ff70 con 0x7f159c0d4e40 2024-09-10T11:39:38.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f159a7fc640 1 -- 172.21.15.99:0/2997292267 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1588003520 con 0x7f159c0d4e40 2024-09-10T11:39:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15927fc640 1 -- 172.21.15.99:0/2997292267 >> v1:172.21.15.196:6789/0 conn(0x7f159c15eb80 legacy=0x7f159c0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15927fc640 1 -- 172.21.15.99:0/2997292267 >> v1:172.21.15.99:6789/0 conn(0x7f159c0d4e40 legacy=0x7f159c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.460+0000 7f15927fc640 1 -- 172.21.15.99:0/2997292267 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c162bc0 con 0x7f159c1626c0 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.461+0000 7f15927fc640 1 --2- 172.21.15.99:0/2997292267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c1626c0 0x7f159c0d4900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f15840097d0 tx=0x7f158402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61e42bc80e97e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.461+0000 7f159a7fc640 1 -- 172.21.15.99:0/2997292267 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584037070 con 0x7f159c1626c0 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.461+0000 7f159a7fc640 1 -- 172.21.15.99:0/2997292267 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1584032440 con 0x7f159c1626c0 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.461+0000 7f159a7fc640 1 -- 172.21.15.99:0/2997292267 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840327f0 con 0x7f159c1626c0 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 -- 172.21.15.99:0/2997292267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c1626c0 msgr2=0x7f159c0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 --2- 172.21.15.99:0/2997292267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c1626c0 0x7f159c0d4900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f15840097d0 tx=0x7f158402a1c0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 -- 172.21.15.99:0/2997292267 shutdown_connections 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 --2- 172.21.15.99:0/2997292267 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c1626c0 0x7f159c0d4900 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 -- 172.21.15.99:0/2997292267 >> 172.21.15.99:0/2997292267 conn(0x7f159c08e680 msgr2=0x7f159c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 -- 172.21.15.99:0/2997292267 shutdown_connections 2024-09-10T11:39:38.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.462+0000 7f15a12f1640 1 -- 172.21.15.99:0/2997292267 wait complete. 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.463+0000 7f15a12f1640 1 Processor -- start 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.463+0000 7f15a12f1640 1 -- start start 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f159c0d4e40 0x7f159c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c15eb80 0x7f159c0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f159c163460 con 0x7f159c1626c0 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f159c163180 con 0x7f159c15eb80 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15a12f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f159c0d5290 con 0x7f159c0d4e40 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58444/0 (socket says 172.21.15.99:58444) 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 -- 172.21.15.99:0/1782366994 learned_addr learned my addr 172.21.15.99:0/1782366994 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f15927fc640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c15eb80 0x7f159c0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159affd640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f159c0d4e40 0x7f159c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 -- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f159c0d4e40 msgr2=0x7f159c15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f159c0d4e40 0x7f159c15ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 -- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c15eb80 msgr2=0x7f159c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c15eb80 0x7f159c0ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 -- 172.21.15.99:0/1782366994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159c15b1d0 con 0x7f159c1626c0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f159b7fe640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f159400caa0 tx=0x7f159400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594017070 con 0x7f159c1626c0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.464+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159400ac00 con 0x7f159c1626c0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.465+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159c0b1ad0 con 0x7f159c1626c0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.465+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594012400 con 0x7f159c1626c0 2024-09-10T11:39:38.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.465+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f159c1f8960 con 0x7f159c1626c0 2024-09-10T11:39:38.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.466+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159c0d4900 con 0x7f159c1626c0 2024-09-10T11:39:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.466+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f159401a070 con 0x7f159c1626c0 2024-09-10T11:39:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.466+0000 7f1593fff640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 0x7f157c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.467+0000 7f159affd640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 0x7f157c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.467+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f159408ab90 con 0x7f159c1626c0 2024-09-10T11:39:38.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.467+0000 7f159affd640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 0x7f157c06e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f158c000f60 tx=0x7f158c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.471+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1594058ef0 con 0x7f159c1626c0 2024-09-10T11:39:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.633+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f159c0ceb50 con 0x7f157c06be30 2024-09-10T11:39:38.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.639+0000 7f1593fff640 1 -- 172.21.15.99:0/1782366994 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247995 (secure 0 0 0) 0x7f159c0ceb50 con 0x7f157c06be30 2024-09-10T11:39:38.643 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:38.644 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 msgr2=0x7f157c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 0x7f157c06e2f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f158c000f60 tx=0x7f158c009040 comp rx=0 tx=0).stop 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 msgr2=0x7f159c1f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f159400caa0 tx=0x7f159400cf60 comp rx=0 tx=0).stop 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 shutdown_connections 2024-09-10T11:39:38.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f157c06be30 0x7f157c06e2f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f159c1626c0 0x7f159c1f7bc0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f159c15eb80 0x7f159c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 --2- 172.21.15.99:0/1782366994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f159c0d4e40 0x7f159c15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 >> 172.21.15.99:0/1782366994 conn(0x7f159c08e680 msgr2=0x7f159c0d0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 shutdown_connections 2024-09-10T11:39:38.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.642+0000 7f15a12f1640 1 -- 172.21.15.99:0/1782366994 wait complete. 2024-09-10T11:39:38.654 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T11:39:37.029925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553355,"statfs":{"total":1600000000,"available":1590636536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000841"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317762+0000","last_change":"2024-09-10T11:39:35.326443+0000","last_active":"2024-09-10T11:39:36.317762+0000","last_peered":"2024-09-10T11:39:36.317762+0000","last_clean":"2024-09-10T11:39:36.317762+0000","last_became_active":"2024-09-10T11:39:35.326226+0000","last_became_peered":"2024-09-10T11:39:35.326226+0000","last_unstale":"2024-09-10T11:39:36.317762+0000","last_undegraded":"2024-09-10T11:39:36.317762+0000","last_fullsized":"2024-09-10T11:39:36.317762+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:18.479271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317628+0000","last_change":"2024-09-10T11:39:35.322939+0000","last_active":"2024-09-10T11:39:36.317628+0000","last_peered":"2024-09-10T11:39:36.317628+0000","last_clean":"2024-09-10T11:39:36.317628+0000","last_became_active":"2024-09-10T11:39:35.322717+0000","last_became_peered":"2024-09-10T11:39:35.322717+0000","last_unstale":"2024-09-10T11:39:36.317628+0000","last_undegraded":"2024-09-10T11:39:36.317628+0000","last_fullsized":"2024-09-10T11:39:36.317628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:10.046604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317819+0000","last_change":"2024-09-10T11:39:35.331052+0000","last_active":"2024-09-10T11:39:36.317819+0000","last_peered":"2024-09-10T11:39:36.317819+0000","last_clean":"2024-09-10T11:39:36.317819+0000","last_became_active":"2024-09-10T11:39:35.330518+0000","last_became_peered":"2024-09-10T11:39:35.330518+0000","last_unstale":"2024-09-10T11:39:36.317819+0000","last_undegraded":"2024-09-10T11:39:36.317819+0000","last_fullsized":"2024-09-10T11:39:36.317819+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:48.123021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318477+0000","last_change":"2024-09-10T11:39:35.334883+0000","last_active":"2024-09-10T11:39:36.318477+0000","last_peered":"2024-09-10T11:39:36.318477+0000","last_clean":"2024-09-10T11:39:36.318477+0000","last_became_active":"2024-09-10T11:39:35.332905+0000","last_became_peered":"2024-09-10T11:39:35.332905+0000","last_unstale":"2024-09-10T11:39:36.318477+0000","last_undegraded":"2024-09-10T11:39:36.318477+0000","last_fullsized":"2024-09-10T11:39:36.318477+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:59.298486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642788+0000","last_change":"2024-09-10T11:39:35.319109+0000","last_active":"2024-09-10T11:39:36.642788+0000","last_peered":"2024-09-10T11:39:36.642788+0000","last_clean":"2024-09-10T11:39:36.642788+0000","last_became_active":"2024-09-10T11:39:35.318825+0000","last_became_peered":"2024-09-10T11:39:35.318825+0000","last_unstale":"2024-09-10T11:39:36.642788+0000","last_undegraded":"2024-09-10T11:39:36.642788+0000","last_fullsized":"2024-09-10T11:39:36.642788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:50.429178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642272+0000","last_change":"2024-09-10T11:39:35.327528+0000","last_active":"2024-09-10T11:39:36.642272+0000","last_peered":"2024-09-10T11:39:36.642272+0000","last_clean":"2024-09-10T11:39:36.642272+0000","last_became_active":"2024-09-10T11:39:35.327150+0000","last_became_peered":"2024-09-10T11:39:35.327150+0000","last_unstale":"2024-09-10T11:39:36.642272+0000","last_undegraded":"2024-09-10T11:39:36.642272+0000","last_fullsized":"2024-09-10T11:39:36.642272+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:12.926848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317623+0000","last_change":"2024-09-10T11:39:35.319852+0000","last_active":"2024-09-10T11:39:36.317623+0000","last_peered":"2024-09-10T11:39:36.317623+0000","last_clean":"2024-09-10T11:39:36.317623+0000","last_became_active":"2024-09-10T11:39:35.319588+0000","last_became_peered":"2024-09-10T11:39:35.319588+0000","last_unstale":"2024-09-10T11:39:36.317623+0000","last_undegraded":"2024-09-10T11:39:36.317623+0000","last_fullsized":"2024-09-10T11:39:36.317623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:57.886470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642915+0000","last_change":"2024-09-10T11:39:35.326991+0000","last_active":"2024-09-10T11:39:36.642915+0000","last_peered":"2024-09-10T11:39:36.642915+0000","last_clean":"2024-09-10T11:39:36.642915+0000","last_became_active":"2024-09-10T11:39:35.326754+0000","last_became_peered":"2024-09-10T11:39:35.326754+0000","last_unstale":"2024-09-10T11:39:36.642915+0000","last_undegraded":"2024-09-10T11:39:36.642915+0000","last_fullsized":"2024-09-10T11:39:36.642915+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:47.594010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642349+0000","last_change":"2024-09-10T11:39:35.328068+0000","last_active":"2024-09-10T11:39:36.642349+0000","last_peered":"2024-09-10T11:39:36.642349+0000","last_clean":"2024-09-10T11:39:36.642349+0000","last_became_active":"2024-09-10T11:39:35.327798+0000","last_became_peered":"2024-09-10T11:39:35.327798+0000","last_unstale":"2024-09-10T11:39:36.642349+0000","last_undegraded":"2024-09-10T11:39:36.642349+0000","last_fullsized":"2024-09-10T11:39:36.642349+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:38.692148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642398+0000","last_change":"2024-09-10T11:39:35.320792+0000","last_active":"2024-09-10T11:39:36.642398+0000","last_peered":"2024-09-10T11:39:36.642398+0000","last_clean":"2024-09-10T11:39:36.642398+0000","last_became_active":"2024-09-10T11:39:35.320542+0000","last_became_peered":"2024-09-10T11:39:35.320542+0000","last_unstale":"2024-09-10T11:39:36.642398+0000","last_undegraded":"2024-09-10T11:39:36.642398+0000","last_fullsized":"2024-09-10T11:39:36.642398+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:21.682381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642728+0000","last_change":"2024-09-10T11:39:35.331226+0000","last_active":"2024-09-10T11:39:36.642728+0000","last_peered":"2024-09-10T11:39:36.642728+0000","last_clean":"2024-09-10T11:39:36.642728+0000","last_became_active":"2024-09-10T11:39:35.330863+0000","last_became_peered":"2024-09-10T11:39:35.330863+0000","last_unstale":"2024-09-10T11:39:36.642728+0000","last_undegraded":"2024-09-10T11:39:36.642728+0000","last_fullsized":"2024-09-10T11:39:36.642728+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:44.289281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330489+0000","last_change":"2024-09-10T11:39:35.322254+0000","last_active":"2024-09-10T11:39:36.330489+0000","last_peered":"2024-09-10T11:39:36.330489+0000","last_clean":"2024-09-10T11:39:36.330489+0000","last_became_active":"2024-09-10T11:39:35.322019+0000","last_became_peered":"2024-09-10T11:39:35.322019+0000","last_unstale":"2024-09-10T11:39:36.330489+0000","last_undegraded":"2024-09-10T11:39:36.330489+0000","last_fullsized":"2024-09-10T11:39:36.330489+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:54.171624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330331+0000","last_change":"2024-09-10T11:39:35.332963+0000","last_active":"2024-09-10T11:39:36.330331+0000","last_peered":"2024-09-10T11:39:36.330331+0000","last_clean":"2024-09-10T11:39:36.330331+0000","last_became_active":"2024-09-10T11:39:35.332724+0000","last_became_peered":"2024-09-10T11:39:35.332724+0000","last_unstale":"2024-09-10T11:39:36.330331+0000","last_undegraded":"2024-09-10T11:39:36.330331+0000","last_fullsized":"2024-09-10T11:39:36.330331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:14.419829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642655+0000","last_change":"2024-09-10T11:39:35.318944+0000","last_active":"2024-09-10T11:39:36.642655+0000","last_peered":"2024-09-10T11:39:36.642655+0000","last_clean":"2024-09-10T11:39:36.642655+0000","last_became_active":"2024-09-10T11:39:35.318718+0000","last_became_peered":"2024-09-10T11:39:35.318718+0000","last_unstale":"2024-09-10T11:39:36.642655+0000","last_undegraded":"2024-09-10T11:39:36.642655+0000","last_fullsized":"2024-09-10T11:39:36.642655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:38.896709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642555+0000","last_change":"2024-09-10T11:39:35.325956+0000","last_active":"2024-09-10T11:39:36.642555+0000","last_peered":"2024-09-10T11:39:36.642555+0000","last_clean":"2024-09-10T11:39:36.642555+0000","last_became_active":"2024-09-10T11:39:35.325412+0000","last_became_peered":"2024-09-10T11:39:35.325412+0000","last_unstale":"2024-09-10T11:39:36.642555+0000","last_undegraded":"2024-09-10T11:39:36.642555+0000","last_fullsized":"2024-09-10T11:39:36.642555+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:20.071561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317607+0000","last_change":"2024-09-10T11:39:35.323009+0000","last_active":"2024-09-10T11:39:36.317607+0000","last_peered":"2024-09-10T11:39:36.317607+0000","last_clean":"2024-09-10T11:39:36.317607+0000","last_became_active":"2024-09-10T11:39:35.322564+0000","last_became_peered":"2024-09-10T11:39:35.322564+0000","last_unstale":"2024-09-10T11:39:36.317607+0000","last_undegraded":"2024-09-10T11:39:36.317607+0000","last_fullsized":"2024-09-10T11:39:36.317607+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:05.739511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642544+0000","last_change":"2024-09-10T11:39:35.327517+0000","last_active":"2024-09-10T11:39:36.642544+0000","last_peered":"2024-09-10T11:39:36.642544+0000","last_clean":"2024-09-10T11:39:36.642544+0000","last_became_active":"2024-09-10T11:39:35.327257+0000","last_became_peered":"2024-09-10T11:39:35.327257+0000","last_unstale":"2024-09-10T11:39:36.642544+0000","last_undegraded":"2024-09-10T11:39:36.642544+0000","last_fullsized":"2024-09-10T11:39:36.642544+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:54.743964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318198+0000","last_change":"2024-09-10T11:39:35.329536+0000","last_active":"2024-09-10T11:39:36.318198+0000","last_peered":"2024-09-10T11:39:36.318198+0000","last_clean":"2024-09-10T11:39:36.318198+0000","last_became_active":"2024-09-10T11:39:35.329282+0000","last_became_peered":"2024-09-10T11:39:35.329282+0000","last_unstale":"2024-09-10T11:39:36.318198+0000","last_undegraded":"2024-09-10T11:39:36.318198+0000","last_fullsized":"2024-09-10T11:39:36.318198+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:32.765304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317547+0000","last_change":"2024-09-10T11:39:35.326704+0000","last_active":"2024-09-10T11:39:36.317547+0000","last_peered":"2024-09-10T11:39:36.317547+0000","last_clean":"2024-09-10T11:39:36.317547+0000","last_became_active":"2024-09-10T11:39:35.326408+0000","last_became_peered":"2024-09-10T11:39:35.326408+0000","last_unstale":"2024-09-10T11:39:36.317547+0000","last_undegraded":"2024-09-10T11:39:36.317547+0000","last_fullsized":"2024-09-10T11:39:36.317547+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:32.033088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330490+0000","last_change":"2024-09-10T11:39:35.328405+0000","last_active":"2024-09-10T11:39:36.330490+0000","last_peered":"2024-09-10T11:39:36.330490+0000","last_clean":"2024-09-10T11:39:36.330490+0000","last_became_active":"2024-09-10T11:39:35.328045+0000","last_became_peered":"2024-09-10T11:39:35.328045+0000","last_unstale":"2024-09-10T11:39:36.330490+0000","last_undegraded":"2024-09-10T11:39:36.330490+0000","last_fullsized":"2024-09-10T11:39:36.330490+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:53.024338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642382+0000","last_change":"2024-09-10T11:39:35.324134+0000","last_active":"2024-09-10T11:39:36.642382+0000","last_peered":"2024-09-10T11:39:36.642382+0000","last_clean":"2024-09-10T11:39:36.642382+0000","last_became_active":"2024-09-10T11:39:35.323860+0000","last_became_peered":"2024-09-10T11:39:35.323860+0000","last_unstale":"2024-09-10T11:39:36.642382+0000","last_undegraded":"2024-09-10T11:39:36.642382+0000","last_fullsized":"2024-09-10T11:39:36.642382+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:56.129995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642747+0000","last_change":"2024-09-10T11:39:35.325448+0000","last_active":"2024-09-10T11:39:36.642747+0000","last_peered":"2024-09-10T11:39:36.642747+0000","last_clean":"2024-09-10T11:39:36.642747+0000","last_became_active":"2024-09-10T11:39:35.325188+0000","last_became_peered":"2024-09-10T11:39:35.325188+0000","last_unstale":"2024-09-10T11:39:36.642747+0000","last_undegraded":"2024-09-10T11:39:36.642747+0000","last_fullsized":"2024-09-10T11:39:36.642747+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:35.145197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457023+0000","last_change":"2024-09-10T11:39:35.333305+0000","last_active":"2024-09-10T11:39:36.457023+0000","last_peered":"2024-09-10T11:39:36.457023+0000","last_clean":"2024-09-10T11:39:36.457023+0000","last_became_active":"2024-09-10T11:39:35.332223+0000","last_became_peered":"2024-09-10T11:39:35.332223+0000","last_unstale":"2024-09-10T11:39:36.457023+0000","last_undegraded":"2024-09-10T11:39:36.457023+0000","last_fullsized":"2024-09-10T11:39:36.457023+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:36.123100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642408+0000","last_change":"2024-09-10T11:39:35.319263+0000","last_active":"2024-09-10T11:39:36.642408+0000","last_peered":"2024-09-10T11:39:36.642408+0000","last_clean":"2024-09-10T11:39:36.642408+0000","last_became_active":"2024-09-10T11:39:35.318878+0000","last_became_peered":"2024-09-10T11:39:35.318878+0000","last_unstale":"2024-09-10T11:39:36.642408+0000","last_undegraded":"2024-09-10T11:39:36.642408+0000","last_fullsized":"2024-09-10T11:39:36.642408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:17.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642901+0000","last_change":"2024-09-10T11:39:35.329244+0000","last_active":"2024-09-10T11:39:36.642901+0000","last_peered":"2024-09-10T11:39:36.642901+0000","last_clean":"2024-09-10T11:39:36.642901+0000","last_became_active":"2024-09-10T11:39:35.328732+0000","last_became_peered":"2024-09-10T11:39:35.328732+0000","last_unstale":"2024-09-10T11:39:36.642901+0000","last_undegraded":"2024-09-10T11:39:36.642901+0000","last_fullsized":"2024-09-10T11:39:36.642901+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:09.845801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457050+0000","last_change":"2024-09-10T11:39:35.325279+0000","last_active":"2024-09-10T11:39:36.457050+0000","last_peered":"2024-09-10T11:39:36.457050+0000","last_clean":"2024-09-10T11:39:36.457050+0000","last_became_active":"2024-09-10T11:39:35.325051+0000","last_became_peered":"2024-09-10T11:39:35.325051+0000","last_unstale":"2024-09-10T11:39:36.457050+0000","last_undegraded":"2024-09-10T11:39:36.457050+0000","last_fullsized":"2024-09-10T11:39:36.457050+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:40.038279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642902+0000","last_change":"2024-09-10T11:39:35.319547+0000","last_active":"2024-09-10T11:39:36.642902+0000","last_peered":"2024-09-10T11:39:36.642902+0000","last_clean":"2024-09-10T11:39:36.642902+0000","last_became_active":"2024-09-10T11:39:35.319268+0000","last_became_peered":"2024-09-10T11:39:35.319268+0000","last_unstale":"2024-09-10T11:39:36.642902+0000","last_undegraded":"2024-09-10T11:39:36.642902+0000","last_fullsized":"2024-09-10T11:39:36.642902+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:39.362543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317214+0000","last_change":"2024-09-10T11:39:35.327878+0000","last_active":"2024-09-10T11:39:36.317214+0000","last_peered":"2024-09-10T11:39:36.317214+0000","last_clean":"2024-09-10T11:39:36.317214+0000","last_became_active":"2024-09-10T11:39:35.327650+0000","last_became_peered":"2024-09-10T11:39:35.327650+0000","last_unstale":"2024-09-10T11:39:36.317214+0000","last_undegraded":"2024-09-10T11:39:36.317214+0000","last_fullsized":"2024-09-10T11:39:36.317214+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:33.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317541+0000","last_change":"2024-09-10T11:39:35.327817+0000","last_active":"2024-09-10T11:39:36.317541+0000","last_peered":"2024-09-10T11:39:36.317541+0000","last_clean":"2024-09-10T11:39:36.317541+0000","last_became_active":"2024-09-10T11:39:35.327566+0000","last_became_peered":"2024-09-10T11:39:35.327566+0000","last_unstale":"2024-09-10T11:39:36.317541+0000","last_undegraded":"2024-09-10T11:39:36.317541+0000","last_fullsized":"2024-09-10T11:39:36.317541+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:25.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642689+0000","last_change":"2024-09-10T11:39:35.319007+0000","last_active":"2024-09-10T11:39:36.642689+0000","last_peered":"2024-09-10T11:39:36.642689+0000","last_clean":"2024-09-10T11:39:36.642689+0000","last_became_active":"2024-09-10T11:39:35.318773+0000","last_became_peered":"2024-09-10T11:39:35.318773+0000","last_unstale":"2024-09-10T11:39:36.642689+0000","last_undegraded":"2024-09-10T11:39:36.642689+0000","last_fullsized":"2024-09-10T11:39:36.642689+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:00.949571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317866+0000","last_change":"2024-09-10T11:39:35.329898+0000","last_active":"2024-09-10T11:39:36.317866+0000","last_peered":"2024-09-10T11:39:36.317866+0000","last_clean":"2024-09-10T11:39:36.317866+0000","last_became_active":"2024-09-10T11:39:35.329588+0000","last_became_peered":"2024-09-10T11:39:35.329588+0000","last_unstale":"2024-09-10T11:39:36.317866+0000","last_undegraded":"2024-09-10T11:39:36.317866+0000","last_fullsized":"2024-09-10T11:39:36.317866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:51.286757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642128+0000","last_change":"2024-09-10T11:39:35.328717+0000","last_active":"2024-09-10T11:39:36.642128+0000","last_peered":"2024-09-10T11:39:36.642128+0000","last_clean":"2024-09-10T11:39:36.642128+0000","last_became_active":"2024-09-10T11:39:35.328154+0000","last_became_peered":"2024-09-10T11:39:35.328154+0000","last_unstale":"2024-09-10T11:39:36.642128+0000","last_undegraded":"2024-09-10T11:39:36.642128+0000","last_fullsized":"2024-09-10T11:39:36.642128+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:21.464233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318644+0000","last_change":"2024-09-10T11:39:35.334614+0000","last_active":"2024-09-10T11:39:36.318644+0000","last_peered":"2024-09-10T11:39:36.318644+0000","last_clean":"2024-09-10T11:39:36.318644+0000","last_became_active":"2024-09-10T11:39:35.332540+0000","last_became_peered":"2024-09-10T11:39:35.332540+0000","last_unstale":"2024-09-10T11:39:36.318644+0000","last_undegraded":"2024-09-10T11:39:36.318644+0000","last_fullsized":"2024-09-10T11:39:36.318644+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:29.547514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"182'4","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317947+0000","last_change":"2024-09-10T11:39:34.312948+0000","last_active":"2024-09-10T11:39:36.317947+0000","last_peered":"2024-09-10T11:39:36.317947+0000","last_clean":"2024-09-10T11:39:36.317947+0000","last_became_active":"2024-09-10T11:39:34.311720+0000","last_became_peered":"2024-09-10T11:39:34.311720+0000","last_unstale":"2024-09-10T11:39:36.317947+0000","last_undegraded":"2024-09-10T11:39:36.317947+0000","last_fullsized":"2024-09-10T11:39:36.317947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:16.170110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317561+0000","last_change":"2024-09-10T11:39:35.333744+0000","last_active":"2024-09-10T11:39:36.317561+0000","last_peered":"2024-09-10T11:39:36.317561+0000","last_clean":"2024-09-10T11:39:36.317561+0000","last_became_active":"2024-09-10T11:39:35.333480+0000","last_became_peered":"2024-09-10T11:39:35.333480+0000","last_unstale":"2024-09-10T11:39:36.317561+0000","last_undegraded":"2024-09-10T11:39:36.317561+0000","last_fullsized":"2024-09-10T11:39:36.317561+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:44.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642754+0000","last_change":"2024-09-10T11:39:34.312061+0000","last_active":"2024-09-10T11:39:36.642754+0000","last_peered":"2024-09-10T11:39:36.642754+0000","last_clean":"2024-09-10T11:39:36.642754+0000","last_became_active":"2024-09-10T11:39:34.311818+0000","last_became_peered":"2024-09-10T11:39:34.311818+0000","last_unstale":"2024-09-10T11:39:36.642754+0000","last_undegraded":"2024-09-10T11:39:36.642754+0000","last_fullsized":"2024-09-10T11:39:36.642754+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:52.795074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642489+0000","last_change":"2024-09-10T11:39:35.326633+0000","last_active":"2024-09-10T11:39:36.642489+0000","last_peered":"2024-09-10T11:39:36.642489+0000","last_clean":"2024-09-10T11:39:36.642489+0000","last_became_active":"2024-09-10T11:39:35.324469+0000","last_became_peered":"2024-09-10T11:39:35.324469+0000","last_unstale":"2024-09-10T11:39:36.642489+0000","last_undegraded":"2024-09-10T11:39:36.642489+0000","last_fullsized":"2024-09-10T11:39:36.642489+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:23.053184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642274+0000","last_change":"2024-09-10T11:39:34.314840+0000","last_active":"2024-09-10T11:39:36.642274+0000","last_peered":"2024-09-10T11:39:36.642274+0000","last_clean":"2024-09-10T11:39:36.642274+0000","last_became_active":"2024-09-10T11:39:34.314346+0000","last_became_peered":"2024-09-10T11:39:34.314346+0000","last_unstale":"2024-09-10T11:39:36.642274+0000","last_undegraded":"2024-09-10T11:39:36.642274+0000","last_fullsized":"2024-09-10T11:39:36.642274+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:49.137667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642818+0000","last_change":"2024-09-10T11:39:35.325664+0000","last_active":"2024-09-10T11:39:36.642818+0000","last_peered":"2024-09-10T11:39:36.642818+0000","last_clean":"2024-09-10T11:39:36.642818+0000","last_became_active":"2024-09-10T11:39:35.325053+0000","last_became_peered":"2024-09-10T11:39:35.325053+0000","last_unstale":"2024-09-10T11:39:36.642818+0000","last_undegraded":"2024-09-10T11:39:36.642818+0000","last_fullsized":"2024-09-10T11:39:36.642818+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:15.341660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457126+0000","last_change":"2024-09-10T11:39:34.309774+0000","last_active":"2024-09-10T11:39:36.457126+0000","last_peered":"2024-09-10T11:39:36.457126+0000","last_clean":"2024-09-10T11:39:36.457126+0000","last_became_active":"2024-09-10T11:39:34.309047+0000","last_became_peered":"2024-09-10T11:39:34.309047+0000","last_unstale":"2024-09-10T11:39:36.457126+0000","last_undegraded":"2024-09-10T11:39:36.457126+0000","last_fullsized":"2024-09-10T11:39:36.457126+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:12.211476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318383+0000","last_change":"2024-09-10T11:39:35.333647+0000","last_active":"2024-09-10T11:39:36.318383+0000","last_peered":"2024-09-10T11:39:36.318383+0000","last_clean":"2024-09-10T11:39:36.318383+0000","last_became_active":"2024-09-10T11:39:35.332951+0000","last_became_peered":"2024-09-10T11:39:35.332951+0000","last_unstale":"2024-09-10T11:39:36.318383+0000","last_undegraded":"2024-09-10T11:39:36.318383+0000","last_fullsized":"2024-09-10T11:39:36.318383+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:18.889369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317792+0000","last_change":"2024-09-10T11:39:34.310066+0000","last_active":"2024-09-10T11:39:36.317792+0000","last_peered":"2024-09-10T11:39:36.317792+0000","last_clean":"2024-09-10T11:39:36.317792+0000","last_became_active":"2024-09-10T11:39:34.309291+0000","last_became_peered":"2024-09-10T11:39:34.309291+0000","last_unstale":"2024-09-10T11:39:36.317792+0000","last_undegraded":"2024-09-10T11:39:36.317792+0000","last_fullsized":"2024-09-10T11:39:36.317792+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:05.569185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642277+0000","last_change":"2024-09-10T11:39:35.329287+0000","last_active":"2024-09-10T11:39:36.642277+0000","last_peered":"2024-09-10T11:39:36.642277+0000","last_clean":"2024-09-10T11:39:36.642277+0000","last_became_active":"2024-09-10T11:39:35.328477+0000","last_became_peered":"2024-09-10T11:39:35.328477+0000","last_unstale":"2024-09-10T11:39:36.642277+0000","last_undegraded":"2024-09-10T11:39:36.642277+0000","last_fullsized":"2024-09-10T11:39:36.642277+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:17.107794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318507+0000","last_change":"2024-09-10T11:39:34.312118+0000","last_active":"2024-09-10T11:39:36.318507+0000","last_peered":"2024-09-10T11:39:36.318507+0000","last_clean":"2024-09-10T11:39:36.318507+0000","last_became_active":"2024-09-10T11:39:34.311890+0000","last_became_peered":"2024-09-10T11:39:34.311890+0000","last_unstale":"2024-09-10T11:39:36.318507+0000","last_undegraded":"2024-09-10T11:39:36.318507+0000","last_fullsized":"2024-09-10T11:39:36.318507+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:48.484800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.456960+0000","last_change":"2024-09-10T11:39:35.327736+0000","last_active":"2024-09-10T11:39:36.456960+0000","last_peered":"2024-09-10T11:39:36.456960+0000","last_clean":"2024-09-10T11:39:36.456960+0000","last_became_active":"2024-09-10T11:39:35.327502+0000","last_became_peered":"2024-09-10T11:39:35.327502+0000","last_unstale":"2024-09-10T11:39:36.456960+0000","last_undegraded":"2024-09-10T11:39:36.456960+0000","last_fullsized":"2024-09-10T11:39:36.456960+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:33.537228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317703+0000","last_change":"2024-09-10T11:39:34.311736+0000","last_active":"2024-09-10T11:39:36.317703+0000","last_peered":"2024-09-10T11:39:36.317703+0000","last_clean":"2024-09-10T11:39:36.317703+0000","last_became_active":"2024-09-10T11:39:34.310903+0000","last_became_peered":"2024-09-10T11:39:34.310903+0000","last_unstale":"2024-09-10T11:39:36.317703+0000","last_undegraded":"2024-09-10T11:39:36.317703+0000","last_fullsized":"2024-09-10T11:39:36.317703+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:50.781728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318799+0000","last_change":"2024-09-10T11:39:35.334390+0000","last_active":"2024-09-10T11:39:36.318799+0000","last_peered":"2024-09-10T11:39:36.318799+0000","last_clean":"2024-09-10T11:39:36.318799+0000","last_became_active":"2024-09-10T11:39:35.331411+0000","last_became_peered":"2024-09-10T11:39:35.331411+0000","last_unstale":"2024-09-10T11:39:36.318799+0000","last_undegraded":"2024-09-10T11:39:36.318799+0000","last_fullsized":"2024-09-10T11:39:36.318799+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:15.788303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457136+0000","last_change":"2024-09-10T11:39:34.308307+0000","last_active":"2024-09-10T11:39:36.457136+0000","last_peered":"2024-09-10T11:39:36.457136+0000","last_clean":"2024-09-10T11:39:36.457136+0000","last_became_active":"2024-09-10T11:39:34.308062+0000","last_became_peered":"2024-09-10T11:39:34.308062+0000","last_unstale":"2024-09-10T11:39:36.457136+0000","last_undegraded":"2024-09-10T11:39:36.457136+0000","last_fullsized":"2024-09-10T11:39:36.457136+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:08.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317781+0000","last_change":"2024-09-10T11:39:35.321895+0000","last_active":"2024-09-10T11:39:36.317781+0000","last_peered":"2024-09-10T11:39:36.317781+0000","last_clean":"2024-09-10T11:39:36.317781+0000","last_became_active":"2024-09-10T11:39:35.321266+0000","last_became_peered":"2024-09-10T11:39:35.321266+0000","last_unstale":"2024-09-10T11:39:36.317781+0000","last_undegraded":"2024-09-10T11:39:36.317781+0000","last_fullsized":"2024-09-10T11:39:36.317781+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:23.624747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642451+0000","last_change":"2024-09-10T11:39:34.314085+0000","last_active":"2024-09-10T11:39:36.642451+0000","last_peered":"2024-09-10T11:39:36.642451+0000","last_clean":"2024-09-10T11:39:36.642451+0000","last_became_active":"2024-09-10T11:39:34.313848+0000","last_became_peered":"2024-09-10T11:39:34.313848+0000","last_unstale":"2024-09-10T11:39:36.642451+0000","last_undegraded":"2024-09-10T11:39:36.642451+0000","last_fullsized":"2024-09-10T11:39:36.642451+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:58.499889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317114+0000","last_change":"2024-09-10T11:39:35.321435+0000","last_active":"2024-09-10T11:39:36.317114+0000","last_peered":"2024-09-10T11:39:36.317114+0000","last_clean":"2024-09-10T11:39:36.317114+0000","last_became_active":"2024-09-10T11:39:35.321158+0000","last_became_peered":"2024-09-10T11:39:35.321158+0000","last_unstale":"2024-09-10T11:39:36.317114+0000","last_undegraded":"2024-09-10T11:39:36.317114+0000","last_fullsized":"2024-09-10T11:39:36.317114+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:34.954100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"182'3","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457429+0000","last_change":"2024-09-10T11:39:34.309865+0000","last_active":"2024-09-10T11:39:36.457429+0000","last_peered":"2024-09-10T11:39:36.457429+0000","last_clean":"2024-09-10T11:39:36.457429+0000","last_became_active":"2024-09-10T11:39:34.309611+0000","last_became_peered":"2024-09-10T11:39:34.309611+0000","last_unstale":"2024-09-10T11:39:36.457429+0000","last_undegraded":"2024-09-10T11:39:36.457429+0000","last_fullsized":"2024-09-10T11:39:36.457429+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:32.198601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317695+0000","last_change":"2024-09-10T11:39:35.331243+0000","last_active":"2024-09-10T11:39:36.317695+0000","last_peered":"2024-09-10T11:39:36.317695+0000","last_clean":"2024-09-10T11:39:36.317695+0000","last_became_active":"2024-09-10T11:39:35.330828+0000","last_became_peered":"2024-09-10T11:39:35.330828+0000","last_unstale":"2024-09-10T11:39:36.317695+0000","last_undegraded":"2024-09-10T11:39:36.317695+0000","last_fullsized":"2024-09-10T11:39:36.317695+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:51.662089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"182'6","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642510+0000","last_change":"2024-09-10T11:39:34.313206+0000","last_active":"2024-09-10T11:39:36.642510+0000","last_peered":"2024-09-10T11:39:36.642510+0000","last_clean":"2024-09-10T11:39:36.642510+0000","last_became_active":"2024-09-10T11:39:34.312484+0000","last_became_peered":"2024-09-10T11:39:34.312484+0000","last_unstale":"2024-09-10T11:39:36.642510+0000","last_undegraded":"2024-09-10T11:39:36.642510+0000","last_fullsized":"2024-09-10T11:39:36.642510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:28.354422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317931+0000","last_change":"2024-09-10T11:39:35.322803+0000","last_active":"2024-09-10T11:39:36.317931+0000","last_peered":"2024-09-10T11:39:36.317931+0000","last_clean":"2024-09-10T11:39:36.317931+0000","last_became_active":"2024-09-10T11:39:35.322259+0000","last_became_peered":"2024-09-10T11:39:35.322259+0000","last_unstale":"2024-09-10T11:39:36.317931+0000","last_undegraded":"2024-09-10T11:39:36.317931+0000","last_fullsized":"2024-09-10T11:39:36.317931+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:11.994600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318494+0000","last_change":"2024-09-10T11:39:34.307625+0000","last_active":"2024-09-10T11:39:36.318494+0000","last_peered":"2024-09-10T11:39:36.318494+0000","last_clean":"2024-09-10T11:39:36.318494+0000","last_became_active":"2024-09-10T11:39:34.307190+0000","last_became_peered":"2024-09-10T11:39:34.307190+0000","last_unstale":"2024-09-10T11:39:36.318494+0000","last_undegraded":"2024-09-10T11:39:36.318494+0000","last_fullsized":"2024-09-10T11:39:36.318494+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:34.150373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457061+0000","last_change":"2024-09-10T11:39:35.326359+0000","last_active":"2024-09-10T11:39:36.457061+0000","last_peered":"2024-09-10T11:39:36.457061+0000","last_clean":"2024-09-10T11:39:36.457061+0000","last_became_active":"2024-09-10T11:39:35.326131+0000","last_became_peered":"2024-09-10T11:39:35.326131+0000","last_unstale":"2024-09-10T11:39:36.457061+0000","last_undegraded":"2024-09-10T11:39:36.457061+0000","last_fullsized":"2024-09-10T11:39:36.457061+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:21.248109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642552+0000","last_change":"2024-09-10T11:39:34.311528+0000","last_active":"2024-09-10T11:39:36.642552+0000","last_peered":"2024-09-10T11:39:36.642552+0000","last_clean":"2024-09-10T11:39:36.642552+0000","last_became_active":"2024-09-10T11:39:34.311293+0000","last_became_peered":"2024-09-10T11:39:34.311293+0000","last_unstale":"2024-09-10T11:39:36.642552+0000","last_undegraded":"2024-09-10T11:39:36.642552+0000","last_fullsized":"2024-09-10T11:39:36.642552+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:54.132543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642217+0000","last_change":"2024-09-10T11:39:35.325616+0000","last_active":"2024-09-10T11:39:36.642217+0000","last_peered":"2024-09-10T11:39:36.642217+0000","last_clean":"2024-09-10T11:39:36.642217+0000","last_became_active":"2024-09-10T11:39:35.325377+0000","last_became_peered":"2024-09-10T11:39:35.325377+0000","last_unstale":"2024-09-10T11:39:36.642217+0000","last_undegraded":"2024-09-10T11:39:36.642217+0000","last_fullsized":"2024-09-10T11:39:36.642217+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:42.425900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317688+0000","last_change":"2024-09-10T11:39:34.307978+0000","last_active":"2024-09-10T11:39:36.317688+0000","last_peered":"2024-09-10T11:39:36.317688+0000","last_clean":"2024-09-10T11:39:36.317688+0000","last_became_active":"2024-09-10T11:39:34.307750+0000","last_became_peered":"2024-09-10T11:39:34.307750+0000","last_unstale":"2024-09-10T11:39:36.317688+0000","last_undegraded":"2024-09-10T11:39:36.317688+0000","last_fullsized":"2024-09-10T11:39:36.317688+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:24.398920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642569+0000","last_change":"2024-09-10T11:39:35.319651+0000","last_active":"2024-09-10T11:39:36.642569+0000","last_peered":"2024-09-10T11:39:36.642569+0000","last_clean":"2024-09-10T11:39:36.642569+0000","last_became_active":"2024-09-10T11:39:35.319408+0000","last_became_peered":"2024-09-10T11:39:35.319408+0000","last_unstale":"2024-09-10T11:39:36.642569+0000","last_undegraded":"2024-09-10T11:39:36.642569+0000","last_fullsized":"2024-09-10T11:39:36.642569+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:10.359966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"182'8","reported_seq":22,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642871+0000","last_change":"2024-09-10T11:39:34.313026+0000","last_active":"2024-09-10T11:39:36.642871+0000","last_peered":"2024-09-10T11:39:36.642871+0000","last_clean":"2024-09-10T11:39:36.642871+0000","last_became_active":"2024-09-10T11:39:34.312811+0000","last_became_peered":"2024-09-10T11:39:34.312811+0000","last_unstale":"2024-09-10T11:39:36.642871+0000","last_undegraded":"2024-09-10T11:39:36.642871+0000","last_fullsized":"2024-09-10T11:39:36.642871+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:52.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317264+0000","last_change":"2024-09-10T11:39:35.327268+0000","last_active":"2024-09-10T11:39:36.317264+0000","last_peered":"2024-09-10T11:39:36.317264+0000","last_clean":"2024-09-10T11:39:36.317264+0000","last_became_active":"2024-09-10T11:39:35.327043+0000","last_became_peered":"2024-09-10T11:39:35.327043+0000","last_unstale":"2024-09-10T11:39:36.317264+0000","last_undegraded":"2024-09-10T11:39:36.317264+0000","last_fullsized":"2024-09-10T11:39:36.317264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:45.663322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642620+0000","last_change":"2024-09-10T11:39:34.313482+0000","last_active":"2024-09-10T11:39:36.642620+0000","last_peered":"2024-09-10T11:39:36.642620+0000","last_clean":"2024-09-10T11:39:36.642620+0000","last_became_active":"2024-09-10T11:39:34.312930+0000","last_became_peered":"2024-09-10T11:39:34.312930+0000","last_unstale":"2024-09-10T11:39:36.642620+0000","last_undegraded":"2024-09-10T11:39:36.642620+0000","last_fullsized":"2024-09-10T11:39:36.642620+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:25.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318925+0000","last_change":"2024-09-10T11:39:35.334005+0000","last_active":"2024-09-10T11:39:36.318925+0000","last_peered":"2024-09-10T11:39:36.318925+0000","last_clean":"2024-09-10T11:39:36.318925+0000","last_became_active":"2024-09-10T11:39:35.330837+0000","last_became_peered":"2024-09-10T11:39:35.330837+0000","last_unstale":"2024-09-10T11:39:36.318925+0000","last_undegraded":"2024-09-10T11:39:36.318925+0000","last_fullsized":"2024-09-10T11:39:36.318925+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:06.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642912+0000","last_change":"2024-09-10T11:39:34.312956+0000","last_active":"2024-09-10T11:39:36.642912+0000","last_peered":"2024-09-10T11:39:36.642912+0000","last_clean":"2024-09-10T11:39:36.642912+0000","last_became_active":"2024-09-10T11:39:34.312733+0000","last_became_peered":"2024-09-10T11:39:34.312733+0000","last_unstale":"2024-09-10T11:39:36.642912+0000","last_undegraded":"2024-09-10T11:39:36.642912+0000","last_fullsized":"2024-09-10T11:39:36.642912+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:58.909280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318666+0000","last_change":"2024-09-10T11:39:35.324599+0000","last_active":"2024-09-10T11:39:36.318666+0000","last_peered":"2024-09-10T11:39:36.318666+0000","last_clean":"2024-09-10T11:39:36.318666+0000","last_became_active":"2024-09-10T11:39:35.324362+0000","last_became_peered":"2024-09-10T11:39:35.324362+0000","last_unstale":"2024-09-10T11:39:36.318666+0000","last_undegraded":"2024-09-10T11:39:36.318666+0000","last_fullsized":"2024-09-10T11:39:36.318666+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:04.988917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642488+0000","last_change":"2024-09-10T11:39:34.307654+0000","last_active":"2024-09-10T11:39:36.642488+0000","last_peered":"2024-09-10T11:39:36.642488+0000","last_clean":"2024-09-10T11:39:36.642488+0000","last_became_active":"2024-09-10T11:39:34.307418+0000","last_became_peered":"2024-09-10T11:39:34.307418+0000","last_unstale":"2024-09-10T11:39:36.642488+0000","last_undegraded":"2024-09-10T11:39:36.642488+0000","last_fullsized":"2024-09-10T11:39:36.642488+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:22.111985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.456937+0000","last_change":"2024-09-10T11:39:35.330555+0000","last_active":"2024-09-10T11:39:36.456937+0000","last_peered":"2024-09-10T11:39:36.456937+0000","last_clean":"2024-09-10T11:39:36.456937+0000","last_became_active":"2024-09-10T11:39:35.329349+0000","last_became_peered":"2024-09-10T11:39:35.329349+0000","last_unstale":"2024-09-10T11:39:36.456937+0000","last_undegraded":"2024-09-10T11:39:36.456937+0000","last_fullsized":"2024-09-10T11:39:36.456937+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:52.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330619+0000","last_change":"2024-09-10T11:39:34.315002+0000","last_active":"2024-09-10T11:39:36.330619+0000","last_peered":"2024-09-10T11:39:36.330619+0000","last_clean":"2024-09-10T11:39:36.330619+0000","last_became_active":"2024-09-10T11:39:34.314276+0000","last_became_peered":"2024-09-10T11:39:34.314276+0000","last_unstale":"2024-09-10T11:39:36.330619+0000","last_undegraded":"2024-09-10T11:39:36.330619+0000","last_fullsized":"2024-09-10T11:39:36.330619+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:34.016844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317649+0000","last_change":"2024-09-10T11:39:35.336786+0000","last_active":"2024-09-10T11:39:36.317649+0000","last_peered":"2024-09-10T11:39:36.317649+0000","last_clean":"2024-09-10T11:39:36.317649+0000","last_became_active":"2024-09-10T11:39:35.336506+0000","last_became_peered":"2024-09-10T11:39:35.336506+0000","last_unstale":"2024-09-10T11:39:36.317649+0000","last_undegraded":"2024-09-10T11:39:36.317649+0000","last_fullsized":"2024-09-10T11:39:36.317649+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:04:50.294733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457526+0000","last_change":"2024-09-10T11:39:34.312714+0000","last_active":"2024-09-10T11:39:36.457526+0000","last_peered":"2024-09-10T11:39:36.457526+0000","last_clean":"2024-09-10T11:39:36.457526+0000","last_became_active":"2024-09-10T11:39:34.312045+0000","last_became_peered":"2024-09-10T11:39:34.312045+0000","last_unstale":"2024-09-10T11:39:36.457526+0000","last_undegraded":"2024-09-10T11:39:36.457526+0000","last_fullsized":"2024-09-10T11:39:36.457526+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:26.566597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457189+0000","last_change":"2024-09-10T11:39:35.332890+0000","last_active":"2024-09-10T11:39:36.457189+0000","last_peered":"2024-09-10T11:39:36.457189+0000","last_clean":"2024-09-10T11:39:36.457189+0000","last_became_active":"2024-09-10T11:39:35.331901+0000","last_became_peered":"2024-09-10T11:39:35.331901+0000","last_unstale":"2024-09-10T11:39:36.457189+0000","last_undegraded":"2024-09-10T11:39:36.457189+0000","last_fullsized":"2024-09-10T11:39:36.457189+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:00.156648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330414+0000","last_change":"2024-09-10T11:39:34.313893+0000","last_active":"2024-09-10T11:39:36.330414+0000","last_peered":"2024-09-10T11:39:36.330414+0000","last_clean":"2024-09-10T11:39:36.330414+0000","last_became_active":"2024-09-10T11:39:34.313640+0000","last_became_peered":"2024-09-10T11:39:34.313640+0000","last_unstale":"2024-09-10T11:39:36.330414+0000","last_undegraded":"2024-09-10T11:39:36.330414+0000","last_fullsized":"2024-09-10T11:39:36.330414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:54.058453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317949+0000","last_change":"2024-09-10T11:39:35.324646+0000","last_active":"2024-09-10T11:39:36.317949+0000","last_peered":"2024-09-10T11:39:36.317949+0000","last_clean":"2024-09-10T11:39:36.317949+0000","last_became_active":"2024-09-10T11:39:35.324412+0000","last_became_peered":"2024-09-10T11:39:35.324412+0000","last_unstale":"2024-09-10T11:39:36.317949+0000","last_undegraded":"2024-09-10T11:39:36.317949+0000","last_fullsized":"2024-09-10T11:39:36.317949+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:40.285906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318639+0000","last_change":"2024-09-10T11:39:34.310970+0000","last_active":"2024-09-10T11:39:36.318639+0000","last_peered":"2024-09-10T11:39:36.318639+0000","last_clean":"2024-09-10T11:39:36.318639+0000","last_became_active":"2024-09-10T11:39:34.310693+0000","last_became_peered":"2024-09-10T11:39:34.310693+0000","last_unstale":"2024-09-10T11:39:36.318639+0000","last_undegraded":"2024-09-10T11:39:36.318639+0000","last_fullsized":"2024-09-10T11:39:36.318639+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:56.261211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317423+0000","last_change":"2024-09-10T11:39:35.319279+0000","last_active":"2024-09-10T11:39:36.317423+0000","last_peered":"2024-09-10T11:39:36.317423+0000","last_clean":"2024-09-10T11:39:36.317423+0000","last_became_active":"2024-09-10T11:39:35.318987+0000","last_became_peered":"2024-09-10T11:39:35.318987+0000","last_unstale":"2024-09-10T11:39:36.317423+0000","last_undegraded":"2024-09-10T11:39:36.317423+0000","last_fullsized":"2024-09-10T11:39:36.317423+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:52.001312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330512+0000","last_change":"2024-09-10T11:39:34.315568+0000","last_active":"2024-09-10T11:39:36.330512+0000","last_peered":"2024-09-10T11:39:36.330512+0000","last_clean":"2024-09-10T11:39:36.330512+0000","last_became_active":"2024-09-10T11:39:34.315331+0000","last_became_peered":"2024-09-10T11:39:34.315331+0000","last_unstale":"2024-09-10T11:39:36.330512+0000","last_undegraded":"2024-09-10T11:39:36.330512+0000","last_fullsized":"2024-09-10T11:39:36.330512+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:09.743526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457331+0000","last_change":"2024-09-10T11:39:35.332647+0000","last_active":"2024-09-10T11:39:36.457331+0000","last_peered":"2024-09-10T11:39:36.457331+0000","last_clean":"2024-09-10T11:39:36.457331+0000","last_became_active":"2024-09-10T11:39:35.331482+0000","last_became_peered":"2024-09-10T11:39:35.331482+0000","last_unstale":"2024-09-10T11:39:36.457331+0000","last_undegraded":"2024-09-10T11:39:36.457331+0000","last_fullsized":"2024-09-10T11:39:36.457331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:27.038068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642616+0000","last_change":"2024-09-10T11:39:34.315221+0000","last_active":"2024-09-10T11:39:36.642616+0000","last_peered":"2024-09-10T11:39:36.642616+0000","last_clean":"2024-09-10T11:39:36.642616+0000","last_became_active":"2024-09-10T11:39:34.314972+0000","last_became_peered":"2024-09-10T11:39:34.314972+0000","last_unstale":"2024-09-10T11:39:36.642616+0000","last_undegraded":"2024-09-10T11:39:36.642616+0000","last_fullsized":"2024-09-10T11:39:36.642616+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:04.755154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318884+0000","last_change":"2024-09-10T11:39:35.324040+0000","last_active":"2024-09-10T11:39:36.318884+0000","last_peered":"2024-09-10T11:39:36.318884+0000","last_clean":"2024-09-10T11:39:36.318884+0000","last_became_active":"2024-09-10T11:39:35.323785+0000","last_became_peered":"2024-09-10T11:39:35.323785+0000","last_unstale":"2024-09-10T11:39:36.318884+0000","last_undegraded":"2024-09-10T11:39:36.318884+0000","last_fullsized":"2024-09-10T11:39:36.318884+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:16.165735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"183'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.331115+0000","last_change":"2024-09-10T11:39:34.314834+0000","last_active":"2024-09-10T11:39:36.331115+0000","last_peered":"2024-09-10T11:39:36.331115+0000","last_clean":"2024-09-10T11:39:36.331115+0000","last_became_active":"2024-09-10T11:39:34.314505+0000","last_became_peered":"2024-09-10T11:39:34.314505+0000","last_unstale":"2024-09-10T11:39:36.331115+0000","last_undegraded":"2024-09-10T11:39:36.331115+0000","last_fullsized":"2024-09-10T11:39:36.331115+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:27.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318366+0000","last_change":"2024-09-10T11:39:35.327114+0000","last_active":"2024-09-10T11:39:36.318366+0000","last_peered":"2024-09-10T11:39:36.318366+0000","last_clean":"2024-09-10T11:39:36.318366+0000","last_became_active":"2024-09-10T11:39:35.326903+0000","last_became_peered":"2024-09-10T11:39:35.326903+0000","last_unstale":"2024-09-10T11:39:36.318366+0000","last_undegraded":"2024-09-10T11:39:36.318366+0000","last_fullsized":"2024-09-10T11:39:36.318366+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.453558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317794+0000","last_change":"2024-09-10T11:39:34.308221+0000","last_active":"2024-09-10T11:39:36.317794+0000","last_peered":"2024-09-10T11:39:36.317794+0000","last_clean":"2024-09-10T11:39:36.317794+0000","last_became_active":"2024-09-10T11:39:34.308015+0000","last_became_peered":"2024-09-10T11:39:34.308015+0000","last_unstale":"2024-09-10T11:39:36.317794+0000","last_undegraded":"2024-09-10T11:39:36.317794+0000","last_fullsized":"2024-09-10T11:39:36.317794+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:02.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.319007+0000","last_change":"2024-09-10T11:39:35.326302+0000","last_active":"2024-09-10T11:39:36.319007+0000","last_peered":"2024-09-10T11:39:36.319007+0000","last_clean":"2024-09-10T11:39:36.319007+0000","last_became_active":"2024-09-10T11:39:35.325997+0000","last_became_peered":"2024-09-10T11:39:35.325997+0000","last_unstale":"2024-09-10T11:39:36.319007+0000","last_undegraded":"2024-09-10T11:39:36.319007+0000","last_fullsized":"2024-09-10T11:39:36.319007+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:47.356895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457195+0000","last_change":"2024-09-10T11:39:34.312217+0000","last_active":"2024-09-10T11:39:36.457195+0000","last_peered":"2024-09-10T11:39:36.457195+0000","last_clean":"2024-09-10T11:39:36.457195+0000","last_became_active":"2024-09-10T11:39:34.311981+0000","last_became_peered":"2024-09-10T11:39:34.311981+0000","last_unstale":"2024-09-10T11:39:36.457195+0000","last_undegraded":"2024-09-10T11:39:36.457195+0000","last_fullsized":"2024-09-10T11:39:36.457195+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:00.570537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642388+0000","last_change":"2024-09-10T11:39:35.329947+0000","last_active":"2024-09-10T11:39:36.642388+0000","last_peered":"2024-09-10T11:39:36.642388+0000","last_clean":"2024-09-10T11:39:36.642388+0000","last_became_active":"2024-09-10T11:39:35.329700+0000","last_became_peered":"2024-09-10T11:39:35.329700+0000","last_unstale":"2024-09-10T11:39:36.642388+0000","last_undegraded":"2024-09-10T11:39:36.642388+0000","last_fullsized":"2024-09-10T11:39:36.642388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:05.526617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642846+0000","last_change":"2024-09-10T11:39:34.305915+0000","last_active":"2024-09-10T11:39:36.642846+0000","last_peered":"2024-09-10T11:39:36.642846+0000","last_clean":"2024-09-10T11:39:36.642846+0000","last_became_active":"2024-09-10T11:39:34.305646+0000","last_became_peered":"2024-09-10T11:39:34.305646+0000","last_unstale":"2024-09-10T11:39:36.642846+0000","last_undegraded":"2024-09-10T11:39:36.642846+0000","last_fullsized":"2024-09-10T11:39:36.642846+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:00.749126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318025+0000","last_change":"2024-09-10T11:39:35.324161+0000","last_active":"2024-09-10T11:39:36.318025+0000","last_peered":"2024-09-10T11:39:36.318025+0000","last_clean":"2024-09-10T11:39:36.318025+0000","last_became_active":"2024-09-10T11:39:35.323918+0000","last_became_peered":"2024-09-10T11:39:35.323918+0000","last_unstale":"2024-09-10T11:39:36.318025+0000","last_undegraded":"2024-09-10T11:39:36.318025+0000","last_fullsized":"2024-09-10T11:39:36.318025+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:41.948128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317746+0000","last_change":"2024-09-10T11:39:34.311986+0000","last_active":"2024-09-10T11:39:36.317746+0000","last_peered":"2024-09-10T11:39:36.317746+0000","last_clean":"2024-09-10T11:39:36.317746+0000","last_became_active":"2024-09-10T11:39:34.311730+0000","last_became_peered":"2024-09-10T11:39:34.311730+0000","last_unstale":"2024-09-10T11:39:36.317746+0000","last_undegraded":"2024-09-10T11:39:36.317746+0000","last_fullsized":"2024-09-10T11:39:36.317746+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:12.149021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317355+0000","last_change":"2024-09-10T11:39:35.320672+0000","last_active":"2024-09-10T11:39:36.317355+0000","last_peered":"2024-09-10T11:39:36.317355+0000","last_clean":"2024-09-10T11:39:36.317355+0000","last_became_active":"2024-09-10T11:39:35.319098+0000","last_became_peered":"2024-09-10T11:39:35.319098+0000","last_unstale":"2024-09-10T11:39:36.317355+0000","last_undegraded":"2024-09-10T11:39:36.317355+0000","last_fullsized":"2024-09-10T11:39:36.317355+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:24.454593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318067+0000","last_change":"2024-09-10T11:39:34.311954+0000","last_active":"2024-09-10T11:39:36.318067+0000","last_peered":"2024-09-10T11:39:36.318067+0000","last_clean":"2024-09-10T11:39:36.318067+0000","last_became_active":"2024-09-10T11:39:34.309642+0000","last_became_peered":"2024-09-10T11:39:34.309642+0000","last_unstale":"2024-09-10T11:39:36.318067+0000","last_undegraded":"2024-09-10T11:39:36.318067+0000","last_fullsized":"2024-09-10T11:39:36.318067+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:49.522881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317891+0000","last_change":"2024-09-10T11:39:35.325320+0000","last_active":"2024-09-10T11:39:36.317891+0000","last_peered":"2024-09-10T11:39:36.317891+0000","last_clean":"2024-09-10T11:39:36.317891+0000","last_became_active":"2024-09-10T11:39:35.325092+0000","last_became_peered":"2024-09-10T11:39:35.325092+0000","last_unstale":"2024-09-10T11:39:36.317891+0000","last_undegraded":"2024-09-10T11:39:36.317891+0000","last_fullsized":"2024-09-10T11:39:36.317891+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:56.346702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317857+0000","last_change":"2024-09-10T11:39:34.310616+0000","last_active":"2024-09-10T11:39:36.317857+0000","last_peered":"2024-09-10T11:39:36.317857+0000","last_clean":"2024-09-10T11:39:36.317857+0000","last_became_active":"2024-09-10T11:39:34.310351+0000","last_became_peered":"2024-09-10T11:39:34.310351+0000","last_unstale":"2024-09-10T11:39:36.317857+0000","last_undegraded":"2024-09-10T11:39:36.317857+0000","last_fullsized":"2024-09-10T11:39:36.317857+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:50.649898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317447+0000","last_change":"2024-09-10T11:39:35.325653+0000","last_active":"2024-09-10T11:39:36.317447+0000","last_peered":"2024-09-10T11:39:36.317447+0000","last_clean":"2024-09-10T11:39:36.317447+0000","last_became_active":"2024-09-10T11:39:35.325248+0000","last_became_peered":"2024-09-10T11:39:35.325248+0000","last_unstale":"2024-09-10T11:39:36.317447+0000","last_undegraded":"2024-09-10T11:39:36.317447+0000","last_fullsized":"2024-09-10T11:39:36.317447+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:52.530151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642367+0000","last_change":"2024-09-10T11:39:34.307130+0000","last_active":"2024-09-10T11:39:36.642367+0000","last_peered":"2024-09-10T11:39:36.642367+0000","last_clean":"2024-09-10T11:39:36.642367+0000","last_became_active":"2024-09-10T11:39:34.306881+0000","last_became_peered":"2024-09-10T11:39:34.306881+0000","last_unstale":"2024-09-10T11:39:36.642367+0000","last_undegraded":"2024-09-10T11:39:36.642367+0000","last_fullsized":"2024-09-10T11:39:36.642367+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:59.755049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":497,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642668+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:39:36.642668+0000","last_peered":"2024-09-10T11:39:36.642668+0000","last_clean":"2024-09-10T11:39:36.642668+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:39:36.642668+0000","last_undegraded":"2024-09-10T11:39:36.642668+0000","last_fullsized":"2024-09-10T11:39:36.642668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980124,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980090,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1573,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193739,"statfs":{"total":200000000,"available":198389084,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1002,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194310,"statfs":{"total":200000000,"available":198973790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1003,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194308,"statfs":{"total":200000000,"available":198972190,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1002,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194309,"statfs":{"total":200000000,"available":198973242,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1574,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193738,"statfs":{"total":200000000,"available":198387962,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:39:38.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:39:38.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 Processor -- start 2024-09-10T11:39:38.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 -- start start 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69a806ece0 0x7f69a8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a8152e60 con 0x7f69a80d1200 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f69a80d15e0 con 0x7f69a806ece0 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.792+0000 7f69ae096640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f69a8072680 con 0x7f69a815c060 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a6ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f69a806ece0 0x7f69a8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a77fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f69a80d1200 0x7f69a806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:37320/0 (socket says 172.21.15.99:37320) 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a77fe640 1 -- 172.21.15.99:0/297715305 learned_addr learned my addr 172.21.15.99:0/297715305 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:38.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a6ffd640 1 -- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 msgr2=0x7f69a815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a6ffd640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a6ffd640 1 -- 172.21.15.99:0/297715305 >> v1:172.21.15.196:6789/0 conn(0x7f69a80d1200 legacy=0x7f69a806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a6ffd640 1 -- 172.21.15.99:0/297715305 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8072960 con 0x7f69a806ece0 2024-09-10T11:39:38.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.793+0000 7f69a7fff640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.794+0000 7f69a6ffd640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8073440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896c760939c3121b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.794+0000 7f69a67fc640 1 -- 172.21.15.99:0/297715305 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990037070 con 0x7f69a806ece0 2024-09-10T11:39:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.794+0000 7f69a67fc640 1 -- 172.21.15.99:0/297715305 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699002ae50 con 0x7f69a806ece0 2024-09-10T11:39:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.794+0000 7f69a67fc640 1 -- 172.21.15.99:0/297715305 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990032660 con 0x7f69a806ece0 2024-09-10T11:39:38.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 -- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 msgr2=0x7f69a8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8073440 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f69900097a0 tx=0x7f699002a100 comp rx=0 tx=0).stop 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 -- 172.21.15.99:0/297715305 shutdown_connections 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 --2- 172.21.15.99:0/297715305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8073440 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 -- 172.21.15.99:0/297715305 >> 172.21.15.99:0/297715305 conn(0x7f69a806bc10 msgr2=0x7f69a806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 -- 172.21.15.99:0/297715305 shutdown_connections 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.795+0000 7f69ae096640 1 -- 172.21.15.99:0/297715305 wait complete. 2024-09-10T11:39:38.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.796+0000 7f69ae096640 1 Processor -- start 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.796+0000 7f69ae096640 1 -- start start 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a806da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f69a815cd90 con 0x7f69a80d1200 2024-09-10T11:39:38.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8072f20 con 0x7f69a806ece0 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f69a815ca00 con 0x7f69a815c060 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58454/0 (socket says 172.21.15.99:58454) 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 -- 172.21.15.99:0/823705905 learned_addr learned my addr 172.21.15.99:0/823705905 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a7fff640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a77fe640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 -- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 msgr2=0x7f69a806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 -- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 msgr2=0x7f69a8149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 -- 172.21.15.99:0/823705905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a8153530 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69a6ffd640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6990035040 tx=0x7f69900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990041070 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699003c460 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.797+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a8153810 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.798+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990037040 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.798+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a8153dd0 con 0x7f69a80d1200 2024-09-10T11:39:38.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.799+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8073440 con 0x7f69a80d1200 2024-09-10T11:39:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.799+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f699003a070 con 0x7f69a80d1200 2024-09-10T11:39:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.799+0000 7f699ffff640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 0x7f698806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.800+0000 7f69a77fe640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 0x7f698806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.800+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f69900aaf50 con 0x7f69a80d1200 2024-09-10T11:39:38.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.800+0000 7f69a77fe640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 0x7f698806e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f69980012e0 tx=0x7f6998009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:38.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.804+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6990079900 con 0x7f69a80d1200 2024-09-10T11:39:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.966+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69a81545d0 con 0x7f698806be30 2024-09-10T11:39:38.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.972+0000 7f699ffff640 1 -- 172.21.15.99:0/823705905 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247995 (secure 0 0 0) 0x7f6998001cb0 con 0x7f698806be30 2024-09-10T11:39:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 msgr2=0x7f698806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 0x7f698806e2f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f69980012e0 tx=0x7f6998009040 comp rx=0 tx=0).stop 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 msgr2=0x7f69a806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f6990035040 tx=0x7f69900046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 shutdown_connections 2024-09-10T11:39:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f698806be30 0x7f698806e2f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69a815c060 0x7f69a806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f69a80d1200 0x7f69a806d4c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 --2- 172.21.15.99:0/823705905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69a806ece0 0x7f69a8149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.975+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 >> 172.21.15.99:0/823705905 conn(0x7f69a806bc10 msgr2=0x7f69a814af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.976+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 shutdown_connections 2024-09-10T11:39:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:38.976+0000 7f69ae096640 1 -- 172.21.15.99:0/823705905 wait complete. 2024-09-10T11:39:38.987 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-10T11:39:37.029925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553355,"statfs":{"total":1600000000,"available":1590636536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000841"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317762+0000","last_change":"2024-09-10T11:39:35.326443+0000","last_active":"2024-09-10T11:39:36.317762+0000","last_peered":"2024-09-10T11:39:36.317762+0000","last_clean":"2024-09-10T11:39:36.317762+0000","last_became_active":"2024-09-10T11:39:35.326226+0000","last_became_peered":"2024-09-10T11:39:35.326226+0000","last_unstale":"2024-09-10T11:39:36.317762+0000","last_undegraded":"2024-09-10T11:39:36.317762+0000","last_fullsized":"2024-09-10T11:39:36.317762+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:18.479271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317628+0000","last_change":"2024-09-10T11:39:35.322939+0000","last_active":"2024-09-10T11:39:36.317628+0000","last_peered":"2024-09-10T11:39:36.317628+0000","last_clean":"2024-09-10T11:39:36.317628+0000","last_became_active":"2024-09-10T11:39:35.322717+0000","last_became_peered":"2024-09-10T11:39:35.322717+0000","last_unstale":"2024-09-10T11:39:36.317628+0000","last_undegraded":"2024-09-10T11:39:36.317628+0000","last_fullsized":"2024-09-10T11:39:36.317628+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:10.046604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317819+0000","last_change":"2024-09-10T11:39:35.331052+0000","last_active":"2024-09-10T11:39:36.317819+0000","last_peered":"2024-09-10T11:39:36.317819+0000","last_clean":"2024-09-10T11:39:36.317819+0000","last_became_active":"2024-09-10T11:39:35.330518+0000","last_became_peered":"2024-09-10T11:39:35.330518+0000","last_unstale":"2024-09-10T11:39:36.317819+0000","last_undegraded":"2024-09-10T11:39:36.317819+0000","last_fullsized":"2024-09-10T11:39:36.317819+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:48.123021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318477+0000","last_change":"2024-09-10T11:39:35.334883+0000","last_active":"2024-09-10T11:39:36.318477+0000","last_peered":"2024-09-10T11:39:36.318477+0000","last_clean":"2024-09-10T11:39:36.318477+0000","last_became_active":"2024-09-10T11:39:35.332905+0000","last_became_peered":"2024-09-10T11:39:35.332905+0000","last_unstale":"2024-09-10T11:39:36.318477+0000","last_undegraded":"2024-09-10T11:39:36.318477+0000","last_fullsized":"2024-09-10T11:39:36.318477+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:59.298486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642788+0000","last_change":"2024-09-10T11:39:35.319109+0000","last_active":"2024-09-10T11:39:36.642788+0000","last_peered":"2024-09-10T11:39:36.642788+0000","last_clean":"2024-09-10T11:39:36.642788+0000","last_became_active":"2024-09-10T11:39:35.318825+0000","last_became_peered":"2024-09-10T11:39:35.318825+0000","last_unstale":"2024-09-10T11:39:36.642788+0000","last_undegraded":"2024-09-10T11:39:36.642788+0000","last_fullsized":"2024-09-10T11:39:36.642788+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:09:50.429178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642272+0000","last_change":"2024-09-10T11:39:35.327528+0000","last_active":"2024-09-10T11:39:36.642272+0000","last_peered":"2024-09-10T11:39:36.642272+0000","last_clean":"2024-09-10T11:39:36.642272+0000","last_became_active":"2024-09-10T11:39:35.327150+0000","last_became_peered":"2024-09-10T11:39:35.327150+0000","last_unstale":"2024-09-10T11:39:36.642272+0000","last_undegraded":"2024-09-10T11:39:36.642272+0000","last_fullsized":"2024-09-10T11:39:36.642272+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:12.926848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317623+0000","last_change":"2024-09-10T11:39:35.319852+0000","last_active":"2024-09-10T11:39:36.317623+0000","last_peered":"2024-09-10T11:39:36.317623+0000","last_clean":"2024-09-10T11:39:36.317623+0000","last_became_active":"2024-09-10T11:39:35.319588+0000","last_became_peered":"2024-09-10T11:39:35.319588+0000","last_unstale":"2024-09-10T11:39:36.317623+0000","last_undegraded":"2024-09-10T11:39:36.317623+0000","last_fullsized":"2024-09-10T11:39:36.317623+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:00:57.886470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642915+0000","last_change":"2024-09-10T11:39:35.326991+0000","last_active":"2024-09-10T11:39:36.642915+0000","last_peered":"2024-09-10T11:39:36.642915+0000","last_clean":"2024-09-10T11:39:36.642915+0000","last_became_active":"2024-09-10T11:39:35.326754+0000","last_became_peered":"2024-09-10T11:39:35.326754+0000","last_unstale":"2024-09-10T11:39:36.642915+0000","last_undegraded":"2024-09-10T11:39:36.642915+0000","last_fullsized":"2024-09-10T11:39:36.642915+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:47.594010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642349+0000","last_change":"2024-09-10T11:39:35.328068+0000","last_active":"2024-09-10T11:39:36.642349+0000","last_peered":"2024-09-10T11:39:36.642349+0000","last_clean":"2024-09-10T11:39:36.642349+0000","last_became_active":"2024-09-10T11:39:35.327798+0000","last_became_peered":"2024-09-10T11:39:35.327798+0000","last_unstale":"2024-09-10T11:39:36.642349+0000","last_undegraded":"2024-09-10T11:39:36.642349+0000","last_fullsized":"2024-09-10T11:39:36.642349+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:38.692148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642398+0000","last_change":"2024-09-10T11:39:35.320792+0000","last_active":"2024-09-10T11:39:36.642398+0000","last_peered":"2024-09-10T11:39:36.642398+0000","last_clean":"2024-09-10T11:39:36.642398+0000","last_became_active":"2024-09-10T11:39:35.320542+0000","last_became_peered":"2024-09-10T11:39:35.320542+0000","last_unstale":"2024-09-10T11:39:36.642398+0000","last_undegraded":"2024-09-10T11:39:36.642398+0000","last_fullsized":"2024-09-10T11:39:36.642398+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:21.682381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642728+0000","last_change":"2024-09-10T11:39:35.331226+0000","last_active":"2024-09-10T11:39:36.642728+0000","last_peered":"2024-09-10T11:39:36.642728+0000","last_clean":"2024-09-10T11:39:36.642728+0000","last_became_active":"2024-09-10T11:39:35.330863+0000","last_became_peered":"2024-09-10T11:39:35.330863+0000","last_unstale":"2024-09-10T11:39:36.642728+0000","last_undegraded":"2024-09-10T11:39:36.642728+0000","last_fullsized":"2024-09-10T11:39:36.642728+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:44.289281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330489+0000","last_change":"2024-09-10T11:39:35.322254+0000","last_active":"2024-09-10T11:39:36.330489+0000","last_peered":"2024-09-10T11:39:36.330489+0000","last_clean":"2024-09-10T11:39:36.330489+0000","last_became_active":"2024-09-10T11:39:35.322019+0000","last_became_peered":"2024-09-10T11:39:35.322019+0000","last_unstale":"2024-09-10T11:39:36.330489+0000","last_undegraded":"2024-09-10T11:39:36.330489+0000","last_fullsized":"2024-09-10T11:39:36.330489+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:54.171624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330331+0000","last_change":"2024-09-10T11:39:35.332963+0000","last_active":"2024-09-10T11:39:36.330331+0000","last_peered":"2024-09-10T11:39:36.330331+0000","last_clean":"2024-09-10T11:39:36.330331+0000","last_became_active":"2024-09-10T11:39:35.332724+0000","last_became_peered":"2024-09-10T11:39:35.332724+0000","last_unstale":"2024-09-10T11:39:36.330331+0000","last_undegraded":"2024-09-10T11:39:36.330331+0000","last_fullsized":"2024-09-10T11:39:36.330331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:14.419829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642655+0000","last_change":"2024-09-10T11:39:35.318944+0000","last_active":"2024-09-10T11:39:36.642655+0000","last_peered":"2024-09-10T11:39:36.642655+0000","last_clean":"2024-09-10T11:39:36.642655+0000","last_became_active":"2024-09-10T11:39:35.318718+0000","last_became_peered":"2024-09-10T11:39:35.318718+0000","last_unstale":"2024-09-10T11:39:36.642655+0000","last_undegraded":"2024-09-10T11:39:36.642655+0000","last_fullsized":"2024-09-10T11:39:36.642655+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:38.896709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642555+0000","last_change":"2024-09-10T11:39:35.325956+0000","last_active":"2024-09-10T11:39:36.642555+0000","last_peered":"2024-09-10T11:39:36.642555+0000","last_clean":"2024-09-10T11:39:36.642555+0000","last_became_active":"2024-09-10T11:39:35.325412+0000","last_became_peered":"2024-09-10T11:39:35.325412+0000","last_unstale":"2024-09-10T11:39:36.642555+0000","last_undegraded":"2024-09-10T11:39:36.642555+0000","last_fullsized":"2024-09-10T11:39:36.642555+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:20.071561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317607+0000","last_change":"2024-09-10T11:39:35.323009+0000","last_active":"2024-09-10T11:39:36.317607+0000","last_peered":"2024-09-10T11:39:36.317607+0000","last_clean":"2024-09-10T11:39:36.317607+0000","last_became_active":"2024-09-10T11:39:35.322564+0000","last_became_peered":"2024-09-10T11:39:35.322564+0000","last_unstale":"2024-09-10T11:39:36.317607+0000","last_undegraded":"2024-09-10T11:39:36.317607+0000","last_fullsized":"2024-09-10T11:39:36.317607+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:05:05.739511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642544+0000","last_change":"2024-09-10T11:39:35.327517+0000","last_active":"2024-09-10T11:39:36.642544+0000","last_peered":"2024-09-10T11:39:36.642544+0000","last_clean":"2024-09-10T11:39:36.642544+0000","last_became_active":"2024-09-10T11:39:35.327257+0000","last_became_peered":"2024-09-10T11:39:35.327257+0000","last_unstale":"2024-09-10T11:39:36.642544+0000","last_undegraded":"2024-09-10T11:39:36.642544+0000","last_fullsized":"2024-09-10T11:39:36.642544+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:54.743964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318198+0000","last_change":"2024-09-10T11:39:35.329536+0000","last_active":"2024-09-10T11:39:36.318198+0000","last_peered":"2024-09-10T11:39:36.318198+0000","last_clean":"2024-09-10T11:39:36.318198+0000","last_became_active":"2024-09-10T11:39:35.329282+0000","last_became_peered":"2024-09-10T11:39:35.329282+0000","last_unstale":"2024-09-10T11:39:36.318198+0000","last_undegraded":"2024-09-10T11:39:36.318198+0000","last_fullsized":"2024-09-10T11:39:36.318198+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:32.765304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317547+0000","last_change":"2024-09-10T11:39:35.326704+0000","last_active":"2024-09-10T11:39:36.317547+0000","last_peered":"2024-09-10T11:39:36.317547+0000","last_clean":"2024-09-10T11:39:36.317547+0000","last_became_active":"2024-09-10T11:39:35.326408+0000","last_became_peered":"2024-09-10T11:39:35.326408+0000","last_unstale":"2024-09-10T11:39:36.317547+0000","last_undegraded":"2024-09-10T11:39:36.317547+0000","last_fullsized":"2024-09-10T11:39:36.317547+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:32.033088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330490+0000","last_change":"2024-09-10T11:39:35.328405+0000","last_active":"2024-09-10T11:39:36.330490+0000","last_peered":"2024-09-10T11:39:36.330490+0000","last_clean":"2024-09-10T11:39:36.330490+0000","last_became_active":"2024-09-10T11:39:35.328045+0000","last_became_peered":"2024-09-10T11:39:35.328045+0000","last_unstale":"2024-09-10T11:39:36.330490+0000","last_undegraded":"2024-09-10T11:39:36.330490+0000","last_fullsized":"2024-09-10T11:39:36.330490+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:53.024338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642382+0000","last_change":"2024-09-10T11:39:35.324134+0000","last_active":"2024-09-10T11:39:36.642382+0000","last_peered":"2024-09-10T11:39:36.642382+0000","last_clean":"2024-09-10T11:39:36.642382+0000","last_became_active":"2024-09-10T11:39:35.323860+0000","last_became_peered":"2024-09-10T11:39:35.323860+0000","last_unstale":"2024-09-10T11:39:36.642382+0000","last_undegraded":"2024-09-10T11:39:36.642382+0000","last_fullsized":"2024-09-10T11:39:36.642382+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:52:56.129995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642747+0000","last_change":"2024-09-10T11:39:35.325448+0000","last_active":"2024-09-10T11:39:36.642747+0000","last_peered":"2024-09-10T11:39:36.642747+0000","last_clean":"2024-09-10T11:39:36.642747+0000","last_became_active":"2024-09-10T11:39:35.325188+0000","last_became_peered":"2024-09-10T11:39:35.325188+0000","last_unstale":"2024-09-10T11:39:36.642747+0000","last_undegraded":"2024-09-10T11:39:36.642747+0000","last_fullsized":"2024-09-10T11:39:36.642747+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:35.145197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457023+0000","last_change":"2024-09-10T11:39:35.333305+0000","last_active":"2024-09-10T11:39:36.457023+0000","last_peered":"2024-09-10T11:39:36.457023+0000","last_clean":"2024-09-10T11:39:36.457023+0000","last_became_active":"2024-09-10T11:39:35.332223+0000","last_became_peered":"2024-09-10T11:39:35.332223+0000","last_unstale":"2024-09-10T11:39:36.457023+0000","last_undegraded":"2024-09-10T11:39:36.457023+0000","last_fullsized":"2024-09-10T11:39:36.457023+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:36.123100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642408+0000","last_change":"2024-09-10T11:39:35.319263+0000","last_active":"2024-09-10T11:39:36.642408+0000","last_peered":"2024-09-10T11:39:36.642408+0000","last_clean":"2024-09-10T11:39:36.642408+0000","last_became_active":"2024-09-10T11:39:35.318878+0000","last_became_peered":"2024-09-10T11:39:35.318878+0000","last_unstale":"2024-09-10T11:39:36.642408+0000","last_undegraded":"2024-09-10T11:39:36.642408+0000","last_fullsized":"2024-09-10T11:39:36.642408+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:17.528974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642901+0000","last_change":"2024-09-10T11:39:35.329244+0000","last_active":"2024-09-10T11:39:36.642901+0000","last_peered":"2024-09-10T11:39:36.642901+0000","last_clean":"2024-09-10T11:39:36.642901+0000","last_became_active":"2024-09-10T11:39:35.328732+0000","last_became_peered":"2024-09-10T11:39:35.328732+0000","last_unstale":"2024-09-10T11:39:36.642901+0000","last_undegraded":"2024-09-10T11:39:36.642901+0000","last_fullsized":"2024-09-10T11:39:36.642901+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:09.845801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457050+0000","last_change":"2024-09-10T11:39:35.325279+0000","last_active":"2024-09-10T11:39:36.457050+0000","last_peered":"2024-09-10T11:39:36.457050+0000","last_clean":"2024-09-10T11:39:36.457050+0000","last_became_active":"2024-09-10T11:39:35.325051+0000","last_became_peered":"2024-09-10T11:39:35.325051+0000","last_unstale":"2024-09-10T11:39:36.457050+0000","last_undegraded":"2024-09-10T11:39:36.457050+0000","last_fullsized":"2024-09-10T11:39:36.457050+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:40.038279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642902+0000","last_change":"2024-09-10T11:39:35.319547+0000","last_active":"2024-09-10T11:39:36.642902+0000","last_peered":"2024-09-10T11:39:36.642902+0000","last_clean":"2024-09-10T11:39:36.642902+0000","last_became_active":"2024-09-10T11:39:35.319268+0000","last_became_peered":"2024-09-10T11:39:35.319268+0000","last_unstale":"2024-09-10T11:39:36.642902+0000","last_undegraded":"2024-09-10T11:39:36.642902+0000","last_fullsized":"2024-09-10T11:39:36.642902+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:39.362543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317214+0000","last_change":"2024-09-10T11:39:35.327878+0000","last_active":"2024-09-10T11:39:36.317214+0000","last_peered":"2024-09-10T11:39:36.317214+0000","last_clean":"2024-09-10T11:39:36.317214+0000","last_became_active":"2024-09-10T11:39:35.327650+0000","last_became_peered":"2024-09-10T11:39:35.327650+0000","last_unstale":"2024-09-10T11:39:36.317214+0000","last_undegraded":"2024-09-10T11:39:36.317214+0000","last_fullsized":"2024-09-10T11:39:36.317214+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:33.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317541+0000","last_change":"2024-09-10T11:39:35.327817+0000","last_active":"2024-09-10T11:39:36.317541+0000","last_peered":"2024-09-10T11:39:36.317541+0000","last_clean":"2024-09-10T11:39:36.317541+0000","last_became_active":"2024-09-10T11:39:35.327566+0000","last_became_peered":"2024-09-10T11:39:35.327566+0000","last_unstale":"2024-09-10T11:39:36.317541+0000","last_undegraded":"2024-09-10T11:39:36.317541+0000","last_fullsized":"2024-09-10T11:39:36.317541+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:48:25.299613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642689+0000","last_change":"2024-09-10T11:39:35.319007+0000","last_active":"2024-09-10T11:39:36.642689+0000","last_peered":"2024-09-10T11:39:36.642689+0000","last_clean":"2024-09-10T11:39:36.642689+0000","last_became_active":"2024-09-10T11:39:35.318773+0000","last_became_peered":"2024-09-10T11:39:35.318773+0000","last_unstale":"2024-09-10T11:39:36.642689+0000","last_undegraded":"2024-09-10T11:39:36.642689+0000","last_fullsized":"2024-09-10T11:39:36.642689+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:00.949571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317866+0000","last_change":"2024-09-10T11:39:35.329898+0000","last_active":"2024-09-10T11:39:36.317866+0000","last_peered":"2024-09-10T11:39:36.317866+0000","last_clean":"2024-09-10T11:39:36.317866+0000","last_became_active":"2024-09-10T11:39:35.329588+0000","last_became_peered":"2024-09-10T11:39:35.329588+0000","last_unstale":"2024-09-10T11:39:36.317866+0000","last_undegraded":"2024-09-10T11:39:36.317866+0000","last_fullsized":"2024-09-10T11:39:36.317866+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:51.286757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642128+0000","last_change":"2024-09-10T11:39:35.328717+0000","last_active":"2024-09-10T11:39:36.642128+0000","last_peered":"2024-09-10T11:39:36.642128+0000","last_clean":"2024-09-10T11:39:36.642128+0000","last_became_active":"2024-09-10T11:39:35.328154+0000","last_became_peered":"2024-09-10T11:39:35.328154+0000","last_unstale":"2024-09-10T11:39:36.642128+0000","last_undegraded":"2024-09-10T11:39:36.642128+0000","last_fullsized":"2024-09-10T11:39:36.642128+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:21.464233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318644+0000","last_change":"2024-09-10T11:39:35.334614+0000","last_active":"2024-09-10T11:39:36.318644+0000","last_peered":"2024-09-10T11:39:36.318644+0000","last_clean":"2024-09-10T11:39:36.318644+0000","last_became_active":"2024-09-10T11:39:35.332540+0000","last_became_peered":"2024-09-10T11:39:35.332540+0000","last_unstale":"2024-09-10T11:39:36.318644+0000","last_undegraded":"2024-09-10T11:39:36.318644+0000","last_fullsized":"2024-09-10T11:39:36.318644+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:29.547514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"182'4","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317947+0000","last_change":"2024-09-10T11:39:34.312948+0000","last_active":"2024-09-10T11:39:36.317947+0000","last_peered":"2024-09-10T11:39:36.317947+0000","last_clean":"2024-09-10T11:39:36.317947+0000","last_became_active":"2024-09-10T11:39:34.311720+0000","last_became_peered":"2024-09-10T11:39:34.311720+0000","last_unstale":"2024-09-10T11:39:36.317947+0000","last_undegraded":"2024-09-10T11:39:36.317947+0000","last_fullsized":"2024-09-10T11:39:36.317947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:16.170110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317561+0000","last_change":"2024-09-10T11:39:35.333744+0000","last_active":"2024-09-10T11:39:36.317561+0000","last_peered":"2024-09-10T11:39:36.317561+0000","last_clean":"2024-09-10T11:39:36.317561+0000","last_became_active":"2024-09-10T11:39:35.333480+0000","last_became_peered":"2024-09-10T11:39:35.333480+0000","last_unstale":"2024-09-10T11:39:36.317561+0000","last_undegraded":"2024-09-10T11:39:36.317561+0000","last_fullsized":"2024-09-10T11:39:36.317561+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:44.976183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642754+0000","last_change":"2024-09-10T11:39:34.312061+0000","last_active":"2024-09-10T11:39:36.642754+0000","last_peered":"2024-09-10T11:39:36.642754+0000","last_clean":"2024-09-10T11:39:36.642754+0000","last_became_active":"2024-09-10T11:39:34.311818+0000","last_became_peered":"2024-09-10T11:39:34.311818+0000","last_unstale":"2024-09-10T11:39:36.642754+0000","last_undegraded":"2024-09-10T11:39:36.642754+0000","last_fullsized":"2024-09-10T11:39:36.642754+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:52.795074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642489+0000","last_change":"2024-09-10T11:39:35.326633+0000","last_active":"2024-09-10T11:39:36.642489+0000","last_peered":"2024-09-10T11:39:36.642489+0000","last_clean":"2024-09-10T11:39:36.642489+0000","last_became_active":"2024-09-10T11:39:35.324469+0000","last_became_peered":"2024-09-10T11:39:35.324469+0000","last_unstale":"2024-09-10T11:39:36.642489+0000","last_undegraded":"2024-09-10T11:39:36.642489+0000","last_fullsized":"2024-09-10T11:39:36.642489+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:23.053184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642274+0000","last_change":"2024-09-10T11:39:34.314840+0000","last_active":"2024-09-10T11:39:36.642274+0000","last_peered":"2024-09-10T11:39:36.642274+0000","last_clean":"2024-09-10T11:39:36.642274+0000","last_became_active":"2024-09-10T11:39:34.314346+0000","last_became_peered":"2024-09-10T11:39:34.314346+0000","last_unstale":"2024-09-10T11:39:36.642274+0000","last_undegraded":"2024-09-10T11:39:36.642274+0000","last_fullsized":"2024-09-10T11:39:36.642274+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:49.137667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642818+0000","last_change":"2024-09-10T11:39:35.325664+0000","last_active":"2024-09-10T11:39:36.642818+0000","last_peered":"2024-09-10T11:39:36.642818+0000","last_clean":"2024-09-10T11:39:36.642818+0000","last_became_active":"2024-09-10T11:39:35.325053+0000","last_became_peered":"2024-09-10T11:39:35.325053+0000","last_unstale":"2024-09-10T11:39:36.642818+0000","last_undegraded":"2024-09-10T11:39:36.642818+0000","last_fullsized":"2024-09-10T11:39:36.642818+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:15.341660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457126+0000","last_change":"2024-09-10T11:39:34.309774+0000","last_active":"2024-09-10T11:39:36.457126+0000","last_peered":"2024-09-10T11:39:36.457126+0000","last_clean":"2024-09-10T11:39:36.457126+0000","last_became_active":"2024-09-10T11:39:34.309047+0000","last_became_peered":"2024-09-10T11:39:34.309047+0000","last_unstale":"2024-09-10T11:39:36.457126+0000","last_undegraded":"2024-09-10T11:39:36.457126+0000","last_fullsized":"2024-09-10T11:39:36.457126+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:12.211476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318383+0000","last_change":"2024-09-10T11:39:35.333647+0000","last_active":"2024-09-10T11:39:36.318383+0000","last_peered":"2024-09-10T11:39:36.318383+0000","last_clean":"2024-09-10T11:39:36.318383+0000","last_became_active":"2024-09-10T11:39:35.332951+0000","last_became_peered":"2024-09-10T11:39:35.332951+0000","last_unstale":"2024-09-10T11:39:36.318383+0000","last_undegraded":"2024-09-10T11:39:36.318383+0000","last_fullsized":"2024-09-10T11:39:36.318383+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:58:18.889369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317792+0000","last_change":"2024-09-10T11:39:34.310066+0000","last_active":"2024-09-10T11:39:36.317792+0000","last_peered":"2024-09-10T11:39:36.317792+0000","last_clean":"2024-09-10T11:39:36.317792+0000","last_became_active":"2024-09-10T11:39:34.309291+0000","last_became_peered":"2024-09-10T11:39:34.309291+0000","last_unstale":"2024-09-10T11:39:36.317792+0000","last_undegraded":"2024-09-10T11:39:36.317792+0000","last_fullsized":"2024-09-10T11:39:36.317792+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:05.569185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642277+0000","last_change":"2024-09-10T11:39:35.329287+0000","last_active":"2024-09-10T11:39:36.642277+0000","last_peered":"2024-09-10T11:39:36.642277+0000","last_clean":"2024-09-10T11:39:36.642277+0000","last_became_active":"2024-09-10T11:39:35.328477+0000","last_became_peered":"2024-09-10T11:39:35.328477+0000","last_unstale":"2024-09-10T11:39:36.642277+0000","last_undegraded":"2024-09-10T11:39:36.642277+0000","last_fullsized":"2024-09-10T11:39:36.642277+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:51:17.107794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318507+0000","last_change":"2024-09-10T11:39:34.312118+0000","last_active":"2024-09-10T11:39:36.318507+0000","last_peered":"2024-09-10T11:39:36.318507+0000","last_clean":"2024-09-10T11:39:36.318507+0000","last_became_active":"2024-09-10T11:39:34.311890+0000","last_became_peered":"2024-09-10T11:39:34.311890+0000","last_unstale":"2024-09-10T11:39:36.318507+0000","last_undegraded":"2024-09-10T11:39:36.318507+0000","last_fullsized":"2024-09-10T11:39:36.318507+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:48.484800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.456960+0000","last_change":"2024-09-10T11:39:35.327736+0000","last_active":"2024-09-10T11:39:36.456960+0000","last_peered":"2024-09-10T11:39:36.456960+0000","last_clean":"2024-09-10T11:39:36.456960+0000","last_became_active":"2024-09-10T11:39:35.327502+0000","last_became_peered":"2024-09-10T11:39:35.327502+0000","last_unstale":"2024-09-10T11:39:36.456960+0000","last_undegraded":"2024-09-10T11:39:36.456960+0000","last_fullsized":"2024-09-10T11:39:36.456960+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:33.537228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317703+0000","last_change":"2024-09-10T11:39:34.311736+0000","last_active":"2024-09-10T11:39:36.317703+0000","last_peered":"2024-09-10T11:39:36.317703+0000","last_clean":"2024-09-10T11:39:36.317703+0000","last_became_active":"2024-09-10T11:39:34.310903+0000","last_became_peered":"2024-09-10T11:39:34.310903+0000","last_unstale":"2024-09-10T11:39:36.317703+0000","last_undegraded":"2024-09-10T11:39:36.317703+0000","last_fullsized":"2024-09-10T11:39:36.317703+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:50.781728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318799+0000","last_change":"2024-09-10T11:39:35.334390+0000","last_active":"2024-09-10T11:39:36.318799+0000","last_peered":"2024-09-10T11:39:36.318799+0000","last_clean":"2024-09-10T11:39:36.318799+0000","last_became_active":"2024-09-10T11:39:35.331411+0000","last_became_peered":"2024-09-10T11:39:35.331411+0000","last_unstale":"2024-09-10T11:39:36.318799+0000","last_undegraded":"2024-09-10T11:39:36.318799+0000","last_fullsized":"2024-09-10T11:39:36.318799+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:15.788303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457136+0000","last_change":"2024-09-10T11:39:34.308307+0000","last_active":"2024-09-10T11:39:36.457136+0000","last_peered":"2024-09-10T11:39:36.457136+0000","last_clean":"2024-09-10T11:39:36.457136+0000","last_became_active":"2024-09-10T11:39:34.308062+0000","last_became_peered":"2024-09-10T11:39:34.308062+0000","last_unstale":"2024-09-10T11:39:36.457136+0000","last_undegraded":"2024-09-10T11:39:36.457136+0000","last_fullsized":"2024-09-10T11:39:36.457136+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:44:08.765279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317781+0000","last_change":"2024-09-10T11:39:35.321895+0000","last_active":"2024-09-10T11:39:36.317781+0000","last_peered":"2024-09-10T11:39:36.317781+0000","last_clean":"2024-09-10T11:39:36.317781+0000","last_became_active":"2024-09-10T11:39:35.321266+0000","last_became_peered":"2024-09-10T11:39:35.321266+0000","last_unstale":"2024-09-10T11:39:36.317781+0000","last_undegraded":"2024-09-10T11:39:36.317781+0000","last_fullsized":"2024-09-10T11:39:36.317781+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:23.624747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642451+0000","last_change":"2024-09-10T11:39:34.314085+0000","last_active":"2024-09-10T11:39:36.642451+0000","last_peered":"2024-09-10T11:39:36.642451+0000","last_clean":"2024-09-10T11:39:36.642451+0000","last_became_active":"2024-09-10T11:39:34.313848+0000","last_became_peered":"2024-09-10T11:39:34.313848+0000","last_unstale":"2024-09-10T11:39:36.642451+0000","last_undegraded":"2024-09-10T11:39:36.642451+0000","last_fullsized":"2024-09-10T11:39:36.642451+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:58.499889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317114+0000","last_change":"2024-09-10T11:39:35.321435+0000","last_active":"2024-09-10T11:39:36.317114+0000","last_peered":"2024-09-10T11:39:36.317114+0000","last_clean":"2024-09-10T11:39:36.317114+0000","last_became_active":"2024-09-10T11:39:35.321158+0000","last_became_peered":"2024-09-10T11:39:35.321158+0000","last_unstale":"2024-09-10T11:39:36.317114+0000","last_undegraded":"2024-09-10T11:39:36.317114+0000","last_fullsized":"2024-09-10T11:39:36.317114+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:34.954100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"182'3","reported_seq":17,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457429+0000","last_change":"2024-09-10T11:39:34.309865+0000","last_active":"2024-09-10T11:39:36.457429+0000","last_peered":"2024-09-10T11:39:36.457429+0000","last_clean":"2024-09-10T11:39:36.457429+0000","last_became_active":"2024-09-10T11:39:34.309611+0000","last_became_peered":"2024-09-10T11:39:34.309611+0000","last_unstale":"2024-09-10T11:39:36.457429+0000","last_undegraded":"2024-09-10T11:39:36.457429+0000","last_fullsized":"2024-09-10T11:39:36.457429+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:26:32.198601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317695+0000","last_change":"2024-09-10T11:39:35.331243+0000","last_active":"2024-09-10T11:39:36.317695+0000","last_peered":"2024-09-10T11:39:36.317695+0000","last_clean":"2024-09-10T11:39:36.317695+0000","last_became_active":"2024-09-10T11:39:35.330828+0000","last_became_peered":"2024-09-10T11:39:35.330828+0000","last_unstale":"2024-09-10T11:39:36.317695+0000","last_undegraded":"2024-09-10T11:39:36.317695+0000","last_fullsized":"2024-09-10T11:39:36.317695+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:27:51.662089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"182'6","reported_seq":20,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642510+0000","last_change":"2024-09-10T11:39:34.313206+0000","last_active":"2024-09-10T11:39:36.642510+0000","last_peered":"2024-09-10T11:39:36.642510+0000","last_clean":"2024-09-10T11:39:36.642510+0000","last_became_active":"2024-09-10T11:39:34.312484+0000","last_became_peered":"2024-09-10T11:39:34.312484+0000","last_unstale":"2024-09-10T11:39:36.642510+0000","last_undegraded":"2024-09-10T11:39:36.642510+0000","last_fullsized":"2024-09-10T11:39:36.642510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:28.354422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317931+0000","last_change":"2024-09-10T11:39:35.322803+0000","last_active":"2024-09-10T11:39:36.317931+0000","last_peered":"2024-09-10T11:39:36.317931+0000","last_clean":"2024-09-10T11:39:36.317931+0000","last_became_active":"2024-09-10T11:39:35.322259+0000","last_became_peered":"2024-09-10T11:39:35.322259+0000","last_unstale":"2024-09-10T11:39:36.317931+0000","last_undegraded":"2024-09-10T11:39:36.317931+0000","last_fullsized":"2024-09-10T11:39:36.317931+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:48:11.994600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318494+0000","last_change":"2024-09-10T11:39:34.307625+0000","last_active":"2024-09-10T11:39:36.318494+0000","last_peered":"2024-09-10T11:39:36.318494+0000","last_clean":"2024-09-10T11:39:36.318494+0000","last_became_active":"2024-09-10T11:39:34.307190+0000","last_became_peered":"2024-09-10T11:39:34.307190+0000","last_unstale":"2024-09-10T11:39:36.318494+0000","last_undegraded":"2024-09-10T11:39:36.318494+0000","last_fullsized":"2024-09-10T11:39:36.318494+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:34.150373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457061+0000","last_change":"2024-09-10T11:39:35.326359+0000","last_active":"2024-09-10T11:39:36.457061+0000","last_peered":"2024-09-10T11:39:36.457061+0000","last_clean":"2024-09-10T11:39:36.457061+0000","last_became_active":"2024-09-10T11:39:35.326131+0000","last_became_peered":"2024-09-10T11:39:35.326131+0000","last_unstale":"2024-09-10T11:39:36.457061+0000","last_undegraded":"2024-09-10T11:39:36.457061+0000","last_fullsized":"2024-09-10T11:39:36.457061+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:21.248109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642552+0000","last_change":"2024-09-10T11:39:34.311528+0000","last_active":"2024-09-10T11:39:36.642552+0000","last_peered":"2024-09-10T11:39:36.642552+0000","last_clean":"2024-09-10T11:39:36.642552+0000","last_became_active":"2024-09-10T11:39:34.311293+0000","last_became_peered":"2024-09-10T11:39:34.311293+0000","last_unstale":"2024-09-10T11:39:36.642552+0000","last_undegraded":"2024-09-10T11:39:36.642552+0000","last_fullsized":"2024-09-10T11:39:36.642552+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:54.132543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642217+0000","last_change":"2024-09-10T11:39:35.325616+0000","last_active":"2024-09-10T11:39:36.642217+0000","last_peered":"2024-09-10T11:39:36.642217+0000","last_clean":"2024-09-10T11:39:36.642217+0000","last_became_active":"2024-09-10T11:39:35.325377+0000","last_became_peered":"2024-09-10T11:39:35.325377+0000","last_unstale":"2024-09-10T11:39:36.642217+0000","last_undegraded":"2024-09-10T11:39:36.642217+0000","last_fullsized":"2024-09-10T11:39:36.642217+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:42.425900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317688+0000","last_change":"2024-09-10T11:39:34.307978+0000","last_active":"2024-09-10T11:39:36.317688+0000","last_peered":"2024-09-10T11:39:36.317688+0000","last_clean":"2024-09-10T11:39:36.317688+0000","last_became_active":"2024-09-10T11:39:34.307750+0000","last_became_peered":"2024-09-10T11:39:34.307750+0000","last_unstale":"2024-09-10T11:39:36.317688+0000","last_undegraded":"2024-09-10T11:39:36.317688+0000","last_fullsized":"2024-09-10T11:39:36.317688+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:12:24.398920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642569+0000","last_change":"2024-09-10T11:39:35.319651+0000","last_active":"2024-09-10T11:39:36.642569+0000","last_peered":"2024-09-10T11:39:36.642569+0000","last_clean":"2024-09-10T11:39:36.642569+0000","last_became_active":"2024-09-10T11:39:35.319408+0000","last_became_peered":"2024-09-10T11:39:35.319408+0000","last_unstale":"2024-09-10T11:39:36.642569+0000","last_undegraded":"2024-09-10T11:39:36.642569+0000","last_fullsized":"2024-09-10T11:39:36.642569+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:20:10.359966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"182'8","reported_seq":22,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642871+0000","last_change":"2024-09-10T11:39:34.313026+0000","last_active":"2024-09-10T11:39:36.642871+0000","last_peered":"2024-09-10T11:39:36.642871+0000","last_clean":"2024-09-10T11:39:36.642871+0000","last_became_active":"2024-09-10T11:39:34.312811+0000","last_became_peered":"2024-09-10T11:39:34.312811+0000","last_unstale":"2024-09-10T11:39:36.642871+0000","last_undegraded":"2024-09-10T11:39:36.642871+0000","last_fullsized":"2024-09-10T11:39:36.642871+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:52.398093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317264+0000","last_change":"2024-09-10T11:39:35.327268+0000","last_active":"2024-09-10T11:39:36.317264+0000","last_peered":"2024-09-10T11:39:36.317264+0000","last_clean":"2024-09-10T11:39:36.317264+0000","last_became_active":"2024-09-10T11:39:35.327043+0000","last_became_peered":"2024-09-10T11:39:35.327043+0000","last_unstale":"2024-09-10T11:39:36.317264+0000","last_undegraded":"2024-09-10T11:39:36.317264+0000","last_fullsized":"2024-09-10T11:39:36.317264+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:41:45.663322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642620+0000","last_change":"2024-09-10T11:39:34.313482+0000","last_active":"2024-09-10T11:39:36.642620+0000","last_peered":"2024-09-10T11:39:36.642620+0000","last_clean":"2024-09-10T11:39:36.642620+0000","last_became_active":"2024-09-10T11:39:34.312930+0000","last_became_peered":"2024-09-10T11:39:34.312930+0000","last_unstale":"2024-09-10T11:39:36.642620+0000","last_undegraded":"2024-09-10T11:39:36.642620+0000","last_fullsized":"2024-09-10T11:39:36.642620+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:25.755875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318925+0000","last_change":"2024-09-10T11:39:35.334005+0000","last_active":"2024-09-10T11:39:36.318925+0000","last_peered":"2024-09-10T11:39:36.318925+0000","last_clean":"2024-09-10T11:39:36.318925+0000","last_became_active":"2024-09-10T11:39:35.330837+0000","last_became_peered":"2024-09-10T11:39:35.330837+0000","last_unstale":"2024-09-10T11:39:36.318925+0000","last_undegraded":"2024-09-10T11:39:36.318925+0000","last_fullsized":"2024-09-10T11:39:36.318925+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:33:06.624591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642912+0000","last_change":"2024-09-10T11:39:34.312956+0000","last_active":"2024-09-10T11:39:36.642912+0000","last_peered":"2024-09-10T11:39:36.642912+0000","last_clean":"2024-09-10T11:39:36.642912+0000","last_became_active":"2024-09-10T11:39:34.312733+0000","last_became_peered":"2024-09-10T11:39:34.312733+0000","last_unstale":"2024-09-10T11:39:36.642912+0000","last_undegraded":"2024-09-10T11:39:36.642912+0000","last_fullsized":"2024-09-10T11:39:36.642912+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:37:58.909280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318666+0000","last_change":"2024-09-10T11:39:35.324599+0000","last_active":"2024-09-10T11:39:36.318666+0000","last_peered":"2024-09-10T11:39:36.318666+0000","last_clean":"2024-09-10T11:39:36.318666+0000","last_became_active":"2024-09-10T11:39:35.324362+0000","last_became_peered":"2024-09-10T11:39:35.324362+0000","last_unstale":"2024-09-10T11:39:36.318666+0000","last_undegraded":"2024-09-10T11:39:36.318666+0000","last_fullsized":"2024-09-10T11:39:36.318666+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:04.988917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"182'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642488+0000","last_change":"2024-09-10T11:39:34.307654+0000","last_active":"2024-09-10T11:39:36.642488+0000","last_peered":"2024-09-10T11:39:36.642488+0000","last_clean":"2024-09-10T11:39:36.642488+0000","last_became_active":"2024-09-10T11:39:34.307418+0000","last_became_peered":"2024-09-10T11:39:34.307418+0000","last_unstale":"2024-09-10T11:39:36.642488+0000","last_undegraded":"2024-09-10T11:39:36.642488+0000","last_fullsized":"2024-09-10T11:39:36.642488+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:24:22.111985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.456937+0000","last_change":"2024-09-10T11:39:35.330555+0000","last_active":"2024-09-10T11:39:36.456937+0000","last_peered":"2024-09-10T11:39:36.456937+0000","last_clean":"2024-09-10T11:39:36.456937+0000","last_became_active":"2024-09-10T11:39:35.329349+0000","last_became_peered":"2024-09-10T11:39:35.329349+0000","last_unstale":"2024-09-10T11:39:36.456937+0000","last_undegraded":"2024-09-10T11:39:36.456937+0000","last_fullsized":"2024-09-10T11:39:36.456937+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:52.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330619+0000","last_change":"2024-09-10T11:39:34.315002+0000","last_active":"2024-09-10T11:39:36.330619+0000","last_peered":"2024-09-10T11:39:36.330619+0000","last_clean":"2024-09-10T11:39:36.330619+0000","last_became_active":"2024-09-10T11:39:34.314276+0000","last_became_peered":"2024-09-10T11:39:34.314276+0000","last_unstale":"2024-09-10T11:39:36.330619+0000","last_undegraded":"2024-09-10T11:39:36.330619+0000","last_fullsized":"2024-09-10T11:39:36.330619+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:34.016844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317649+0000","last_change":"2024-09-10T11:39:35.336786+0000","last_active":"2024-09-10T11:39:36.317649+0000","last_peered":"2024-09-10T11:39:36.317649+0000","last_clean":"2024-09-10T11:39:36.317649+0000","last_became_active":"2024-09-10T11:39:35.336506+0000","last_became_peered":"2024-09-10T11:39:35.336506+0000","last_unstale":"2024-09-10T11:39:36.317649+0000","last_undegraded":"2024-09-10T11:39:36.317649+0000","last_fullsized":"2024-09-10T11:39:36.317649+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:04:50.294733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457526+0000","last_change":"2024-09-10T11:39:34.312714+0000","last_active":"2024-09-10T11:39:36.457526+0000","last_peered":"2024-09-10T11:39:36.457526+0000","last_clean":"2024-09-10T11:39:36.457526+0000","last_became_active":"2024-09-10T11:39:34.312045+0000","last_became_peered":"2024-09-10T11:39:34.312045+0000","last_unstale":"2024-09-10T11:39:36.457526+0000","last_undegraded":"2024-09-10T11:39:36.457526+0000","last_fullsized":"2024-09-10T11:39:36.457526+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:26.566597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457189+0000","last_change":"2024-09-10T11:39:35.332890+0000","last_active":"2024-09-10T11:39:36.457189+0000","last_peered":"2024-09-10T11:39:36.457189+0000","last_clean":"2024-09-10T11:39:36.457189+0000","last_became_active":"2024-09-10T11:39:35.331901+0000","last_became_peered":"2024-09-10T11:39:35.331901+0000","last_unstale":"2024-09-10T11:39:36.457189+0000","last_undegraded":"2024-09-10T11:39:36.457189+0000","last_fullsized":"2024-09-10T11:39:36.457189+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:44:00.156648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330414+0000","last_change":"2024-09-10T11:39:34.313893+0000","last_active":"2024-09-10T11:39:36.330414+0000","last_peered":"2024-09-10T11:39:36.330414+0000","last_clean":"2024-09-10T11:39:36.330414+0000","last_became_active":"2024-09-10T11:39:34.313640+0000","last_became_peered":"2024-09-10T11:39:34.313640+0000","last_unstale":"2024-09-10T11:39:36.330414+0000","last_undegraded":"2024-09-10T11:39:36.330414+0000","last_fullsized":"2024-09-10T11:39:36.330414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:54.058453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317949+0000","last_change":"2024-09-10T11:39:35.324646+0000","last_active":"2024-09-10T11:39:36.317949+0000","last_peered":"2024-09-10T11:39:36.317949+0000","last_clean":"2024-09-10T11:39:36.317949+0000","last_became_active":"2024-09-10T11:39:35.324412+0000","last_became_peered":"2024-09-10T11:39:35.324412+0000","last_unstale":"2024-09-10T11:39:36.317949+0000","last_undegraded":"2024-09-10T11:39:36.317949+0000","last_fullsized":"2024-09-10T11:39:36.317949+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:40.285906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318639+0000","last_change":"2024-09-10T11:39:34.310970+0000","last_active":"2024-09-10T11:39:36.318639+0000","last_peered":"2024-09-10T11:39:36.318639+0000","last_clean":"2024-09-10T11:39:36.318639+0000","last_became_active":"2024-09-10T11:39:34.310693+0000","last_became_peered":"2024-09-10T11:39:34.310693+0000","last_unstale":"2024-09-10T11:39:36.318639+0000","last_undegraded":"2024-09-10T11:39:36.318639+0000","last_fullsized":"2024-09-10T11:39:36.318639+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:56.261211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317423+0000","last_change":"2024-09-10T11:39:35.319279+0000","last_active":"2024-09-10T11:39:36.317423+0000","last_peered":"2024-09-10T11:39:36.317423+0000","last_clean":"2024-09-10T11:39:36.317423+0000","last_became_active":"2024-09-10T11:39:35.318987+0000","last_became_peered":"2024-09-10T11:39:35.318987+0000","last_unstale":"2024-09-10T11:39:36.317423+0000","last_undegraded":"2024-09-10T11:39:36.317423+0000","last_fullsized":"2024-09-10T11:39:36.317423+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:52.001312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.330512+0000","last_change":"2024-09-10T11:39:34.315568+0000","last_active":"2024-09-10T11:39:36.330512+0000","last_peered":"2024-09-10T11:39:36.330512+0000","last_clean":"2024-09-10T11:39:36.330512+0000","last_became_active":"2024-09-10T11:39:34.315331+0000","last_became_peered":"2024-09-10T11:39:34.315331+0000","last_unstale":"2024-09-10T11:39:36.330512+0000","last_undegraded":"2024-09-10T11:39:36.330512+0000","last_fullsized":"2024-09-10T11:39:36.330512+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:09.743526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457331+0000","last_change":"2024-09-10T11:39:35.332647+0000","last_active":"2024-09-10T11:39:36.457331+0000","last_peered":"2024-09-10T11:39:36.457331+0000","last_clean":"2024-09-10T11:39:36.457331+0000","last_became_active":"2024-09-10T11:39:35.331482+0000","last_became_peered":"2024-09-10T11:39:35.331482+0000","last_unstale":"2024-09-10T11:39:36.457331+0000","last_undegraded":"2024-09-10T11:39:36.457331+0000","last_fullsized":"2024-09-10T11:39:36.457331+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:15:27.038068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"182'2","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642616+0000","last_change":"2024-09-10T11:39:34.315221+0000","last_active":"2024-09-10T11:39:36.642616+0000","last_peered":"2024-09-10T11:39:36.642616+0000","last_clean":"2024-09-10T11:39:36.642616+0000","last_became_active":"2024-09-10T11:39:34.314972+0000","last_became_peered":"2024-09-10T11:39:34.314972+0000","last_unstale":"2024-09-10T11:39:36.642616+0000","last_undegraded":"2024-09-10T11:39:36.642616+0000","last_fullsized":"2024-09-10T11:39:36.642616+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:36:04.755154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318884+0000","last_change":"2024-09-10T11:39:35.324040+0000","last_active":"2024-09-10T11:39:36.318884+0000","last_peered":"2024-09-10T11:39:36.318884+0000","last_clean":"2024-09-10T11:39:36.318884+0000","last_became_active":"2024-09-10T11:39:35.323785+0000","last_became_peered":"2024-09-10T11:39:35.323785+0000","last_unstale":"2024-09-10T11:39:36.318884+0000","last_undegraded":"2024-09-10T11:39:36.318884+0000","last_fullsized":"2024-09-10T11:39:36.318884+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:16.165735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"183'1","reported_seq":15,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.331115+0000","last_change":"2024-09-10T11:39:34.314834+0000","last_active":"2024-09-10T11:39:36.331115+0000","last_peered":"2024-09-10T11:39:36.331115+0000","last_clean":"2024-09-10T11:39:36.331115+0000","last_became_active":"2024-09-10T11:39:34.314505+0000","last_became_peered":"2024-09-10T11:39:34.314505+0000","last_unstale":"2024-09-10T11:39:36.331115+0000","last_undegraded":"2024-09-10T11:39:36.331115+0000","last_fullsized":"2024-09-10T11:39:36.331115+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:27.125417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318366+0000","last_change":"2024-09-10T11:39:35.327114+0000","last_active":"2024-09-10T11:39:36.318366+0000","last_peered":"2024-09-10T11:39:36.318366+0000","last_clean":"2024-09-10T11:39:36.318366+0000","last_became_active":"2024-09-10T11:39:35.326903+0000","last_became_peered":"2024-09-10T11:39:35.326903+0000","last_unstale":"2024-09-10T11:39:36.318366+0000","last_undegraded":"2024-09-10T11:39:36.318366+0000","last_fullsized":"2024-09-10T11:39:36.318366+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.453558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317794+0000","last_change":"2024-09-10T11:39:34.308221+0000","last_active":"2024-09-10T11:39:36.317794+0000","last_peered":"2024-09-10T11:39:36.317794+0000","last_clean":"2024-09-10T11:39:36.317794+0000","last_became_active":"2024-09-10T11:39:34.308015+0000","last_became_peered":"2024-09-10T11:39:34.308015+0000","last_unstale":"2024-09-10T11:39:36.317794+0000","last_undegraded":"2024-09-10T11:39:36.317794+0000","last_fullsized":"2024-09-10T11:39:36.317794+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:02.801280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.319007+0000","last_change":"2024-09-10T11:39:35.326302+0000","last_active":"2024-09-10T11:39:36.319007+0000","last_peered":"2024-09-10T11:39:36.319007+0000","last_clean":"2024-09-10T11:39:36.319007+0000","last_became_active":"2024-09-10T11:39:35.325997+0000","last_became_peered":"2024-09-10T11:39:35.325997+0000","last_unstale":"2024-09-10T11:39:36.319007+0000","last_undegraded":"2024-09-10T11:39:36.319007+0000","last_fullsized":"2024-09-10T11:39:36.319007+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:47.356895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.457195+0000","last_change":"2024-09-10T11:39:34.312217+0000","last_active":"2024-09-10T11:39:36.457195+0000","last_peered":"2024-09-10T11:39:36.457195+0000","last_clean":"2024-09-10T11:39:36.457195+0000","last_became_active":"2024-09-10T11:39:34.311981+0000","last_became_peered":"2024-09-10T11:39:34.311981+0000","last_unstale":"2024-09-10T11:39:36.457195+0000","last_undegraded":"2024-09-10T11:39:36.457195+0000","last_fullsized":"2024-09-10T11:39:36.457195+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:00.570537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642388+0000","last_change":"2024-09-10T11:39:35.329947+0000","last_active":"2024-09-10T11:39:36.642388+0000","last_peered":"2024-09-10T11:39:36.642388+0000","last_clean":"2024-09-10T11:39:36.642388+0000","last_became_active":"2024-09-10T11:39:35.329700+0000","last_became_peered":"2024-09-10T11:39:35.329700+0000","last_unstale":"2024-09-10T11:39:36.642388+0000","last_undegraded":"2024-09-10T11:39:36.642388+0000","last_fullsized":"2024-09-10T11:39:36.642388+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:05.526617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642846+0000","last_change":"2024-09-10T11:39:34.305915+0000","last_active":"2024-09-10T11:39:36.642846+0000","last_peered":"2024-09-10T11:39:36.642846+0000","last_clean":"2024-09-10T11:39:36.642846+0000","last_became_active":"2024-09-10T11:39:34.305646+0000","last_became_peered":"2024-09-10T11:39:34.305646+0000","last_unstale":"2024-09-10T11:39:36.642846+0000","last_undegraded":"2024-09-10T11:39:36.642846+0000","last_fullsized":"2024-09-10T11:39:36.642846+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:30:00.749126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318025+0000","last_change":"2024-09-10T11:39:35.324161+0000","last_active":"2024-09-10T11:39:36.318025+0000","last_peered":"2024-09-10T11:39:36.318025+0000","last_clean":"2024-09-10T11:39:36.318025+0000","last_became_active":"2024-09-10T11:39:35.323918+0000","last_became_peered":"2024-09-10T11:39:35.323918+0000","last_unstale":"2024-09-10T11:39:36.318025+0000","last_undegraded":"2024-09-10T11:39:36.318025+0000","last_fullsized":"2024-09-10T11:39:36.318025+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:41.948128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317746+0000","last_change":"2024-09-10T11:39:34.311986+0000","last_active":"2024-09-10T11:39:36.317746+0000","last_peered":"2024-09-10T11:39:36.317746+0000","last_clean":"2024-09-10T11:39:36.317746+0000","last_became_active":"2024-09-10T11:39:34.311730+0000","last_became_peered":"2024-09-10T11:39:34.311730+0000","last_unstale":"2024-09-10T11:39:36.317746+0000","last_undegraded":"2024-09-10T11:39:36.317746+0000","last_fullsized":"2024-09-10T11:39:36.317746+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:12.149021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317355+0000","last_change":"2024-09-10T11:39:35.320672+0000","last_active":"2024-09-10T11:39:36.317355+0000","last_peered":"2024-09-10T11:39:36.317355+0000","last_clean":"2024-09-10T11:39:36.317355+0000","last_became_active":"2024-09-10T11:39:35.319098+0000","last_became_peered":"2024-09-10T11:39:35.319098+0000","last_unstale":"2024-09-10T11:39:36.317355+0000","last_undegraded":"2024-09-10T11:39:36.317355+0000","last_fullsized":"2024-09-10T11:39:36.317355+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:24.454593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.318067+0000","last_change":"2024-09-10T11:39:34.311954+0000","last_active":"2024-09-10T11:39:36.318067+0000","last_peered":"2024-09-10T11:39:36.318067+0000","last_clean":"2024-09-10T11:39:36.318067+0000","last_became_active":"2024-09-10T11:39:34.309642+0000","last_became_peered":"2024-09-10T11:39:34.309642+0000","last_unstale":"2024-09-10T11:39:36.318067+0000","last_undegraded":"2024-09-10T11:39:36.318067+0000","last_fullsized":"2024-09-10T11:39:36.318067+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:49.522881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317891+0000","last_change":"2024-09-10T11:39:35.325320+0000","last_active":"2024-09-10T11:39:36.317891+0000","last_peered":"2024-09-10T11:39:36.317891+0000","last_clean":"2024-09-10T11:39:36.317891+0000","last_became_active":"2024-09-10T11:39:35.325092+0000","last_became_peered":"2024-09-10T11:39:35.325092+0000","last_unstale":"2024-09-10T11:39:36.317891+0000","last_undegraded":"2024-09-10T11:39:36.317891+0000","last_fullsized":"2024-09-10T11:39:36.317891+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:56.346702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317857+0000","last_change":"2024-09-10T11:39:34.310616+0000","last_active":"2024-09-10T11:39:36.317857+0000","last_peered":"2024-09-10T11:39:36.317857+0000","last_clean":"2024-09-10T11:39:36.317857+0000","last_became_active":"2024-09-10T11:39:34.310351+0000","last_became_peered":"2024-09-10T11:39:34.310351+0000","last_unstale":"2024-09-10T11:39:36.317857+0000","last_undegraded":"2024-09-10T11:39:36.317857+0000","last_fullsized":"2024-09-10T11:39:36.317857+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:14:50.649898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.317447+0000","last_change":"2024-09-10T11:39:35.325653+0000","last_active":"2024-09-10T11:39:36.317447+0000","last_peered":"2024-09-10T11:39:36.317447+0000","last_clean":"2024-09-10T11:39:36.317447+0000","last_became_active":"2024-09-10T11:39:35.325248+0000","last_became_peered":"2024-09-10T11:39:35.325248+0000","last_unstale":"2024-09-10T11:39:36.317447+0000","last_undegraded":"2024-09-10T11:39:36.317447+0000","last_fullsized":"2024-09-10T11:39:36.317447+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:34.297539+0000","last_clean_scrub_stamp":"2024-09-10T11:39:34.297539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:32:52.530151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642367+0000","last_change":"2024-09-10T11:39:34.307130+0000","last_active":"2024-09-10T11:39:36.642367+0000","last_peered":"2024-09-10T11:39:36.642367+0000","last_clean":"2024-09-10T11:39:36.642367+0000","last_became_active":"2024-09-10T11:39:34.306881+0000","last_became_peered":"2024-09-10T11:39:34.306881+0000","last_unstale":"2024-09-10T11:39:36.642367+0000","last_undegraded":"2024-09-10T11:39:36.642367+0000","last_fullsized":"2024-09-10T11:39:36.642367+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:39:33.290851+0000","last_clean_scrub_stamp":"2024-09-10T11:39:33.290851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:12:59.755049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":497,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-10T11:39:36.642668+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:39:36.642668+0000","last_peered":"2024-09-10T11:39:36.642668+0000","last_clean":"2024-09-10T11:39:36.642668+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:39:36.642668+0000","last_undegraded":"2024-09-10T11:39:36.642668+0000","last_fullsized":"2024-09-10T11:39:36.642668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574962,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":995,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980124,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980090,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1573,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193739,"statfs":{"total":200000000,"available":198389084,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1002,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194310,"statfs":{"total":200000000,"available":198973790,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1003,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194308,"statfs":{"total":200000000,"available":198972190,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1002,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194309,"statfs":{"total":200000000,"available":198973242,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1574,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193738,"statfs":{"total":200000000,"available":198387962,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198980054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:39:38.991 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:39:38.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T11:39:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.133+0000 7f66bd186640 1 Processor -- start 2024-09-10T11:39:39.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.133+0000 7f66bd186640 1 -- start start 2024-09-10T11:39:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66bd186640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66b806ece0 0x7f66b8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66bd186640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b8152e60 con 0x7f66b80d1200 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66bd186640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b80d15e0 con 0x7f66b815c060 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66bd186640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f66b8072680 con 0x7f66b806ece0 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66b6d9d640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f66b80d1200 0x7f66b806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40216/0 (socket says 172.21.15.99:40216) 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66b6d9d640 1 -- 172.21.15.99:0/3918711985 learned_addr learned my addr 172.21.15.99:0/3918711985 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66b659c640 1 --2- 172.21.15.99:0/3918711985 >> v2:172.21.15.196:3300/0 conn(0x7f66b806ece0 0x7f66b8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.134+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453170440 0 0) 0x7f66b8152e60 con 0x7f66b80d1200 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a0003520 con 0x7f66b80d1200 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407063600 0 0) 0x7f66b0001110 con 0x7f66b815c060 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66b8152e60 con 0x7f66b815c060 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b659c640 1 -- 172.21.15.99:0/3918711985 >> v1:172.21.15.196:6789/0 conn(0x7f66b815c060 legacy=0x7f66b815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 125810593 0 0) 0x7f66a0003520 con 0x7f66b80d1200 2024-09-10T11:39:39.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b659c640 1 -- 172.21.15.99:0/3918711985 >> v1:172.21.15.99:6789/0 conn(0x7f66b80d1200 legacy=0x7f66b806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.135+0000 7f66b659c640 1 -- 172.21.15.99:0/3918711985 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b8072960 con 0x7f66b806ece0 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66b659c640 1 --2- 172.21.15.99:0/3918711985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b806ece0 0x7f66b8073440 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0097a0 tx=0x7f66ac02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3cfb6cb84986314 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac037070 con 0x7f66b806ece0 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac02abc0 con 0x7f66b806ece0 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66b5d9b640 1 -- 172.21.15.99:0/3918711985 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a80019f0 con 0x7f66b806ece0 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 -- 172.21.15.99:0/3918711985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b806ece0 msgr2=0x7f66b8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 --2- 172.21.15.99:0/3918711985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b806ece0 0x7f66b8073440 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0097a0 tx=0x7f66ac02a130 comp rx=0 tx=0).stop 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 -- 172.21.15.99:0/3918711985 shutdown_connections 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 --2- 172.21.15.99:0/3918711985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b806ece0 0x7f66b8073440 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 -- 172.21.15.99:0/3918711985 >> 172.21.15.99:0/3918711985 conn(0x7f66b806bc10 msgr2=0x7f66b806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:39.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 -- 172.21.15.99:0/3918711985 shutdown_connections 2024-09-10T11:39:39.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.136+0000 7f66bd186640 1 -- 172.21.15.99:0/3918711985 wait complete. 2024-09-10T11:39:39.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 Processor -- start 2024-09-10T11:39:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 -- start start 2024-09-10T11:39:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b806ece0 0x7f66b8147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b815c060 0x7f66b8157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66b815cd90 con 0x7f66b80d1200 2024-09-10T11:39:39.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66b8072f20 con 0x7f66b815c060 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66bd186640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f66b8159b50 con 0x7f66b806ece0 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.138+0000 7f66b659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58462/0 (socket says 172.21.15.99:58462) 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 -- 172.21.15.99:0/3707042027 learned_addr learned my addr 172.21.15.99:0/3707042027 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b6d9d640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b806ece0 0x7f66b8147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b759e640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b815c060 0x7f66b8157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 -- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b806ece0 msgr2=0x7f66b8147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b806ece0 0x7f66b8147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 -- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b815c060 msgr2=0x7f66b8157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b815c060 0x7f66b8157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 -- 172.21.15.99:0/3707042027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b8148810 con 0x7f66b80d1200 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66b659c640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f66ac035040 tx=0x7f66ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac041020 con 0x7f66b80d1200 2024-09-10T11:39:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac03d620 con 0x7f66b80d1200 2024-09-10T11:39:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b8147e40 con 0x7f66b80d1200 2024-09-10T11:39:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac037040 con 0x7f66b80d1200 2024-09-10T11:39:39.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.139+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b81585f0 con 0x7f66b80d1200 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.140+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b80d0d80 con 0x7f66b80d1200 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.141+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66ac044020 con 0x7f66b80d1200 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.141+0000 7f669f7fe640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 0x7f66a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.141+0000 7f66b6d9d640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 0x7f66a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.141+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f66ac0ab3f0 con 0x7f66b80d1200 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.142+0000 7f66b6d9d640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 0x7f66a406e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f66a8001560 tx=0x7f66a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.145+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66b80d0d80 con 0x7f66b80d1200 2024-09-10T11:39:39.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.373+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f66b8073440 con 0x7f66b80d1200 2024-09-10T11:39:39.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.375+0000 7f669f7fe640 1 -- 172.21.15.99:0/3707042027 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f66ac04b030 con 0x7f66b80d1200 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQC4LeBm2QdYHhAAMHdDcvJnufKPs8FiWhGN6g== 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: caps mds = "allow" 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: caps mon = "allow r" 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T11:39:39.376 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 msgr2=0x7f66a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 0x7f66a406e2f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f66a8001560 tx=0x7f66a8009040 comp rx=0 tx=0).stop 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 msgr2=0x7f66b81578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f66ac035040 tx=0x7f66ac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 shutdown_connections 2024-09-10T11:39:39.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f66a406be30 0x7f66a406e2f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66b815c060 0x7f66b8157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66b80d1200 0x7f66b81578b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 --2- 172.21.15.99:0/3707042027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66b806ece0 0x7f66b8147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 >> 172.21.15.99:0/3707042027 conn(0x7f66b806bc10 msgr2=0x7f66b8169f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 shutdown_connections 2024-09-10T11:39:39.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.377+0000 7f66bd186640 1 -- 172.21.15.99:0/3707042027 wait complete. 2024-09-10T11:39:39.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.487+0000 7f5eed80a640 1 Processor -- start 2024-09-10T11:39:39.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.487+0000 7f5eed80a640 1 -- start start 2024-09-10T11:39:39.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5eed80a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5ee815d5c0 0x7f5ee815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5eed80a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ee806c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5eed80a640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee8070b70 con 0x7f5ee8161160 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5eed80a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee8160cd0 con 0x7f5ee815d5c0 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5eed80a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee806c580 con 0x7f5ee806bc10 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee67fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5ee8161160 0x7f5ee8070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40232/0 (socket says 172.21.15.99:40232) 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee67fc640 1 -- 172.21.15.99:0/2781990914 learned_addr learned my addr 172.21.15.99:0/2781990914 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ee806c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/2781990914 >> v2:172.21.15.196:3300/0 conn(0x7f5ee815d5c0 0x7f5ee815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1330432996 0 0) 0x7f5ee8070b70 con 0x7f5ee8161160 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.488+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ed4003520 con 0x7f5ee8161160 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1033551558 0 0) 0x7f5ed4003520 con 0x7f5ee8161160 2024-09-10T11:39:39.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 msgr2=0x7f5ee806c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ee806c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 >> v2:172.21.15.196:3300/0 conn(0x7f5ee815d5c0 msgr2=0x7f5ee815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 --2- 172.21.15.99:0/2781990914 >> v2:172.21.15.196:3300/0 conn(0x7f5ee815d5c0 0x7f5ee815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee8070e00 con 0x7f5ee8161160 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f5ed0001670 con 0x7f5ee8161160 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 >> v1:172.21.15.99:6789/0 conn(0x7f5ee8161160 legacy=0x7f5ee8070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ed40039b0 0x7f5ed4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ed40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8160cd0 con 0x7f5ed40039b0 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee806c580 con 0x7f5ed4004800 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed400b9c0 con 0x7f5ee806bc10 2024-09-10T11:39:39.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ed40039b0 0x7f5ed4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee67fc640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ed40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 -- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 msgr2=0x7f5ed40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ed40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 -- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ed4004800 msgr2=0x7f5ed4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ed4004800 0x7f5ed4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee6ffd640 1 -- 172.21.15.99:0/2781990914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed4003520 con 0x7f5ed40039b0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.489+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ee806bc10 0x7f5ed40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ed40039b0 0x7f5ed4003dd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009fc0 tx=0x7f5ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5096a86d63be116a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8009c10 con 0x7f5ed40039b0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8036070 con 0x7f5ed40039b0 2024-09-10T11:39:39.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5ee5ffb640 1 -- 172.21.15.99:0/2781990914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed802d070 con 0x7f5ed40039b0 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 -- 172.21.15.99:0/2781990914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ed40039b0 msgr2=0x7f5ed4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 --2- 172.21.15.99:0/2781990914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ed40039b0 0x7f5ed4003dd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009fc0 tx=0x7f5ed4005830 comp rx=0 tx=0).stop 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5ee77fe640 1 -- 172.21.15.99:0/2781990914 reap_dead start 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 -- 172.21.15.99:0/2781990914 shutdown_connections 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 -- 172.21.15.99:0/2781990914 >> 172.21.15.99:0/2781990914 conn(0x7f5ee80d1900 msgr2=0x7f5ee80d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 -- 172.21.15.99:0/2781990914 shutdown_connections 2024-09-10T11:39:39.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.490+0000 7f5eed80a640 1 -- 172.21.15.99:0/2781990914 wait complete. 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 Processor -- start 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 -- start start 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ed40039b0 0x7f5ee81584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 0x7f5ee80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee806c040 con 0x7f5ee814cd20 2024-09-10T11:39:39.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8160960 con 0x7f5ee80cbd20 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.492+0000 7f5eed80a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee806c1c0 con 0x7f5ed40039b0 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58486/0 (socket says 172.21.15.99:58486) 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 -- 172.21.15.99:0/774321943 learned_addr learned my addr 172.21.15.99:0/774321943 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee67fc640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 0x7f5ee80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ed40039b0 0x7f5ee81584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 -- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ed40039b0 msgr2=0x7f5ee81584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ed40039b0 0x7f5ee81584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 -- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 msgr2=0x7f5ee80cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 0x7f5ee80cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 -- 172.21.15.99:0/774321943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee80c9e80 con 0x7f5ee814cd20 2024-09-10T11:39:39.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee67fc640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 0x7f5ee80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5ee77fe640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806dc20 tx=0x7f5ee0011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004440 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee001b070 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee80ca160 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0003850 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.493+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee80ca720 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.494+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee815d9a0 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.495+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ee00039f0 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.495+0000 7f5edffff640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 0x7f5ec806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.495+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 0x7f5ec806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.496+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f5ee008ae70 con 0x7f5ee814cd20 2024-09-10T11:39:39.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.496+0000 7f5ee6ffd640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 0x7f5ec806e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8035000 tx=0x7f5ed802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:39.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.500+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ee0025020 con 0x7f5ee814cd20 2024-09-10T11:39:39.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.698+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ee806fd10 con 0x7f5ee814cd20 2024-09-10T11:39:39.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.699+0000 7f5edffff640 1 -- 172.21.15.99:0/774321943 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f5ee0059790 con 0x7f5ee814cd20 2024-09-10T11:39:39.700 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:39.700 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:39:39.700 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 msgr2=0x7f5ec806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 0x7f5ec806e320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8035000 tx=0x7f5ed802f000 comp rx=0 tx=0).stop 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 msgr2=0x7f5ee814d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806dc20 tx=0x7f5ee0011860 comp rx=0 tx=0).stop 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 shutdown_connections 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5ec806be60 0x7f5ec806e320 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ee814cd20 0x7f5ee814d180 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ee80cbd20 0x7f5ee80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 --2- 172.21.15.99:0/774321943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ed40039b0 0x7f5ee81584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:39.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 >> 172.21.15.99:0/774321943 conn(0x7f5ee80d1900 msgr2=0x7f5ee815e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:39.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 shutdown_connections 2024-09-10T11:39:39.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:39.701+0000 7f5eed80a640 1 -- 172.21.15.99:0/774321943 wait complete. 2024-09-10T11:39:40.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T11:39:34.642019+0000', 'modified': '2024-09-10T11:39:37.051865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7956}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7956': {'gid': 7956, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6837/2972885632', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6835', 'nonce': 2972885632}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 2972885632}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7956, 'qdb_cluster': [7956]} 2024-09-10T11:39:40.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T11:39:40.116 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T11:39:40.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.217+0000 7f61e425e640 1 Processor -- start 2024-09-10T11:39:40.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.217+0000 7f61e425e640 1 -- start start 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e425e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61dc15ee00 0x7f61dc0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e425e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61dc0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e425e640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61dc0b40e0 con 0x7f61dc0b0090 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e425e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f61dc0b5ab0 con 0x7f61dc15ee00 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e425e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc073c50 con 0x7f61dc15f1e0 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e27fb640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f61dc0b0090 0x7f61dc0b1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40240/0 (socket says 172.21.15.99:40240) 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.218+0000 7f61e27fb640 1 -- 172.21.15.99:0/772466473 learned_addr learned my addr 172.21.15.99:0/772466473 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:40.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463460085 0 0) 0x7f61dc0b40e0 con 0x7f61dc0b0090 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e17f9640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61dc0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c8003520 con 0x7f61dc0b0090 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/772466473 >> v2:172.21.15.196:3300/0 conn(0x7f61dc15ee00 0x7f61dc0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1140988132 0 0) 0x7f61c8003520 con 0x7f61dc0b0090 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 msgr2=0x7f61dc0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61dc0b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 >> v2:172.21.15.196:3300/0 conn(0x7f61dc15ee00 msgr2=0x7f61dc0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 --2- 172.21.15.99:0/772466473 >> v2:172.21.15.196:3300/0 conn(0x7f61dc15ee00 0x7f61dc0b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc0b4380 con 0x7f61dc0b0090 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f61d4001670 con 0x7f61dc0b0090 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 >> v1:172.21.15.99:6789/0 conn(0x7f61dc0b0090 legacy=0x7f61dc0b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c8004d40 0x7f61c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b5ab0 con 0x7f61c80039b0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc073c50 con 0x7f61c8004d40 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61c800b9c0 con 0x7f61dc15f1e0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e17f9640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e27fb640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c8004d40 0x7f61c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 -- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 msgr2=0x7f61c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc15f1e0 0x7f61c8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 -- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c8004d40 msgr2=0x7f61c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c8004d40 0x7f61c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.219+0000 7f61e1ffa640 1 -- 172.21.15.99:0/772466473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c8003520 con 0x7f61c80039b0 2024-09-10T11:39:40.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.220+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61c8003dd0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e170 tx=0x7f61c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290fd58c78db4d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.220+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8034020 con 0x7f61c80039b0 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.220+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8030430 con 0x7f61c80039b0 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.220+0000 7f61e0ff8640 1 -- 172.21.15.99:0/772466473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8009c10 con 0x7f61c80039b0 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 -- 172.21.15.99:0/772466473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 msgr2=0x7f61c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 --2- 172.21.15.99:0/772466473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61c8003dd0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e170 tx=0x7f61c8005830 comp rx=0 tx=0).stop 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e27fb640 1 -- 172.21.15.99:0/772466473 reap_dead start 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 -- 172.21.15.99:0/772466473 shutdown_connections 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 -- 172.21.15.99:0/772466473 >> 172.21.15.99:0/772466473 conn(0x7f61dc0730c0 msgr2=0x7f61dc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 -- 172.21.15.99:0/772466473 shutdown_connections 2024-09-10T11:39:40.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.221+0000 7f61e425e640 1 -- 172.21.15.99:0/772466473 wait complete. 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.222+0000 7f61e425e640 1 Processor -- start 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.222+0000 7f61e425e640 1 -- start start 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 0x7f61dc14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc06ec60 0x7f61dc14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0b21b0 con 0x7f61c80039b0 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc0afd60 con 0x7f61dc06e800 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc0b0600 con 0x7f61dc06ec60 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58504/0 (socket says 172.21.15.99:58504) 2024-09-10T11:39:40.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 -- 172.21.15.99:0/3661062970 learned_addr learned my addr 172.21.15.99:0/3661062970 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e27fb640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc06ec60 0x7f61dc14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 0x7f61dc14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 -- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc06ec60 msgr2=0x7f61dc14d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc06ec60 0x7f61dc14d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 -- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 msgr2=0x7f61dc14cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 0x7f61dc14cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 -- 172.21.15.99:0/3661062970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc0ca4f0 con 0x7f61c80039b0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 0x7f61dc14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e290 tx=0x7f61d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61d27fc640 1 -- 172.21.15.99:0/3661062970 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8040070 con 0x7f61c80039b0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61d27fc640 1 -- 172.21.15.99:0/3661062970 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d8003990 con 0x7f61c80039b0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc0ca7d0 con 0x7f61c80039b0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61d27fc640 1 -- 172.21.15.99:0/3661062970 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d8034040 con 0x7f61c80039b0 2024-09-10T11:39:40.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.223+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc0cad90 con 0x7f61c80039b0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.225+0000 7f61d27fc640 1 -- 172.21.15.99:0/3661062970 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61d8037020 con 0x7f61c80039b0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.226+0000 7f61d27fc640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 0x7f61c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.226+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 0x7f61c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.226+0000 7f61d27fc640 1 -- 172.21.15.99:0/3661062970 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f61d80ab750 con 0x7f61c80039b0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.226+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 0x7f61c006e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cd4a0 tx=0x7f61cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 Processor -- start 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 -- start start 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57402a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f573b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559f5731f520 con 0x559f573baa80 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f572c73e0 con 0x559f573bd2c0 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.227+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f5731b4e0 con 0x559f573bea50 2024-09-10T11:39:40.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58514/0 (socket says 172.21.15.99:58514) 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 -- 172.21.15.99:0/720562030 learned_addr learned my addr 172.21.15.99:0/720562030 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e27fb640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f573b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57402a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 -- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 msgr2=0x559f573b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f573b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 -- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 msgr2=0x559f57402a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57402a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 -- 172.21.15.99:0/720562030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f572e22f0 con 0x559f573baa80 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57402a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61e17f9640 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f61cc013e00 tx=0x7f61cc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf1273724c9dd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61bf7fe640 1 -- 172.21.15.99:0/720562030 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc023440 con 0x559f573baa80 2024-09-10T11:39:40.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61bf7fe640 1 -- 172.21.15.99:0/720562030 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc028070 con 0x559f573baa80 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.228+0000 7f61bf7fe640 1 -- 172.21.15.99:0/720562030 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0237e0 con 0x559f573baa80 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- 172.21.15.99:0/720562030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 msgr2=0x559f573bd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f61cc013e00 tx=0x7f61cc016000 comp rx=0 tx=0).stop 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- 172.21.15.99:0/720562030 shutdown_connections 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f573b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573baa80 0x559f573bd6a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/720562030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57402a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- 172.21.15.99:0/720562030 >> 172.21.15.99:0/720562030 conn(0x559f573b9a80 msgr2=0x559f57401f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- 172.21.15.99:0/720562030 shutdown_connections 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- 172.21.15.99:0/720562030 wait complete. 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 -- start start 2024-09-10T11:39:40.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.229+0000 7f61e6ffd740 1 Processor -- start 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f573b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573dd9b0 0x559f573b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559f573b8f20 con 0x559f573bea50 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f572a8450 con 0x559f573bd2c0 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f57403290 con 0x559f573dd9b0 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e6ffd740 10 client.0 _ll_register_callbacks cb 0x559f57031ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:39:40.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e17f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e17f9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58518/0 (socket says 172.21.15.99:58518) 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.230+0000 7f61e17f9640 1 -- 172.21.15.99:0/657184263 learned_addr learned my addr 172.21.15.99:0/657184263 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e27fb640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573dd9b0 0x559f573b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f573b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 -- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573dd9b0 msgr2=0x559f573b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573dd9b0 0x559f573b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 -- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 msgr2=0x559f573b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f573b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 -- 172.21.15.99:0/657184263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f570322d0 con 0x559f573bea50 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e17f9640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f61cc025040 tx=0x7f61cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d72c7503f2e3bd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc03a060 con 0x559f573bea50 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f61cc023440 con 0x559f573bea50 2024-09-10T11:39:40.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc028070 con 0x559f573bea50 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f61cc040050 con 0x559f573bea50 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 10 client.8064 fetch_fsmap learned FSMap version 74 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x559f570fc6b0 con 0x559f573bea50 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61cc028070 con 0x559f573bea50 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f61cc043020 con 0x559f573bea50 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 10 client.8064 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 10 client.8064 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.231+0000 7f61e6ffd740 20 client.8064 populate_metadata read hostname 'smithi099' 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f573e1060 0x559f573e1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61e6ffd740 4 client.8064 mds_command: new command op to 7956 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 --> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f5732cc50 con 0x559f573e1060 2024-09-10T11:39:40.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f573e1060 0x559f573e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61bcff9640 10 client.8064 ms_handle_connect on v2:172.21.15.99:6835/2972885632 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.232+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f573e1060 0x559f573e1440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479b1f6e13edafd8 server_cookie=13428d589f9511a in_seq=0 out_seq=0 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61bcff9640 1 -- 172.21.15.99:0/657184263 <== mds.0 v2:172.21.15.99:6835/2972885632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f5732cc50 con 0x559f573e1060 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61bcff9640 10 client.8064 handle_command_reply: tid=0 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 client.8064 shutdown 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 msgr2=0x559f573dc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f61cc025040 tx=0x7f61cc004730 comp rx=0 tx=0).stop 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 shutdown_connections 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f573e1060 0x559f573e1440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573dd9b0 0x559f573b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61bcff9640 0 client.8064 ms_handle_reset on v2:172.21.15.99:6835/2972885632 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573bea50 0x559f573dc4e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/657184263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f573b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 >> 172.21.15.99:0/657184263 conn(0x559f573baa80 msgr2=0x559f573bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 shutdown_connections 2024-09-10T11:39:40.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 1 -- 172.21.15.99:0/657184263 wait complete. 2024-09-10T11:39:40.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.233+0000 7f61e6ffd740 20 client.8064 trim_cache size 0 max 16384 2024-09-10T11:39:40.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 Processor -- start 2024-09-10T11:39:40.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 -- start start 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bd2c0 0x559f573e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 0x559f573dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc2009f0 con 0x559f573dd9b0 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc14ee80 con 0x559f573bea50 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f61dc2003a0 con 0x559f573bd2c0 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e27fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e27fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58528/0 (socket says 172.21.15.99:58528) 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.251+0000 7f61e27fb640 1 -- 172.21.15.99:0/3025424033 learned_addr learned my addr 172.21.15.99:0/3025424033 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bd2c0 0x559f573e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 -- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bd2c0 msgr2=0x559f573e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bd2c0 0x559f573e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 -- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 msgr2=0x559f573dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 0x559f573dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 0x559f573dbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 -- 172.21.15.99:0/3025424033 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f573b2470 con 0x559f573dd9b0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e17f9640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 0x559f573dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e27fb640 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0721a0 tx=0x7f61d4001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8ffea97af22289 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61beffd640 1 -- 172.21.15.99:0/3025424033 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4016da0 con 0x559f573dd9b0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61beffd640 1 -- 172.21.15.99:0/3025424033 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d401b020 con 0x559f573dd9b0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61beffd640 1 -- 172.21.15.99:0/3025424033 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017430 con 0x559f573dd9b0 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 -- 172.21.15.99:0/3025424033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 msgr2=0x559f573dc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0721a0 tx=0x7f61d4001cc0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 -- 172.21.15.99:0/3025424033 shutdown_connections 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dc3d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bea50 0x559f573dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/3025424033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bd2c0 0x559f573e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.252+0000 7f61e6ffd740 1 -- 172.21.15.99:0/3025424033 >> 172.21.15.99:0/3025424033 conn(0x559f573baa80 msgr2=0x559f57400560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.253+0000 7f61e6ffd740 1 -- 172.21.15.99:0/3025424033 shutdown_connections 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.253+0000 7f61e6ffd740 1 -- 172.21.15.99:0/3025424033 wait complete. 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.253+0000 7f61e6ffd740 1 -- start start 2024-09-10T11:39:40.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.253+0000 7f61e6ffd740 1 Processor -- start 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57480be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f57465da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559f573dded0 con 0x559f573dd9b0 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559f573b8f20 con 0x559f573bd2c0 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x559f573dcb10 con 0x559f573bea50 2024-09-10T11:39:40.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e6ffd740 10 client.0 _ll_register_callbacks cb 0x7f61d4001670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58530/0 (socket says 172.21.15.99:58530) 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 -- 172.21.15.99:0/2500310866 learned_addr learned my addr 172.21.15.99:0/2500310866 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57480be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e17f9640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f57465da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 -- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 msgr2=0x559f57465da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f57465da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 -- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 msgr2=0x559f57480be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57480be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e27fb640 1 -- 172.21.15.99:0/2500310866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f573e26d0 con 0x559f573dd9b0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.254+0000 7f61e17f9640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f57465da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e27fb640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f61dc071c60 tx=0x7f61d4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae3a76e447e5bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017430 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f61d40066e0 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d401b020 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f61d40068e0 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 10 client.8070 fetch_fsmap learned FSMap version 74 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x559f57466370 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d401b020 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f61d4006080 con 0x559f573dd9b0 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 10 client.8070 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 20 client.8070 populate_metadata read hostname 'smithi099' 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f57483d80 0x559f573dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 4 client.8070 mds_command: new command op to 7956 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T11:39:40.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 --> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] -- command(tid 0: {"prefix": "status"}) -- 0x559f57326fa0 con 0x559f57483d80 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.255+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f57483d80 0x559f573dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61bcff9640 10 client.8070 ms_handle_connect on v2:172.21.15.99:6835/2972885632 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e1ffa640 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f57483d80 0x559f573dab60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d643b40394b33fc8 server_cookie=f47e40eb47ba904 in_seq=0 out_seq=0 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61bcff9640 1 -- 172.21.15.99:0/2500310866 <== mds.0 v2:172.21.15.99:6835/2972885632 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x559f57326fa0 con 0x559f57483d80 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61bcff9640 10 client.8070 handle_command_reply: tid=0 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 client.8070 shutdown 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 msgr2=0x559f573dbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f61dc071c60 tx=0x7f61d4001a40 comp rx=0 tx=0).stop 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 shutdown_connections 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:6835/2972885632,v1:172.21.15.99:6837/2972885632] conn(0x559f57483d80 0x559f573dab60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559f573dd9b0 0x559f573dbc70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61bcff9640 0 client.8070 ms_handle_reset on v2:172.21.15.99:6835/2972885632 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559f573bea50 0x559f57465da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 --2- 172.21.15.99:0/2500310866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559f573bd2c0 0x559f57480be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 >> 172.21.15.99:0/2500310866 conn(0x559f573baa80 msgr2=0x559f57403c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 shutdown_connections 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 1 -- 172.21.15.99:0/2500310866 wait complete. 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.256+0000 7f61e6ffd740 20 client.8070 trim_cache size 0 max 16384 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 7956, 2024-09-10T11:39:40.264 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 4.9322730310000003, 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 74, 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 183, 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 183, 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 9.0116849840000004 2024-09-10T11:39:40.265 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 msgr2=0x7f61c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 0x7f61c006e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0cd4a0 tx=0x7f61cc008040 comp rx=0 tx=0).stop 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 msgr2=0x7f61dc156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f61d800e290 tx=0x7f61d8009750 comp rx=0 tx=0).stop 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 shutdown_connections 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f61c006be30 0x7f61c006e2f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61dc06ec60 0x7f61dc14d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61dc06e800 0x7f61dc14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 --2- 172.21.15.99:0/3661062970 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61c80039b0 0x7f61dc156bd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 >> 172.21.15.99:0/3661062970 conn(0x7f61dc0730c0 msgr2=0x7f61dc0af560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 shutdown_connections 2024-09-10T11:39:40.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.257+0000 7f61e425e640 1 -- 172.21.15.99:0/3661062970 wait complete. 2024-09-10T11:39:40.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:39:40.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.366+0000 7f0c03fc4640 1 Processor -- start 2024-09-10T11:39:40.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.367+0000 7f0c03fc4640 1 -- start start 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.367+0000 7f0c03fc4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0bfc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.367+0000 7f0c03fc4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bfc0d52f0 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.367+0000 7f0c03fc4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bfc15ff70 con 0x7f0bfc15eb80 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.367+0000 7f0c03fc4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc0d30c0 con 0x7f0bfc1626c0 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c02561640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0bfc0d4e40 0x7f0bfc0d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40254/0 (socket says 172.21.15.99:40254) 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c02561640 1 -- 172.21.15.99:0/2146844657 learned_addr learned my addr 172.21.15.99:0/2146844657 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:40.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c0155f640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0bfc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211572925 0 0) 0x7f0bf4001490 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be8003520 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3947434043 0 0) 0x7f0be8003520 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 msgr2=0x7f0bfc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0bfc0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 >> v1:172.21.15.196:6789/0 conn(0x7f0bfc15eb80 legacy=0x7f0bfc0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0d1620 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f0bf4001bc0 con 0x7f0bfc0d4e40 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 >> v1:172.21.15.99:6789/0 conn(0x7f0bfc0d4e40 legacy=0x7f0bfc0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0be80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8003f20 0x7f0be8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0d30c0 con 0x7f0be8004d40 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0be800bad0 con 0x7f0be8003f20 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0be800bcd0 con 0x7f0bfc1626c0 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c02561640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c0155f640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8003f20 0x7f0be8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.368+0000 7f0c01d60640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0be80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 -- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 msgr2=0x7f0be80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1626c0 0x7f0be80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 -- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8003f20 msgr2=0x7f0be8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8003f20 0x7f0be8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 -- 172.21.15.99:0/2146844657 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8003520 con 0x7f0be8004d40 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c02561640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400aa70 tx=0x7f0be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2124db6d71a97d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400a690 con 0x7f0be8004d40 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf402f070 con 0x7f0be8004d40 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.369+0000 7f0c00d5e640 1 -- 172.21.15.99:0/2146844657 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4034470 con 0x7f0be8004d40 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 -- 172.21.15.99:0/2146844657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0be8004d40 msgr2=0x7f0be80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/2146844657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400aa70 tx=0x7f0be8005830 comp rx=0 tx=0).stop 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c02561640 1 -- 172.21.15.99:0/2146844657 reap_dead start 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 -- 172.21.15.99:0/2146844657 shutdown_connections 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 -- 172.21.15.99:0/2146844657 >> 172.21.15.99:0/2146844657 conn(0x7f0bfc08e680 msgr2=0x7f0bfc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 -- 172.21.15.99:0/2146844657 shutdown_connections 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.370+0000 7f0c03fc4640 1 -- 172.21.15.99:0/2146844657 wait complete. 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.371+0000 7f0c03fc4640 1 Processor -- start 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.371+0000 7f0c03fc4640 1 -- start start 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 0x7f0bfc0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1e8b60 0x7f0bfc1e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc163180 con 0x7f0bfc161520 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc162ea0 con 0x7f0be8004d40 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c03fc4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc163460 con 0x7f0bfc1e8b60 2024-09-10T11:39:40.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58552/0 (socket says 172.21.15.99:58552) 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 -- 172.21.15.99:0/3148067890 learned_addr learned my addr 172.21.15.99:0/3148067890 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c01d60640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 0x7f0bfc0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c02561640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1e8b60 0x7f0bfc1e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 -- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1e8b60 msgr2=0x7f0bfc1e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1e8b60 0x7f0bfc1e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 -- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 msgr2=0x7f0bfc0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 0x7f0bfc0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 -- 172.21.15.99:0/3148067890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0b2f40 con 0x7f0bfc161520 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c01d60640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 0x7f0bfc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.372+0000 7f0c0155f640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011910 tx=0x7f0bec011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.373+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00faf0 con 0x7f0bfc161520 2024-09-10T11:39:40.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.373+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bec00fc90 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.373+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec014440 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.373+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc0b3220 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.373+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc0b37e0 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.374+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc0d4900 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.374+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0bec012040 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.374+0000 7f0bf27fc640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 0x7f0be006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.375+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f0bec08b480 con 0x7f0bfc161520 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.375+0000 7f0c01d60640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 0x7f0be006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.375+0000 7f0c01d60640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 0x7f0be006e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8003700 tx=0x7f0bf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:40.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.379+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0bec019030 con 0x7f0bfc161520 2024-09-10T11:39:40.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.581+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bfc06c110 con 0x7f0bfc161520 2024-09-10T11:39:40.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.581+0000 7f0bf27fc640 1 -- 172.21.15.99:0/3148067890 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f0bec059ec0 con 0x7f0bfc161520 2024-09-10T11:39:40.582 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 msgr2=0x7f0be006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 0x7f0be006e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8003700 tx=0x7f0bf8009040 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 msgr2=0x7f0bfc15a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011910 tx=0x7f0bec011dd0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 shutdown_connections 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0be006be30 0x7f0be006e2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bfc1e8b60 0x7f0bfc1e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0bfc161520 0x7f0bfc15a100 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 --2- 172.21.15.99:0/3148067890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be8004d40 0x7f0bfc0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 >> 172.21.15.99:0/3148067890 conn(0x7f0bfc08e680 msgr2=0x7f0bfc15cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.583+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 shutdown_connections 2024-09-10T11:39:40.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40.584+0000 7f0c03fc4640 1 -- 172.21.15.99:0/3148067890 wait complete. 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:39:40.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:39:40.596 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:39:40.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:40.596 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-09-10T11:39:40.612 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42268sec preferred_lft 42268sec 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link noprefixroute 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::44fb:e7ff:feb7:eff3/64 scope link 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether da:07:35:cd:35:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::d807:35ff:fecd:35ed/64 scope link 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 36:5d:76:62:77:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::345d:76ff:fe62:77e6/64 scope link 2024-09-10T11:39:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:40.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:40.615 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:39:40.676 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-10T11:39:40.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:40.678 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-09-10T11:39:40.734 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-09-10T11:39:40.734 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:39:40.734 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:39:40.735 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:39:40.735 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:39:40.735 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-09-10T11:39:40.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:39:40.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:39:40.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:40.861 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:39:40.876 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:39:40.876 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:39:40.876 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:39:40.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:40.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-09-10T11:39:40.941 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:39:40.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:39:40.942 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:40.997 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:39:40.997 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:39:40.997 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:41.054 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:39:41.123 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:39:41.184 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:39:41.185 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:39:41.186 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:39:41.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:41.187 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:39:41.241 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:39:41.241 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:39:41.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-09-10T11:39:41.310 DEBUG:teuthology.orchestra.run.smithi099:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:39:41.365 INFO:teuthology.orchestra.run.smithi099.stdout:/proc 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/dev 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/security 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/shm 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/pts 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/run 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/cgroup 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/pstore 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/bpf 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/ 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/selinux 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:39:41.366 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/mqueue 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/tracing 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/debug 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/dev/hugepages 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/fs/fuse/connections 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/sys/kernel/config 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/run/user/1000 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T11:39:41.367 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run.smithi099.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:39:41.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:41.368 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-09-10T11:39:41.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-09-10T11:39:41.365+0000 7f9c90520480 -1 init, newargv = 0x55c67bd80140 newargc=15 2024-09-10T11:39:41.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[63134]: starting ceph client 2024-09-10T11:39:41.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[63134]: starting fuse 2024-09-10T11:39:41.384 INFO:teuthology.orchestra.run.smithi099.stdout:42 2024-09-10T11:39:41.384 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-09-10T11:39:41.385 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-09-10T11:39:41.386 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-09-10T11:39:41.501 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.63134.asok 2024-09-10T11:39:41.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:39:41.512 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.63134.asok 2024-09-10T11:39:41.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:41.512 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.63134.asok status 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "63134", 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-09-10T11:39:41.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 8085, 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 8085 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:39:41.646 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1964546962 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1964546962 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.8085 192.168.0.1:0/1964546962", 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/1964546962", 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 74, 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 183, 2024-09-10T11:39:41.647 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:39:41.648 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-09-10T11:39:41.648 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-09-10T11:39:41.648 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:41.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:41.653 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-09-10T11:39:41.653 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:41.669 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-09-10T11:39:41.669 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:41.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:41.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:39:41.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T11:39:41.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 Processor -- start 2024-09-10T11:39:41.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 -- start start 2024-09-10T11:39:41.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb16415c7b0 0x7fb1640d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1640d0fb0 0x7fb16406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb16406ee30 0x7fb1641604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb1640d1420 con 0x7fb1640d0fb0 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.872+0000 7fb16ab42640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fb16406f280 con 0x7fb16415c7b0 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb16ab42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640b3840 con 0x7fb16406ee30 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1640d0fb0 0x7fb16406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1640d0fb0 0x7fb16406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58568/0 (socket says 172.21.15.99:58568) 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 -- 172.21.15.99:0/1557673925 learned_addr learned my addr 172.21.15.99:0/1557673925 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 -- 172.21.15.99:0/1557673925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb16406ee30 msgr2=0x7fb1641604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 --2- 172.21.15.99:0/1557673925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb16406ee30 0x7fb1641604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 -- 172.21.15.99:0/1557673925 >> v2:172.21.15.196:3300/0 conn(0x7fb16415c7b0 msgr2=0x7fb1640d0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 --2- 172.21.15.99:0/1557673925 >> v2:172.21.15.196:3300/0 conn(0x7fb16415c7b0 0x7fb1640d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.873+0000 7fb163fff640 1 -- 172.21.15.99:0/1557673925 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640ceb10 con 0x7fb1640d0fb0 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb163fff640 1 --2- 172.21.15.99:0/1557673925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1640d0fb0 0x7fb16406e8f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cd7c0 tx=0x7fb15402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c5e6ae1a4280f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:41.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb1637fe640 1 -- 172.21.15.99:0/1557673925 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154036070 con 0x7fb1640d0fb0 2024-09-10T11:39:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb1637fe640 1 -- 172.21.15.99:0/1557673925 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154031600 con 0x7fb1640d0fb0 2024-09-10T11:39:41.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb1637fe640 1 -- 172.21.15.99:0/1557673925 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1540319d0 con 0x7fb1640d0fb0 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 -- 172.21.15.99:0/1557673925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1640d0fb0 msgr2=0x7fb16406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 --2- 172.21.15.99:0/1557673925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1640d0fb0 0x7fb16406e8f0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb1640cd7c0 tx=0x7fb15402a6a0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 -- 172.21.15.99:0/1557673925 shutdown_connections 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 --2- 172.21.15.99:0/1557673925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb16406ee30 0x7fb1641604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 --2- 172.21.15.99:0/1557673925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1640d0fb0 0x7fb16406e8f0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 --2- 172.21.15.99:0/1557673925 >> v2:172.21.15.196:3300/0 conn(0x7fb16415c7b0 0x7fb1640d0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 -- 172.21.15.99:0/1557673925 >> 172.21.15.99:0/1557673925 conn(0x7fb16406bc10 msgr2=0x7fb16406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 -- 172.21.15.99:0/1557673925 shutdown_connections 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.874+0000 7fb16ab42640 1 -- 172.21.15.99:0/1557673925 wait complete. 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.875+0000 7fb16ab42640 1 Processor -- start 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 -- start start 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1640d0fb0 0x7fb16413cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16415c7b0 0x7fb16414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640b4050 con 0x7fb16406ee30 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640b3d70 con 0x7fb16415c7b0 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb16ab42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640b4330 con 0x7fb1640d0fb0 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.876+0000 7fb1688de640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58572/0 (socket says 172.21.15.99:58572) 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 -- 172.21.15.99:0/2041786353 learned_addr learned my addr 172.21.15.99:0/2041786353 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:41.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb163fff640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1640d0fb0 0x7fb16413cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1690df640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16415c7b0 0x7fb16414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 -- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1640d0fb0 msgr2=0x7fb16413cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1640d0fb0 0x7fb16413cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 -- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16415c7b0 msgr2=0x7fb16414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16415c7b0 0x7fb16414e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 -- 172.21.15.99:0/2041786353 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb164140380 con 0x7fb16406ee30 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1688de640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b410 tx=0x7fb15c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c015020 con 0x7fb16406ee30 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c011850 con 0x7fb16406ee30 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c011c40 con 0x7fb16406ee30 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16413d060 con 0x7fb16406ee30 2024-09-10T11:39:41.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.877+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb164150a40 con 0x7fb16406ee30 2024-09-10T11:39:41.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.878+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16415cb90 con 0x7fb16406ee30 2024-09-10T11:39:41.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.879+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb15c018020 con 0x7fb16406ee30 2024-09-10T11:39:41.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.879+0000 7fb1617fa640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 0x7fb14406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:41.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.879+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7fb15c08acd0 con 0x7fb16406ee30 2024-09-10T11:39:41.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.879+0000 7fb163fff640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 0x7fb14406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:41.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.880+0000 7fb163fff640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 0x7fb14406e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb154033000 tx=0x7fb154009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:41.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:41.883+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb15c01f020 con 0x7fb16406ee30 2024-09-10T11:39:42.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.084+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1640b5300 con 0x7fb16406ee30 2024-09-10T11:39:42.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.084+0000 7fb1617fa640 1 -- 172.21.15.99:0/2041786353 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7fb15c058fa0 con 0x7fb16406ee30 2024-09-10T11:39:42.085 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 msgr2=0x7fb14406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 0x7fb14406e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb154033000 tx=0x7fb154009000 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 msgr2=0x7fb164151520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b410 tx=0x7fb15c00b8d0 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 shutdown_connections 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb14406be30 0x7fb14406e2f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb16415c7b0 0x7fb16414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1640d0fb0 0x7fb16413cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 --2- 172.21.15.99:0/2041786353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb16406ee30 0x7fb164151520 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 >> 172.21.15.99:0/2041786353 conn(0x7fb16406bc10 msgr2=0x7fb1640d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 shutdown_connections 2024-09-10T11:39:42.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:42.086+0000 7fb16ab42640 1 -- 172.21.15.99:0/2041786353 wait complete. 2024-09-10T11:39:42.097 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:39:42.097 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:39:42.097 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:39:42.098 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:39:42.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.098 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-09-10T11:39:42.118 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:39:42.118 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:39:42.118 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:42.119 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42282sec preferred_lft 42282sec 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link noprefixroute 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::4cd5:d2ff:fe58:fd11/64 scope link 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 2a:bf:7e:9d:36:82 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::28bf:7eff:fe9d:3682/64 scope link 2024-09-10T11:39:42.120 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.121 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T11:39:42.121 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether ae:f5:7f:64:82:69 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:39:42.121 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::acf5:7fff:fe64:8269/64 scope link 2024-09-10T11:39:42.121 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:39:42.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.121 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:39:42.186 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-09-10T11:39:42.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.188 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-09-10T11:39:42.247 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-09-10T11:39:42.247 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:39:42.248 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:39:42.248 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:39:42.248 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:39:42.248 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-09-10T11:39:42.343 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:39:42.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:39:42.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.384 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:39:42.399 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:39:42.400 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:39:42.400 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:39:42.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.400 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-09-10T11:39:42.464 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:39:42.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:39:42.464 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:42.520 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:39:42.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:39:42.520 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:42.576 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:39:42.641 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:39:42.697 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:39:42.698 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:39:42.699 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:39:42.700 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:39:42.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.700 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:39:42.753 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:39:42.753 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:39:42.795 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-09-10T11:39:42.820 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-09-10T11:39:42.876 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-09-10T11:39:42.877 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-09-10T11:39:42.878 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:39:42.879 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:39:42.879 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:39:42.879 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:39:42.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:42.879 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-09-10T11:39:42.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-09-10T11:39:42.874+0000 7fed75b56480 -1 init, newargv = 0x56113d105210 newargc=15 2024-09-10T11:39:42.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[40256]: starting ceph client 2024-09-10T11:39:42.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[40256]: starting fuse 2024-09-10T11:39:42.895 INFO:teuthology.orchestra.run.smithi196.stdout:41 2024-09-10T11:39:42.895 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-10T11:39:42.895 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:39:42.895 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-09-10T11:39:42.895 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-09-10T11:39:42.895 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-09-10T11:39:42.895 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:39:42.896 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-09-10T11:39:42.897 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-09-10T11:39:43.012 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.40256.asok 2024-09-10T11:39:43.015 INFO:teuthology.orchestra.run.smithi196.stderr:2024-09-10T11:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:39:43.023 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40256.asok 2024-09-10T11:39:43.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:43.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40256.asok status 2024-09-10T11:39:43.157 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-10T11:39:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-09-10T11:39:43.157 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "40256", 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-09-10T11:39:43.158 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 7321, 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 7321 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3206862696 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-09-10T11:39:43.159 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 3206862696 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.7321 192.168.0.1:0/3206862696", 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/3206862696", 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 74, 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 183, 2024-09-10T11:39:43.160 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:39:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-09-10T11:39:43.161 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-09-10T11:39:43.161 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-10T11:39:43.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:43.166 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-09-10T11:39:43.166 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:43.182 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-09-10T11:39:43.182 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:43.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:43.182 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:39:43.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:43.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.357+0000 7f6b5445c640 1 Processor -- start 2024-09-10T11:39:43.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.357+0000 7f6b5445c640 1 -- start start 2024-09-10T11:39:43.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.358+0000 7f6b5445c640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b4c15eb80 0x7f6b4c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.358+0000 7f6b5445c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b4c1626c0 0x7f6b4c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.358+0000 7f6b5445c640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4c15ff70 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.358+0000 7f6b5445c640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4c0d1620 con 0x7f6b4c15eb80 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.358+0000 7f6b5445c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4c162bc0 con 0x7f6b4c1626c0 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b521f8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b4c15eb80 0x7f6b4c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b529f9640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6b4c0d4e40 0x7f6b4c0d1b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32828/0 (socket says 172.21.15.99:32828) 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b529f9640 1 -- 172.21.15.99:0/2795074342 learned_addr learned my addr 172.21.15.99:0/2795074342 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b519f7640 1 --2- 172.21.15.99:0/2795074342 >> v2:172.21.15.196:3300/0 conn(0x7f6b4c1626c0 0x7f6b4c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b521f8640 1 -- 172.21.15.99:0/2795074342 >> v2:172.21.15.196:3300/0 conn(0x7f6b4c1626c0 msgr2=0x7f6b4c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b521f8640 1 --2- 172.21.15.99:0/2795074342 >> v2:172.21.15.196:3300/0 conn(0x7f6b4c1626c0 0x7f6b4c0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b521f8640 1 -- 172.21.15.99:0/2795074342 >> v1:172.21.15.99:6789/0 conn(0x7f6b4c0d4e40 legacy=0x7f6b4c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.359+0000 7f6b521f8640 1 -- 172.21.15.99:0/2795074342 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c162ea0 con 0x7f6b4c15eb80 2024-09-10T11:39:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b521f8640 1 --2- 172.21.15.99:0/2795074342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0d29b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097a0 tx=0x7f6b4802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7364ac5be1b9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b511f6640 1 -- 172.21.15.99:0/2795074342 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48037070 con 0x7f6b4c15eb80 2024-09-10T11:39:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b511f6640 1 -- 172.21.15.99:0/2795074342 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4802ae50 con 0x7f6b4c15eb80 2024-09-10T11:39:43.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b511f6640 1 -- 172.21.15.99:0/2795074342 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48032660 con 0x7f6b4c15eb80 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 -- 172.21.15.99:0/2795074342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c15eb80 msgr2=0x7f6b4c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2795074342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0d29b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097a0 tx=0x7f6b4802a140 comp rx=0 tx=0).stop 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 -- 172.21.15.99:0/2795074342 shutdown_connections 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2795074342 >> v2:172.21.15.196:3300/0 conn(0x7f6b4c1626c0 0x7f6b4c0d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2795074342 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0d29b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 -- 172.21.15.99:0/2795074342 >> 172.21.15.99:0/2795074342 conn(0x7f6b4c08e680 msgr2=0x7f6b4c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:43.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 -- 172.21.15.99:0/2795074342 shutdown_connections 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.360+0000 7f6b5445c640 1 -- 172.21.15.99:0/2795074342 wait complete. 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 Processor -- start 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 -- start start 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 0x7f6b4c14ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c0d5290 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4c163460 con 0x7f6b4c15eb80 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b5445c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4c0d2ef0 con 0x7f6b4c1626c0 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b521f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b521f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45368/0 (socket says 172.21.15.99:45368) 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.362+0000 7f6b521f8640 1 -- 172.21.15.99:0/2622731004 learned_addr learned my addr 172.21.15.99:0/2622731004 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b519f7640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b529f9640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 0x7f6b4c14ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 -- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 msgr2=0x7f6b4c14ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 0x7f6b4c14ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 -- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b4c15eb80 msgr2=0x7f6b4c0b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 -- 172.21.15.99:0/2622731004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4c15aa30 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b529f9640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 0x7f6b4c14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b521f8640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6b480098c0 tx=0x7f6b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48041030 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4802ac50 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b4c0b8030 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48037040 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.363+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b4c161ea0 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.364+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b4c0d29b0 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.365+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b48044070 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.365+0000 7f6b42ffd640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 0x7f6b3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.365+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7f6b480ab420 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.365+0000 7f6b519f7640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 0x7f6b3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.366+0000 7f6b519f7640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 0x7f6b3006e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c1f7ed0 tx=0x7f6b3c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.370+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b48079ef0 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.567+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b4c0c8530 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.568+0000 7f6b42ffd640 1 -- 172.21.15.99:0/2622731004 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f6b480797f0 con 0x7f6b4c0d4e40 2024-09-10T11:39:43.570 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:43.570 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:39:43.570 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 msgr2=0x7f6b3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 0x7f6b3006e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c1f7ed0 tx=0x7f6b3c009190 comp rx=0 tx=0).stop 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 msgr2=0x7f6b4c0cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6b480098c0 tx=0x7f6b480046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 shutdown_connections 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6b3006be30 0x7f6b3006e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b4c1626c0 0x7f6b4c14ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b4c15eb80 0x7f6b4c0b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 --2- 172.21.15.99:0/2622731004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b4c0d4e40 0x7f6b4c0cb2a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 >> 172.21.15.99:0/2622731004 conn(0x7f6b4c08e680 msgr2=0x7f6b4c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:43.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 shutdown_connections 2024-09-10T11:39:43.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.570+0000 7f6b5445c640 1 -- 172.21.15.99:0/2622731004 wait complete. 2024-09-10T11:39:43.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:39:43.698 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":7956,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":8.3649315170000005,"mdsmap_epoch":74,"osdmap_epoch":183,"osdmap_epoch_barrier":183,"uptime":12.444344731999999} 2024-09-10T11:39:43.698 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 7956, "mdsmap_epoch": 74, "osdmap_epoch": 183, "osdmap_epoch_barrier": 183, "rank_uptime": 8.364931517, "state": "up:active", "uptime": 12.444344732, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:39:43.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-09-10T11:39:43.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 Processor -- start 2024-09-10T11:39:43.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 -- start start 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d5740 0x7fb2b8163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b80d2150 con 0x7fb2b80d16f0 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b80d0d30 con 0x7fb2b80b59b0 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.800+0000 7fb2be18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80d5bb0 con 0x7fb2b80d5740 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45382/0 (socket says 172.21.15.99:45382) 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 -- 172.21.15.99:0/3521641346 learned_addr learned my addr 172.21.15.99:0/3521641346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2b7fff640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d5740 0x7fb2b8163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 -- 172.21.15.99:0/3521641346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d5740 msgr2=0x7fb2b8163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d5740 0x7fb2b8163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 -- 172.21.15.99:0/3521641346 >> v1:172.21.15.196:6789/0 conn(0x7fb2b80d16f0 legacy=0x7fb2b8162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:43.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.801+0000 7fb2aeffd640 1 -- 172.21.15.99:0/3521641346 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80b5780 con 0x7fb2b80b59b0 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2aeffd640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009870 tx=0x7fb2a002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25374a0c0d1e43e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2b6ffd640 1 -- 172.21.15.99:0/3521641346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0037070 con 0x7fb2b80b59b0 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2b6ffd640 1 -- 172.21.15.99:0/3521641346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a0032440 con 0x7fb2b80b59b0 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2b6ffd640 1 -- 172.21.15.99:0/3521641346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0032810 con 0x7fb2b80b59b0 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 -- 172.21.15.99:0/3521641346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80b59b0 msgr2=0x7fb2b80d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009870 tx=0x7fb2a002a180 comp rx=0 tx=0).stop 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 -- 172.21.15.99:0/3521641346 shutdown_connections 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d5740 0x7fb2b8163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3521641346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80b59b0 0x7fb2b80d1ad0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 -- 172.21.15.99:0/3521641346 >> 172.21.15.99:0/3521641346 conn(0x7fb2b80d4c20 msgr2=0x7fb2b80d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:43.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 -- 172.21.15.99:0/3521641346 shutdown_connections 2024-09-10T11:39:43.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.802+0000 7fb2be18d640 1 -- 172.21.15.99:0/3521641346 wait complete. 2024-09-10T11:39:43.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.803+0000 7fb2be18d640 1 Processor -- start 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 -- start start 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b80b59b0 0x7fb2b815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d16f0 0x7fb2b815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b52f0 con 0x7fb2b80d5740 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80b4d30 con 0x7fb2b80b59b0 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2be18d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80b4f80 con 0x7fb2b80d16f0 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.804+0000 7fb2b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45398/0 (socket says 172.21.15.99:45398) 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 -- 172.21.15.99:0/3362101817 learned_addr learned my addr 172.21.15.99:0/3362101817 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2aeffd640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d16f0 0x7fb2b815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b77fe640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b80b59b0 0x7fb2b815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 -- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d16f0 msgr2=0x7fb2b815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d16f0 0x7fb2b815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 -- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b80b59b0 msgr2=0x7fb2b815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b80b59b0 0x7fb2b815b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 -- 172.21.15.99:0/3362101817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b815d450 con 0x7fb2b80d5740 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b7fff640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000cc30 tx=0x7fb2b000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000ce40 con 0x7fb2b80d5740 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0003990 con 0x7fb2b80d5740 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b815b6f0 con 0x7fb2b80d5740 2024-09-10T11:39:43.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0003d40 con 0x7fb2b80d5740 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.805+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b815d9d0 con 0x7fb2b80d5740 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.806+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b81639f0 con 0x7fb2b80d5740 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.807+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2b0004000 con 0x7fb2b80d5740 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.807+0000 7fb2b4ff9640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 0x7fb29806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.807+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b008b210 con 0x7fb2b80d5740 2024-09-10T11:39:43.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.807+0000 7fb2b77fe640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 0x7fb29806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:43.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.808+0000 7fb2b77fe640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 0x7fb29806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80d1b80 tx=0x7fb2a8009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:43.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.811+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2b0014030 con 0x7fb2b80d5740 2024-09-10T11:39:43.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:43.976+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7fb2b8163370 con 0x7fb2b80d5740 2024-09-10T11:39:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.739+0000 7fb2b4ff9640 1 -- 172.21.15.99:0/3362101817 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v184) ==== 181+0+0 (secure 0 0 0) 0x7fb2b0059600 con 0x7fb2b80d5740 2024-09-10T11:39:44.741 INFO:teuthology.orchestra.run.smithi099.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 msgr2=0x7fb29806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 0x7fb29806e2f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80d1b80 tx=0x7fb2a8009060 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 msgr2=0x7fb2b815bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fb2b000cc30 tx=0x7fb2b000a5c0 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 shutdown_connections 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb29806be30 0x7fb29806e2f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2b80d5740 0x7fb2b815bf10 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2b80d16f0 0x7fb2b815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 --2- 172.21.15.99:0/3362101817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2b80b59b0 0x7fb2b815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 >> 172.21.15.99:0/3362101817 conn(0x7fb2b80d4c20 msgr2=0x7fb2b80b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:44.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 shutdown_connections 2024-09-10T11:39:44.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:44.742+0000 7fb2be18d640 1 -- 172.21.15.99:0/3362101817 wait complete. 2024-09-10T11:39:44.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-09-10T11:39:44.874 INFO:teuthology.orchestra.run.smithi099.stdout:{"osd_mon_report_interval":"5"} 2024-09-10T11:39:44.874 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-09-10T11:39:44.875 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-09-10T11:39:44.875 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:44.875 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:44.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:44.891 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-09-10T11:39:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records in 2024-09-10T11:39:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:16+0 records out 2024-09-10T11:39:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0872073 s, 192 MB/s 2024-09-10T11:39:45.038 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:45.038 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:45.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:45.054 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-09-10T11:39:45.192 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records in 2024-09-10T11:39:45.192 INFO:teuthology.orchestra.run.smithi099.stderr:17+0 records out 2024-09-10T11:39:45.192 INFO:teuthology.orchestra.run.smithi099.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.0806834 s, 221 MB/s 2024-09-10T11:39:45.192 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-09-10T11:39:45.193 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:39:45.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 Processor -- start 2024-09-10T11:39:45.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 -- start start 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f79500d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f795015ff70 con 0x7f79501626c0 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79500d1620 con 0x7f79500d5010 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.289+0000 7f7956186640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7950162bc0 con 0x7f795015eb80 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f7946ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f79501626c0 0x7f79500d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32832/0 (socket says 172.21.15.99:32832) 2024-09-10T11:39:45.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f7946ffd640 1 -- 172.21.15.99:0/3906512507 learned_addr learned my addr 172.21.15.99:0/3906512507 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794f7fe640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f79500d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041738321 0 0) 0x7f795015ff70 con 0x7f79501626c0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7930003520 con 0x7f79501626c0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1830546709 0 0) 0x7f7930003520 con 0x7f79501626c0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 msgr2=0x7f79500d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f79500d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 >> v1:172.21.15.196:6789/0 conn(0x7f79500d5010 legacy=0x7f79500d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7950162ea0 con 0x7f79501626c0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7938001670 con 0x7f79501626c0 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 >> v1:172.21.15.99:6789/0 conn(0x7f79501626c0 legacy=0x7f79500d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79300039e0 0x7f7930003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f7930004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7930004d40 0x7f79300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7950162bc0 con 0x7f7930004d40 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f793000ba70 con 0x7f79300039e0 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f793000bc70 con 0x7f795015eb80 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794ffff640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7930004d40 0x7f79300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f794f7fe640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79300039e0 0x7f7930003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.290+0000 7f7946ffd640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f7930004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 -- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 msgr2=0x7f7930004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f795015eb80 0x7f7930004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 -- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79300039e0 msgr2=0x7f7930003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f79300039e0 0x7f7930003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 -- 172.21.15.99:0/3906512507 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7930003520 con 0x7f7930004d40 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794ffff640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7930004d40 0x7f79300051c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7948001f30 tx=0x7f7930005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a9161b8b332a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948001b30 con 0x7f7930004d40 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7948031620 con 0x7f7930004d40 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.291+0000 7f794effd640 1 -- 172.21.15.99:0/3906512507 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948031a10 con 0x7f7930004d40 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 -- 172.21.15.99:0/3906512507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7930004d40 msgr2=0x7f79300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 --2- 172.21.15.99:0/3906512507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7930004d40 0x7f79300051c0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f7948001f30 tx=0x7f7930005830 comp rx=0 tx=0).stop 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f794ffff640 1 -- 172.21.15.99:0/3906512507 reap_dead start 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 -- 172.21.15.99:0/3906512507 shutdown_connections 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 -- 172.21.15.99:0/3906512507 >> 172.21.15.99:0/3906512507 conn(0x7f795008e680 msgr2=0x7f795009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 -- 172.21.15.99:0/3906512507 shutdown_connections 2024-09-10T11:39:45.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.292+0000 7f7956186640 1 -- 172.21.15.99:0/3906512507 wait complete. 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.293+0000 7f7956186640 1 Processor -- start 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.293+0000 7f7956186640 1 -- start start 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 0x7f79500cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950159960 0x7f7950159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79500d2230 con 0x7f79500ba770 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7950163460 con 0x7f7930004d40 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7956186640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f79500d23b0 con 0x7f7950159960 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45410/0 (socket says 172.21.15.99:45410) 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 -- 172.21.15.99:0/1883924121 learned_addr learned my addr 172.21.15.99:0/1883924121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f794f7fe640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 0x7f79500cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f794ffff640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950159960 0x7f7950159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 -- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950159960 msgr2=0x7f7950159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950159960 0x7f7950159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 -- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 msgr2=0x7f79500cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 0x7f79500cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 -- 172.21.15.99:0/1883924121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79500cc650 con 0x7f79500ba770 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f794f7fe640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 0x7f79500cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.294+0000 7f7946ffd640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7938007370 tx=0x7f7938007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.295+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793800f940 con 0x7f79500ba770 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.295+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7938018070 con 0x7f79500ba770 2024-09-10T11:39:45.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.295+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79500cc930 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.295+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793800fd60 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.295+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79500ccef0 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.296+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79500d1b20 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.296+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f793801d460 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.297+0000 7f794cff9640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.297+0000 7f794f7fe640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.297+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5709+0+0 (secure 0 0 0) 0x7f793808af40 con 0x7f79500ba770 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.297+0000 7f794f7fe640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 0x7f793406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7940009bc0 tx=0x7f7940008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:45.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.301+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f79380598f0 con 0x7f79500ba770 2024-09-10T11:39:45.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.468+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f795006c560 con 0x7f79500ba770 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.469+0000 7f794cff9640 1 -- 172.21.15.99:0/1883924121 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v184) ==== 81+0+34278 (secure 0 0 0) 0x7f79380591f0 con 0x7f79500ba770 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 184, 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:39:44.739450+0000", 2024-09-10T11:39:45.471 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:39:45.472 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:39:45.473 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:39:45.474 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:45.475 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:45.476 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.477 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:39:45.478 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:45.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:45.480 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:45.481 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:45.482 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:45.483 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:45.484 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:39:45.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:45.487 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:45.488 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "184", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:45.489 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:45.490 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:45.491 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:39:45.492 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.493 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.496 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.499 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.500 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:39:45.501 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.502 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.503 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:39:45.504 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.505 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.507 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:39:45.510 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.511 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:39:45.514 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.516 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.517 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.518 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.519 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.522 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.523 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.524 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:39:45.526 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:39:45.527 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.529 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.532 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:45.533 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.534 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:45.536 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:45.537 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:45.538 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:39:45.539 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.540 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:39:45.541 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.542 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.543 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:39:45.544 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:45.545 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:39:45.546 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:39:45.547 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:45.548 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 msgr2=0x7f793406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 0x7f793406e2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7940009bc0 tx=0x7f7940008040 comp rx=0 tx=0).stop 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 msgr2=0x7f795015bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f7938007370 tx=0x7f7938007830 comp rx=0 tx=0).stop 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 shutdown_connections 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f793406be30 0x7f793406e2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7950159960 0x7f7950159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79500ba770 0x7f795015bcb0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 --2- 172.21.15.99:0/1883924121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7930004d40 0x7f79500cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:45.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 >> 172.21.15.99:0/1883924121 conn(0x7f795008e680 msgr2=0x7f79500d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:45.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 shutdown_connections 2024-09-10T11:39:45.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:45.471+0000 7f7956186640 1 -- 172.21.15.99:0/1883924121 wait complete. 2024-09-10T11:39:45.551 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:39:50.553 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:39:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.661+0000 7f20af998640 1 Processor -- start 2024-09-10T11:39:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.662+0000 7f20af998640 1 -- start start 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20af998640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20a815b2d0 0x7f20a80b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20af998640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20a80afbb0 0x7f20a80714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20af998640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a80b0020 con 0x7f20a815aef0 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20af998640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f20a806ee10 con 0x7f20a815b2d0 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20af998640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f20a80cd9a0 con 0x7f20a80afbb0 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20a815b2d0 0x7f20a80b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20a815b2d0 0x7f20a80b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45412/0 (socket says 172.21.15.99:45412) 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 -- 172.21.15.99:0/1397069690 learned_addr learned my addr 172.21.15.99:0/1397069690 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:50.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20ad734640 1 --1- 172.21.15.99:0/1397069690 >> v1:172.21.15.99:6789/0 conn(0x7f20a815aef0 0x7f20a80cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32838/0 (socket says 172.21.15.99:32838) 2024-09-10T11:39:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20adf35640 1 --2- 172.21.15.99:0/1397069690 >> v2:172.21.15.196:3300/0 conn(0x7f20a80afbb0 0x7f20a80714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 -- 172.21.15.99:0/1397069690 >> v2:172.21.15.196:3300/0 conn(0x7f20a80afbb0 msgr2=0x7f20a80714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 --2- 172.21.15.99:0/1397069690 >> v2:172.21.15.196:3300/0 conn(0x7f20a80afbb0 0x7f20a80714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 -- 172.21.15.99:0/1397069690 >> v1:172.21.15.99:6789/0 conn(0x7f20a815aef0 legacy=0x7f20a80cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.663+0000 7f20acf33640 1 -- 172.21.15.99:0/1397069690 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a80b0290 con 0x7f20a815b2d0 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f20acf33640 1 --2- 172.21.15.99:0/1397069690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a815b2d0 0x7f20a80b4510 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009870 tx=0x7f20a402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26c787715038233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f209bfff640 1 -- 172.21.15.99:0/1397069690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4037070 con 0x7f20a815b2d0 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f209bfff640 1 -- 172.21.15.99:0/1397069690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a4032440 con 0x7f20a815b2d0 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f209bfff640 1 -- 172.21.15.99:0/1397069690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4032810 con 0x7f20a815b2d0 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f20af998640 1 -- 172.21.15.99:0/1397069690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a815b2d0 msgr2=0x7f20a80b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.664+0000 7f20af998640 1 --2- 172.21.15.99:0/1397069690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a815b2d0 0x7f20a80b4510 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009870 tx=0x7f20a402a2a0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 -- 172.21.15.99:0/1397069690 shutdown_connections 2024-09-10T11:39:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 --2- 172.21.15.99:0/1397069690 >> v2:172.21.15.196:3300/0 conn(0x7f20a80afbb0 0x7f20a80714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 --2- 172.21.15.99:0/1397069690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a815b2d0 0x7f20a80b4510 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 -- 172.21.15.99:0/1397069690 >> 172.21.15.99:0/1397069690 conn(0x7f20a806bc10 msgr2=0x7f20a806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 -- 172.21.15.99:0/1397069690 shutdown_connections 2024-09-10T11:39:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.665+0000 7f20af998640 1 -- 172.21.15.99:0/1397069690 wait complete. 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.666+0000 7f20af998640 1 Processor -- start 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.666+0000 7f20af998640 1 -- start start 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20a815aef0 0x7f20a8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a815b2d0 0x7f20a81f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f20a806eb30 con 0x7f20a80afbb0 2024-09-10T11:39:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20a806e850 con 0x7f20a815b2d0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f20a80719e0 con 0x7f20a815aef0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45428/0 (socket says 172.21.15.99:45428) 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 -- 172.21.15.99:0/1471696255 learned_addr learned my addr 172.21.15.99:0/1471696255 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20adf35640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a815b2d0 0x7f20a81f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20acf33640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20a815aef0 0x7f20a8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 -- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20a815aef0 msgr2=0x7f20a8158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20a815aef0 0x7f20a8158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 -- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a815b2d0 msgr2=0x7f20a81f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a815b2d0 0x7f20a81f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 -- 172.21.15.99:0/1471696255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a815cdc0 con 0x7f20a80afbb0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20ad734640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f209c00c430 tx=0x7f209c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0074e0 con 0x7f20a80afbb0 2024-09-10T11:39:50.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c007680 con 0x7f20a80afbb0 2024-09-10T11:39:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a815d0a0 con 0x7f20a80afbb0 2024-09-10T11:39:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.667+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c004a00 con 0x7f20a80afbb0 2024-09-10T11:39:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.668+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a815d660 con 0x7f20a80afbb0 2024-09-10T11:39:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.669+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a8071d70 con 0x7f20a80afbb0 2024-09-10T11:39:50.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.669+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f209c01a020 con 0x7f20a80afbb0 2024-09-10T11:39:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.670+0000 7f2099ffb640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 0x7f208c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.670+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5709+0+0 (secure 0 0 0) 0x7f209c08a5e0 con 0x7f20a80afbb0 2024-09-10T11:39:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.670+0000 7f20acf33640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 0x7f208c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.670+0000 7f20acf33640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 0x7f208c06e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f20a4034000 tx=0x7f20a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:50.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.674+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f209c059020 con 0x7f20a80afbb0 2024-09-10T11:39:50.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.841+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f20a80b38e0 con 0x7f20a80afbb0 2024-09-10T11:39:50.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.843+0000 7f2099ffb640 1 -- 172.21.15.99:0/1471696255 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) ==== 81+0+34297 (secure 0 0 0) 0x7f209c058920 con 0x7f20a80afbb0 2024-09-10T11:39:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:50.845 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 185, 2024-09-10T11:39:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:50.845 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:39:50.720623+0000", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:39:50.846 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:39:50.847 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:50.848 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:50.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:50.850 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:50.851 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:50.852 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:39:50.853 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:50.854 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:39:50.855 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:50.856 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:50.857 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:50.858 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:50.859 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:50.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:50.861 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:50.862 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:50.863 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:50.864 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:50.865 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:39:50.866 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.867 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.869 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:39:50.872 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.873 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:39:50.877 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:39:50.878 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.879 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.880 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:39:50.883 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:39:50.884 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.885 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:39:50.888 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.889 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.890 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.891 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.892 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.893 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.895 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:39:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.897 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:39:50.900 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.901 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.902 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.903 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.908 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:50.909 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:50.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:39:50.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.915 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:39:50.916 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.917 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.918 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.919 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:50.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:39:50.921 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:39:50.922 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:39:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:39:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:39:50.923 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:50.923 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:50.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 msgr2=0x7f208c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 0x7f208c06e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f20a4034000 tx=0x7f20a402e000 comp rx=0 tx=0).stop 2024-09-10T11:39:50.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 msgr2=0x7f20a814d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f209c00c430 tx=0x7f209c00c8f0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 shutdown_connections 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f208c06be30 0x7f208c06e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20a815b2d0 0x7f20a81f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20a815aef0 0x7f20a8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 --2- 172.21.15.99:0/1471696255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20a80afbb0 0x7f20a814d2a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 >> 172.21.15.99:0/1471696255 conn(0x7f20a806bc10 msgr2=0x7f20a815a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 shutdown_connections 2024-09-10T11:39:50.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:50.845+0000 7f20af998640 1 -- 172.21.15.99:0/1471696255 wait complete. 2024-09-10T11:39:50.926 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T11:39:50.926 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:39:50.926 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:39:50.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:50.942 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-09-10T11:39:51.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:39:51.043 INFO:teuthology.orchestra.run.smithi099.stderr:dd: error writing 'large_file_b': No space left on device 2024-09-10T11:39:51.043 INFO:teuthology.orchestra.run.smithi099.stderr:dd: closing output file 'large_file_b': No space left on device 2024-09-10T11:39:51.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:39:51.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 Processor -- start 2024-09-10T11:39:51.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 -- start start 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 0x7f63a80d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63a80d5010 0x7f63a80d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a815ff70 con 0x7f63a81626c0 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f63a80d1620 con 0x7f63a80d5010 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.140+0000 7f63ae98d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f63a8162bc0 con 0x7f63a815eb80 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63acf2a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f63a80d5010 0x7f63a80d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63a77fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f63a81626c0 0x7f63a80d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:32840/0 (socket says 172.21.15.99:32840) 2024-09-10T11:39:51.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63a77fe640 1 -- 172.21.15.99:0/419706695 learned_addr learned my addr 172.21.15.99:0/419706695 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63a7fff640 1 --2- 172.21.15.99:0/419706695 >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 0x7f63a80d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63acf2a640 1 -- 172.21.15.99:0/419706695 >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 msgr2=0x7f63a80d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63acf2a640 1 --2- 172.21.15.99:0/419706695 >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 0x7f63a80d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63acf2a640 1 -- 172.21.15.99:0/419706695 >> v1:172.21.15.99:6789/0 conn(0x7f63a81626c0 legacy=0x7f63a80d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63acf2a640 1 -- 172.21.15.99:0/419706695 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a8162ea0 con 0x7f63a80d5010 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63a6ffd640 1 -- 172.21.15.99:0/419706695 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910695781 0 0) 0x7f63a815ff70 con 0x7f63a81626c0 2024-09-10T11:39:51.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.141+0000 7f63a7fff640 1 --2- 172.21.15.99:0/419706695 >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 0x7f63a80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:51.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63acf2a640 1 --2- 172.21.15.99:0/419706695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a80d5010 0x7f63a80d1b20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec94e86815fc2adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63a6ffd640 1 -- 172.21.15.99:0/419706695 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0031600 con 0x7f63a80d5010 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63a6ffd640 1 -- 172.21.15.99:0/419706695 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a00317a0 con 0x7f63a80d5010 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63a6ffd640 1 -- 172.21.15.99:0/419706695 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0031b70 con 0x7f63a80d5010 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 -- 172.21.15.99:0/419706695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a80d5010 msgr2=0x7f63a80d1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 --2- 172.21.15.99:0/419706695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a80d5010 0x7f63a80d1b20 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a002a610 comp rx=0 tx=0).stop 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 -- 172.21.15.99:0/419706695 shutdown_connections 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 --2- 172.21.15.99:0/419706695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a80d5010 0x7f63a80d1b20 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 --2- 172.21.15.99:0/419706695 >> v2:172.21.15.196:3300/0 conn(0x7f63a815eb80 0x7f63a80d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 -- 172.21.15.99:0/419706695 >> 172.21.15.99:0/419706695 conn(0x7f63a808e680 msgr2=0x7f63a809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 -- 172.21.15.99:0/419706695 shutdown_connections 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.142+0000 7f63ae98d640 1 -- 172.21.15.99:0/419706695 wait complete. 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.143+0000 7f63ae98d640 1 Processor -- start 2024-09-10T11:39:51.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 -- start start 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a80d5010 0x7f63a81530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63a815eb80 0x7f63a80b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80d2ef0 con 0x7f63a81626c0 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f63a8163460 con 0x7f63a80d5010 2024-09-10T11:39:51.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.144+0000 7f63ae98d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f63a80d3070 con 0x7f63a815eb80 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45446/0 (socket says 172.21.15.99:45446) 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 -- 172.21.15.99:0/2081460579 learned_addr learned my addr 172.21.15.99:0/2081460579 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63a77fe640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63a815eb80 0x7f63a80b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63a7fff640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a80d5010 0x7f63a81530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 -- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63a815eb80 msgr2=0x7f63a80b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63a815eb80 0x7f63a80b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 -- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a80d5010 msgr2=0x7f63a81530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a80d5010 0x7f63a81530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a80b1c80 con 0x7f63a81626c0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63acf2a640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a003f070 con 0x7f63a81626c0 2024-09-10T11:39:51.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a0030b60 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a80b8320 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0009450 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.145+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a80b88e0 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.146+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a80b1e50 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.147+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63a003b410 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.147+0000 7f63a4ff9640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.147+0000 7f63a7fff640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.147+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f63a00ac030 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.147+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=186}) -- 0x7f6394073960 con 0x7f63a81626c0 2024-09-10T11:39:51.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.148+0000 7f63a7fff640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cead0 tx=0x7f6398008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.152+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f63a007a390 con 0x7f63a81626c0 2024-09-10T11:39:51.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.318+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f63a80b62f0 con 0x7f63a81626c0 2024-09-10T11:39:51.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.319+0000 7f63a4ff9640 1 -- 172.21.15.99:0/2081460579 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) ==== 81+0+34297 (secure 0 0 0) 0x7f63a007e690 con 0x7f63a81626c0 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 185, 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:39:50.720623+0000", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:39:51.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:39:51.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:39:51.324 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:51.325 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:51.326 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:51.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:51.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:51.329 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:51.330 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.331 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:51.332 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:51.333 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:39:51.334 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.335 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:51.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:51.337 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:51.338 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:39:51.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:51.340 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:51.341 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:39:51.342 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:51.343 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:51.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:39:51.346 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:39:51.347 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:51.348 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.349 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:39:51.351 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:39:51.353 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:39:51.354 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.355 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:39:51.357 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.359 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.361 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.362 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:39:51.365 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.366 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.368 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.372 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:39:51.373 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.374 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:39:51.376 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.378 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:39:51.379 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.381 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.384 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.385 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.386 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.387 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.388 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:39:51.389 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:39:51.390 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.392 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:51.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:51.395 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:39:51.396 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:39:51.399 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:39:51.400 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.401 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.402 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:51.405 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:51.406 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:51.407 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:39:51.408 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:39:51.409 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:39:51.410 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:39:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:39:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:39:51.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:51.411 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:51.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 msgr2=0x7f639406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cead0 tx=0x7f6398008040 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 msgr2=0x7f63a8152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009d90 tx=0x7f63a0004a40 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 shutdown_connections 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f639406be30 0x7f639406e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63a81626c0 0x7f63a8152850 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f63a815eb80 0x7f63a80b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 --2- 172.21.15.99:0/2081460579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63a80d5010 0x7f63a81530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 >> 172.21.15.99:0/2081460579 conn(0x7f63a808e680 msgr2=0x7f63a809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:51.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 shutdown_connections 2024-09-10T11:39:51.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.321+0000 7f63ae98d640 1 -- 172.21.15.99:0/2081460579 wait complete. 2024-09-10T11:39:51.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:39:51.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.511+0000 7ff716586640 1 Processor -- start 2024-09-10T11:39:51.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.511+0000 7ff716586640 1 -- start start 2024-09-10T11:39:51.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.512+0000 7ff716586640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.512+0000 7ff716586640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.512+0000 7ff716586640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff710152e60 con 0x7ff71006e5d0 2024-09-10T11:39:51.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.512+0000 7ff716586640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff71006ef50 con 0x7ff71015c770 2024-09-10T11:39:51.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.512+0000 7ff716586640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff7100d15e0 con 0x7ff7100d1200 2024-09-10T11:39:51.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff714b23640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70f7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff71006e5d0 0x7ff71015c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:46730/0 (socket says 172.21.15.99:46730) 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff714b23640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:56254/0 (socket says 172.21.15.99:56254) 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:54088/0 (socket says 172.21.15.99:54088) 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70f7fe640 1 -- 172.21.15.99:0/1950051020 learned_addr learned my addr 172.21.15.99:0/1950051020 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 -- 172.21.15.99:0/1950051020 >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 msgr2=0x7ff71006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 --2- 172.21.15.99:0/1950051020 >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 -- 172.21.15.99:0/1950051020 >> v1:172.21.15.196:6789/0 conn(0x7ff71006e5d0 legacy=0x7ff71015c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70effd640 1 -- 172.21.15.99:0/1950051020 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508297157 0 0) 0x7ff710152e60 con 0x7ff71006e5d0 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff70ffff640 1 -- 172.21.15.99:0/1950051020 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff710072680 con 0x7ff7100d1200 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.513+0000 7ff714b23640 1 --2- 172.21.15.99:0/1950051020 >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:39:51.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff70ffff640 1 --2- 172.21.15.99:0/1950051020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff7000097a0 tx=0x7ff70002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1135c1c5fbb2a88a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff70effd640 1 -- 172.21.15.99:0/1950051020 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700037070 con 0x7ff7100d1200 2024-09-10T11:39:51.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff70effd640 1 -- 172.21.15.99:0/1950051020 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff70002ae50 con 0x7ff7100d1200 2024-09-10T11:39:51.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff70effd640 1 -- 172.21.15.99:0/1950051020 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700032660 con 0x7ff7100d1200 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff716586640 1 -- 172.21.15.99:0/1950051020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 msgr2=0x7ff710073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.514+0000 7ff716586640 1 --2- 172.21.15.99:0/1950051020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff7000097a0 tx=0x7ff70002a100 comp rx=0 tx=0).stop 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 -- 172.21.15.99:0/1950051020 shutdown_connections 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 --2- 172.21.15.99:0/1950051020 >> v2:172.21.15.196:3300/0 conn(0x7ff71015c770 0x7ff71006ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 --2- 172.21.15.99:0/1950051020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff7100d1200 0x7ff710073440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 -- 172.21.15.99:0/1950051020 >> 172.21.15.99:0/1950051020 conn(0x7ff71006bc10 msgr2=0x7ff71006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 -- 172.21.15.99:0/1950051020 shutdown_connections 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.515+0000 7ff716586640 1 -- 172.21.15.99:0/1950051020 wait complete. 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.516+0000 7ff716586640 1 Processor -- start 2024-09-10T11:39:51.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.516+0000 7ff716586640 1 -- start start 2024-09-10T11:39:51.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff71006e5d0 0x7ff7101f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff71015c770 0x7ff7101f55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff710072f20 con 0x7ff7100d1200 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff710072c40 con 0x7ff71006e5d0 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff71015cd90 con 0x7ff71015c770 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45456/0 (socket says 172.21.15.99:45456) 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 -- 172.21.15.99:0/3667472501 learned_addr learned my addr 172.21.15.99:0/3667472501 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70ffff640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff71006e5d0 0x7ff7101f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff714b23640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff71015c770 0x7ff7101f55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 -- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff71015c770 msgr2=0x7ff7101f55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff71015c770 0x7ff7101f55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 -- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff71006e5d0 msgr2=0x7ff7101f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff71006e5d0 0x7ff7101f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7100ce000 con 0x7ff7100d1200 2024-09-10T11:39:51.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70f7fe640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff70400c440 tx=0x7ff70400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7040074e0 con 0x7ff7100d1200 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7040047e0 con 0x7ff7100d1200 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff704004bd0 con 0x7ff7100d1200 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7100ce2e0 con 0x7ff7100d1200 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.517+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7100ce8a0 con 0x7ff7100d1200 2024-09-10T11:39:51.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.519+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff710073440 con 0x7ff7100d1200 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.519+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff704011070 con 0x7ff7100d1200 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.520+0000 7ff70cff9640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 0x7ff6fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.520+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7ff70408a8e0 con 0x7ff7100d1200 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.520+0000 7ff70ffff640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 0x7ff6fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.520+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=186}) -- 0x7ff6fc073960 con 0x7ff7100d1200 2024-09-10T11:39:51.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.520+0000 7ff70ffff640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 0x7ff6fc06e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff700034000 tx=0x7ff70002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:51.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.524+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff704059320 con 0x7ff7100d1200 2024-09-10T11:39:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.722+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7100d0bf0 con 0x7ff7100d1200 2024-09-10T11:39:51.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.722+0000 7ff70cff9640 1 -- 172.21.15.99:0/3667472501 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7ff704058c20 con 0x7ff7100d1200 2024-09-10T11:39:51.724 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:51.725 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:39:51.725 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:39:51.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 msgr2=0x7ff6fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 0x7ff6fc06e2f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff700034000 tx=0x7ff70002e000 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 msgr2=0x7ff710155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ff70400c440 tx=0x7ff70400c840 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 shutdown_connections 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff6fc06be30 0x7ff6fc06e2f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff71015c770 0x7ff7101f55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7100d1200 0x7ff710155860 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 --2- 172.21.15.99:0/3667472501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff71006e5d0 0x7ff7101f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 >> 172.21.15.99:0/3667472501 conn(0x7ff71006bc10 msgr2=0x7ff71015e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 shutdown_connections 2024-09-10T11:39:51.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:51.724+0000 7ff716586640 1 -- 172.21.15.99:0/3667472501 wait complete. 2024-09-10T11:39:51.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:39:51.855 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":7956,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.521880531000001,"mdsmap_epoch":74,"osdmap_epoch":185,"osdmap_epoch_barrier":183,"uptime":20.601294618000001} 2024-09-10T11:39:51.856 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 7956, "mdsmap_epoch": 74, "osdmap_epoch": 185, "osdmap_epoch_barrier": 183, "rank_uptime": 16.521880531, "state": "up:active", "uptime": 20.601294618, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:39:51.856 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:39:51.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:51.856 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-09-10T11:39:51.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:39:51.875 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-09-10T11:39:51.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:39:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.073+0000 7ff33ff4b640 1 Processor -- start 2024-09-10T11:39:52.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.073+0000 7ff33ff4b640 1 -- start start 2024-09-10T11:39:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33ff4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33ff4b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff33815c770 0x7ff33806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33ff4b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff338152e60 con 0x7ff33806e5d0 2024-09-10T11:39:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33ff4b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff33806ef50 con 0x7ff33815c770 2024-09-10T11:39:52.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33ff4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380d15e0 con 0x7ff3380d1200 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33dce7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33dce7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:54096/0 (socket says 172.21.15.99:54096) 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33dce7640 1 -- 172.21.15.99:0/396462512 learned_addr learned my addr 172.21.15.99:0/396462512 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.074+0000 7ff33e4e8640 1 --2- 172.21.15.99:0/396462512 >> v2:172.21.15.196:3300/0 conn(0x7ff33815c770 0x7ff33806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.075+0000 7ff33dce7640 1 -- 172.21.15.99:0/396462512 >> v2:172.21.15.196:3300/0 conn(0x7ff33815c770 msgr2=0x7ff33806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:52.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.075+0000 7ff33dce7640 1 --2- 172.21.15.99:0/396462512 >> v2:172.21.15.196:3300/0 conn(0x7ff33815c770 0x7ff33806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.075+0000 7ff33dce7640 1 -- 172.21.15.99:0/396462512 >> v1:172.21.15.196:6789/0 conn(0x7ff33806e5d0 legacy=0x7ff33815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.075+0000 7ff33cce5640 1 -- 172.21.15.99:0/396462512 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2927785051 0 0) 0x7ff338152e60 con 0x7ff33806e5d0 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.075+0000 7ff33dce7640 1 -- 172.21.15.99:0/396462512 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338072680 con 0x7ff3380d1200 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33dce7640 1 --2- 172.21.15.99:0/396462512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff328009e50 tx=0x7ff32802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0f4f007ddac616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33cce5640 1 -- 172.21.15.99:0/396462512 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328036070 con 0x7ff3380d1200 2024-09-10T11:39:52.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33cce5640 1 -- 172.21.15.99:0/396462512 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff328031600 con 0x7ff3380d1200 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33cce5640 1 -- 172.21.15.99:0/396462512 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280319d0 con 0x7ff3380d1200 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 -- 172.21.15.99:0/396462512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 msgr2=0x7ff338073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/396462512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff328009e50 tx=0x7ff32802a680 comp rx=0 tx=0).stop 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 -- 172.21.15.99:0/396462512 shutdown_connections 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/396462512 >> v2:172.21.15.196:3300/0 conn(0x7ff33815c770 0x7ff33806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/396462512 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff3380d1200 0x7ff338073440 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 -- 172.21.15.99:0/396462512 >> 172.21.15.99:0/396462512 conn(0x7ff33806bc10 msgr2=0x7ff33806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 -- 172.21.15.99:0/396462512 shutdown_connections 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.076+0000 7ff33ff4b640 1 -- 172.21.15.99:0/396462512 wait complete. 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.077+0000 7ff33ff4b640 1 Processor -- start 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 -- start start 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 0x7ff3381f1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 0x7ff338153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff338072f20 con 0x7ff3380d1200 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff338072c40 con 0x7ff33815c770 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33ff4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff33815cd90 con 0x7ff33806e5d0 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33d4e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33d4e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45458/0 (socket says 172.21.15.99:45458) 2024-09-10T11:39:52.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.078+0000 7ff33d4e6640 1 -- 172.21.15.99:0/1206797779 learned_addr learned my addr 172.21.15.99:0/1206797779 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:52.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33dce7640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 0x7ff3381f1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33e4e8640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 0x7ff338153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 -- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 msgr2=0x7ff3381f1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 0x7ff3381f1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 -- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 msgr2=0x7ff338153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 0x7ff338153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33814dce0 con 0x7ff3380d1200 2024-09-10T11:39:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33dce7640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 0x7ff3381f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:52.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33e4e8640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 0x7ff338153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33d4e6640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c3e0 tx=0x7ff32c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c017070 con 0x7ff3380d1200 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c0035f0 con 0x7ff3380d1200 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33814dfc0 con 0x7ff3380d1200 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003a00 con 0x7ff3380d1200 2024-09-10T11:39:52.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.079+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33814e580 con 0x7ff3380d1200 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.080+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff338073440 con 0x7ff3380d1200 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.080+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff32c01a070 con 0x7ff3380d1200 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.081+0000 7ff3367fc640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.081+0000 7ff33dce7640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.081+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7ff32c08a290 con 0x7ff3380d1200 2024-09-10T11:39:52.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.081+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=186}) -- 0x7ff3240739c0 con 0x7ff3380d1200 2024-09-10T11:39:52.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.081+0000 7ff33dce7640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 0x7ff32406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff328033000 tx=0x7ff328009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:52.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.085+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff32c058c40 con 0x7ff3380d1200 2024-09-10T11:39:52.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.250+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ff33815fe40 con 0x7ff3380d1200 2024-09-10T11:39:52.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.252+0000 7ff3367fc640 1 -- 172.21.15.99:0/1206797779 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) ==== 81+0+34297 (secure 0 0 0) 0x7ff32c058540 con 0x7ff3380d1200 2024-09-10T11:39:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:52.254 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 185, 2024-09-10T11:39:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:52.254 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:39:50.720623+0000", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:39:52.255 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:39:52.256 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:39:52.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:39:52.258 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:52.259 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:52.260 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:52.261 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:52.262 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:39:52.263 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:52.264 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:39:52.265 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:52.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:52.267 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:52.268 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:52.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:52.270 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.271 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:52.273 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:52.274 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:52.275 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:52.276 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:52.277 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.278 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:52.279 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.280 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.282 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.283 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.286 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:39:52.287 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:39:52.288 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.290 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:39:52.291 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.292 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:39:52.294 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:39:52.295 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:39:52.296 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.297 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.298 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:52.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.303 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.305 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.306 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.307 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.308 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.309 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.314 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.315 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.318 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:52.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.320 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:52.321 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:39:52.322 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:39:52.323 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.324 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:52.326 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.328 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.329 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.330 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.331 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:52.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:39:52.333 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:39:52.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:52.335 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:52.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 msgr2=0x7ff32406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 0x7ff32406e2f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff328033000 tx=0x7ff328009000 comp rx=0 tx=0).stop 2024-09-10T11:39:52.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 msgr2=0x7ff338155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:52.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00c3e0 tx=0x7ff32c00c8a0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 shutdown_connections 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff32406be30 0x7ff32406e2f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff33815c770 0x7ff338153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3380d1200 0x7ff338155730 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 --2- 172.21.15.99:0/1206797779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff33806e5d0 0x7ff3381f1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 >> 172.21.15.99:0/1206797779 conn(0x7ff33806bc10 msgr2=0x7ff3380ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 shutdown_connections 2024-09-10T11:39:52.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:52.254+0000 7ff33ff4b640 1 -- 172.21.15.99:0/1206797779 wait complete. 2024-09-10T11:39:52.338 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:39:57.338 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:39:57.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.447+0000 7f047bbd6640 1 Processor -- start 2024-09-10T11:39:57.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.447+0000 7f047bbd6640 1 -- start start 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f047415c7b0 0x7f04740d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f04740d0fb0 0x7f047406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047406ee30 0x7f04741604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f04740d1420 con 0x7f047415c7b0 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f047406f280 con 0x7f04740d0fb0 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047bbd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04740b3840 con 0x7f047406ee30 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f047415c7b0 0x7f04740d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f047415c7b0 0x7f04740d0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36924/0 (socket says 172.21.15.99:36924) 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 -- 172.21.15.99:0/2861172390 learned_addr learned my addr 172.21.15.99:0/2861172390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479171640 1 --2- 172.21.15.99:0/2861172390 >> v2:172.21.15.196:3300/0 conn(0x7f04740d0fb0 0x7f047406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f047a173640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047406ee30 0x7f04741604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 -- 172.21.15.99:0/2861172390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047406ee30 msgr2=0x7f04741604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047406ee30 0x7f04741604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 -- 172.21.15.99:0/2861172390 >> v2:172.21.15.196:3300/0 conn(0x7f04740d0fb0 msgr2=0x7f047406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 --2- 172.21.15.99:0/2861172390 >> v2:172.21.15.196:3300/0 conn(0x7f04740d0fb0 0x7f047406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.448+0000 7f0479972640 1 -- 172.21.15.99:0/2861172390 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04740ceb10 con 0x7f047415c7b0 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.449+0000 7f0479972640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047415c7b0 0x7f04740d0a70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f04740d0250 tx=0x7f047002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f475047a8de9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.449+0000 7f0478970640 1 -- 172.21.15.99:0/2861172390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470037070 con 0x7f047415c7b0 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.449+0000 7f0478970640 1 -- 172.21.15.99:0/2861172390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0470032440 con 0x7f047415c7b0 2024-09-10T11:39:57.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.449+0000 7f0478970640 1 -- 172.21.15.99:0/2861172390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470032810 con 0x7f047415c7b0 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 -- 172.21.15.99:0/2861172390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047415c7b0 msgr2=0x7f04740d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047415c7b0 0x7f04740d0a70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f04740d0250 tx=0x7f047002a1e0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 -- 172.21.15.99:0/2861172390 shutdown_connections 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047406ee30 0x7f04741604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 --2- 172.21.15.99:0/2861172390 >> v2:172.21.15.196:3300/0 conn(0x7f04740d0fb0 0x7f047406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 --2- 172.21.15.99:0/2861172390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047415c7b0 0x7f04740d0a70 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 -- 172.21.15.99:0/2861172390 >> 172.21.15.99:0/2861172390 conn(0x7f047406bc10 msgr2=0x7f047406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 -- 172.21.15.99:0/2861172390 shutdown_connections 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.450+0000 7f047bbd6640 1 -- 172.21.15.99:0/2861172390 wait complete. 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.451+0000 7f047bbd6640 1 Processor -- start 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.451+0000 7f047bbd6640 1 -- start start 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04740d0fb0 0x7f04741e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 0x7f0474159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f04740b4050 con 0x7f047406ee30 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04740b3d70 con 0x7f04740d0fb0 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047bbd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f04740b4330 con 0x7f047415c7b0 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36928/0 (socket says 172.21.15.99:36928) 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 -- 172.21.15.99:0/718692556 learned_addr learned my addr 172.21.15.99:0/718692556 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479171640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04740d0fb0 0x7f04741e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047a173640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 0x7f0474159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 -- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 msgr2=0x7f0474159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 0x7f0474159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 -- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04740d0fb0 msgr2=0x7f04741e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04740d0fb0 0x7f04741e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741e7c10 con 0x7f047406ee30 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f047a173640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 0x7f0474159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f0479972640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f047002a220 tx=0x7f04700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.452+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470041020 con 0x7f047406ee30 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.453+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04700306c0 con 0x7f047406ee30 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.453+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470037040 con 0x7f047406ee30 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.453+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04741e7370 con 0x7f047406ee30 2024-09-10T11:39:57.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.453+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04741e7930 con 0x7f047406ee30 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.453+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047415cb90 con 0x7f047406ee30 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.454+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0470044020 con 0x7f047406ee30 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.455+0000 7f046a7fc640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.455+0000 7f0479171640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.455+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5709+0+0 (secure 0 0 0) 0x7f04700ab5a0 con 0x7f047406ee30 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.455+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=186}) -- 0x7f0458073960 con 0x7f047406ee30 2024-09-10T11:39:57.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.455+0000 7f0479171640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 0x7f045806e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0474158300 tx=0x7f0464008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:57.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.459+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f047007ee20 con 0x7f047406ee30 2024-09-10T11:39:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.630+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f0474157d70 con 0x7f047406ee30 2024-09-10T11:39:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.632+0000 7f046a7fc640 1 -- 172.21.15.99:0/718692556 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v185) ==== 81+0+34297 (secure 0 0 0) 0x7f047004b030 con 0x7f047406ee30 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 185, 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:39:50.720623+0000", 2024-09-10T11:39:57.634 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:39:57.635 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:57.636 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:39:57.637 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:57.638 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:57.639 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:57.640 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:57.641 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:57.642 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.643 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:57.644 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:57.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:57.646 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:57.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:57.648 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:39:57.649 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:39:57.650 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1027, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool,full,full_quota", 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:39:57.651 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:39:57.652 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "185", 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:39:57.653 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:39:57.654 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:39:57.655 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:39:57.656 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:39:57.657 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.658 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.659 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:39:57.661 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:39:57.662 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:39:57.663 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.664 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.665 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.669 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.672 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.673 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:39:57.674 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.676 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.678 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:39:57.679 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:39:57.680 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:39:57.681 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:39:57.682 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.684 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.685 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.686 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.687 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:39:57.689 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.690 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:39:57.691 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.692 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:57.696 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.698 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.699 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.700 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:39:57.701 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:39:57.702 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:39:57.707 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:39:57.708 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.711 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.712 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:39:57.714 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:39:57.715 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:39:57.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:39:57.717 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:39:57.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 msgr2=0x7f045806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:57.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 0x7f045806e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0474158300 tx=0x7f0464008040 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 msgr2=0x7f0474154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f047002a220 tx=0x7f04700046c0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 shutdown_connections 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f045806be30 0x7f045806e2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f047415c7b0 0x7f0474159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04740d0fb0 0x7f04741e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 --2- 172.21.15.99:0/718692556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f047406ee30 0x7f0474154420 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 >> 172.21.15.99:0/718692556 conn(0x7f047406bc10 msgr2=0x7f04740afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 shutdown_connections 2024-09-10T11:39:57.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:39:57.634+0000 7f047bbd6640 1 -- 172.21.15.99:0/718692556 wait complete. 2024-09-10T11:39:57.720 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-09-10T11:40:00.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:00.047 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:02.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:40:02.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.832+0000 7f9eb96dd640 1 Processor -- start 2024-09-10T11:40:02.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.832+0000 7f9eb96dd640 1 -- start start 2024-09-10T11:40:02.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb96dd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eb40d1200 0x7f9eb4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb96dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 0x7f9eb406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb96dd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eb4152e60 con 0x7f9eb415c060 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb96dd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f9eb406ef50 con 0x7f9eb40d1200 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb96dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb40d15e0 con 0x7f9eb406e5d0 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eb40d1200 0x7f9eb4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9eb40d1200 0x7f9eb4073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33960/0 (socket says 172.21.15.99:33960) 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2810996375 learned_addr learned my addr 172.21.15.99:0/2810996375 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2810996375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 msgr2=0x7f9eb406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 0x7f9eb406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 0x7f9eb406ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2810996375 >> v1:172.21.15.196:6789/0 conn(0x7f9eb415c060 legacy=0x7f9eb415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2810996375 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb4072680 con 0x7f9eb40d1200 2024-09-10T11:40:02.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.833+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 0x7f9eb406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.834+0000 7f9eb2ffd640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb40d1200 0x7f9eb4073440 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400bb10 tx=0x7f9ea402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb259e1b98fc32a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.834+0000 7f9eb27fc640 1 -- 172.21.15.99:0/2810996375 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4032790 con 0x7f9eb40d1200 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.834+0000 7f9eb27fc640 1 -- 172.21.15.99:0/2810996375 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea4038070 con 0x7f9eb40d1200 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.834+0000 7f9eb27fc640 1 -- 172.21.15.99:0/2810996375 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4032b50 con 0x7f9eb40d1200 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2810996375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb40d1200 msgr2=0x7f9eb4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb40d1200 0x7f9eb4073440 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400bb10 tx=0x7f9ea402c890 comp rx=0 tx=0).stop 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2810996375 shutdown_connections 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb406e5d0 0x7f9eb406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2810996375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb40d1200 0x7f9eb4073440 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:02.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2810996375 >> 172.21.15.99:0/2810996375 conn(0x7f9eb406bc10 msgr2=0x7f9eb406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:02.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2810996375 shutdown_connections 2024-09-10T11:40:02.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.835+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2810996375 wait complete. 2024-09-10T11:40:02.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.836+0000 7f9eb96dd640 1 Processor -- start 2024-09-10T11:40:02.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.836+0000 7f9eb96dd640 1 -- start start 2024-09-10T11:40:02.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 0x7f9eb4155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb415c060 0x7f9eb414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4072f20 con 0x7f9eb406e5d0 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb4072c40 con 0x7f9eb415c060 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb96dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb415cd90 con 0x7f9eb40d1200 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33976/0 (socket says 172.21.15.99:33976) 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2383137351 learned_addr learned my addr 172.21.15.99:0/2383137351 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 0x7f9eb4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb37fe640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb415c060 0x7f9eb414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 msgr2=0x7f9eb4155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 0x7f9eb4155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb415c060 msgr2=0x7f9eb414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb415c060 0x7f9eb414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eb2ffd640 1 -- 172.21.15.99:0/2383137351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb41548c0 con 0x7f9eb406e5d0 2024-09-10T11:40:02.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.837+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 0x7f9eb4155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eb2ffd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400b2c0 tx=0x7f9ea40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4033620 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea40337e0 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4033bd0 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb4154ba0 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.838+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb4155160 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.839+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eb406ea10 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.839+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ea404d060 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.840+0000 7f9eabfff640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 0x7f9e9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.840+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f9ea40ad630 con 0x7f9eb406e5d0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.840+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 0x7f9e9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:02.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.840+0000 7f9eaa7fc640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 0x7f9e9406e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9eb415f3d0 tx=0x7f9e9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:02.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:02.844+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ea4041070 con 0x7f9eb406e5d0 2024-09-10T11:40:03.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.009+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f9eb40b43d0 con 0x7f9eb406e5d0 2024-09-10T11:40:03.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.010+0000 7f9eabfff640 1 -- 172.21.15.99:0/2383137351 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) ==== 81+0+34278 (secure 0 0 0) 0x7f9ea4080e20 con 0x7f9eb406e5d0 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 186, 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:40:00.722829+0000", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:40:03.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:40:03.014 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.015 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:40:03.016 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.017 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.018 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.019 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:40:03.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.021 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.022 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:40:03.023 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.024 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.025 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.026 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.027 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.028 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.029 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:40:03.030 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "186", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.031 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.032 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.033 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.034 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.036 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.037 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.038 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:40:03.039 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:40:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.041 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.043 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.044 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.045 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.046 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:40:03.047 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.048 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:40:03.052 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:40:03.053 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.054 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:40:03.055 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:40:03.056 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.057 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.060 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.064 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.065 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.066 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.067 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:40:03.070 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.071 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:40:03.073 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:40:03.074 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:40:03.075 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:40:03.076 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.077 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:40:03.078 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.079 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.080 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:40:03.082 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.083 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:40:03.084 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.085 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.086 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:40:03.087 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:40:03.088 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:40:03.089 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:40:03.090 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.091 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 msgr2=0x7f9e9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 0x7f9e9406e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9eb415f3d0 tx=0x7f9e9c008040 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 msgr2=0x7f9eb414abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400b2c0 tx=0x7f9ea40046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 shutdown_connections 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f9e9406be30 0x7f9e9406e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9eb415c060 0x7f9eb414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9eb40d1200 0x7f9eb4155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.012+0000 7f9eb96dd640 1 --2- 172.21.15.99:0/2383137351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9eb406e5d0 0x7f9eb414abd0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.013+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 >> 172.21.15.99:0/2383137351 conn(0x7f9eb406bc10 msgr2=0x7f9eb40b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:03.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.013+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 shutdown_connections 2024-09-10T11:40:03.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.013+0000 7f9eb96dd640 1 -- 172.21.15.99:0/2383137351 wait complete. 2024-09-10T11:40:03.094 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-09-10T11:40:03.094 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-09-10T11:40:03.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.190+0000 7f4a9df9f640 1 Processor -- start 2024-09-10T11:40:03.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.190+0000 7f4a9df9f640 1 -- start start 2024-09-10T11:40:03.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a9df9f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4a9815f1e0 0x7f4a980b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a9df9f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 0x7f4a980b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a9df9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a980b40e0 con 0x7f4a9815ee00 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a9df9f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f4a980b5ab0 con 0x7f4a9815f1e0 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a9df9f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f4a98073c50 con 0x7f4a980b0090 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4a9815f1e0 0x7f4a980b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4a9815f1e0 0x7f4a980b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33980/0 (socket says 172.21.15.99:33980) 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 -- 172.21.15.99:0/4031202241 learned_addr learned my addr 172.21.15.99:0/4031202241 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a97fff640 1 --2- 172.21.15.99:0/4031202241 >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 0x7f4a980b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 -- 172.21.15.99:0/4031202241 >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 msgr2=0x7f4a980b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:03.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 --2- 172.21.15.99:0/4031202241 >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 0x7f4a980b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 -- 172.21.15.99:0/4031202241 >> v1:172.21.15.196:6789/0 conn(0x7f4a9815ee00 legacy=0x7f4a980b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a96ffd640 1 -- 172.21.15.99:0/4031202241 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a980b4380 con 0x7f4a9815f1e0 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.191+0000 7f4a97fff640 1 --2- 172.21.15.99:0/4031202241 >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 0x7f4a980b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a96ffd640 1 --2- 172.21.15.99:0/4031202241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a980b5f90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4a98070c30 tx=0x7f4a8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b01e18c4519141 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a967fc640 1 -- 172.21.15.99:0/4031202241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c037070 con 0x7f4a9815f1e0 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a967fc640 1 -- 172.21.15.99:0/4031202241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8c032440 con 0x7f4a9815f1e0 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a967fc640 1 -- 172.21.15.99:0/4031202241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c032810 con 0x7f4a9815f1e0 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a9df9f640 1 -- 172.21.15.99:0/4031202241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 msgr2=0x7f4a980b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/4031202241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a980b5f90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4a98070c30 tx=0x7f4a8c02a230 comp rx=0 tx=0).stop 2024-09-10T11:40:03.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a9df9f640 1 -- 172.21.15.99:0/4031202241 shutdown_connections 2024-09-10T11:40:03.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/4031202241 >> v2:172.21.15.196:3300/0 conn(0x7f4a980b0090 0x7f4a980b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.192+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/4031202241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a980b5f90 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.193+0000 7f4a9df9f640 1 -- 172.21.15.99:0/4031202241 >> 172.21.15.99:0/4031202241 conn(0x7f4a980730c0 msgr2=0x7f4a980734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:03.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.193+0000 7f4a9df9f640 1 -- 172.21.15.99:0/4031202241 shutdown_connections 2024-09-10T11:40:03.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.193+0000 7f4a9df9f640 1 -- 172.21.15.99:0/4031202241 wait complete. 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.194+0000 7f4a9df9f640 1 Processor -- start 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.194+0000 7f4a9df9f640 1 -- start start 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980b0090 0x7f4a980ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9815ee00 0x7f4a981583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4a980b21b0 con 0x7f4a9815f1e0 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a980afd60 con 0x7f4a980b0090 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4a980b0600 con 0x7f4a9815ee00 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33984/0 (socket says 172.21.15.99:33984) 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 -- 172.21.15.99:0/1359216156 learned_addr learned my addr 172.21.15.99:0/1359216156 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a96ffd640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9815ee00 0x7f4a981583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a977fe640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980b0090 0x7f4a980ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 -- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9815ee00 msgr2=0x7f4a981583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9815ee00 0x7f4a981583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 -- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980b0090 msgr2=0x7f4a980ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980b0090 0x7f4a980ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 -- 172.21.15.99:0/1359216156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a980cadd0 con 0x7f4a9815f1e0 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a97fff640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cef0 tx=0x7f4a9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9001a070 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9000d860 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000dc50 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a980cb0b0 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.195+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a980cb670 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.196+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a980cbe30 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.197+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a9001d070 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.197+0000 7f4a7ffff640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 0x7f4a8406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.197+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f4a9008b6e0 con 0x7f4a9815f1e0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.197+0000 7f4a977fe640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 0x7f4a8406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.198+0000 7f4a977fe640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 0x7f4a8406e320 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a88004f60 tx=0x7f4a88007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.202+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a980cbe30 con 0x7f4a9815f1e0 2024-09-10T11:40:03.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.370+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f4a980b5f90 con 0x7f4a9815f1e0 2024-09-10T11:40:03.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.371+0000 7f4a7ffff640 1 -- 172.21.15.99:0/1359216156 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v186) ==== 81+0+34278 (secure 0 0 0) 0x7f4a90059a70 con 0x7f4a9815f1e0 2024-09-10T11:40:03.373 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:03.373 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "epoch": 186, 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "created": "2024-09-10T11:30:10.853690+0000", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "modified": "2024-09-10T11:40:00.722829+0000", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "last_up_change": "2024-09-10T11:30:15.930782+0000", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "last_in_change": "2024-09-10T11:30:12.922527+0000", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_num": 5799936, 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_set": [ 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "pglog_hardlimit", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "purged_snapdirs", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_deletes", 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: "sortbitwise" 2024-09-10T11:40:03.374 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_version": 4, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "full_ratio": 0.69999998807907104, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "nearfull_ratio": 0.60000002384185791, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_snapshot": "", 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_max": 23, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "max_osd": 8, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "require_min_compat_client": "luminous", 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "min_compat_client": "jewel", 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "require_osd_release": "squid", 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "allow_crimson": false, 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "pools": [ 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.375 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 1, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": ".mgr", 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:30:16.935517+0000", 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.376 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 1, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 1, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 1, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 1, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 1, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.377 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "111", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.379 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.380 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_max": 32, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 1 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": {} 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 8, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 8, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 0.25, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 2, 2024-09-10T11:40:03.381 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 2, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 22, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_metadata", 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.255122+0000", 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.382 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 32, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 32, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 32, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 32, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 32, 2024-09-10T11:40:03.383 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "182", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.384 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 0, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.385 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.386 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_bias": 4, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 16, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_priority": 5 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.387 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": "cephfs" 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.5, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.5, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.5, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.5, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.388 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pool": 23, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_name": "cephfs_data", 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "create_time": "2024-09-10T11:39:33.614490+0000", 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags": 1, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flags_names": "hashpspool", 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type": 1, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 2, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "min_size": 1, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_rule": 0, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_count": 0, 2024-09-10T11:40:03.389 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_target": 0, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_barrier": 0, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "is_stretch_pool": false, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "object_hash": 2, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_autoscale_mode": "off", 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num": 64, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num": 64, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_placement_num_target": 64, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_target": 64, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_pending": 64, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_pg_merge_meta": { 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "source_pgid": "0.0", 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "ready_epoch": 0, 2024-09-10T11:40:03.390 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_started": 0, 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "last_epoch_clean": 0, 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "source_version": "0'0", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "target_version": "0'0" 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "last_change": "186", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend": "0", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "last_force_op_resend_preluminous": "0", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "auid": 0, 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_mode": "selfmanaged", 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_seq": 0, 2024-09-10T11:40:03.391 INFO:teuthology.orchestra.run.smithi099.stdout: "snap_epoch": 0, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "pool_snaps": [], 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps": "[]", 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_bytes": 33554432, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "quota_max_objects": 0, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tiers": [], 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tier_of": -1, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "read_tier": -1, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "write_tier": -1, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_mode": "none", 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_bytes": 0, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "target_max_objects": 0, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-10T11:40:03.392 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_flush_age": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "cache_min_evict_age": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profile": "", 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_params": { 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "none" 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_period": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_count": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "use_gmt_hitset": true, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min_read_recency_for_promote": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "min_write_recency_for_promote": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_grade_decay_rate": 0, 2024-09-10T11:40:03.393 INFO:teuthology.orchestra.run.smithi099.stdout: "hit_set_search_last_n": 0, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "grade_table": [], 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "stripe_width": 0, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "expected_num_objects": 0, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "fast_read": false, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "options": { 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_num_min": 64, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "target_size_ratio": 0.89999997615814209 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "application_metadata": { 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "cephfs": { 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "data": "cephfs" 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "read_balance": { 2024-09-10T11:40:03.394 INFO:teuthology.orchestra.run.smithi099.stdout: "score_type": "Fair distribution", 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "score_acting": 1.3799999952316284, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "score_stable": 1.3799999952316284, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "optimal_score": 1, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity_weighted": 1, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity": 1, 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "average_primary_affinity_weighted": 1 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "osds": [ 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.395 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.396 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6800", 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6801", 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.397 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6802", 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6803", 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.398 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6806", 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6807", 2024-09-10T11:40:03.399 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6804", 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6805", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 291490777 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6801/291490777", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6803/291490777", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6807/291490777", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6805/291490777", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.402 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6808", 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6809", 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.403 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6810", 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6811", 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.404 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6814", 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6815", 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.405 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6812", 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6813", 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2120836022 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.406 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6809/2120836022", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6811/2120836022", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6815/2120836022", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6813/2120836022", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.407 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6817", 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6819", 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6822", 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6823", 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6830", 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6831", 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.411 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6825", 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6828", 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1090025270 2024-09-10T11:40:03.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6819/1090025270", 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6823/1090025270", 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6831/1090025270", 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6828/1090025270", 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.413 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6816", 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6818", 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6820", 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6821", 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6827", 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6829", 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.417 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6824", 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.99:6826", 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1027928425 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.99:6818/1027928425", 2024-09-10T11:40:03.418 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.99:6821/1027928425", 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.99:6829/1027928425", 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.99:6826/1027928425", 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.419 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6800", 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6801", 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6802", 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6803", 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6807", 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6809", 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6804", 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6805", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3885371275 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6801/3885371275", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6803/3885371275", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6809/3885371275", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6805/3885371275", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.424 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.425 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6824", 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6825", 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.426 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6826", 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6827", 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.427 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6830", 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.428 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6831", 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6828", 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.429 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6829", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3797029994 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6825/3797029994", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6827/3797029994", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6831/3797029994", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6829/3797029994", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.431 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6806", 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6808", 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.432 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6810", 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6811", 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6814", 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6815", 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6812", 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6813", 2024-09-10T11:40:03.435 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4190591731 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6808/4190591731", 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6811/4190591731", 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6815/4190591731", 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6813/4190591731", 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.436 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up": 1, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "in": 1, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "weight": 1, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_affinity": 1, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_begin": 0, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "last_clean_end": 0, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up_from": 13, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "up_thru": 181, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "down_at": 0, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "lost_at": 0, 2024-09-10T11:40:03.437 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addrs": { 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6816", 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6817", 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.438 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addrs": { 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6818", 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6819", 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.439 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addrs": { 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6822", 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6823", 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addrs": { 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addrvec": [ 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v2", 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6820", 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "172.21.15.196:6821", 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 106140719 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.441 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "public_addr": "172.21.15.196:6817/106140719", 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_addr": "172.21.15.196:6819/106140719", 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_back_addr": "172.21.15.196:6823/106140719", 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "heartbeat_front_addr": "172.21.15.196:6821/106140719", 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "state": [ 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "exists", 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "up" 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_xinfo": [ 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.442 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:13.995895+0000", 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 1, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.443 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.035476+0000", 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 2, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.058077+0000", 2024-09-10T11:40:03.444 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 3, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.028749+0000", 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.445 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 4, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.009275+0000", 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 5, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.446 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.060859+0000", 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 6, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.094664+0000", 2024-09-10T11:40:03.447 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": 7, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "down_stamp": "0.000000", 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_probability": 0, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "laggy_interval": 0, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "features": 4540701547738038271, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "old_weight": 0, 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "last_purged_snaps_scrub": "2024-09-10T11:30:14.020827+0000", 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "dead_epoch": 0 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap": [], 2024-09-10T11:40:03.448 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_items": [], 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_upmap_primaries": [], 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "pg_temp": [], 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "primary_temp": [], 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklist": {}, 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "range_blocklist": {}, 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "erasure_code_profiles": { 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "default": { 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "crush-failure-domain": "osd", 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "k": "2", 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "m": "1", 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "plugin": "jerasure", 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: "technique": "reed_sol_van" 2024-09-10T11:40:03.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "removed_snaps_queue": [], 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "new_removed_snaps": [], 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "new_purged_snaps": [], 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "crush_node_flags": {}, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "device_class_flags": {}, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode": { 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_enabled": false, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_bucket_count": 0, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "degraded_stretch_mode": 0, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "recovering_stretch_mode": 0, 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: "stretch_mode_bucket": 0 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-10T11:40:03.450 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 msgr2=0x7f4a8406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 0x7f4a8406e320 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4a88004f60 tx=0x7f4a88007e30 comp rx=0 tx=0).stop 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 msgr2=0x7f4a98158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000cef0 tx=0x7f4a9000a710 comp rx=0 tx=0).stop 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 shutdown_connections 2024-09-10T11:40:03.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4a8406be60 0x7f4a8406e320 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4a9815f1e0 0x7f4a98158900 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4a9815ee00 0x7f4a981583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 --2- 172.21.15.99:0/1359216156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a980b0090 0x7f4a980ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.373+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 >> 172.21.15.99:0/1359216156 conn(0x7f4a980730c0 msgr2=0x7f4a9815d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.374+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 shutdown_connections 2024-09-10T11:40:03.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.374+0000 7f4a9df9f640 1 -- 172.21.15.99:0/1359216156 wait complete. 2024-09-10T11:40:03.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:03.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.549+0000 7fddecfa6640 1 Processor -- start 2024-09-10T11:40:03.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.549+0000 7fddecfa6640 1 -- start start 2024-09-10T11:40:03.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 0x7fdde8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde806e5d0 0x7fdde806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde815c060 0x7fdde815c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fdde806ef50 con 0x7fdde815c060 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fdde8152e60 con 0x7fdde80d1200 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.550+0000 7fddecfa6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdde80d15e0 con 0x7fdde806e5d0 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde815c060 0x7fdde815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdde815c060 0x7fdde815c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33996/0 (socket says 172.21.15.99:33996) 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 -- 172.21.15.99:0/37561779 learned_addr learned my addr 172.21.15.99:0/37561779 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde659c640 1 --2- 172.21.15.99:0/37561779 >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 0x7fdde8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde5d9b640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde806e5d0 0x7fdde806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 -- 172.21.15.99:0/37561779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde806e5d0 msgr2=0x7fdde806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde806e5d0 0x7fdde806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 -- 172.21.15.99:0/37561779 >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 msgr2=0x7fdde8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:03.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 --2- 172.21.15.99:0/37561779 >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 0x7fdde8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 -- 172.21.15.99:0/37561779 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde8072680 con 0x7fdde815c060 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde659c640 1 --2- 172.21.15.99:0/37561779 >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 0x7fdde8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.551+0000 7fdde6d9d640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde815c060 0x7fdde815c480 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdde80d1850 tx=0x7fddd802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c271db807892e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fdde559a640 1 -- 172.21.15.99:0/37561779 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8037070 con 0x7fdde815c060 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fdde559a640 1 -- 172.21.15.99:0/37561779 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddd8032440 con 0x7fdde815c060 2024-09-10T11:40:03.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fdde559a640 1 -- 172.21.15.99:0/37561779 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8032810 con 0x7fdde815c060 2024-09-10T11:40:03.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 -- 172.21.15.99:0/37561779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde815c060 msgr2=0x7fdde815c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde815c060 0x7fdde815c480 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdde80d1850 tx=0x7fddd802a140 comp rx=0 tx=0).stop 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 -- 172.21.15.99:0/37561779 shutdown_connections 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde815c060 0x7fdde815c480 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 --2- 172.21.15.99:0/37561779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde806e5d0 0x7fdde806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 --2- 172.21.15.99:0/37561779 >> v2:172.21.15.196:3300/0 conn(0x7fdde80d1200 0x7fdde8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 -- 172.21.15.99:0/37561779 >> 172.21.15.99:0/37561779 conn(0x7fdde806bc10 msgr2=0x7fdde806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 -- 172.21.15.99:0/37561779 shutdown_connections 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.552+0000 7fddecfa6640 1 -- 172.21.15.99:0/37561779 wait complete. 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 Processor -- start 2024-09-10T11:40:03.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 -- start start 2024-09-10T11:40:03.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 0x7fdde8143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde815c060 0x7fdde81587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8072f20 con 0x7fdde80d1200 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8072c40 con 0x7fdde806e5d0 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fddecfa6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdde815cd90 con 0x7fdde815c060 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fdde5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.554+0000 7fdde5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34002/0 (socket says 172.21.15.99:34002) 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 -- 172.21.15.99:0/790077095 learned_addr learned my addr 172.21.15.99:0/790077095 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde6d9d640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde815c060 0x7fdde81587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde659c640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 0x7fdde8143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 -- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde815c060 msgr2=0x7fdde81587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde815c060 0x7fdde81587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 -- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 msgr2=0x7fdde8143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 0x7fdde8143200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 -- 172.21.15.99:0/790077095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde8143740 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde659c640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 0x7fdde8143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fdde5d9b640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007e40 tx=0x7fdddc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc014620 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdddc019070 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde815aa80 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc014a00 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.555+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde815b040 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.556+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde815e900 con 0x7fdde80d1200 2024-09-10T11:40:03.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.557+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdddc014ba0 con 0x7fdde80d1200 2024-09-10T11:40:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.557+0000 7fddd6ffd640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 0x7fddc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.557+0000 7fdde659c640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 0x7fddc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.557+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7fdddc08b620 con 0x7fdde80d1200 2024-09-10T11:40:03.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.558+0000 7fdde659c640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 0x7fddc806e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006fd0 tx=0x7fdde0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:03.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.562+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdde815e900 con 0x7fdde80d1200 2024-09-10T11:40:03.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.758+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdde8071c50 con 0x7fdde80d1200 2024-09-10T11:40:03.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.758+0000 7fddd6ffd640 1 -- 172.21.15.99:0/790077095 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7fdde8071c50 con 0x7fdde80d1200 2024-09-10T11:40:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:40:03.761 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:40:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 msgr2=0x7fddc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 0x7fddc806e2f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdde0006fd0 tx=0x7fdde0008040 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 msgr2=0x7fdde81582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fdddc007e40 tx=0x7fdddc00e4b0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 shutdown_connections 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fddc806be30 0x7fddc806e2f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdde815c060 0x7fdde81587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdde80d1200 0x7fdde81582b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 --2- 172.21.15.99:0/790077095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdde806e5d0 0x7fdde8143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.760+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 >> 172.21.15.99:0/790077095 conn(0x7fdde806bc10 msgr2=0x7fdde815d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.761+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 shutdown_connections 2024-09-10T11:40:03.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:03.761+0000 7fddecfa6640 1 -- 172.21.15.99:0/790077095 wait complete. 2024-09-10T11:40:03.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-09-10T11:40:03.889 INFO:teuthology.orchestra.run.smithi099.stdout:{"cluster_fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","whoami":0,"id":7956,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.555812266,"mdsmap_epoch":74,"osdmap_epoch":186,"osdmap_epoch_barrier":185,"uptime":32.635226516000003} 2024-09-10T11:40:03.889 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "fb5864c6-feee-49d0-8ee4-0b85b9c6d632", "fs_name": "cephfs", "id": 7956, "mdsmap_epoch": 74, "osdmap_epoch": 186, "osdmap_epoch_barrier": 185, "rank_uptime": 28.555812266, "state": "up:active", "uptime": 32.635226516, "want_state": "up:active", "whoami": 0 } 2024-09-10T11:40:03.889 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T11:40:03.889 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:40:03.890 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:03.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:03.906 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-09-10T11:40:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records in 2024-09-10T11:40:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:50+0 records out 2024-09-10T11:40:04.175 INFO:teuthology.orchestra.run.smithi099.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.211863 s, 247 MB/s 2024-09-10T11:40:04.175 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T11:40:04.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:40:04.215 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T11:40:04.257 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T11:40:04.258 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:40:04.258 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:40:04.258 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:04.274 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:40:04.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.413+0000 7f80c16f8640 1 Processor -- start 2024-09-10T11:40:04.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.413+0000 7f80c16f8640 1 -- start start 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80c16f8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f80bc0d5850 0x7f80bc0d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80c16f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1b40 0x7f80bc0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80c16f8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80bc0d1f50 con 0x7f80bc15f020 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80c16f8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d3b40 con 0x7f80bc0d5850 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80c16f8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f80bc0d3e20 con 0x7f80bc0d1b40 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80b27fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f80bc0d5850 0x7f80bc0d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80baffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f80bc15f020 0x7f80bc0d32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41106/0 (socket says 172.21.15.99:41106) 2024-09-10T11:40:04.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80baffd640 1 -- 172.21.15.99:0/3724265183 learned_addr learned my addr 172.21.15.99:0/3724265183 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80bb7fe640 1 --2- 172.21.15.99:0/3724265183 >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1b40 0x7f80bc0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80b27fc640 1 -- 172.21.15.99:0/3724265183 >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1b40 msgr2=0x7f80bc0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80b27fc640 1 --2- 172.21.15.99:0/3724265183 >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1b40 0x7f80bc0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80b27fc640 1 -- 172.21.15.99:0/3724265183 >> v1:172.21.15.99:6789/0 conn(0x7f80bc15f020 legacy=0x7f80bc0d32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80b27fc640 1 -- 172.21.15.99:0/3724265183 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc0b4810 con 0x7f80bc0d5850 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.414+0000 7f80ba7fc640 1 -- 172.21.15.99:0/3724265183 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795215628 0 0) 0x7f80ac001110 con 0x7f80bc15f020 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.415+0000 7f80b27fc640 1 --2- 172.21.15.99:0/3724265183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0d1600 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0cbd70 tx=0x7f80a402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554457dc9eb8994c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.415+0000 7f80ba7fc640 1 -- 172.21.15.99:0/3724265183 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4037070 con 0x7f80bc0d5850 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.415+0000 7f80ba7fc640 1 -- 172.21.15.99:0/3724265183 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a4032440 con 0x7f80bc0d5850 2024-09-10T11:40:04.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.415+0000 7f80ba7fc640 1 -- 172.21.15.99:0/3724265183 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4032810 con 0x7f80bc0d5850 2024-09-10T11:40:04.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 -- 172.21.15.99:0/3724265183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 msgr2=0x7f80bc0d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 --2- 172.21.15.99:0/3724265183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0d1600 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0cbd70 tx=0x7f80a402a1a0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 -- 172.21.15.99:0/3724265183 shutdown_connections 2024-09-10T11:40:04.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 --2- 172.21.15.99:0/3724265183 >> v2:172.21.15.196:3300/0 conn(0x7f80bc0d1b40 0x7f80bc0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 --2- 172.21.15.99:0/3724265183 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0d1600 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 -- 172.21.15.99:0/3724265183 >> 172.21.15.99:0/3724265183 conn(0x7f80bc09dfa0 msgr2=0x7f80bc0d4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 -- 172.21.15.99:0/3724265183 shutdown_connections 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.416+0000 7f80c16f8640 1 -- 172.21.15.99:0/3724265183 wait complete. 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.417+0000 7f80c16f8640 1 Processor -- start 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.417+0000 7f80c16f8640 1 -- start start 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 0x7f80bc15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc15f020 0x7f80bc161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b5830 con 0x7f80bc0d5850 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f80bc0b4dd0 con 0x7f80bc15f020 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80c16f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f80bc0b59b0 con 0x7f80bc0d1b40 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34026/0 (socket says 172.21.15.99:34026) 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 -- 172.21.15.99:0/4261479789 learned_addr learned my addr 172.21.15.99:0/4261479789 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80bb7fe640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc15f020 0x7f80bc161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80baffd640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 0x7f80bc15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 -- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 msgr2=0x7f80bc15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 0x7f80bc15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 -- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc15f020 msgr2=0x7f80bc161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc15f020 0x7f80bc161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 -- 172.21.15.99:0/4261479789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80bc162330 con 0x7f80bc0d5850 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80baffd640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 0x7f80bc15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b27fc640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.418+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4037040 con 0x7f80bc0d5850 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.419+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a4042070 con 0x7f80bc0d5850 2024-09-10T11:40:04.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.419+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80bc15f530 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.419+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4030bd0 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.419+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80bc0ca0b0 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.420+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80bc0b52f0 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.420+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f80a4047460 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.420+0000 7f80b3fff640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 0x7f809c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.420+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f80a40ab970 con 0x7f80bc0d5850 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.420+0000 7f80baffd640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 0x7f809c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.421+0000 7f80baffd640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 0x7f809c06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000f60 tx=0x7f80ac007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.425+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f80a4032e20 con 0x7f80bc0d5850 2024-09-10T11:40:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.601+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80bc0b4450 con 0x7f80bc0d5850 2024-09-10T11:40:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.601+0000 7f80b3fff640 1 -- 172.21.15.99:0/4261479789 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v186) ==== 80+0+0 (secure 0 0 0) 0x7f80a407a3b0 con 0x7f80bc0d5850 2024-09-10T11:40:04.603 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 msgr2=0x7f809c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 0x7f809c06e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f80ac000f60 tx=0x7f80ac007ca0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 msgr2=0x7f80bc0c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000c00 tx=0x7f80a40046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 shutdown_connections 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f809c06be30 0x7f809c06e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80bc15f020 0x7f80bc161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80bc0d5850 0x7f80bc0c9b70 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 --2- 172.21.15.99:0/4261479789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80bc0d1b40 0x7f80bc15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 >> 172.21.15.99:0/4261479789 conn(0x7f80bc09dfa0 msgr2=0x7f80bc06d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 shutdown_connections 2024-09-10T11:40:04.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.603+0000 7f80c16f8640 1 -- 172.21.15.99:0/4261479789 wait complete. 2024-09-10T11:40:04.615 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-09-10T11:40:04.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:04.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T11:40:04.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[63134]: fuse finished with error 0 and tester_r 0 2024-09-10T11:40:04.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:04.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:04.645 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T11:40:04.785 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:40:04.786 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:40:04.786 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:04.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T11:40:04.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.903+0000 7f0dd53e8640 1 Processor -- start 2024-09-10T11:40:04.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.903+0000 7f0dd53e8640 1 -- start start 2024-09-10T11:40:04.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dd53e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0dd0071400 0x7f0dd01678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dd53e8640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd00d18d0 con 0x7f0dd00ce0e0 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dd53e8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dd00ce6c0 con 0x7f0dd01603c0 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dd53e8640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f0dd00ce9a0 con 0x7f0dd0071400 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dc67fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0dd00ce0e0 0x7f0dd00d11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41118/0 (socket says 172.21.15.99:41118) 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dcf7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0dd0071400 0x7f0dd01678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.904+0000 7f0dc67fc640 1 -- 172.21.15.99:0/1599057437 learned_addr learned my addr 172.21.15.99:0/1599057437 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1599057437 >> v1:172.21.15.196:6789/0 conn(0x7f0dd01603c0 legacy=0x7f0dd00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1599057437 >> v1:172.21.15.99:6789/0 conn(0x7f0dd00ce0e0 legacy=0x7f0dd00d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1599057437 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156040755 0 0) 0x7f0dd00d18d0 con 0x7f0dd00ce0e0 2024-09-10T11:40:04.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dcf7fe640 1 -- 172.21.15.99:0/1599057437 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd006f910 con 0x7f0dd0071400 2024-09-10T11:40:04.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/1599057437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd01678e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009d90 tx=0x7f0dc802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b3eb35155f6d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1599057437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007740 con 0x7f0dd0071400 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1599057437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc80078e0 con 0x7f0dd0071400 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.905+0000 7f0dce7fc640 1 -- 172.21.15.99:0/1599057437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8007cb0 con 0x7f0dd0071400 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 -- 172.21.15.99:0/1599057437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 msgr2=0x7f0dd01678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/1599057437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd01678e0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009d90 tx=0x7f0dc802a580 comp rx=0 tx=0).stop 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 -- 172.21.15.99:0/1599057437 shutdown_connections 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/1599057437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd01678e0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 -- 172.21.15.99:0/1599057437 >> 172.21.15.99:0/1599057437 conn(0x7f0dd00b45c0 msgr2=0x7f0dd00b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 -- 172.21.15.99:0/1599057437 shutdown_connections 2024-09-10T11:40:04.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.906+0000 7f0dd53e8640 1 -- 172.21.15.99:0/1599057437 wait complete. 2024-09-10T11:40:04.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.907+0000 7f0dd53e8640 1 Processor -- start 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.907+0000 7f0dd53e8640 1 -- start start 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 0x7f0dd00b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd01603c0 0x7f0dd00cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0167e20 con 0x7f0dd0071400 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd006fed0 con 0x7f0dd00ce0e0 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dd53e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd0167fa0 con 0x7f0dd01603c0 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34048/0 (socket says 172.21.15.99:34048) 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 -- 172.21.15.99:0/723048790 learned_addr learned my addr 172.21.15.99:0/723048790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dc67fc640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 0x7f0dd00b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dcf7fe640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd01603c0 0x7f0dd00cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 -- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd01603c0 msgr2=0x7f0dd00cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd01603c0 0x7f0dd00cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 -- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 msgr2=0x7f0dd00b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 0x7f0dd00b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dceffd640 1 -- 172.21.15.99:0/723048790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd014c530 con 0x7f0dd0071400 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.908+0000 7f0dc67fc640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 0x7f0dd00b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dceffd640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000b3f0 tx=0x7f0dc000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0015020 con 0x7f0dd0071400 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc0011830 con 0x7f0dd0071400 2024-09-10T11:40:04.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd0153b10 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011c20 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.909+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd0153db0 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.910+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd00761a0 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.910+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0dc0018020 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.911+0000 7f0dc7fff640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 0x7f0db006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.911+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f0dc008aa60 con 0x7f0dd0071400 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.911+0000 7f0dc67fc640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 0x7f0db006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:04.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.911+0000 7f0dc67fc640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 0x7f0db006e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0dd01f6280 tx=0x7f0db8007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:04.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:04.915+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0dc001f030 con 0x7f0dd0071400 2024-09-10T11:40:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.092+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0dd00cf220 con 0x7f0dd0071400 2024-09-10T11:40:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.093+0000 7f0dc7fff640 1 -- 172.21.15.99:0/723048790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v186) ==== 80+0+0 (secure 0 0 0) 0x7f0dd014c530 con 0x7f0dd0071400 2024-09-10T11:40:05.095 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 msgr2=0x7f0db006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 0x7f0db006e2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0dd01f6280 tx=0x7f0db8007c10 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 msgr2=0x7f0dd014bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000b3f0 tx=0x7f0dc000b8b0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 shutdown_connections 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f0db006be30 0x7f0db006e2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dd01603c0 0x7f0dd00cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dd00ce0e0 0x7f0dd00b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 --2- 172.21.15.99:0/723048790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0dd0071400 0x7f0dd014bff0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 >> 172.21.15.99:0/723048790 conn(0x7f0dd00b45c0 msgr2=0x7f0dd0155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 shutdown_connections 2024-09-10T11:40:05.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.095+0000 7f0dd53e8640 1 -- 172.21.15.99:0/723048790 wait complete. 2024-09-10T11:40:05.108 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-09-10T11:40:05.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:05.108 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-10T11:40:05.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[40256]: fuse finished with error 0 and tester_r 0 2024-09-10T11:40:05.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:05.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:05.139 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T11:40:05.296 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:40:05.296 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:40:05.296 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:05.315 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:40:05.315 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:05.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:05.315 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T11:40:05.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:05.370 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-10T11:40:05.370 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T11:40:05.370 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:40:05.370 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:05.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:40:05.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:05.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:05.389 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T11:40:05.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:05.444 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-10T11:40:05.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:05.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.540+0000 7f29e8b2a640 1 Processor -- start 2024-09-10T11:40:05.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.540+0000 7f29e8b2a640 1 -- start start 2024-09-10T11:40:05.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e8b2a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e8b2a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f29e415c770 0x7f29e406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e8b2a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e4152e60 con 0x7f29e406e5d0 2024-09-10T11:40:05.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e8b2a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f29e406ef50 con 0x7f29e415c770 2024-09-10T11:40:05.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e8b2a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f29e40d15e0 con 0x7f29e40d1200 2024-09-10T11:40:05.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e1d9b640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f29e406e5d0 0x7f29e415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:48598/0 (socket says 172.21.15.99:48598) 2024-09-10T11:40:05.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e2d9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f29e415c770 0x7f29e406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.541+0000 7f29e1d9b640 1 -- 172.21.15.99:0/68941299 learned_addr learned my addr 172.21.15.99:0/68941299 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:05.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e159a640 1 -- 172.21.15.99:0/68941299 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 373143951 0 0) 0x7f29e4152e60 con 0x7f29e406e5d0 2024-09-10T11:40:05.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e159a640 1 -- 172.21.15.99:0/68941299 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29cc005730 con 0x7f29e406e5d0 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e2d9d640 1 -- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 msgr2=0x7f29e4073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e2d9d640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e2d9d640 1 -- 172.21.15.99:0/68941299 >> v1:172.21.15.196:6789/0 conn(0x7f29e406e5d0 legacy=0x7f29e415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e2d9d640 1 -- 172.21.15.99:0/68941299 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e4072680 con 0x7f29e415c770 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.542+0000 7f29e259c640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e2d9d640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e406ea10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d73eae2b4b42085a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e159a640 1 -- 172.21.15.99:0/68941299 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc031600 con 0x7f29e415c770 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e159a640 1 -- 172.21.15.99:0/68941299 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc0317a0 con 0x7f29e415c770 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e159a640 1 -- 172.21.15.99:0/68941299 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc031b50 con 0x7f29e415c770 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 -- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 msgr2=0x7f29e406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e406ea10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009d90 tx=0x7f29dc02a5d0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 -- 172.21.15.99:0/68941299 shutdown_connections 2024-09-10T11:40:05.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e406ea10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/68941299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 -- 172.21.15.99:0/68941299 >> 172.21.15.99:0/68941299 conn(0x7f29e406bc10 msgr2=0x7f29e406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 -- 172.21.15.99:0/68941299 shutdown_connections 2024-09-10T11:40:05.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.543+0000 7f29e8b2a640 1 -- 172.21.15.99:0/68941299 wait complete. 2024-09-10T11:40:05.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 Processor -- start 2024-09-10T11:40:05.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 -- start start 2024-09-10T11:40:05.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e41442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4072f20 con 0x7f29e406e5d0 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f29e4072c40 con 0x7f29e415c770 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e8b2a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f29e415cd90 con 0x7f29e40d1200 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.545+0000 7f29e259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34054/0 (socket says 172.21.15.99:34054) 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 -- 172.21.15.99:0/3480147442 learned_addr learned my addr 172.21.15.99:0/3480147442 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e1d9b640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e2d9d640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e41442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 -- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 msgr2=0x7f29e4142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 -- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 msgr2=0x7f29e41442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e41442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 -- 172.21.15.99:0/3480147442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41413b0 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e259c640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f29d400e990 tx=0x7f29d400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4017070 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d400c9e0 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e4141690 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d400cdb0 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.546+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e4141c50 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.547+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4071c50 con 0x7f29e406e5d0 2024-09-10T11:40:05.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.548+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f29d401a070 con 0x7f29e406e5d0 2024-09-10T11:40:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.548+0000 7f29d2ffd640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 0x7f29c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.548+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f29d408af70 con 0x7f29e406e5d0 2024-09-10T11:40:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.548+0000 7f29e1d9b640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 0x7f29c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.549+0000 7f29e1d9b640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 0x7f29c406e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f29e4151e80 tx=0x7f29d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.552+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f29e4071c50 con 0x7f29e406e5d0 2024-09-10T11:40:05.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.751+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29e4073440 con 0x7f29e406e5d0 2024-09-10T11:40:05.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.752+0000 7f29d2ffd640 1 -- 172.21.15.99:0/3480147442 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f29d4059360 con 0x7f29e406e5d0 2024-09-10T11:40:05.754 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:05.754 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:40:05.754 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:40:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 msgr2=0x7f29c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 0x7f29c406e2f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f29e4151e80 tx=0x7f29d8009040 comp rx=0 tx=0).stop 2024-09-10T11:40:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 msgr2=0x7f29e40b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f29d400e990 tx=0x7f29d400ee50 comp rx=0 tx=0).stop 2024-09-10T11:40:05.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 shutdown_connections 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f29c406be30 0x7f29c406e2f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29e415c770 0x7f29e41442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29e40d1200 0x7f29e4142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 --2- 172.21.15.99:0/3480147442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29e406e5d0 0x7f29e40b0ce0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 >> 172.21.15.99:0/3480147442 conn(0x7f29e406bc10 msgr2=0x7f29e40cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:05.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 shutdown_connections 2024-09-10T11:40:05.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.755+0000 7f29e8b2a640 1 -- 172.21.15.99:0/3480147442 wait complete. 2024-09-10T11:40:05.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.863+0000 7febd816f640 1 Processor -- start 2024-09-10T11:40:05.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.863+0000 7febd816f640 1 -- start start 2024-09-10T11:40:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd816f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febd00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd816f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd006ee10 con 0x7febd00afbb0 2024-09-10T11:40:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd816f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd00afff0 con 0x7febd015b2d0 2024-09-10T11:40:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd816f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7febd00cd9a0 con 0x7febd015aef0 2024-09-10T11:40:05.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd670c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7febd00afbb0 0x7febd00714a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41120/0 (socket says 172.21.15.99:41120) 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd670c640 1 -- 172.21.15.99:0/641228637 learned_addr learned my addr 172.21.15.99:0/641228637 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd5f0b640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febd00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786005897 0 0) 0x7febd006ee10 con 0x7febd00afbb0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febbc003440 con 0x7febd00afbb0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744240023 0 0) 0x7febd00afff0 con 0x7febd015b2d0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd006ee10 con 0x7febd015b2d0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.864+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2512555951 0 0) 0x7febbc003440 con 0x7febd00afbb0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 msgr2=0x7febd00b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febd00b4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 >> v1:172.21.15.196:6789/0 conn(0x7febd015b2d0 legacy=0x7febd00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd00b0290 con 0x7febd00afbb0 2024-09-10T11:40:05.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7febc80019f0 con 0x7febd00afbb0 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 >> v1:172.21.15.99:6789/0 conn(0x7febd00afbb0 legacy=0x7febd00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febbc003c60 0x7febbc008850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc004160 0x7febbc00ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febbc00d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7febd00cd9a0 con 0x7febbc003c60 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febbc00dc60 con 0x7febbc004160 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7febbc00de60 con 0x7febd015aef0 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febbc003c60 0x7febbc008850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd570a640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc004160 0x7febbc00ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd670c640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febbc00d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 -- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 msgr2=0x7febbc00d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febd015aef0 0x7febbc00d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 -- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc004160 msgr2=0x7febbc00ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc004160 0x7febbc00ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.865+0000 7febd5f0b640 1 -- 172.21.15.99:0/641228637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc003440 con 0x7febbc003c60 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd5f0b640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febbc003c60 0x7febbc008850 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7febcc009ac0 tx=0x7febbc003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dcab18df57953a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc036070 con 0x7febbc003c60 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc031400 con 0x7febbc003c60 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd4f09640 1 -- 172.21.15.99:0/641228637 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc009690 con 0x7febbc003c60 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd816f640 1 -- 172.21.15.99:0/641228637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febbc003c60 msgr2=0x7febbc008850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd816f640 1 --2- 172.21.15.99:0/641228637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febbc003c60 0x7febbc008850 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7febcc009ac0 tx=0x7febbc003730 comp rx=0 tx=0).stop 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd670c640 1 -- 172.21.15.99:0/641228637 reap_dead start 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd816f640 1 -- 172.21.15.99:0/641228637 shutdown_connections 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.866+0000 7febd816f640 1 -- 172.21.15.99:0/641228637 >> 172.21.15.99:0/641228637 conn(0x7febd006bc10 msgr2=0x7febd006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.867+0000 7febd816f640 1 -- 172.21.15.99:0/641228637 shutdown_connections 2024-09-10T11:40:05.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.867+0000 7febd816f640 1 -- 172.21.15.99:0/641228637 wait complete. 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 Processor -- start 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 -- start start 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febbc003c60 0x7febd0150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 0x7febd014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7febd006eb30 con 0x7febd0153cd0 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febd006e850 con 0x7febd0154150 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.868+0000 7febd816f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7febd0071bb0 con 0x7febbc003c60 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34070/0 (socket says 172.21.15.99:34070) 2024-09-10T11:40:05.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 -- 172.21.15.99:0/2277662189 learned_addr learned my addr 172.21.15.99:0/2277662189 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd5f0b640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febbc003c60 0x7febd0150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd670c640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 0x7febd014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 -- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febbc003c60 msgr2=0x7febd0150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febbc003c60 0x7febd0150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 -- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 msgr2=0x7febd014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 0x7febd014f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 -- 172.21.15.99:0/2277662189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd0154c70 con 0x7febd0153cd0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd670c640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 0x7febd014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd570a640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7febc000f800 tx=0x7febc000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00052e0 con 0x7febd0153cd0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc0018070 con 0x7febd0153cd0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd01546b0 con 0x7febd0153cd0 2024-09-10T11:40:05.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000d970 con 0x7febd0153cd0 2024-09-10T11:40:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.869+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd014e820 con 0x7febd0153cd0 2024-09-10T11:40:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.870+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd00b4510 con 0x7febd0153cd0 2024-09-10T11:40:05.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.871+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febc000db10 con 0x7febd0153cd0 2024-09-10T11:40:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.871+0000 7febc67fc640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.871+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7febc008ad00 con 0x7febd0153cd0 2024-09-10T11:40:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.871+0000 7febd5f0b640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.872+0000 7febd5f0b640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 0x7febb406e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7febcc033000 tx=0x7febcc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:05.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:05.875+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7febc0059740 con 0x7febd0153cd0 2024-09-10T11:40:06.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.076+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd015a350 con 0x7febd0153cd0 2024-09-10T11:40:06.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.077+0000 7febc67fc640 1 -- 172.21.15.99:0/2277662189 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7febc0059040 con 0x7febd0153cd0 2024-09-10T11:40:06.079 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:06.079 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:40:06.079 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:40:06.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 msgr2=0x7febb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 0x7febb406e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7febcc033000 tx=0x7febcc008030 comp rx=0 tx=0).stop 2024-09-10T11:40:06.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 msgr2=0x7febd014f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7febc000f800 tx=0x7febc000fcc0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 shutdown_connections 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7febb406be30 0x7febb406e2f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febd0154150 0x7febd014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7febd0153cd0 0x7febd014f020 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 --2- 172.21.15.99:0/2277662189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7febbc003c60 0x7febd0150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 >> 172.21.15.99:0/2277662189 conn(0x7febd006bc10 msgr2=0x7febd00714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 shutdown_connections 2024-09-10T11:40:06.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.079+0000 7febd816f640 1 -- 172.21.15.99:0/2277662189 wait complete. 2024-09-10T11:40:06.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:40:06.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.188+0000 7fb2f3080640 1 Processor -- start 2024-09-10T11:40:06.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.188+0000 7fb2f3080640 1 -- start start 2024-09-10T11:40:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f3080640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2ec0cee30 0x7fb2ec0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f3080640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ec0cf240 con 0x7fb2ec15cc30 2024-09-10T11:40:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f3080640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ec0cea00 con 0x7fb2ec0cf410 2024-09-10T11:40:06.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f3080640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fb2ec0d1540 con 0x7fb2ec0cee30 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f161d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2ec0cee30 0x7fb2ec0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2ebfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb2ec15cc30 0x7fb2ec0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41122/0 (socket says 172.21.15.99:41122) 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2ebfff640 1 -- 172.21.15.99:0/770854567 learned_addr learned my addr 172.21.15.99:0/770854567 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f161d640 1 -- 172.21.15.99:0/770854567 >> v1:172.21.15.196:6789/0 conn(0x7fb2ec0cf410 legacy=0x7fb2ec0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f161d640 1 -- 172.21.15.99:0/770854567 >> v1:172.21.15.99:6789/0 conn(0x7fb2ec15cc30 legacy=0x7fb2ec0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2f161d640 1 -- 172.21.15.99:0/770854567 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec0d1770 con 0x7fb2ec0cee30 2024-09-10T11:40:06.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.189+0000 7fb2eb7fe640 1 -- 172.21.15.99:0/770854567 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2573903030 0 0) 0x7fb2ec0cf240 con 0x7fb2ec15cc30 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.190+0000 7fb2f161d640 1 --2- 172.21.15.99:0/770854567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0d1000 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e402a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad2771c93f3377 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.190+0000 7fb2eb7fe640 1 -- 172.21.15.99:0/770854567 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4007740 con 0x7fb2ec0cee30 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.190+0000 7fb2eb7fe640 1 -- 172.21.15.99:0/770854567 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e40078e0 con 0x7fb2ec0cee30 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.190+0000 7fb2eb7fe640 1 -- 172.21.15.99:0/770854567 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4007cb0 con 0x7fb2ec0cee30 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 -- 172.21.15.99:0/770854567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cee30 msgr2=0x7fb2ec0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 --2- 172.21.15.99:0/770854567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0d1000 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e402a580 comp rx=0 tx=0).stop 2024-09-10T11:40:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 -- 172.21.15.99:0/770854567 shutdown_connections 2024-09-10T11:40:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 --2- 172.21.15.99:0/770854567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0d1000 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 -- 172.21.15.99:0/770854567 >> 172.21.15.99:0/770854567 conn(0x7fb2ec0b4570 msgr2=0x7fb2ec0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 -- 172.21.15.99:0/770854567 shutdown_connections 2024-09-10T11:40:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.191+0000 7fb2f3080640 1 -- 172.21.15.99:0/770854567 wait complete. 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.192+0000 7fb2f3080640 1 Processor -- start 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.192+0000 7fb2f3080640 1 -- start start 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec15cc30 0x7fb2ec15f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0cd4c0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0cdaf0 con 0x7fb2ec0cee30 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f3080640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b5870 con 0x7fb2ec15cc30 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34086/0 (socket says 172.21.15.99:34086) 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 -- 172.21.15.99:0/3523330551 learned_addr learned my addr 172.21.15.99:0/3523330551 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f0e1c640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f161d640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec15cc30 0x7fb2ec15f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 -- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec15cc30 msgr2=0x7fb2ec15f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec15cc30 0x7fb2ec15f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 -- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 msgr2=0x7fb2ec0c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 -- 172.21.15.99:0/3523330551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec0cadf0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2f0e1c640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.193+0000 7fb2ebfff640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c3e0 tx=0x7fb2e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.194+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00074e0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.194+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e00047e0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.194+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004bd0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.194+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec0cb0d0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.194+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec0cb690 con 0x7fb2ec0cf410 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.195+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ec0d1000 con 0x7fb2ec0cf410 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.195+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb2e0011070 con 0x7fb2ec0cf410 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.195+0000 7fb2e97fa640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 0x7fb2d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.195+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7fb2e008a9f0 con 0x7fb2ec0cf410 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.195+0000 7fb2f0e1c640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 0x7fb2d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.196+0000 7fb2f0e1c640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 0x7fb2d806e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc003eb0 tx=0x7fb2dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.200+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2e0059430 con 0x7fb2ec0cf410 2024-09-10T11:40:06.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.370+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2ec070e30 con 0x7fb2ec0cf410 2024-09-10T11:40:06.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.371+0000 7fb2e97fa640 1 -- 172.21.15.99:0/3523330551 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+16747 (secure 0 0 0) 0x7fb2e0058d30 con 0x7fb2ec0cf410 2024-09-10T11:40:06.372 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:06.372 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":186,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:40:00.722829+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:39:33.255122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T11:39:33.614490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:40:06.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 msgr2=0x7fb2d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 0x7fb2d806e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc003eb0 tx=0x7fb2dc009040 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 msgr2=0x7fb2ec153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c3e0 tx=0x7fb2e000c8a0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 shutdown_connections 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fb2d806be30 0x7fb2d806e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec15cc30 0x7fb2ec15f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2ec0cf410 0x7fb2ec153da0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 --2- 172.21.15.99:0/3523330551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0cee30 0x7fb2ec0c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 >> 172.21.15.99:0/3523330551 conn(0x7fb2ec0b4570 msgr2=0x7fb2ec0b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 shutdown_connections 2024-09-10T11:40:06.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.373+0000 7fb2f3080640 1 -- 172.21.15.99:0/3523330551 wait complete. 2024-09-10T11:40:06.385 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:40:06.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:06.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.483+0000 7f72619b7640 1 Processor -- start 2024-09-10T11:40:06.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.483+0000 7f72619b7640 1 -- start start 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f72619b7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f725c15eb80 0x7f725c0d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f72619b7640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c15ff70 con 0x7f725c1626c0 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f72619b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c0d1620 con 0x7f725c0d5010 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f72619b7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f725c162bc0 con 0x7f725c15eb80 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f7253fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f725c1626c0 0x7f725c0d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41136/0 (socket says 172.21.15.99:41136) 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f7253fff640 1 -- 172.21.15.99:0/2230490600 learned_addr learned my addr 172.21.15.99:0/2230490600 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:06.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725affd640 1 --2- 172.21.15.99:0/2230490600 >> v2:172.21.15.196:3300/0 conn(0x7f725c15eb80 0x7f725c0d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725a7fc640 1 -- 172.21.15.99:0/2230490600 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422560350 0 0) 0x7f725c15ff70 con 0x7f725c1626c0 2024-09-10T11:40:06.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725a7fc640 1 -- 172.21.15.99:0/2230490600 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7248003520 con 0x7f725c1626c0 2024-09-10T11:40:06.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725affd640 1 -- 172.21.15.99:0/2230490600 >> v1:172.21.15.196:6789/0 conn(0x7f725c0d5010 legacy=0x7f725c0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725affd640 1 -- 172.21.15.99:0/2230490600 >> v1:172.21.15.99:6789/0 conn(0x7f725c1626c0 legacy=0x7f725c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.484+0000 7f725affd640 1 -- 172.21.15.99:0/2230490600 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c162ea0 con 0x7f725c15eb80 2024-09-10T11:40:06.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.485+0000 7f725affd640 1 --2- 172.21.15.99:0/2230490600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c15eb80 0x7f725c0d29b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f724c0097d0 tx=0x7f724c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2edb2e35b4016f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.485+0000 7f725a7fc640 1 -- 172.21.15.99:0/2230490600 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c037070 con 0x7f725c15eb80 2024-09-10T11:40:06.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.485+0000 7f725a7fc640 1 -- 172.21.15.99:0/2230490600 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c032440 con 0x7f725c15eb80 2024-09-10T11:40:06.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.485+0000 7f725a7fc640 1 -- 172.21.15.99:0/2230490600 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0327f0 con 0x7f725c15eb80 2024-09-10T11:40:06.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 -- 172.21.15.99:0/2230490600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c15eb80 msgr2=0x7f725c0d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 --2- 172.21.15.99:0/2230490600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c15eb80 0x7f725c0d29b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f724c0097d0 tx=0x7f724c02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 -- 172.21.15.99:0/2230490600 shutdown_connections 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 --2- 172.21.15.99:0/2230490600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c15eb80 0x7f725c0d29b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 -- 172.21.15.99:0/2230490600 >> 172.21.15.99:0/2230490600 conn(0x7f725c08e680 msgr2=0x7f725c09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 -- 172.21.15.99:0/2230490600 shutdown_connections 2024-09-10T11:40:06.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.486+0000 7f72619b7640 1 -- 172.21.15.99:0/2230490600 wait complete. 2024-09-10T11:40:06.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.487+0000 7f72619b7640 1 Processor -- start 2024-09-10T11:40:06.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.487+0000 7f72619b7640 1 -- start start 2024-09-10T11:40:06.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f725c0d5010 0x7f725c1f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c1626c0 0x7f725c0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0d2ef0 con 0x7f725c15eb80 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f725c163460 con 0x7f725c1626c0 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f72619b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f725c0d3070 con 0x7f725c0d5010 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34092/0 (socket says 172.21.15.99:34092) 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 -- 172.21.15.99:0/283281327 learned_addr learned my addr 172.21.15.99:0/283281327 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f725affd640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f725c0d5010 0x7f725c1f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f725b7fe640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c1626c0 0x7f725c0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 -- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f725c0d5010 msgr2=0x7f725c1f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f725c0d5010 0x7f725c1f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 -- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c1626c0 msgr2=0x7f725c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c1626c0 0x7f725c0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.488+0000 7f7253fff640 1 -- 172.21.15.99:0/283281327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0cb730 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f7253fff640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f724400c3e0 tx=0x7f724400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244017070 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7244005ce0 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c1f7e90 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440072d0 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.489+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1f8130 con 0x7f725c15eb80 2024-09-10T11:40:06.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.490+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c0d29b0 con 0x7f725c15eb80 2024-09-10T11:40:06.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.490+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f724401a070 con 0x7f725c15eb80 2024-09-10T11:40:06.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.491+0000 7f7258ff9640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 0x7f723c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.491+0000 7f725affd640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 0x7f723c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.491+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f724408a0b0 con 0x7f725c15eb80 2024-09-10T11:40:06.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.491+0000 7f725affd640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 0x7f723c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f724c034000 tx=0x7f724c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.495+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72440584a0 con 0x7f725c15eb80 2024-09-10T11:40:06.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.695+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f725c15e120 con 0x7f725c15eb80 2024-09-10T11:40:06.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.696+0000 7f7258ff9640 1 -- 172.21.15.99:0/283281327 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f724405c7a0 con 0x7f725c15eb80 2024-09-10T11:40:06.697 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:06.697 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:40:06.697 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 msgr2=0x7f723c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 0x7f723c06e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f724c034000 tx=0x7f724c02e000 comp rx=0 tx=0).stop 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 msgr2=0x7f725c0caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f724400c3e0 tx=0x7f724400c8a0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 shutdown_connections 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f723c06be30 0x7f723c06e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725c1626c0 0x7f725c0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f725c15eb80 0x7f725c0caae0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 --2- 172.21.15.99:0/283281327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f725c0d5010 0x7f725c1f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 >> 172.21.15.99:0/283281327 conn(0x7f725c08e680 msgr2=0x7f725c15d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 shutdown_connections 2024-09-10T11:40:06.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.698+0000 7f72619b7640 1 -- 172.21.15.99:0/283281327 wait complete. 2024-09-10T11:40:06.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:06.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.807+0000 7f12d37cd640 1 Processor -- start 2024-09-10T11:40:06.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.807+0000 7f12d37cd640 1 -- start start 2024-09-10T11:40:06.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d37cd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12cc1610b0 0x7f12cc15e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d37cd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d37cd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12cc0d1bd0 con 0x7f12cc1605c0 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d37cd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f12cc075620 con 0x7f12cc1610b0 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d37cd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc0752b0 con 0x7f12cc0ae020 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d0d68640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12cc1610b0 0x7f12cc15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d0d68640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12cc1610b0 0x7f12cc15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34100/0 (socket says 172.21.15.99:34100) 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.808+0000 7f12d0d68640 1 -- 172.21.15.99:0/1140763075 learned_addr learned my addr 172.21.15.99:0/1140763075 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d1d6a640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d0d68640 1 -- 172.21.15.99:0/1140763075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 msgr2=0x7f12cc15e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d0d68640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc15e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d0d68640 1 -- 172.21.15.99:0/1140763075 >> v1:172.21.15.196:6789/0 conn(0x7f12cc1605c0 legacy=0x7f12cc1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d0d68640 1 -- 172.21.15.99:0/1140763075 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0d1eb0 con 0x7f12cc1610b0 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12d0d68640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1610b0 0x7f12cc15e200 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009870 tx=0x7f12bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2a2d26434d8943 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.809+0000 7f12c3fff640 1 -- 172.21.15.99:0/1140763075 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc037070 con 0x7f12cc1610b0 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12c3fff640 1 -- 172.21.15.99:0/1140763075 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc032440 con 0x7f12cc1610b0 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12c3fff640 1 -- 172.21.15.99:0/1140763075 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc032810 con 0x7f12cc1610b0 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 -- 172.21.15.99:0/1140763075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1610b0 msgr2=0x7f12cc15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1610b0 0x7f12cc15e200 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009870 tx=0x7f12bc02a270 comp rx=0 tx=0).stop 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 -- 172.21.15.99:0/1140763075 shutdown_connections 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc15e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 --2- 172.21.15.99:0/1140763075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1610b0 0x7f12cc15e200 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 -- 172.21.15.99:0/1140763075 >> 172.21.15.99:0/1140763075 conn(0x7f12cc0730c0 msgr2=0x7f12cc0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 -- 172.21.15.99:0/1140763075 shutdown_connections 2024-09-10T11:40:06.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.810+0000 7f12d37cd640 1 -- 172.21.15.99:0/1140763075 wait complete. 2024-09-10T11:40:06.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.811+0000 7f12d37cd640 1 Processor -- start 2024-09-10T11:40:06.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.811+0000 7f12d37cd640 1 -- start start 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc1610b0 0x7f12cc15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc073e80 con 0x7f12cc1605c0 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc073d00 con 0x7f12cc1610b0 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d37cd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc15ec80 con 0x7f12cc0ae020 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d0d68640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d0d68640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34106/0 (socket says 172.21.15.99:34106) 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d0d68640 1 -- 172.21.15.99:0/2608454049 learned_addr learned my addr 172.21.15.99:0/2608454049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d1d6a640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc1610b0 0x7f12cc15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d1569640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.812+0000 7f12d0d68640 1 -- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 msgr2=0x7f12cc0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d0d68640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc0b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d0d68640 1 -- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc1610b0 msgr2=0x7f12cc15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d0d68640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc1610b0 0x7f12cc15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d0d68640 1 -- 172.21.15.99:0/2608454049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0b1fb0 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d0d68640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc037040 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc042070 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc0b2290 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0308a0 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.813+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc14fc40 con 0x7f12cc1605c0 2024-09-10T11:40:06.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.814+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc15efb0 con 0x7f12cc1605c0 2024-09-10T11:40:06.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.815+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12bc030a40 con 0x7f12cc1605c0 2024-09-10T11:40:06.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.815+0000 7f12c1ffb640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 0x7f12b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.815+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f12bc0ab530 con 0x7f12cc1605c0 2024-09-10T11:40:06.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.815+0000 7f12d1569640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 0x7f12b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.815+0000 7f12d1569640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 0x7f12b006e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001780 tx=0x7f12c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:06.819+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12bc079890 con 0x7f12cc1605c0 2024-09-10T11:40:07.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.017+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12cc15f2f0 con 0x7f12cc1605c0 2024-09-10T11:40:07.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.017+0000 7f12c1ffb640 1 -- 172.21.15.99:0/2608454049 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5233 (secure 0 0 0) 0x7f12bc07db90 con 0x7f12cc1605c0 2024-09-10T11:40:07.018 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:07.018 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":74,"btime":"2024-09-10T11:39:37:721061+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T11:39:34.642019+0000","modified":"2024-09-10T11:39:37.051865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7956":{"gid":7956,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2972885632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2972885632},{"type":"v1","addr":"172.21.15.99:6837","nonce":2972885632}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7956,"qdb_cluster":[7956]},"id":11}]} 2024-09-10T11:40:07.019 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 74 2024-09-10T11:40:07.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 msgr2=0x7f12b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 0x7f12b006e2f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001780 tx=0x7f12c8009040 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 msgr2=0x7f12cc1f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000c00 tx=0x7f12bc0046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 shutdown_connections 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f12b006be30 0x7f12b006e2f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc1610b0 0x7f12cc15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12cc1605c0 0x7f12cc1f2510 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 --2- 172.21.15.99:0/2608454049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0ae020 0x7f12cc0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.019+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 >> 172.21.15.99:0/2608454049 conn(0x7f12cc0730c0 msgr2=0x7f12cc06f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.020+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 shutdown_connections 2024-09-10T11:40:07.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.020+0000 7f12d37cd640 1 -- 172.21.15.99:0/2608454049 wait complete. 2024-09-10T11:40:07.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:40:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.128+0000 7fed079e4640 1 Processor -- start 2024-09-10T11:40:07.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 -- start start 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fed00163580 0x7fed000b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed000b3960 0x7fed000d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed000b3620 con 0x7fed0006cd20 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fed0006d960 con 0x7fed00163580 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.129+0000 7fed079e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fed0006dc40 con 0x7fed000b3960 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05780640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fed00163580 0x7fed000b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05f81640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fed0006cd20 0x7fed0006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41152/0 (socket says 172.21.15.99:41152) 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05f81640 1 -- 172.21.15.99:0/1214291739 learned_addr learned my addr 172.21.15.99:0/1214291739 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed04f7f640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed000b3960 0x7fed000d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05780640 1 -- 172.21.15.99:0/1214291739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed000b3960 msgr2=0x7fed000d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05780640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed000b3960 0x7fed000d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05780640 1 -- 172.21.15.99:0/1214291739 >> v1:172.21.15.99:6789/0 conn(0x7fed0006cd20 legacy=0x7fed0006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fed05780640 1 -- 172.21.15.99:0/1214291739 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed0006df20 con 0x7fed00163580 2024-09-10T11:40:07.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.130+0000 7fecf3fff640 1 -- 172.21.15.99:0/1214291739 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3299078569 0 0) 0x7fed000b3620 con 0x7fed0006cd20 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed05780640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00163580 0x7fed000b5090 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097a0 tx=0x7fecfc02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145b365e8f3c353 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fecf3fff640 1 -- 172.21.15.99:0/1214291739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc037070 con 0x7fed00163580 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fecf3fff640 1 -- 172.21.15.99:0/1214291739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc02ae50 con 0x7fed00163580 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fecf3fff640 1 -- 172.21.15.99:0/1214291739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc032660 con 0x7fed00163580 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 -- 172.21.15.99:0/1214291739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00163580 msgr2=0x7fed000b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00163580 0x7fed000b5090 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097a0 tx=0x7fecfc02a100 comp rx=0 tx=0).stop 2024-09-10T11:40:07.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 -- 172.21.15.99:0/1214291739 shutdown_connections 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed000b3960 0x7fed000d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 --2- 172.21.15.99:0/1214291739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed00163580 0x7fed000b5090 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 -- 172.21.15.99:0/1214291739 >> 172.21.15.99:0/1214291739 conn(0x7fed000b4570 msgr2=0x7fed000b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 -- 172.21.15.99:0/1214291739 shutdown_connections 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.131+0000 7fed079e4640 1 -- 172.21.15.99:0/1214291739 wait complete. 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 Processor -- start 2024-09-10T11:40:07.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 -- start start 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed0006cd20 0x7fed000cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed00163580 0x7fed0015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fed000b3160 con 0x7fed000b3960 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fed000d2120 con 0x7fed0006cd20 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed079e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fed000b32e0 con 0x7fed00163580 2024-09-10T11:40:07.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed04f7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed04f7f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34130/0 (socket says 172.21.15.99:34130) 2024-09-10T11:40:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.133+0000 7fed04f7f640 1 -- 172.21.15.99:0/326912010 learned_addr learned my addr 172.21.15.99:0/326912010 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed05f81640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed00163580 0x7fed0015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed05780640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed0006cd20 0x7fed000cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 -- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed00163580 msgr2=0x7fed0015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed00163580 0x7fed0015d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 -- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed0006cd20 msgr2=0x7fed000cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed0006cd20 0x7fed000cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 -- 172.21.15.99:0/326912010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed000cd510 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed04f7f640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d830 tx=0x7fecf400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400b6c0 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf4019070 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed000cd7f0 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400ba80 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.134+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed000cf4e0 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.135+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed000d1be0 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.136+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecf401e460 con 0x7fed000b3960 2024-09-10T11:40:07.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.136+0000 7fecf1ffb640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 0x7fece406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.136+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7fecf408b4b0 con 0x7fed000b3960 2024-09-10T11:40:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.136+0000 7fed05780640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 0x7fece406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.136+0000 7fed05780640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 0x7fece406e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fecfc034000 tx=0x7fecfc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.140+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fecf4059ef0 con 0x7fed000b3960 2024-09-10T11:40:07.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.307+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed000b89a0 con 0x7fed000b3960 2024-09-10T11:40:07.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.308+0000 7fecf1ffb640 1 -- 172.21.15.99:0/326912010 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+16747 (secure 0 0 0) 0x7fecf40597f0 con 0x7fed000b3960 2024-09-10T11:40:07.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:07.309 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":186,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:40:00.722829+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-10T11:39:33.255122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-10T11:39:33.614490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.309+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 msgr2=0x7fece406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.309+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 0x7fece406e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fecfc034000 tx=0x7fecfc02e000 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 msgr2=0x7fed00070e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fecf400d830 tx=0x7fecf400dcf0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 shutdown_connections 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fece406be30 0x7fece406e2f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed00163580 0x7fed0015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed000b3960 0x7fed00070e00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 --2- 172.21.15.99:0/326912010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed0006cd20 0x7fed000cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 >> 172.21.15.99:0/326912010 conn(0x7fed000b4570 msgr2=0x7fed000cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 shutdown_connections 2024-09-10T11:40:07.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.310+0000 7fed079e4640 1 -- 172.21.15.99:0/326912010 wait complete. 2024-09-10T11:40:07.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T11:40:07.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.421+0000 7f2db6c5e640 1 Processor -- start 2024-09-10T11:40:07.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.421+0000 7f2db6c5e640 1 -- start start 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db6c5e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2db015aef0 0x7f2db00b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db6c5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015b2d0 0x7f2db00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db6c5e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2db006ee10 con 0x7f2db00cd290 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db6c5e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2db00b0290 con 0x7f2db015aef0 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db6c5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2db006e570 con 0x7f2db015b2d0 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2db015aef0 0x7f2db00b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2db015aef0 0x7f2db00b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34142/0 (socket says 172.21.15.99:34142) 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 -- 172.21.15.99:0/3452266205 learned_addr learned my addr 172.21.15.99:0/3452266205 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 -- 172.21.15.99:0/3452266205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015b2d0 msgr2=0x7f2db00afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2da7fff640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015b2d0 0x7f2db00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015b2d0 0x7f2db00afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 -- 172.21.15.99:0/3452266205 >> v1:172.21.15.196:6789/0 conn(0x7f2db00cd290 legacy=0x7f2db00714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:07.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.422+0000 7f2db49fa640 1 -- 172.21.15.99:0/3452266205 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db006e850 con 0x7f2db015aef0 2024-09-10T11:40:07.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.423+0000 7f2db49fa640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db015aef0 0x7f2db00b4510 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00aa10 tx=0x7f2dac02b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d50faea89d03fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.423+0000 7f2da77fe640 1 -- 172.21.15.99:0/3452266205 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac00b040 con 0x7f2db015aef0 2024-09-10T11:40:07.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.423+0000 7f2da77fe640 1 -- 172.21.15.99:0/3452266205 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac030710 con 0x7f2db015aef0 2024-09-10T11:40:07.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.423+0000 7f2da77fe640 1 -- 172.21.15.99:0/3452266205 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac030ae0 con 0x7f2db015aef0 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3452266205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db015aef0 msgr2=0x7f2db00b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db015aef0 0x7f2db00b4510 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00aa10 tx=0x7f2dac02b5d0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3452266205 shutdown_connections 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015b2d0 0x7f2db00afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3452266205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db015aef0 0x7f2db00b4510 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3452266205 >> 172.21.15.99:0/3452266205 conn(0x7f2db006bc10 msgr2=0x7f2db006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3452266205 shutdown_connections 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.424+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3452266205 wait complete. 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.425+0000 7f2db6c5e640 1 Processor -- start 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.425+0000 7f2db6c5e640 1 -- start start 2024-09-10T11:40:07.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 0x7f2db0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 0x7f2db01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0071d30 con 0x7f2db00cd290 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2db0071bb0 con 0x7f2db015b2d0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2db0071eb0 con 0x7f2db015aef0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34156/0 (socket says 172.21.15.99:34156) 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 -- 172.21.15.99:0/3957409449 learned_addr learned my addr 172.21.15.99:0/3957409449 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2da7fff640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 0x7f2db0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db51fb640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 0x7f2db01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 -- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 msgr2=0x7f2db0153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 0x7f2db0153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 -- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 msgr2=0x7f2db01549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 0x7f2db01549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 -- 172.21.15.99:0/3957409449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db01541c0 con 0x7f2db00cd290 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db51fb640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 0x7f2db01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2da7fff640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 0x7f2db0153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db49fa640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c00 tx=0x7f2dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac030e30 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac03f070 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03a3f0 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.426+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db0152be0 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.427+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01531a0 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.428+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db00cd760 con 0x7f2db00cd290 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.428+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dac03a590 con 0x7f2db00cd290 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.428+0000 7f2da57fa640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.428+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f2dac0ab540 con 0x7f2db00cd290 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.428+0000 7f2da7fff640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.429+0000 7f2da7fff640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2db0154600 tx=0x7f2da0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.433+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2dac079f80 con 0x7f2db00cd290 2024-09-10T11:40:07.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.631+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2db00d0e60 con 0x7f2db00cd290 2024-09-10T11:40:07.632 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-09-10T11:40:07.632+0000 7f2664076640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:40:07.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.633+0000 7f2da57fa640 1 -- 172.21.15.99:0/3957409449 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7f2dac079880 con 0x7f2db00cd290 2024-09-10T11:40:07.634 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.636+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 msgr2=0x7f2d9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.636+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f2db0154600 tx=0x7f2da0008040 comp rx=0 tx=0).stop 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.636+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 msgr2=0x7f2db00aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.636+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c00 tx=0x7f2dac0046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 shutdown_connections 2024-09-10T11:40:07.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2d9c06be30 0x7f2d9c06e2f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2db015b2d0 0x7f2db01549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2db015aef0 0x7f2db0153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 --2- 172.21.15.99:0/3957409449 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db00cd290 0x7f2db00aa540 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 >> 172.21.15.99:0/3957409449 conn(0x7f2db006bc10 msgr2=0x7f2db0159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 shutdown_connections 2024-09-10T11:40:07.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.637+0000 7f2db6c5e640 1 -- 172.21.15.99:0/3957409449 wait complete. 2024-09-10T11:40:07.642 INFO:tasks.ceph.mds.a.smithi099.stderr: -7793> 2024-09-10T11:39:36.580+0000 7fc46b1d0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T11:40:07.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T11:40:07.679 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-09-10T11:40:07.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.754+0000 7f58e0a7b640 1 Processor -- start 2024-09-10T11:40:07.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.754+0000 7f58e0a7b640 1 -- start start 2024-09-10T11:40:07.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58e0a7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 0x7f58dc0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58e0a7b640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc15ff70 con 0x7f58dc15eb80 2024-09-10T11:40:07.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58e0a7b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58dc0d1620 con 0x7f58dc1626c0 2024-09-10T11:40:07.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58e0a7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0d30c0 con 0x7f58dc0d5010 2024-09-10T11:40:07.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58da59c640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f58dc15eb80 0x7f58dc0d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41168/0 (socket says 172.21.15.99:41168) 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58da59c640 1 -- 172.21.15.99:0/170942781 learned_addr learned my addr 172.21.15.99:0/170942781 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58dad9d640 1 --2- 172.21.15.99:0/170942781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 0x7f58dc0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737761543 0 0) 0x7f58dc15ff70 con 0x7f58dc15eb80 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58c8003520 con 0x7f58dc15eb80 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276749950 0 0) 0x7f58dc0d1620 con 0x7f58dc1626c0 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58dc15ff70 con 0x7f58dc1626c0 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1188110163 0 0) 0x7f58c8003520 con 0x7f58dc15eb80 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58dad9d640 1 -- 172.21.15.99:0/170942781 >> v1:172.21.15.196:6789/0 conn(0x7f58dc1626c0 legacy=0x7f58dc0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58dad9d640 1 -- 172.21.15.99:0/170942781 >> v1:172.21.15.99:6789/0 conn(0x7f58dc15eb80 legacy=0x7f58dc0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.755+0000 7f58dad9d640 1 -- 172.21.15.99:0/170942781 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc162bc0 con 0x7f58dc0d5010 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.756+0000 7f58dad9d640 1 --2- 172.21.15.99:0/170942781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 0x7f58dc0d0770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009df0 tx=0x7f58d402a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a90c6760b67efdfc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.756+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0015f0 con 0x7f58dc0d5010 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.756+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4007540 con 0x7f58dc0d5010 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.756+0000 7f58d9d9b640 1 -- 172.21.15.99:0/170942781 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4007910 con 0x7f58dc0d5010 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 -- 172.21.15.99:0/170942781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 msgr2=0x7f58dc0d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/170942781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 0x7f58dc0d0770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009df0 tx=0x7f58d402a610 comp rx=0 tx=0).stop 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 -- 172.21.15.99:0/170942781 shutdown_connections 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/170942781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc0d5010 0x7f58dc0d0770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 -- 172.21.15.99:0/170942781 >> 172.21.15.99:0/170942781 conn(0x7f58dc08e680 msgr2=0x7f58dc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 -- 172.21.15.99:0/170942781 shutdown_connections 2024-09-10T11:40:07.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.757+0000 7f58e0a7b640 1 -- 172.21.15.99:0/170942781 wait complete. 2024-09-10T11:40:07.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.758+0000 7f58e0a7b640 1 Processor -- start 2024-09-10T11:40:07.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.758+0000 7f58e0a7b640 1 -- start start 2024-09-10T11:40:07.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 0x7f58dc1e8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc15eb80 0x7f58dc14a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc163460 con 0x7f58dc1626c0 2024-09-10T11:40:07.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc163180 con 0x7f58dc0d5010 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58e0a7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58dc0d0cb0 con 0x7f58dc15eb80 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34194/0 (socket says 172.21.15.99:34194) 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 -- 172.21.15.99:0/2869882972 learned_addr learned my addr 172.21.15.99:0/2869882972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58da59c640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 0x7f58dc1e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58d1d9b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc15eb80 0x7f58dc14a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 -- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc15eb80 msgr2=0x7f58dc14a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc15eb80 0x7f58dc14a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 -- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 msgr2=0x7f58dc1e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 0x7f58dc1e8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58dad9d640 1 -- 172.21.15.99:0/2869882972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc14a5f0 con 0x7f58dc1626c0 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.759+0000 7f58da59c640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 0x7f58dc1e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58dad9d640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009f10 tx=0x7f58d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4040070 con 0x7f58dc1626c0 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d4031bc0 con 0x7f58dc1626c0 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4009450 con 0x7f58dc1626c0 2024-09-10T11:40:07.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc0b2420 con 0x7f58dc1626c0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.760+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc0b29e0 con 0x7f58dc1626c0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.761+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc0d29b0 con 0x7f58dc1626c0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.761+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58d4039040 con 0x7f58dc1626c0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.762+0000 7f58d37fe640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 0x7f58bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.762+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5795+0+0 (secure 0 0 0) 0x7f58d40ac610 con 0x7f58dc1626c0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.762+0000 7f58da59c640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 0x7f58bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.762+0000 7f58da59c640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 0x7f58bc06e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f60 tx=0x7f58cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.766+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58dc14a5f0 con 0x7f58dc1626c0 2024-09-10T11:40:07.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:07.969+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f58dc0cfa40 con 0x7f58dc1626c0 2024-09-10T11:40:08.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.635+0000 7f58d37fe640 1 -- 172.21.15.99:0/2869882972 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7f58d407aee0 con 0x7f58dc1626c0 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.637+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 msgr2=0x7f58bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.637+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 0x7f58bc06e2f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f58cc000f60 tx=0x7f58cc009040 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.637+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 msgr2=0x7f58dc14a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.637+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009f10 tx=0x7f58d4004a40 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 shutdown_connections 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58bc06be30 0x7f58bc06e2f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58dc1626c0 0x7f58dc14a8d0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58dc15eb80 0x7f58dc14a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 --2- 172.21.15.99:0/2869882972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58dc0d5010 0x7f58dc1e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 >> 172.21.15.99:0/2869882972 conn(0x7f58dc08e680 msgr2=0x7f58dc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 shutdown_connections 2024-09-10T11:40:08.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.638+0000 7f58e0a7b640 1 -- 172.21.15.99:0/2869882972 wait complete. 2024-09-10T11:40:08.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T11:40:08.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.804+0000 7f446959f640 1 Processor -- start 2024-09-10T11:40:08.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.804+0000 7f446959f640 1 -- start start 2024-09-10T11:40:08.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f446959f640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f44640d16f0 0x7f44640d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f446959f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44640d5b70 con 0x7f44640b59b0 2024-09-10T11:40:08.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f446959f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4464163b30 con 0x7f44640d5740 2024-09-10T11:40:08.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f446959f640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f44640d0d30 con 0x7f44640d16f0 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f44627fc640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f44640b59b0 0x7f4464162d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41174/0 (socket says 172.21.15.99:41174) 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f44627fc640 1 -- 172.21.15.99:0/3367839614 learned_addr learned my addr 172.21.15.99:0/3367839614 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f4462ffd640 1 --2- 172.21.15.99:0/3367839614 >> v2:172.21.15.99:3300/0 conn(0x7f44640d16f0 0x7f44640d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f4462ffd640 1 -- 172.21.15.99:0/3367839614 >> v1:172.21.15.196:6789/0 conn(0x7f44640d5740 legacy=0x7f4464163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f4462ffd640 1 -- 172.21.15.99:0/3367839614 >> v1:172.21.15.99:6789/0 conn(0x7f44640b59b0 legacy=0x7f4464162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.805+0000 7f4462ffd640 1 -- 172.21.15.99:0/3367839614 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44640d2150 con 0x7f44640d16f0 2024-09-10T11:40:08.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.806+0000 7f4462ffd640 1 --2- 172.21.15.99:0/3367839614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d16f0 0x7f44640d1ad0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f445c0097a0 tx=0x7f445c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887a34f17c6d39d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.807+0000 7f4461ffb640 1 -- 172.21.15.99:0/3367839614 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c037070 con 0x7f44640d16f0 2024-09-10T11:40:08.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.807+0000 7f4461ffb640 1 -- 172.21.15.99:0/3367839614 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f445c02ae50 con 0x7f44640d16f0 2024-09-10T11:40:08.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.807+0000 7f4461ffb640 1 -- 172.21.15.99:0/3367839614 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445c032660 con 0x7f44640d16f0 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 -- 172.21.15.99:0/3367839614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d16f0 msgr2=0x7f44640d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 --2- 172.21.15.99:0/3367839614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d16f0 0x7f44640d1ad0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f445c0097a0 tx=0x7f445c02a140 comp rx=0 tx=0).stop 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 -- 172.21.15.99:0/3367839614 shutdown_connections 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 --2- 172.21.15.99:0/3367839614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d16f0 0x7f44640d1ad0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 -- 172.21.15.99:0/3367839614 >> 172.21.15.99:0/3367839614 conn(0x7f44640d4c20 msgr2=0x7f44640d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 -- 172.21.15.99:0/3367839614 shutdown_connections 2024-09-10T11:40:08.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.808+0000 7f446959f640 1 -- 172.21.15.99:0/3367839614 wait complete. 2024-09-10T11:40:08.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.809+0000 7f446959f640 1 Processor -- start 2024-09-10T11:40:08.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 -- start start 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44640b59b0 0x7f44641494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44640d16f0 0x7f446406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44640b52f0 con 0x7f44640d5740 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f44640b4d30 con 0x7f44640d16f0 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f446959f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f44640d23f0 con 0x7f44640b59b0 2024-09-10T11:40:08.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f44637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f44637fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34226/0 (socket says 172.21.15.99:34226) 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.810+0000 7f44637fe640 1 -- 172.21.15.99:0/2895729407 learned_addr learned my addr 172.21.15.99:0/2895729407 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 -- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44640b59b0 msgr2=0x7f44641494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44640b59b0 0x7f44641494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 -- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44640d16f0 msgr2=0x7f446406c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44640d16f0 0x7f446406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 -- 172.21.15.99:0/2895729407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4464149a00 con 0x7f44640d5740 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f44637fe640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f445400b410 tx=0x7f445400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454015030 con 0x7f44640d5740 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4454004020 con 0x7f44640d5740 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44641f8be0 con 0x7f44640d5740 2024-09-10T11:40:08.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004410 con 0x7f44640d5740 2024-09-10T11:40:08.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.811+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4464148cc0 con 0x7f44640d5740 2024-09-10T11:40:08.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.812+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44640d1ad0 con 0x7f44640d5740 2024-09-10T11:40:08.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.812+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4454018070 con 0x7f44640d5740 2024-09-10T11:40:08.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.813+0000 7f445b7fe640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 0x7f444406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.813+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5795+0+0 (secure 0 0 0) 0x7f445408a9a0 con 0x7f44640d5740 2024-09-10T11:40:08.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.813+0000 7f4462ffd640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 0x7f444406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.813+0000 7f4462ffd640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 0x7f444406e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f445c034000 tx=0x7f445c02c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.817+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44540592f0 con 0x7f44640d5740 2024-09-10T11:40:08.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:08.982+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44640d10c0 con 0x7f44640d5740 2024-09-10T11:40:09.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.812+0000 7f445b7fe640 1 -- 172.21.15.99:0/2895729407 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v188) ==== 181+0+0 (secure 0 0 0) 0x7f4454058bf0 con 0x7f44640d5740 2024-09-10T11:40:09.813 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 msgr2=0x7f444406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 0x7f444406e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f445c034000 tx=0x7f445c02c000 comp rx=0 tx=0).stop 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 msgr2=0x7f44641f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f445400b410 tx=0x7f445400b8d0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 shutdown_connections 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f444406be30 0x7f444406e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44640d5740 0x7f44641f8140 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44640d16f0 0x7f446406c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 --2- 172.21.15.99:0/2895729407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44640b59b0 0x7f44641494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 >> 172.21.15.99:0/2895729407 conn(0x7f44640d4c20 msgr2=0x7f446406dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:09.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 shutdown_connections 2024-09-10T11:40:09.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.816+0000 7f446959f640 1 -- 172.21.15.99:0/2895729407 wait complete. 2024-09-10T11:40:09.831 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T11:40:09.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.938+0000 7fbc087c4640 1 Processor -- start 2024-09-10T11:40:09.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.938+0000 7fbc087c4640 1 -- start start 2024-09-10T11:40:09.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc087c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbc000d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc087c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbc000d5740 0x7fbc00163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc087c4640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000d2150 con 0x7fbc000b59b0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc087c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fbc000d0d30 con 0x7fbc000d5740 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc087c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbc000d5bd0 con 0x7fbc000d16f0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc05d5f640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbc000b59b0 0x7fbc00162d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41186/0 (socket says 172.21.15.99:41186) 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc05d5f640 1 -- 172.21.15.99:0/1773070350 learned_addr learned my addr 172.21.15.99:0/1773070350 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1773070350 >> v2:172.21.15.196:3300/0 conn(0x7fbc000d5740 0x7fbc00163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc06560640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbc000d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271003199 0 0) 0x7fbc000d2150 con 0x7fbc000b59b0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.939+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbec003520 con 0x7fbc000b59b0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 568234780 0 0) 0x7fbbec003520 con 0x7fbc000b59b0 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 msgr2=0x7fbc000d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbc000d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 >> v2:172.21.15.196:3300/0 conn(0x7fbc000d5740 msgr2=0x7fbc00163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 --2- 172.21.15.99:0/1773070350 >> v2:172.21.15.196:3300/0 conn(0x7fbc000d5740 0x7fbc00163420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc000b5780 con 0x7fbc000b59b0 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fbbf0001670 con 0x7fbc000b59b0 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 >> v1:172.21.15.99:6789/0 conn(0x7fbc000b59b0 legacy=0x7fbc00162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbbec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbec003f30 0x7fbbec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbbec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000d0d30 con 0x7fbbec004d40 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000d5bd0 con 0x7fbbec003f30 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbbec00b9c0 con 0x7fbc000d16f0 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbbec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06560640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbbec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc05d5f640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbec003f30 0x7fbbec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 -- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 msgr2=0x7fbbec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000d16f0 0x7fbbec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 -- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbec003f30 msgr2=0x7fbbec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbbec003f30 0x7fbbec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.940+0000 7fbc06d61640 1 -- 172.21.15.99:0/1773070350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbec003520 con 0x7fbbec004d40 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbbec0051c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009fc0 tx=0x7fbbec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f60a105445626fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8009c10 con 0x7fbbec004d40 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf8036070 con 0x7fbbec004d40 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc0555e640 1 -- 172.21.15.99:0/1773070350 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf802d070 con 0x7fbbec004d40 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc087c4640 1 -- 172.21.15.99:0/1773070350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 msgr2=0x7fbbec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1773070350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbbec0051c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009fc0 tx=0x7fbbec005830 comp rx=0 tx=0).stop 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.941+0000 7fbc06d61640 1 -- 172.21.15.99:0/1773070350 reap_dead start 2024-09-10T11:40:09.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.942+0000 7fbc087c4640 1 -- 172.21.15.99:0/1773070350 shutdown_connections 2024-09-10T11:40:09.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.942+0000 7fbc087c4640 1 -- 172.21.15.99:0/1773070350 >> 172.21.15.99:0/1773070350 conn(0x7fbc000d4c20 msgr2=0x7fbc000d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:09.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.942+0000 7fbc087c4640 1 -- 172.21.15.99:0/1773070350 shutdown_connections 2024-09-10T11:40:09.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.942+0000 7fbc087c4640 1 -- 172.21.15.99:0/1773070350 wait complete. 2024-09-10T11:40:09.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 Processor -- start 2024-09-10T11:40:09.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 -- start start 2024-09-10T11:40:09.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000b2cd0 0x7fbc000b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00161080 0x7fbc00161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000b52f0 con 0x7fbbec004d40 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000b4d30 con 0x7fbc00161080 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.943+0000 7fbc087c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fbc000b4f80 con 0x7fbc000b2cd0 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34248/0 (socket says 172.21.15.99:34248) 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 -- 172.21.15.99:0/1186653090 learned_addr learned my addr 172.21.15.99:0/1186653090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc05d5f640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000b2cd0 0x7fbc000b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06d61640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00161080 0x7fbc00161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 -- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000b2cd0 msgr2=0x7fbc000b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000b2cd0 0x7fbc000b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 -- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00161080 msgr2=0x7fbc00161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00161080 0x7fbc00161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 -- 172.21.15.99:0/1186653090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc000b2230 con 0x7fbbec004d40 2024-09-10T11:40:09.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc06560640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0119d0 tx=0x7fbbfc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00faf0 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbfc00fc90 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc014440 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc000b88f0 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.944+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc000b27b0 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.945+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc00163a40 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.946+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbbfc012040 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.946+0000 7fbbf6ffd640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 0x7fbbe406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.946+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 5357+0+0 (secure 0 0 0) 0x7fbbfc08b390 con 0x7fbbec004d40 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.946+0000 7fbc05d5f640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 0x7fbbe406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:09.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.947+0000 7fbc05d5f640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 0x7fbbe406e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:09.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:09.950+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbbfc019030 con 0x7fbbec004d40 2024-09-10T11:40:10.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.116+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc00163640 con 0x7fbbec004d40 2024-09-10T11:40:10.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.726+0000 7fbbf6ffd640 1 -- 172.21.15.99:0/1186653090 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v189) ==== 169+0+0 (secure 0 0 0) 0x7fbbfc059ea0 con 0x7fbbec004d40 2024-09-10T11:40:10.727 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 msgr2=0x7fbbe406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 0x7fbbe406e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0009040 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 msgr2=0x7fbc00157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0119d0 tx=0x7fbbfc011e90 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 shutdown_connections 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fbbe406be30 0x7fbbe406e2f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc00161080 0x7fbc00161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc000b2cd0 0x7fbc000b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 --2- 172.21.15.99:0/1186653090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbec004d40 0x7fbc00157250 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 >> 172.21.15.99:0/1186653090 conn(0x7fbc000d4c20 msgr2=0x7fbc0006cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:10.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 shutdown_connections 2024-09-10T11:40:10.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.729+0000 7fbc087c4640 1 -- 172.21.15.99:0/1186653090 wait complete. 2024-09-10T11:40:10.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T11:40:10.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.846+0000 7f58ef6a5640 1 Processor -- start 2024-09-10T11:40:10.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.846+0000 7f58ef6a5640 1 -- start start 2024-09-10T11:40:10.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ef6a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ef6a5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f58e806e5d0 0x7f58e806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ef6a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e8152e60 con 0x7f58e815c060 2024-09-10T11:40:10.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ef6a5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f58e806ef50 con 0x7f58e806e5d0 2024-09-10T11:40:10.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ef6a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58e80d15e0 con 0x7f58e80d1200 2024-09-10T11:40:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ed441640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ed441640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:32964/0 (socket says 172.21.15.99:32964) 2024-09-10T11:40:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ed441640 1 -- 172.21.15.99:0/2465415144 learned_addr learned my addr 172.21.15.99:0/2465415144 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.847+0000 7f58ecc40640 1 --2- 172.21.15.99:0/2465415144 >> v2:172.21.15.196:3300/0 conn(0x7f58e806e5d0 0x7f58e806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.849+0000 7f58ed441640 1 -- 172.21.15.99:0/2465415144 >> v2:172.21.15.196:3300/0 conn(0x7f58e806e5d0 msgr2=0x7f58e806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.849+0000 7f58ed441640 1 --2- 172.21.15.99:0/2465415144 >> v2:172.21.15.196:3300/0 conn(0x7f58e806e5d0 0x7f58e806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.849+0000 7f58ed441640 1 -- 172.21.15.99:0/2465415144 >> v1:172.21.15.196:6789/0 conn(0x7f58e815c060 legacy=0x7f58e815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.849+0000 7f58ed441640 1 -- 172.21.15.99:0/2465415144 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e8072680 con 0x7f58e80d1200 2024-09-10T11:40:10.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.851+0000 7f58ed441640 1 --2- 172.21.15.99:0/2465415144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e402c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef387d0a0e540a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:10.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.852+0000 7f58dbfff640 1 -- 172.21.15.99:0/2465415144 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4038070 con 0x7f58e80d1200 2024-09-10T11:40:10.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.852+0000 7f58dbfff640 1 -- 172.21.15.99:0/2465415144 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e4031710 con 0x7f58e80d1200 2024-09-10T11:40:10.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.852+0000 7f58dbfff640 1 -- 172.21.15.99:0/2465415144 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4038070 con 0x7f58e80d1200 2024-09-10T11:40:10.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 -- 172.21.15.99:0/2465415144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 msgr2=0x7f58e8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/2465415144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009870 tx=0x7f58e402c3d0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 -- 172.21.15.99:0/2465415144 shutdown_connections 2024-09-10T11:40:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/2465415144 >> v2:172.21.15.196:3300/0 conn(0x7f58e806e5d0 0x7f58e806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/2465415144 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e8073440 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 -- 172.21.15.99:0/2465415144 >> 172.21.15.99:0/2465415144 conn(0x7f58e806bc10 msgr2=0x7f58e806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.853+0000 7f58ef6a5640 1 -- 172.21.15.99:0/2465415144 shutdown_connections 2024-09-10T11:40:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.854+0000 7f58ef6a5640 1 -- 172.21.15.99:0/2465415144 wait complete. 2024-09-10T11:40:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.855+0000 7f58ef6a5640 1 Processor -- start 2024-09-10T11:40:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.855+0000 7f58ef6a5640 1 -- start start 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e81f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e815c060 0x7f58e81f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8072f20 con 0x7f58e806e5d0 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58e8072c40 con 0x7f58e815c060 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ef6a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f58e815cd90 con 0x7f58e80d1200 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34260/0 (socket says 172.21.15.99:34260) 2024-09-10T11:40:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 -- 172.21.15.99:0/4064229652 learned_addr learned my addr 172.21.15.99:0/4064229652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58edc42640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e815c060 0x7f58e81f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ecc40640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e81f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 -- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 msgr2=0x7f58e81f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e81f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 -- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e815c060 msgr2=0x7f58e81f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e815c060 0x7f58e81f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 -- 172.21.15.99:0/4064229652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e80ae510 con 0x7f58e806e5d0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.856+0000 7f58ed441640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f58e402c410 tx=0x7f58e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.857+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4042070 con 0x7f58e806e5d0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.857+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e4033a40 con 0x7f58e806e5d0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.857+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4038070 con 0x7f58e806e5d0 2024-09-10T11:40:10.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.857+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e80ae7f0 con 0x7f58e806e5d0 2024-09-10T11:40:10.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.857+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e80aedb0 con 0x7f58e806e5d0 2024-09-10T11:40:10.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.858+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e80008d0 con 0x7f58e806e5d0 2024-09-10T11:40:10.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.858+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58e4045070 con 0x7f58e806e5d0 2024-09-10T11:40:10.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.859+0000 7f58d9ffb640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 0x7f58cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:10.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.859+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f58e40ad150 con 0x7f58e806e5d0 2024-09-10T11:40:10.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.859+0000 7f58ecc40640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 0x7f58cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:10.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.859+0000 7f58ecc40640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 0x7f58cc06e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b1540 tx=0x7f58dc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:10.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:10.863+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58e407b7b0 con 0x7f58e806e5d0 2024-09-10T11:40:11.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.028+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f58e80ad820 con 0x7f58e806e5d0 2024-09-10T11:40:11.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.031+0000 7f58d9ffb640 1 -- 172.21.15.99:0/4064229652 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f58e407fab0 con 0x7f58e806e5d0 2024-09-10T11:40:11.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.033+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 msgr2=0x7f58cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.033+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 0x7f58cc06e2f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b1540 tx=0x7f58dc009190 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.033+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 msgr2=0x7f58e80ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.033+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f58e402c410 tx=0x7f58e40046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.033+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 shutdown_connections 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f58cc06be30 0x7f58cc06e2f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58e815c060 0x7f58e81f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58e80d1200 0x7f58e81f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 --2- 172.21.15.99:0/4064229652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58e806e5d0 0x7f58e80ad2e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 >> 172.21.15.99:0/4064229652 conn(0x7f58e806bc10 msgr2=0x7f58e80b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 shutdown_connections 2024-09-10T11:40:11.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.034+0000 7f58ef6a5640 1 -- 172.21.15.99:0/4064229652 wait complete. 2024-09-10T11:40:11.046 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-09-10T11:40:11.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.143+0000 7ff3a01bd640 1 Processor -- start 2024-09-10T11:40:11.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.144+0000 7ff3a01bd640 1 -- start start 2024-09-10T11:40:11.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.144+0000 7ff3a01bd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff39806ece0 0x7ff398073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.144+0000 7ff3a01bd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff39815c060 0x7ff39815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.144+0000 7ff3a01bd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff398152e60 con 0x7ff3980d1200 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.144+0000 7ff3a01bd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7ff3980d15e0 con 0x7ff39806ece0 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff3a01bd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff398072680 con 0x7ff39815c060 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff39806ece0 0x7ff398073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff39806ece0 0x7ff398073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34270/0 (socket says 172.21.15.99:34270) 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 -- 172.21.15.99:0/3166479159 learned_addr learned my addr 172.21.15.99:0/3166479159 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39e75a640 1 --2- 172.21.15.99:0/3166479159 >> v2:172.21.15.196:3300/0 conn(0x7ff39815c060 0x7ff39815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 -- 172.21.15.99:0/3166479159 >> v2:172.21.15.196:3300/0 conn(0x7ff39815c060 msgr2=0x7ff39815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 --2- 172.21.15.99:0/3166479159 >> v2:172.21.15.196:3300/0 conn(0x7ff39815c060 0x7ff39815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 -- 172.21.15.99:0/3166479159 >> v1:172.21.15.196:6789/0 conn(0x7ff3980d1200 legacy=0x7ff39806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:11.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.145+0000 7ff39d758640 1 -- 172.21.15.99:0/3166479159 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff398072960 con 0x7ff39806ece0 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff39d758640 1 --2- 172.21.15.99:0/3166479159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398073440 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff3980cd570 tx=0x7ff38c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be2b825d3c56db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff39cf57640 1 -- 172.21.15.99:0/3166479159 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c036070 con 0x7ff39806ece0 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff39cf57640 1 -- 172.21.15.99:0/3166479159 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c031600 con 0x7ff39806ece0 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff39cf57640 1 -- 172.21.15.99:0/3166479159 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0319d0 con 0x7ff39806ece0 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 -- 172.21.15.99:0/3166479159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 msgr2=0x7ff398073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/3166479159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398073440 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff3980cd570 tx=0x7ff38c02a660 comp rx=0 tx=0).stop 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 -- 172.21.15.99:0/3166479159 shutdown_connections 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/3166479159 >> v2:172.21.15.196:3300/0 conn(0x7ff39815c060 0x7ff39815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/3166479159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398073440 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 -- 172.21.15.99:0/3166479159 >> 172.21.15.99:0/3166479159 conn(0x7ff39806bc10 msgr2=0x7ff39806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 -- 172.21.15.99:0/3166479159 shutdown_connections 2024-09-10T11:40:11.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.146+0000 7ff3a01bd640 1 -- 172.21.15.99:0/3166479159 wait complete. 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 Processor -- start 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 -- start start 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980d1200 0x7ff398151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39815c060 0x7ff3981599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff39815cd90 con 0x7ff39806ece0 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff398072f20 con 0x7ff3980d1200 2024-09-10T11:40:11.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff3a01bd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff39815ca30 con 0x7ff39815c060 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff39df59640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff39df59640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:34286/0 (socket says 172.21.15.99:34286) 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.148+0000 7ff39df59640 1 -- 172.21.15.99:0/1393378027 learned_addr learned my addr 172.21.15.99:0/1393378027 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39d758640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980d1200 0x7ff398151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39e75a640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39815c060 0x7ff3981599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 -- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39815c060 msgr2=0x7ff3981599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39815c060 0x7ff3981599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 -- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980d1200 msgr2=0x7ff398151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980d1200 0x7ff398151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 -- 172.21.15.99:0/1393378027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff398157c50 con 0x7ff39806ece0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff39df59640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff39000b3f0 tx=0x7ff39000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390015020 con 0x7ff39806ece0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff390011830 con 0x7ff39806ece0 2024-09-10T11:40:11.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff398157f30 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390011c20 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.149+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3981584f0 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.150+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff398073440 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.151+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff390018020 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.151+0000 7ff3967fc640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 0x7ff38406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.151+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7ff39008a6f0 con 0x7ff39806ece0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.151+0000 7ff39d758640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 0x7ff38406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:11.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.151+0000 7ff39d758640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 0x7ff38406e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff38c033000 tx=0x7ff38c009000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:11.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.155+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff39001f030 con 0x7ff39806ece0 2024-09-10T11:40:11.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.381+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7ff39806df30 con 0x7ff39806ece0 2024-09-10T11:40:11.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.816+0000 7ff3967fc640 1 -- 172.21.15.99:0/1393378027 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v466) ==== 135+0+0 (secure 0 0 0) 0x7ff390059430 con 0x7ff39806ece0 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 msgr2=0x7ff38406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 0x7ff38406e2f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff38c033000 tx=0x7ff38c009000 comp rx=0 tx=0).stop 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 msgr2=0x7ff398157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff39000b3f0 tx=0x7ff39000b8b0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 shutdown_connections 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff38406be30 0x7ff38406e2f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff39815c060 0x7ff3981599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff3980d1200 0x7ff398151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 --2- 172.21.15.99:0/1393378027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39806ece0 0x7ff398157350 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 >> 172.21.15.99:0/1393378027 conn(0x7ff39806bc10 msgr2=0x7ff3980cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 shutdown_connections 2024-09-10T11:40:11.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:11.818+0000 7ff3a01bd640 1 -- 172.21.15.99:0/1393378027 wait complete. 2024-09-10T11:40:11.831 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-09-10T11:40:11.831 INFO:tasks.cephfs_test_runner: 2024-09-10T11:40:11.831 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T11:40:11.831 INFO:tasks.cephfs_test_runner:Ran 10 tests in 562.001s 2024-09-10T11:40:11.831 INFO:tasks.cephfs_test_runner: 2024-09-10T11:40:11.832 INFO:tasks.cephfs_test_runner:OK 2024-09-10T11:40:11.832 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T11:40:11.841 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T11:40:11.865 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T11:40:11.866 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:40:11.866 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:11.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:40:11.883 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-10T11:40:11.884 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:11.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16636 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T11:40:11.940 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:40:11.940 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:11.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:40:11.960 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-10T11:40:11.960 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T11:40:12.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056015,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17054 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248720k,nr_inodes=812180,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497444k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T11:40:12.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.020 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-09-10T11:40:12.037 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T11:40:12.037 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T11:40:12.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T11:40:12.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.105 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:40:12.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.172 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-09-10T11:40:12.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.263 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-09-10T11:40:12.283 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T11:40:12.283 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T11:40:12.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.283 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T11:40:12.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.354 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:40:12.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:40:12.424 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-09-10T11:40:12.521 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T11:40:12.533 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T11:40:12.533 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:40:12.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.633+0000 7f3b9139f640 1 Processor -- start 2024-09-10T11:40:12.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.633+0000 7f3b9139f640 1 -- start start 2024-09-10T11:40:12.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b9139f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3b8c0d1200 0x7f3b8c073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b9139f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b9139f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b8c152e60 con 0x7f3b8c15c060 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b9139f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f3b8c06ef50 con 0x7f3b8c0d1200 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b9139f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8c0d15e0 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b827fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b8affd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3b8c0d1200 0x7f3b8c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3b8c0d1200 0x7f3b8c073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:43334/0 (socket says 172.21.15.99:43334) 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b827fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.99:52372/0 (socket says 172.21.15.99:52372) 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.634+0000 7f3b8b7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3b8c15c060 0x7f3b8c15c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:56198/0 (socket says 172.21.15.99:56198) 2024-09-10T11:40:12.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 -- 172.21.15.99:0/3338214687 learned_addr learned my addr 172.21.15.99:0/3338214687 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:12.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 -- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 msgr2=0x7f3b8c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:12.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 -- 172.21.15.99:0/3338214687 >> v1:172.21.15.196:6789/0 conn(0x7f3b8c15c060 legacy=0x7f3b8c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b8affd640 1 -- 172.21.15.99:0/3338214687 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c072680 con 0x7f3b8c0d1200 2024-09-10T11:40:12.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.635+0000 7f3b827fc640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:12.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b8affd640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0d1200 0x7f3b8c073440 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ce48eb8af3421e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b8a7fc640 1 -- 172.21.15.99:0/3338214687 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037070 con 0x7f3b8c0d1200 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b8a7fc640 1 -- 172.21.15.99:0/3338214687 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c032440 con 0x7f3b8c0d1200 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b8a7fc640 1 -- 172.21.15.99:0/3338214687 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c032810 con 0x7f3b8c0d1200 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 -- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0d1200 msgr2=0x7f3b8c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0d1200 0x7f3b8c073440 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009870 tx=0x7f3b7c02a1c0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 -- 172.21.15.99:0/3338214687 shutdown_connections 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c06e5d0 0x7f3b8c06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 --2- 172.21.15.99:0/3338214687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c0d1200 0x7f3b8c073440 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 -- 172.21.15.99:0/3338214687 >> 172.21.15.99:0/3338214687 conn(0x7f3b8c06bc10 msgr2=0x7f3b8c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:12.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 -- 172.21.15.99:0/3338214687 shutdown_connections 2024-09-10T11:40:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.636+0000 7f3b9139f640 1 -- 172.21.15.99:0/3338214687 wait complete. 2024-09-10T11:40:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.637+0000 7f3b9139f640 1 Processor -- start 2024-09-10T11:40:12.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 -- start start 2024-09-10T11:40:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c0d1200 0x7f3b8c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c15c060 0x7f3b8c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c072f20 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8c072c40 con 0x7f3b8c15c060 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b9139f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8c15cd90 con 0x7f3b8c0d1200 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.638+0000 7f3b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35518/0 (socket says 172.21.15.99:35518) 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 -- 172.21.15.99:0/1225727358 learned_addr learned my addr 172.21.15.99:0/1225727358 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8b7fe640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c15c060 0x7f3b8c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b827fc640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c0d1200 0x7f3b8c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 -- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c0d1200 msgr2=0x7f3b8c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c0d1200 0x7f3b8c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 -- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c15c060 msgr2=0x7f3b8c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c15c060 0x7f3b8c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 -- 172.21.15.99:0/1225727358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b8c150cf0 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b8affd640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c000c00 tx=0x7f3b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c037040 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c042070 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c030880 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8c150fd0 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.639+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8c151590 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.640+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8c06ea10 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.641+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b7c02ad10 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.641+0000 7f3b83fff640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 0x7f3b6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.641+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f3b7c0aaf70 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.641+0000 7f3b827fc640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 0x7f3b6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.641+0000 7f3b827fc640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 0x7f3b6c06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009560 tx=0x7f3b74009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.645+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b7c0795d0 con 0x7f3b8c06e5d0 2024-09-10T11:40:12.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.807+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b8c0d0c70 con 0x7f3b6c06be30 2024-09-10T11:40:12.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.807+0000 7f3b83fff640 1 -- 172.21.15.99:0/1225727358 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12597 (secure 0 0 0) 0x7f3b8c0d0c70 con 0x7f3b6c06be30 2024-09-10T11:40:12.809 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:40:12.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 msgr2=0x7f3b6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 0x7f3b6c06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3b74009560 tx=0x7f3b74009190 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 msgr2=0x7f3b8c14e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c000c00 tx=0x7f3b7c0046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 shutdown_connections 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f3b6c06be30 0x7f3b6c06e2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b8c15c060 0x7f3b8c0ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b8c0d1200 0x7f3b8c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 --2- 172.21.15.99:0/1225727358 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3b8c06e5d0 0x7f3b8c14e790 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.809+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 >> 172.21.15.99:0/1225727358 conn(0x7f3b8c06bc10 msgr2=0x7f3b8c155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.810+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 shutdown_connections 2024-09-10T11:40:12.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.810+0000 7f3b9139f640 1 -- 172.21.15.99:0/1225727358 wait complete. 2024-09-10T11:40:12.821 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-10T11:40:11.037310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":111828,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450671,"statfs":{"total":1600000000,"available":1485487614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-42718,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-104,"num_write_kb":-33863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001021"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":503,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-10T11:40:00.731619+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:40:00.731619+0000","last_peered":"2024-09-10T11:40:00.731619+0000","last_clean":"2024-09-10T11:40:00.731619+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:40:00.731619+0000","last_undegraded":"2024-09-10T11:40:00.731619+0000","last_fullsized":"2024-09-10T11:40:00.731619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5116,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190195,"statfs":{"total":200000000,"available":194760620,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":17404,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":177907,"statfs":{"total":200000000,"available":182177472,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22077,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173234,"statfs":{"total":200000000,"available":177392364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9212,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186099,"statfs":{"total":200000000,"available":190566282,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574969,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":11315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":183997,"statfs":{"total":200000000,"available":188413421,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":5117,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":190195,"statfs":{"total":200000000,"available":194759836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":22131,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173180,"statfs":{"total":200000000,"available":177337133,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19452,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175859,"statfs":{"total":200000000,"available":180080486,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:40:12.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:40:12.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.920+0000 7fa03f877640 1 Processor -- start 2024-09-10T11:40:12.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.920+0000 7fa03f877640 1 -- start start 2024-09-10T11:40:12.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03f877640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa0380b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03f877640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0380b3620 con 0x7fa0380b3960 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03f877640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa03806d960 con 0x7fa038163580 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03f877640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa03806dc40 con 0x7fa0380b3160 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03ce12640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa0380b3960 0x7fa03806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35874/0 (socket says 172.21.15.99:35874) 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03ce12640 1 -- 172.21.15.99:0/1933939222 learned_addr learned my addr 172.21.15.99:0/1933939222 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa03de14640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa0380b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492952031 0 0) 0x7fa0380b3620 con 0x7fa0380b3960 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa024003520 con 0x7fa0380b3960 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247525927 0 0) 0x7fa03806d960 con 0x7fa038163580 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0380b3620 con 0x7fa038163580 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 487742600 0 0) 0x7fa024003520 con 0x7fa0380b3960 2024-09-10T11:40:12.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 msgr2=0x7fa0380b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa0380b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 >> v1:172.21.15.196:6789/0 conn(0x7fa038163580 legacy=0x7fa0380d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03806df20 con 0x7fa0380b3960 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fa028001670 con 0x7fa0380b3960 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 >> v1:172.21.15.99:6789/0 conn(0x7fa0380b3960 legacy=0x7fa03806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa024003b70 0x7fa024003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024004800 0x7fa024004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa024005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa03806dc40 con 0x7fa024003b70 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa02400bbb0 con 0x7fa024004800 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.921+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa02400bdb0 con 0x7fa0380b3160 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa024003b70 0x7fa024003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03ce12640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024004800 0x7fa024004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03de14640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa024005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 -- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 msgr2=0x7fa024005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380b3160 0x7fa024005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 -- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024004800 msgr2=0x7fa024004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024004800 0x7fa024004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.922+0000 7fa03d613640 1 -- 172.21.15.99:0/1933939222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa024003520 con 0x7fa024003b70 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03d613640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa024003b70 0x7fa024003f70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa0340018d0 tx=0x7fa024005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea20ef8a883db2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0340014f0 con 0x7fa024003b70 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0340052d0 con 0x7fa024003b70 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa02ffff640 1 -- 172.21.15.99:0/1933939222 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03402aca0 con 0x7fa024003b70 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 -- 172.21.15.99:0/1933939222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa024003b70 msgr2=0x7fa024003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 --2- 172.21.15.99:0/1933939222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa024003b70 0x7fa024003f70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fa0340018d0 tx=0x7fa024005830 comp rx=0 tx=0).stop 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03de14640 1 -- 172.21.15.99:0/1933939222 reap_dead start 2024-09-10T11:40:12.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 -- 172.21.15.99:0/1933939222 shutdown_connections 2024-09-10T11:40:12.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 -- 172.21.15.99:0/1933939222 >> 172.21.15.99:0/1933939222 conn(0x7fa0380b4570 msgr2=0x7fa0380b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:12.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 -- 172.21.15.99:0/1933939222 shutdown_connections 2024-09-10T11:40:12.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.923+0000 7fa03f877640 1 -- 172.21.15.99:0/1933939222 wait complete. 2024-09-10T11:40:12.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.924+0000 7fa03f877640 1 Processor -- start 2024-09-10T11:40:12.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.924+0000 7fa03f877640 1 -- start start 2024-09-10T11:40:12.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024003b70 0x7fa038155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 0x7fa0380c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa03806d5b0 con 0x7fa0380c6550 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa03806d430 con 0x7fa024003b70 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03f877640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa0380c7b20 con 0x7fa0380c69d0 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03ce12640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03ce12640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35546/0 (socket says 172.21.15.99:35546) 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03ce12640 1 -- 172.21.15.99:0/883238474 learned_addr learned my addr 172.21.15.99:0/883238474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03d613640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024003b70 0x7fa038155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.925+0000 7fa03de14640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 0x7fa0380c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 -- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 msgr2=0x7fa0380c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 0x7fa0380c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 -- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024003b70 msgr2=0x7fa038155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:12.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024003b70 0x7fa038155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 -- 172.21.15.99:0/883238474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0380c6f10 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03de14640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 0x7fa0380c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03ce12640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa028004ed0 tx=0x7fa028007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800f750 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa028018070 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0380c71f0 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800fb30 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.926+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0380c77b0 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.927+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0380761a0 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.927+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa02800fcd0 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.928+0000 7fa02dffb640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 0x7fa01c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.928+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7fa02808a9b0 con 0x7fa0380c6550 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.928+0000 7fa03d613640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 0x7fa01c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:12.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.928+0000 7fa03d613640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 0x7fa01c06e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa034032000 tx=0x7fa034009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:12.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:12.932+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa0280596f0 con 0x7fa0380c6550 2024-09-10T11:40:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.094+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0380b5090 con 0x7fa01c06be30 2024-09-10T11:40:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.095+0000 7fa02dffb640 1 -- 172.21.15.99:0/883238474 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12591 (secure 0 0 0) 0x7fa034045140 con 0x7fa01c06be30 2024-09-10T11:40:13.096 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:13.096 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:40:13.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.096+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 msgr2=0x7fa01c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.096+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 0x7fa01c06e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa034032000 tx=0x7fa034009030 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 msgr2=0x7fa0380c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fa028004ed0 tx=0x7fa028007430 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 shutdown_connections 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa01c06be30 0x7fa01c06e2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0380c69d0 0x7fa0380c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0380c6550 0x7fa0380c86b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 --2- 172.21.15.99:0/883238474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa024003b70 0x7fa038155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 >> 172.21.15.99:0/883238474 conn(0x7fa0380b4570 msgr2=0x7fa0380ca400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 shutdown_connections 2024-09-10T11:40:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.097+0000 7fa03f877640 1 -- 172.21.15.99:0/883238474 wait complete. 2024-09-10T11:40:13.108 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T11:40:13.037566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9458,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553041,"statfs":{"total":1600000000,"available":1590314336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485521,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000988"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":509,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-10T11:40:10.729344+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:40:10.729344+0000","last_peered":"2024-09-10T11:40:10.729344+0000","last_clean":"2024-09-10T11:40:10.729344+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:40:10.729344+0000","last_undegraded":"2024-09-10T11:40:10.729344+0000","last_fullsized":"2024-09-10T11:40:10.729344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:40:13.109 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T11:40:13.110 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:40:13.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.206+0000 7ff55f3c6640 1 Processor -- start 2024-09-10T11:40:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 -- start start 2024-09-10T11:40:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 0x7ff5580cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5580d0130 0x7ff5580cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5580ce430 con 0x7ff558160030 2024-09-10T11:40:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7ff5580ce710 con 0x7ff5580d0130 2024-09-10T11:40:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.207+0000 7ff55f3c6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff5580d1240 con 0x7ff5580d1760 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5580d0130 0x7ff5580cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff5580d0130 0x7ff5580cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:43370/0 (socket says 172.21.15.99:43370) 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55c961640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 0x7ff5580cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 -- 172.21.15.99:0/2795239654 learned_addr learned my addr 172.21.15.99:0/2795239654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d162640 1 --1- 172.21.15.99:0/2795239654 >> v1:172.21.15.196:6789/0 conn(0x7ff558160030 0x7ff5580b38f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:56226/0 (socket says 172.21.15.99:56226) 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 -- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 msgr2=0x7ff5580cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 0x7ff5580cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 -- 172.21.15.99:0/2795239654 >> v1:172.21.15.196:6789/0 conn(0x7ff558160030 legacy=0x7ff5580b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff54bfff640 1 -- 172.21.15.99:0/2795239654 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4041421881 0 0) 0x7ff5580ce430 con 0x7ff558160030 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55d963640 1 -- 172.21.15.99:0/2795239654 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5580ce030 con 0x7ff5580d0130 2024-09-10T11:40:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.208+0000 7ff55c961640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 0x7ff5580cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:13.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.209+0000 7ff55d963640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d0130 0x7ff5580cdaf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e50 tx=0x7ff55002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb982c6458e30ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.209+0000 7ff54bfff640 1 -- 172.21.15.99:0/2795239654 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550036070 con 0x7ff5580d0130 2024-09-10T11:40:13.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.209+0000 7ff54bfff640 1 -- 172.21.15.99:0/2795239654 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff550031600 con 0x7ff5580d0130 2024-09-10T11:40:13.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.209+0000 7ff54bfff640 1 -- 172.21.15.99:0/2795239654 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500319d0 con 0x7ff5580d0130 2024-09-10T11:40:13.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 -- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d0130 msgr2=0x7ff5580cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d0130 0x7ff5580cdaf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e50 tx=0x7ff55002a710 comp rx=0 tx=0).stop 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 -- 172.21.15.99:0/2795239654 shutdown_connections 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d0130 0x7ff5580cdaf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/2795239654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d1760 0x7ff5580cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 -- 172.21.15.99:0/2795239654 >> 172.21.15.99:0/2795239654 conn(0x7ff5580b4650 msgr2=0x7ff5580b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 -- 172.21.15.99:0/2795239654 shutdown_connections 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.210+0000 7ff55f3c6640 1 -- 172.21.15.99:0/2795239654 wait complete. 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.211+0000 7ff55f3c6640 1 Processor -- start 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.211+0000 7ff55f3c6640 1 -- start start 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d0130 0x7ff5580c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d1760 0x7ff5580b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff558160660 con 0x7ff558160030 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff5580b5950 con 0x7ff5580d1760 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7ff55815a310 con 0x7ff5580d0130 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35554/0 (socket says 172.21.15.99:35554) 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 -- 172.21.15.99:0/3853721023 learned_addr learned my addr 172.21.15.99:0/3853721023 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d162640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d0130 0x7ff5580c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55c961640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d1760 0x7ff5580b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 -- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d0130 msgr2=0x7ff5580c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d0130 0x7ff5580c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 -- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d1760 msgr2=0x7ff5580b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d1760 0x7ff5580b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 -- 172.21.15.99:0/3853721023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5580c8610 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55d963640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e50 tx=0x7ff550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550041020 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff550037040 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550036050 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.212+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5580c88f0 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.213+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5580c8eb0 con 0x7ff558160030 2024-09-10T11:40:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.214+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55815a4c0 con 0x7ff558160030 2024-09-10T11:40:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.214+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff550007bd0 con 0x7ff558160030 2024-09-10T11:40:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.215+0000 7ff549ffb640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 0x7ff53c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.215+0000 7ff55d162640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 0x7ff53c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.215+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7ff5500ab010 con 0x7ff558160030 2024-09-10T11:40:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.215+0000 7ff55d162640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 0x7ff53c06e320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff554007fd0 tx=0x7ff554007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.219+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff55004b060 con 0x7ff558160030 2024-09-10T11:40:13.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.379+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5580b38f0 con 0x7ff53c06be60 2024-09-10T11:40:13.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.380+0000 7ff549ffb640 1 -- 172.21.15.99:0/3853721023 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12591 (secure 0 0 0) 0x7ff5580b38f0 con 0x7ff53c06be60 2024-09-10T11:40:13.381 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:13.381 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 msgr2=0x7ff53c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 0x7ff53c06e320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff554007fd0 tx=0x7ff554007de0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 msgr2=0x7ff5580b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff550009e50 tx=0x7ff550004a40 comp rx=0 tx=0).stop 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 shutdown_connections 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7ff53c06be60 0x7ff53c06e320 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff558160030 0x7ff5580b14d0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff5580d1760 0x7ff5580b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 --2- 172.21.15.99:0/3853721023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5580d0130 0x7ff5580c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 >> 172.21.15.99:0/3853721023 conn(0x7ff5580b4650 msgr2=0x7ff558070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 shutdown_connections 2024-09-10T11:40:13.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.382+0000 7ff55f3c6640 1 -- 172.21.15.99:0/3853721023 wait complete. 2024-09-10T11:40:13.394 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-10T11:40:13.037566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9458,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553041,"statfs":{"total":1600000000,"available":1590314336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485521,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000988"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":509,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-10T11:40:10.729344+0000","last_change":"2024-09-10T11:30:18.077228+0000","last_active":"2024-09-10T11:40:10.729344+0000","last_peered":"2024-09-10T11:40:10.729344+0000","last_clean":"2024-09-10T11:40:10.729344+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:40:10.729344+0000","last_undegraded":"2024-09-10T11:40:10.729344+0000","last_fullsized":"2024-09-10T11:40:10.729344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:30:16.936443+0000","last_clean_scrub_stamp":"2024-09-10T11:30:16.936443+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:40.050393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574969,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574970,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574969,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:40:13.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T11:40:13.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.491+0000 7f8ac23c0640 1 Processor -- start 2024-09-10T11:40:13.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.491+0000 7f8ac23c0640 1 -- start start 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac23c0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8abc073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac23c0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8abc15c060 0x7f8abc15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac23c0640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8abc152e60 con 0x7f8abc0d1200 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac23c0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f8abc0d15e0 con 0x7f8abc15c060 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac23c0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc072680 con 0x7f8abc06ece0 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abbfff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8abc0d1200 0x7f8abc06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35880/0 (socket says 172.21.15.99:35880) 2024-09-10T11:40:13.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abbfff640 1 -- 172.21.15.99:0/2987909091 learned_addr learned my addr 172.21.15.99:0/2987909091 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2987909091 >> v2:172.21.15.196:3300/0 conn(0x7f8abc15c060 0x7f8abc15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115396337 0 0) 0x7f8abc152e60 con 0x7f8abc0d1200 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aa4003520 con 0x7f8abc0d1200 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abb7fe640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8abc073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1539431244 0 0) 0x7f8aa4003520 con 0x7f8abc0d1200 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 msgr2=0x7f8abc073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8abc073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 >> v2:172.21.15.196:3300/0 conn(0x7f8abc15c060 msgr2=0x7f8abc15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 --2- 172.21.15.99:0/2987909091 >> v2:172.21.15.196:3300/0 conn(0x7f8abc15c060 0x7f8abc15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc072960 con 0x7f8abc0d1200 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f8aac001670 con 0x7f8abc0d1200 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2987909091 >> v2:172.21.15.196:3300/0 conn(0x7f8abc15c060 0x7f8abc15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:13.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 >> v1:172.21.15.99:6789/0 conn(0x7f8abc0d1200 legacy=0x7f8abc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8aa4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4004d40 0x7f8aa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc0d15e0 con 0x7f8aa40039b0 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc072680 con 0x7f8aa4004d40 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa400bb50 con 0x7f8abc06ece0 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.492+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4004d40 0x7f8aa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abb7fe640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8aa4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 -- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 msgr2=0x7f8aa4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc06ece0 0x7f8aa4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 -- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4004d40 msgr2=0x7f8aa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4004d40 0x7f8aa40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 -- 172.21.15.99:0/2987909091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa4003520 con 0x7f8aa40039b0 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa4004d40 0x7f8aa40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c210 tx=0x7f8aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9aa0a970ecf047 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007d50 con 0x7f8aa40039b0 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.493+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac005ad0 con 0x7f8aa40039b0 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8abaffd640 1 -- 172.21.15.99:0/2987909091 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007070 con 0x7f8aa40039b0 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2987909091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa40039b0 msgr2=0x7f8aa4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2987909091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c210 tx=0x7f8aa4005830 comp rx=0 tx=0).stop 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac095d640 1 -- 172.21.15.99:0/2987909091 reap_dead start 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2987909091 shutdown_connections 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2987909091 >> 172.21.15.99:0/2987909091 conn(0x7f8abc06bc10 msgr2=0x7f8abc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2987909091 shutdown_connections 2024-09-10T11:40:13.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.494+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2987909091 wait complete. 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.495+0000 7f8ac23c0640 1 Processor -- start 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.495+0000 7f8ac23c0640 1 -- start start 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa40039b0 0x7f8abc154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc153610 0x7f8abc1441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc15cd90 con 0x7f8abc155a40 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8abc072f20 con 0x7f8aa40039b0 2024-09-10T11:40:13.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac23c0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f8abc15c9e0 con 0x7f8abc153610 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35568/0 (socket says 172.21.15.99:35568) 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 -- 172.21.15.99:0/2277286209 learned_addr learned my addr 172.21.15.99:0/2277286209 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 -- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc153610 msgr2=0x7f8abc1441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8abb7fe640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc153610 0x7f8abc1441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc153610 0x7f8abc1441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 -- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa40039b0 msgr2=0x7f8abc154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa40039b0 0x7f8abc154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.496+0000 7f8ac095d640 1 -- 172.21.15.99:0/2277286209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8abc153b40 con 0x7f8abc155a40 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ac095d640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0b53d0 tx=0x7f8ab4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4014410 con 0x7f8abc155a40 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab4019070 con 0x7f8abc155a40 2024-09-10T11:40:13.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8abc153e20 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40147f0 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.497+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8abc1543e0 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.498+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8abc15c4a0 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.498+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ab4014990 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.499+0000 7f8ab8ff9640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 0x7f8aa806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.499+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f8ab408b080 con 0x7f8abc155a40 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.499+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 0x7f8aa806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.499+0000 7f8abbfff640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 0x7f8aa806e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8aac033000 tx=0x7f8aac007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:13.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.503+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ab4059d30 con 0x7f8abc155a40 2024-09-10T11:40:13.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.667+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8abc0cdae0 con 0x7f8abc155a40 2024-09-10T11:40:13.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.668+0000 7f8ab8ff9640 1 -- 172.21.15.99:0/2277286209 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) ==== 74+0+13043 (secure 0 0 0) 0x7f8ab4059630 con 0x7f8abc155a40 2024-09-10T11:40:13.669 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:13.669 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":189,"fsid":"fb5864c6-feee-49d0-8ee4-0b85b9c6d632","created":"2024-09-10T11:30:10.853690+0000","modified":"2024-09-10T11:40:10.724890+0000","last_up_change":"2024-09-10T11:30:15.930782+0000","last_in_change":"2024-09-10T11:30:12.922527+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:30:16.935517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6801","nonce":291490777}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6803","nonce":291490777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6807","nonce":291490777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":291490777},{"type":"v1","addr":"172.21.15.99:6805","nonce":291490777}]},"public_addr":"172.21.15.99:6801/291490777","cluster_addr":"172.21.15.99:6803/291490777","heartbeat_back_addr":"172.21.15.99:6807/291490777","heartbeat_front_addr":"172.21.15.99:6805/291490777","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6809","nonce":2120836022}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6811","nonce":2120836022}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6815","nonce":2120836022}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":2120836022},{"type":"v1","addr":"172.21.15.99:6813","nonce":2120836022}]},"public_addr":"172.21.15.99:6809/2120836022","cluster_addr":"172.21.15.99:6811/2120836022","heartbeat_back_addr":"172.21.15.99:6815/2120836022","heartbeat_front_addr":"172.21.15.99:6813/2120836022","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6817","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6819","nonce":1090025270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6823","nonce":1090025270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6831","nonce":1090025270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6825","nonce":1090025270},{"type":"v1","addr":"172.21.15.99:6828","nonce":1090025270}]},"public_addr":"172.21.15.99:6819/1090025270","cluster_addr":"172.21.15.99:6823/1090025270","heartbeat_back_addr":"172.21.15.99:6831/1090025270","heartbeat_front_addr":"172.21.15.99:6828/1090025270","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6818","nonce":1027928425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6821","nonce":1027928425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6827","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6829","nonce":1027928425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":1027928425},{"type":"v1","addr":"172.21.15.99:6826","nonce":1027928425}]},"public_addr":"172.21.15.99:6818/1027928425","cluster_addr":"172.21.15.99:6821/1027928425","heartbeat_back_addr":"172.21.15.99:6829/1027928425","heartbeat_front_addr":"172.21.15.99:6826/1027928425","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6801","nonce":3885371275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6803","nonce":3885371275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6809","nonce":3885371275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3885371275},{"type":"v1","addr":"172.21.15.196:6805","nonce":3885371275}]},"public_addr":"172.21.15.196:6801/3885371275","cluster_addr":"172.21.15.196:6803/3885371275","heartbeat_back_addr":"172.21.15.196:6809/3885371275","heartbeat_front_addr":"172.21.15.196:6805/3885371275","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6825","nonce":3797029994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6827","nonce":3797029994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6831","nonce":3797029994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":3797029994},{"type":"v1","addr":"172.21.15.196:6829","nonce":3797029994}]},"public_addr":"172.21.15.196:6825/3797029994","cluster_addr":"172.21.15.196:6827/3797029994","heartbeat_back_addr":"172.21.15.196:6831/3797029994","heartbeat_front_addr":"172.21.15.196:6829/3797029994","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6808","nonce":4190591731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6811","nonce":4190591731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6815","nonce":4190591731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":4190591731},{"type":"v1","addr":"172.21.15.196:6813","nonce":4190591731}]},"public_addr":"172.21.15.196:6808/4190591731","cluster_addr":"172.21.15.196:6811/4190591731","heartbeat_back_addr":"172.21.15.196:6815/4190591731","heartbeat_front_addr":"172.21.15.196:6813/4190591731","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6817","nonce":106140719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6819","nonce":106140719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6823","nonce":106140719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":106140719},{"type":"v1","addr":"172.21.15.196:6821","nonce":106140719}]},"public_addr":"172.21.15.196:6817/106140719","cluster_addr":"172.21.15.196:6819/106140719","heartbeat_back_addr":"172.21.15.196:6823/106140719","heartbeat_front_addr":"172.21.15.196:6821/106140719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:13.995895+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.035476+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.058077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.028749+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.009275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.060859+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.094664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:30:14.020827+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6837/2972885632":"2024-09-11T11:40:07.632232+0000","172.21.15.99:6835/2972885632":"2024-09-11T11:40:07.632232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 msgr2=0x7f8aa806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 0x7f8aa806e320 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8aac033000 tx=0x7f8aac007000 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 msgr2=0x7f8abc06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0b53d0 tx=0x7f8ab4007700 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 shutdown_connections 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f8aa806be60 0x7f8aa806e320 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8abc155a40 0x7f8abc06d4f0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8abc153610 0x7f8abc1441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 --2- 172.21.15.99:0/2277286209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8aa40039b0 0x7f8abc154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:13.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 >> 172.21.15.99:0/2277286209 conn(0x7f8abc06bc10 msgr2=0x7f8abc0cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:13.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 shutdown_connections 2024-09-10T11:40:13.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:13.670+0000 7f8ac23c0640 1 -- 172.21.15.99:0/2277286209 wait complete. 2024-09-10T11:40:14.683 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T11:40:14.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:14.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.800+0000 7f54ef7fd640 1 Processor -- start 2024-09-10T11:40:14.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.800+0000 7f54ef7fd640 1 -- start start 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ef7fd640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54e80b62a0 0x7f54e80733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ef7fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ef7fd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e8145790 con 0x7f54e81453b0 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ef7fd640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f54e80d1380 con 0x7f54e80b62a0 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ef7fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54e80ce9f0 con 0x7f54e80701c0 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ecd98640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54e80b62a0 0x7f54e80733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ecd98640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54e80b62a0 0x7f54e80733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35570/0 (socket says 172.21.15.99:35570) 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.801+0000 7f54ecd98640 1 -- 172.21.15.99:0/307573781 learned_addr learned my addr 172.21.15.99:0/307573781 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.802+0000 7f54edd9a640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.802+0000 7f54ecd98640 1 -- 172.21.15.99:0/307573781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 msgr2=0x7f54e80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.802+0000 7f54ecd98640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.802+0000 7f54ecd98640 1 -- 172.21.15.99:0/307573781 >> v1:172.21.15.196:6789/0 conn(0x7f54e81453b0 legacy=0x7f54e80b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.802+0000 7f54ecd98640 1 -- 172.21.15.99:0/307573781 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e80d15d0 con 0x7f54e80b62a0 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ecd98640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80733d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009870 tx=0x7f54e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f619849e341f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54dffff640 1 -- 172.21.15.99:0/307573781 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4037070 con 0x7f54e80b62a0 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54dffff640 1 -- 172.21.15.99:0/307573781 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4032440 con 0x7f54e80b62a0 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54dffff640 1 -- 172.21.15.99:0/307573781 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4032810 con 0x7f54e80b62a0 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 -- 172.21.15.99:0/307573781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 msgr2=0x7f54e80733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80733d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009870 tx=0x7f54e402a270 comp rx=0 tx=0).stop 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 -- 172.21.15.99:0/307573781 shutdown_connections 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/307573781 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80733d0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 -- 172.21.15.99:0/307573781 >> 172.21.15.99:0/307573781 conn(0x7f54e8072680 msgr2=0x7f54e8072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:14.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.803+0000 7f54ef7fd640 1 -- 172.21.15.99:0/307573781 shutdown_connections 2024-09-10T11:40:14.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.804+0000 7f54ef7fd640 1 -- 172.21.15.99:0/307573781 wait complete. 2024-09-10T11:40:14.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 Processor -- start 2024-09-10T11:40:14.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 -- start start 2024-09-10T11:40:14.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e81453b0 0x7f54e81e1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54e80ce430 con 0x7f54e80b62a0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8073910 con 0x7f54e81453b0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.805+0000 7f54ef7fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f54e80ce710 con 0x7f54e80701c0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35574/0 (socket says 172.21.15.99:35574) 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 -- 172.21.15.99:0/2390628156 learned_addr learned my addr 172.21.15.99:0/2390628156 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ed599640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54edd9a640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e81453b0 0x7f54e81e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 -- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 msgr2=0x7f54e80cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54e80701c0 0x7f54e80cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 -- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e81453b0 msgr2=0x7f54e81e1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54e81453b0 0x7f54e81e1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e8140430 con 0x7f54e80b62a0 2024-09-10T11:40:14.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ecd98640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80b15f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c00 tx=0x7f54e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4037040 con 0x7f54e80b62a0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e4042070 con 0x7f54e80b62a0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e80b2890 con 0x7f54e80b62a0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.806+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4030880 con 0x7f54e80b62a0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.807+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81409b0 con 0x7f54e80b62a0 2024-09-10T11:40:14.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.807+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54e8071020 con 0x7f54e80b62a0 2024-09-10T11:40:14.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.808+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54e4030a20 con 0x7f54e80b62a0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.808+0000 7f54ddffb640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f54cc06be60 0x7f54cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f54e40ab240 con 0x7f54e80b62a0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ed599640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f54cc06be60 0x7f54cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ddffb640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7f54cc071d20 0x7f54cc074140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54cc0747f0 con 0x7f54cc071d20 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f54e40b2050 con 0x7f54e80b62a0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54edd9a640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7f54cc071d20 0x7f54cc074140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54ed599640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f54cc06be60 0x7f54cc06e320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000f60 tx=0x7f54d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.809+0000 7f54edd9a640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7f54cc071d20 0x7f54cc074140 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.810+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== osd.0 v2:172.21.15.99:6800/291490777 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f54cc0747f0 con 0x7f54cc071d20 2024-09-10T11:40:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.821+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 --> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54e81413f0 con 0x7f54cc071d20 2024-09-10T11:40:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.821+0000 7f54ddffb640 1 -- 172.21.15.99:0/2390628156 <== osd.0 v2:172.21.15.99:6800/291490777 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f54e81413f0 con 0x7f54cc071d20 2024-09-10T11:40:14.822 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:14.822 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:14.822 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7f54cc071d20 msgr2=0x7f54cc074140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6800/291490777,v1:172.21.15.99:6801/291490777] conn(0x7f54cc071d20 0x7f54cc074140 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f54cc06be60 msgr2=0x7f54cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f54cc06be60 0x7f54cc06e320 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f54d8000f60 tx=0x7f54d8009040 comp rx=0 tx=0).stop 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 msgr2=0x7f54e80b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 --2- 172.21.15.99:0/2390628156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54e80b62a0 0x7f54e80b15f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f54e4000c00 tx=0x7f54e40046c0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54edd9a640 1 -- 172.21.15.99:0/2390628156 reap_dead start 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 shutdown_connections 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 >> 172.21.15.99:0/2390628156 conn(0x7f54e8072680 msgr2=0x7f54e81e8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 shutdown_connections 2024-09-10T11:40:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.822+0000 7f54ef7fd640 1 -- 172.21.15.99:0/2390628156 wait complete. 2024-09-10T11:40:14.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T11:40:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.934+0000 7f732dbaf640 1 Processor -- start 2024-09-10T11:40:14.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.934+0000 7f732dbaf640 1 -- start start 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f732dbaf640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73281626c0 0x7f73280d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f732dbaf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 0x7f73280d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f732dbaf640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732815ff70 con 0x7f732815eb80 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f732dbaf640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f73280d30c0 con 0x7f73281626c0 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f732dbaf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f73280d1620 con 0x7f73280d4e40 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73281626c0 0x7f73280d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f73281626c0 0x7f73280d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35590/0 (socket says 172.21.15.99:35590) 2024-09-10T11:40:14.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 -- 172.21.15.99:0/1009787592 learned_addr learned my addr 172.21.15.99:0/1009787592 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f7327fff640 1 --2- 172.21.15.99:0/1009787592 >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 0x7f73280d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 -- 172.21.15.99:0/1009787592 >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 msgr2=0x7f73280d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 --2- 172.21.15.99:0/1009787592 >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 0x7f73280d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 -- 172.21.15.99:0/1009787592 >> v1:172.21.15.196:6789/0 conn(0x7f732815eb80 legacy=0x7f73280d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f731ffff640 1 -- 172.21.15.99:0/1009787592 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7328162bc0 con 0x7f73281626c0 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.935+0000 7f7327fff640 1 --2- 172.21.15.99:0/1009787592 >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 0x7f73280d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.936+0000 7f731ffff640 1 --2- 172.21.15.99:0/1009787592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73281626c0 0x7f73280d4900 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0f4a967999621f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.936+0000 7f7326ffd640 1 -- 172.21.15.99:0/1009787592 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310037070 con 0x7f73281626c0 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.936+0000 7f7326ffd640 1 -- 172.21.15.99:0/1009787592 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310032440 con 0x7f73281626c0 2024-09-10T11:40:14.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.936+0000 7f7326ffd640 1 -- 172.21.15.99:0/1009787592 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310032810 con 0x7f73281626c0 2024-09-10T11:40:14.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 -- 172.21.15.99:0/1009787592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73281626c0 msgr2=0x7f73280d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 --2- 172.21.15.99:0/1009787592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73281626c0 0x7f73280d4900 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7310009870 tx=0x7f731002a210 comp rx=0 tx=0).stop 2024-09-10T11:40:14.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 -- 172.21.15.99:0/1009787592 shutdown_connections 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 --2- 172.21.15.99:0/1009787592 >> v2:172.21.15.196:3300/0 conn(0x7f73280d4e40 0x7f73280d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 --2- 172.21.15.99:0/1009787592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73281626c0 0x7f73280d4900 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 -- 172.21.15.99:0/1009787592 >> 172.21.15.99:0/1009787592 conn(0x7f732808e680 msgr2=0x7f732809dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 -- 172.21.15.99:0/1009787592 shutdown_connections 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.937+0000 7f732dbaf640 1 -- 172.21.15.99:0/1009787592 wait complete. 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 Processor -- start 2024-09-10T11:40:14.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 -- start start 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f732815eb80 0x7f7328161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73281626c0 0x7f732815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7328163460 con 0x7f73280d4e40 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7328163180 con 0x7f73281626c0 2024-09-10T11:40:14.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f732dbaf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f73280d5290 con 0x7f732815eb80 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.939+0000 7f73277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35592/0 (socket says 172.21.15.99:35592) 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 -- 172.21.15.99:0/2771319313 learned_addr learned my addr 172.21.15.99:0/2771319313 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f7327fff640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73281626c0 0x7f732815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f731ffff640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f732815eb80 0x7f7328161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 -- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f732815eb80 msgr2=0x7f7328161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f732815eb80 0x7f7328161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 -- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73281626c0 msgr2=0x7f732815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73281626c0 0x7f732815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 -- 172.21.15.99:0/2771319313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73281623f0 con 0x7f73280d4e40 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f73277fe640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f731800c3c0 tx=0x7f731800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73180052d0 con 0x7f73280d4e40 2024-09-10T11:40:14.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7318005490 con 0x7f73280d4e40 2024-09-10T11:40:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73280b4f40 con 0x7f73280d4e40 2024-09-10T11:40:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7318007260 con 0x7f73280d4e40 2024-09-10T11:40:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.940+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7328161990 con 0x7f73280d4e40 2024-09-10T11:40:14.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.941+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73280d4900 con 0x7f73280d4e40 2024-09-10T11:40:14.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.942+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7318007400 con 0x7f73280d4e40 2024-09-10T11:40:14.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.942+0000 7f7324ff9640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 0x7f730806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:14.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.942+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f7318089f90 con 0x7f73280d4e40 2024-09-10T11:40:14.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.942+0000 7f731ffff640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 0x7f730806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:14.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.943+0000 7f731ffff640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 0x7f730806e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:14.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:14.947+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f73180585f0 con 0x7f73280d4e40 2024-09-10T11:40:15.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.120+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f73280d1260 con 0x7f730806be30 2024-09-10T11:40:15.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.120+0000 7f7324ff9640 1 -- 172.21.15.99:0/2771319313 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f73280d1260 con 0x7f730806be30 2024-09-10T11:40:15.121 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 msgr2=0x7f730806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 0x7f730806e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7310034000 tx=0x7f731002e000 comp rx=0 tx=0).stop 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 msgr2=0x7f732815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f731800c3c0 tx=0x7f731800c880 comp rx=0 tx=0).stop 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 shutdown_connections 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f730806be30 0x7f730806e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73281626c0 0x7f732815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f732815eb80 0x7f7328161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 --2- 172.21.15.99:0/2771319313 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73280d4e40 0x7f732815a390 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 >> 172.21.15.99:0/2771319313 conn(0x7f732808e680 msgr2=0x7f732806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 shutdown_connections 2024-09-10T11:40:15.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.122+0000 7f732dbaf640 1 -- 172.21.15.99:0/2771319313 wait complete. 2024-09-10T11:40:15.134 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T11:40:15.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 Processor -- start 2024-09-10T11:40:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 -- start start 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ac40d16f0 0x7f2ac40d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 0x7f2ac4163420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac40d5b70 con 0x7f2ac40b59b0 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac40d0d30 con 0x7f2ac40d16f0 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.242+0000 7f2aca359640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac40b4d30 con 0x7f2ac40d5740 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac37fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2ac40b59b0 0x7f2ac4162d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35894/0 (socket says 172.21.15.99:35894) 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac3fff640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2ac40d16f0 0x7f2ac40d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac37fe640 1 -- 172.21.15.99:0/2131602955 learned_addr learned my addr 172.21.15.99:0/2131602955 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac88f6640 1 --2- 172.21.15.99:0/2131602955 >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 0x7f2ac4163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac3fff640 1 -- 172.21.15.99:0/2131602955 >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 msgr2=0x7f2ac4163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac3fff640 1 --2- 172.21.15.99:0/2131602955 >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 0x7f2ac4163420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac3fff640 1 -- 172.21.15.99:0/2131602955 >> v1:172.21.15.99:6789/0 conn(0x7f2ac40b59b0 legacy=0x7f2ac4162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac2ffd640 1 -- 172.21.15.99:0/2131602955 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766036261 0 0) 0x7f2ab8001110 con 0x7f2ac40b59b0 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac3fff640 1 -- 172.21.15.99:0/2131602955 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac40d2150 con 0x7f2ac40d16f0 2024-09-10T11:40:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.243+0000 7f2ac88f6640 1 --2- 172.21.15.99:0/2131602955 >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 0x7f2ac4163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:15.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2ac3fff640 1 --2- 172.21.15.99:0/2131602955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d16f0 0x7f2ac40d1ad0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40b63c0 tx=0x7f2ab402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7760c9c70d2c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2ac2ffd640 1 -- 172.21.15.99:0/2131602955 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4037070 con 0x7f2ac40d16f0 2024-09-10T11:40:15.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2ac2ffd640 1 -- 172.21.15.99:0/2131602955 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4032440 con 0x7f2ac40d16f0 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2ac2ffd640 1 -- 172.21.15.99:0/2131602955 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402ae60 con 0x7f2ac40d16f0 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2aca359640 1 -- 172.21.15.99:0/2131602955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d16f0 msgr2=0x7f2ac40d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.244+0000 7f2aca359640 1 --2- 172.21.15.99:0/2131602955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d16f0 0x7f2ac40d1ad0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40b63c0 tx=0x7f2ab402a110 comp rx=0 tx=0).stop 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 -- 172.21.15.99:0/2131602955 shutdown_connections 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 --2- 172.21.15.99:0/2131602955 >> v2:172.21.15.196:3300/0 conn(0x7f2ac40d5740 0x7f2ac4163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 --2- 172.21.15.99:0/2131602955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d16f0 0x7f2ac40d1ad0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 -- 172.21.15.99:0/2131602955 >> 172.21.15.99:0/2131602955 conn(0x7f2ac40d4c20 msgr2=0x7f2ac40d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 -- 172.21.15.99:0/2131602955 shutdown_connections 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.245+0000 7f2aca359640 1 -- 172.21.15.99:0/2131602955 wait complete. 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.246+0000 7f2aca359640 1 Processor -- start 2024-09-10T11:40:15.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.246+0000 7f2aca359640 1 -- start start 2024-09-10T11:40:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac40b59b0 0x7f2ac415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ac40d16f0 0x7f2ac4159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 0x7f2ac41f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac40d23f0 con 0x7f2ac40d5740 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac40b52f0 con 0x7f2ac40b59b0 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2aca359640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac40b4f80 con 0x7f2ac40d16f0 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 0x7f2ac41f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac3fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac40b59b0 0x7f2ac415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 0x7f2ac41f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35614/0 (socket says 172.21.15.99:35614) 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 -- 172.21.15.99:0/200680782 learned_addr learned my addr 172.21.15.99:0/200680782 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac37fe640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ac40d16f0 0x7f2ac4159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 -- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ac40d16f0 msgr2=0x7f2ac4159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ac40d16f0 0x7f2ac4159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 -- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac40b59b0 msgr2=0x7f2ac415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac40b59b0 0x7f2ac415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac415c580 con 0x7f2ac40d5740 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac3fff640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ac40b59b0 0x7f2ac415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.247+0000 7f2ac88f6640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 0x7f2ac41f8800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2abc007e20 tx=0x7f2abc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.248+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc01a070 con 0x7f2ac40d5740 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.248+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc003b90 con 0x7f2ac40d5740 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.248+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac4159b50 con 0x7f2ac40d5740 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.248+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc00de80 con 0x7f2ac40d5740 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.248+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac415b560 con 0x7f2ac40d5740 2024-09-10T11:40:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.249+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ac40d1ad0 con 0x7f2ac40d5740 2024-09-10T11:40:15.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.249+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2abc01d070 con 0x7f2ac40d5740 2024-09-10T11:40:15.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.249+0000 7f2ac0ff9640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab006be30 0x7f2ab006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f2abc08ad40 con 0x7f2ac40d5740 2024-09-10T11:40:15.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac3fff640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab006be30 0x7f2ab006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac0ff9640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2ab0071cf0 0x7f2ab0074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2abc00be50 con 0x7f2ab0071cf0 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f2abc08d440 con 0x7f2ac40d5740 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac37fe640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2ab0071cf0 0x7f2ab0074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac3fff640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab006be30 0x7f2ab006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4034000 tx=0x7f2ab402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.250+0000 7f2ac37fe640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2ab0071cf0 0x7f2ab0074110 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.251+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== osd.1 v2:172.21.15.99:6808/2120836022 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f2abc00be50 con 0x7f2ab0071cf0 2024-09-10T11:40:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.262+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 --> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2ac40d3710 con 0x7f2ab0071cf0 2024-09-10T11:40:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.262+0000 7f2ac0ff9640 1 -- 172.21.15.99:0/200680782 <== osd.1 v2:172.21.15.99:6808/2120836022 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f2ac40d3710 con 0x7f2ab0071cf0 2024-09-10T11:40:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2ab0071cf0 msgr2=0x7f2ab0074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6808/2120836022,v1:172.21.15.99:6809/2120836022] conn(0x7f2ab0071cf0 0x7f2ab0074110 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab006be30 msgr2=0x7f2ab006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f2ab006be30 0x7f2ab006e2f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4034000 tx=0x7f2ab402e000 comp rx=0 tx=0).stop 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 msgr2=0x7f2ac41f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 --2- 172.21.15.99:0/200680782 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ac40d5740 0x7f2ac41f8800 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f2abc007e20 tx=0x7f2abc00d4a0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2ac88f6640 1 -- 172.21.15.99:0/200680782 reap_dead start 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 shutdown_connections 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 >> 172.21.15.99:0/200680782 conn(0x7f2ac40d4c20 msgr2=0x7f2ac40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 shutdown_connections 2024-09-10T11:40:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.263+0000 7f2aca359640 1 -- 172.21.15.99:0/200680782 wait complete. 2024-09-10T11:40:15.275 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T11:40:15.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 Processor -- start 2024-09-10T11:40:15.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 -- start start 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b8806df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7b88163160 0x7f7b880d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b881635f0 con 0x7f7b880cfb10 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7b88163860 con 0x7f7b88163160 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.376+0000 7f7b8e677640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7b880b50e0 con 0x7f7b8806ea60 2024-09-10T11:40:15.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b87fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7b880cfb10 0x7f7b88162a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35898/0 (socket says 172.21.15.99:35898) 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b87fff640 1 -- 172.21.15.99:0/137843905 learned_addr learned my addr 172.21.15.99:0/137843905 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b8cc14640 1 --2- 172.21.15.99:0/137843905 >> v2:172.21.15.196:3300/0 conn(0x7f7b88163160 0x7f7b880d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b8806df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1101631846 0 0) 0x7f7b881635f0 con 0x7f7b880cfb10 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b74003520 con 0x7f7b880cfb10 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1088116579 0 0) 0x7f7b74003520 con 0x7f7b880cfb10 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 msgr2=0x7f7b8806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b8806df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 >> v2:172.21.15.196:3300/0 conn(0x7f7b88163160 msgr2=0x7f7b880d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 --2- 172.21.15.99:0/137843905 >> v2:172.21.15.196:3300/0 conn(0x7f7b88163160 0x7f7b880d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880b53c0 con 0x7f7b880cfb10 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f7b78001670 con 0x7f7b880cfb10 2024-09-10T11:40:15.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 >> v1:172.21.15.99:6789/0 conn(0x7f7b880cfb10 legacy=0x7f7b88162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88163860 con 0x7f7b74004800 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880b50e0 con 0x7f7b740039b0 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7400b9c0 con 0x7f7b8806ea60 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b87fff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.377+0000 7f7b8cc14640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b7ffff640 1 -- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 msgr2=0x7f7b740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8806ea60 0x7f7b740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b7ffff640 1 -- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740039b0 msgr2=0x7f7b74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b7ffff640 1 -- 172.21.15.99:0/137843905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b74003520 con 0x7f7b74004800 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.378+0000 7f7b87fff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b740039b0 0x7f7b74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009a90 tx=0x7f7b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ec86bbe1543d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70036070 con 0x7f7b74004800 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b70031400 con 0x7f7b74004800 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b877fe640 1 -- 172.21.15.99:0/137843905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70009690 con 0x7f7b74004800 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 -- 172.21.15.99:0/137843905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 msgr2=0x7f7b74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 --2- 172.21.15.99:0/137843905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b74004c80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009a90 tx=0x7f7b74005830 comp rx=0 tx=0).stop 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8cc14640 1 -- 172.21.15.99:0/137843905 reap_dead start 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 -- 172.21.15.99:0/137843905 shutdown_connections 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 -- 172.21.15.99:0/137843905 >> 172.21.15.99:0/137843905 conn(0x7f7b880b45c0 msgr2=0x7f7b880b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 -- 172.21.15.99:0/137843905 shutdown_connections 2024-09-10T11:40:15.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.379+0000 7f7b8e677640 1 -- 172.21.15.99:0/137843905 wait complete. 2024-09-10T11:40:15.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.381+0000 7f7b8e677640 1 Processor -- start 2024-09-10T11:40:15.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.381+0000 7f7b8e677640 1 -- start start 2024-09-10T11:40:15.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b880b58e0 0x7f7b88160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b880b5d60 0x7f7b881612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880709d0 con 0x7f7b74004800 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7b880704b0 con 0x7f7b880b58e0 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7b880b62c0 con 0x7f7b880b5d60 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35632/0 (socket says 172.21.15.99:35632) 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 -- 172.21.15.99:0/1013869858 learned_addr learned my addr 172.21.15.99:0/1013869858 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b880b58e0 0x7f7b88160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8cc14640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b880b5d60 0x7f7b881612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 -- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b880b5d60 msgr2=0x7f7b881612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b880b5d60 0x7f7b881612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 -- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b880b58e0 msgr2=0x7f7b88160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b880b58e0 0x7f7b88160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 -- 172.21.15.99:0/1013869858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b880b65a0 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b87fff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7b78007b60 tx=0x7f7b78007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800fa20 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b78018070 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7800fdc0 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b880b6880 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.382+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881f4650 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.384+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b880b6ea0 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.384+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b7801d460 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.384+0000 7f7b857fa640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 0x7f7b6806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.385+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f7b7808ac80 con 0x7f7b74004800 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.385+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 0x7f7b6806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.385+0000 7f7b7ffff640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 0x7f7b6806e320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7b70033000 tx=0x7f7b70008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.393+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b780592e0 con 0x7f7b74004800 2024-09-10T11:40:15.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.565+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7b8806df60 con 0x7f7b6806be60 2024-09-10T11:40:15.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.566+0000 7f7b857fa640 1 -- 172.21.15.99:0/1013869858 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7b8806df60 con 0x7f7b6806be60 2024-09-10T11:40:15.566 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 msgr2=0x7f7b6806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 0x7f7b6806e320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7b70033000 tx=0x7f7b70008030 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 msgr2=0x7f7b8815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f7b78007b60 tx=0x7f7b78007720 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 shutdown_connections 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7b6806be60 0x7f7b6806e320 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b880b5d60 0x7f7b881612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b880b58e0 0x7f7b88160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 --2- 172.21.15.99:0/1013869858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b74004800 0x7f7b8815bf30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 >> 172.21.15.99:0/1013869858 conn(0x7f7b880b45c0 msgr2=0x7f7b880d25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 shutdown_connections 2024-09-10T11:40:15.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.568+0000 7f7b8e677640 1 -- 172.21.15.99:0/1013869858 wait complete. 2024-09-10T11:40:15.579 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T11:40:15.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.682+0000 7f6e4bd8d640 1 Processor -- start 2024-09-10T11:40:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.682+0000 7f6e4bd8d640 1 -- start start 2024-09-10T11:40:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4bd8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e4415cab0 0x7f6e440b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4bd8d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e440b6510 0x7f6e440b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4bd8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e440b2510 con 0x7f6e440b1fa0 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4bd8d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f6e440b6e80 con 0x7f6e440b6510 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4bd8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e440b27f0 con 0x7f6e4415cab0 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4a32a640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e440b6510 0x7f6e440b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e49328640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6e440b1fa0 0x7f6e4415ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35906/0 (socket says 172.21.15.99:35906) 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e49328640 1 -- 172.21.15.99:0/3618701463 learned_addr learned my addr 172.21.15.99:0/3618701463 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e49b29640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e4415cab0 0x7f6e440b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4a32a640 1 -- 172.21.15.99:0/3618701463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e4415cab0 msgr2=0x7f6e440b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4a32a640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e4415cab0 0x7f6e440b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4a32a640 1 -- 172.21.15.99:0/3618701463 >> v1:172.21.15.99:6789/0 conn(0x7f6e440b1fa0 legacy=0x7f6e4415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e4a32a640 1 -- 172.21.15.99:0/3618701463 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e440b2ad0 con 0x7f6e440b6510 2024-09-10T11:40:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.683+0000 7f6e48b27640 1 -- 172.21.15.99:0/3618701463 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258594730 0 0) 0x7f6e440b2510 con 0x7f6e440b1fa0 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.684+0000 7f6e4a32a640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e440b6510 0x7f6e440b6940 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009d90 tx=0x7f6e3c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18799de94f30f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.684+0000 7f6e48b27640 1 -- 172.21.15.99:0/3618701463 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c007740 con 0x7f6e440b6510 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.684+0000 7f6e48b27640 1 -- 172.21.15.99:0/3618701463 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0078e0 con 0x7f6e440b6510 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.684+0000 7f6e48b27640 1 -- 172.21.15.99:0/3618701463 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c007cb0 con 0x7f6e440b6510 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/3618701463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e440b6510 msgr2=0x7f6e440b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e440b6510 0x7f6e440b6940 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009d90 tx=0x7f6e3c02a6f0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/3618701463 shutdown_connections 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e440b6510 0x7f6e440b6940 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/3618701463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e4415cab0 0x7f6e440b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/3618701463 >> 172.21.15.99:0/3618701463 conn(0x7f6e44073c60 msgr2=0x7f6e4409ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/3618701463 shutdown_connections 2024-09-10T11:40:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.685+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/3618701463 wait complete. 2024-09-10T11:40:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.686+0000 7f6e4bd8d640 1 Processor -- start 2024-09-10T11:40:15.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.686+0000 7f6e4bd8d640 1 -- start start 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e440b1fa0 0x7f6e4414cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e440b6510 0x7f6e4406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 0x7f6e441572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440b6120 con 0x7f6e4415cab0 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6e440b3090 con 0x7f6e440b6510 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4bd8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f6e440b35b0 con 0x7f6e440b1fa0 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 0x7f6e441572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 0x7f6e441572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35654/0 (socket says 172.21.15.99:35654) 2024-09-10T11:40:15.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 -- 172.21.15.99:0/851156475 learned_addr learned my addr 172.21.15.99:0/851156475 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e49b29640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e440b1fa0 0x7f6e4414cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e49328640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e440b6510 0x7f6e4406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 -- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e440b1fa0 msgr2=0x7f6e4414cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e440b1fa0 0x7f6e4414cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 -- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e440b6510 msgr2=0x7f6e4406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e440b6510 0x7f6e4406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e44151710 con 0x7f6e4415cab0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e49b29640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e440b1fa0 0x7f6e4414cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.687+0000 7f6e4a32a640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 0x7f6e441572c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02ace0 tx=0x7f6e3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.688+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c040070 con 0x7f6e4415cab0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.688+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e3c031bc0 con 0x7f6e4415cab0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.688+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c009450 con 0x7f6e4415cab0 2024-09-10T11:40:15.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.688+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e441519f0 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.688+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e44151fb0 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.689+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e440b6940 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.689+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6e3c039040 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.689+0000 7f6e3a7fc640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e2806be30 0x7f6e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f6e3c0abc10 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e49b29640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e2806be30 0x7f6e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e3a7fc640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f6e28071cf0 0x7f6e28074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e3c039e50 con 0x7f6e28071cf0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f6e3c0b3050 con 0x7f6e4415cab0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e49328640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f6e28071cf0 0x7f6e28074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e49b29640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e2806be30 0x7f6e2806e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009e90 tx=0x7f6e34009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.690+0000 7f6e49328640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f6e28071cf0 0x7f6e28074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.691+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== osd.2 v2:172.21.15.99:6817/1090025270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f6e3c039e50 con 0x7f6e28071cf0 2024-09-10T11:40:15.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 --> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6e440b0fd0 con 0x7f6e28071cf0 2024-09-10T11:40:15.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e3a7fc640 1 -- 172.21.15.99:0/851156475 <== osd.2 v2:172.21.15.99:6817/1090025270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f6e440b0fd0 con 0x7f6e28071cf0 2024-09-10T11:40:15.703 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:15.703 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:15.703 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f6e28071cf0 msgr2=0x7f6e28074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6817/1090025270,v1:172.21.15.99:6819/1090025270] conn(0x7f6e28071cf0 0x7f6e28074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e2806be30 msgr2=0x7f6e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f6e2806be30 0x7f6e2806e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009e90 tx=0x7f6e34009190 comp rx=0 tx=0).stop 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 msgr2=0x7f6e441572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 --2- 172.21.15.99:0/851156475 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e4415cab0 0x7f6e441572c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02ace0 tx=0x7f6e3c004a40 comp rx=0 tx=0).stop 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4a32a640 1 -- 172.21.15.99:0/851156475 reap_dead start 2024-09-10T11:40:15.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.702+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 shutdown_connections 2024-09-10T11:40:15.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.703+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 >> 172.21.15.99:0/851156475 conn(0x7f6e44073c60 msgr2=0x7f6e440b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.703+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 shutdown_connections 2024-09-10T11:40:15.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.703+0000 7f6e4bd8d640 1 -- 172.21.15.99:0/851156475 wait complete. 2024-09-10T11:40:15.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T11:40:15.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 Processor -- start 2024-09-10T11:40:15.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 -- start start 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2774161160 0x7f277415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2774070b70 con 0x7f277415d5c0 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f2774160cd0 con 0x7f277406bc10 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.813+0000 7f27798c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f277406c580 con 0x7f2774161160 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f27737fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2774161160 0x7f277415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f2772ffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f277415d5c0 0x7f2774070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35916/0 (socket says 172.21.15.99:35916) 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2774161160 0x7f277415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.99:43516/0 (socket says 172.21.15.99:43516) 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f2772ffd640 1 -- 172.21.15.99:0/4059470270 learned_addr learned my addr 172.21.15.99:0/4059470270 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:15.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f27737fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35668/0 (socket says 172.21.15.99:35668) 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 -- 172.21.15.99:0/4059470270 >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 msgr2=0x7f277406c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 --2- 172.21.15.99:0/4059470270 >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 -- 172.21.15.99:0/4059470270 >> v1:172.21.15.99:6789/0 conn(0x7f277415d5c0 legacy=0x7f2774070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f27727fc640 1 -- 172.21.15.99:0/4059470270 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3844697424 0 0) 0x7f2774070b70 con 0x7f277415d5c0 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f276bfff640 1 -- 172.21.15.99:0/4059470270 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2774070e00 con 0x7f2774161160 2024-09-10T11:40:15.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.814+0000 7f27737fe640 1 --2- 172.21.15.99:0/4059470270 >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:15.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f276bfff640 1 --2- 172.21.15.99:0/4059470270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f277415d9a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f277406ccd0 tx=0x7f275c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c78df93f5931c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f27727fc640 1 -- 172.21.15.99:0/4059470270 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c037070 con 0x7f2774161160 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f27727fc640 1 -- 172.21.15.99:0/4059470270 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c032440 con 0x7f2774161160 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f27727fc640 1 -- 172.21.15.99:0/4059470270 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c02ae60 con 0x7f2774161160 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f27798c9640 1 -- 172.21.15.99:0/4059470270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 msgr2=0x7f277415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.815+0000 7f27798c9640 1 --2- 172.21.15.99:0/4059470270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f277415d9a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f277406ccd0 tx=0x7f275c02a110 comp rx=0 tx=0).stop 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 -- 172.21.15.99:0/4059470270 shutdown_connections 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 --2- 172.21.15.99:0/4059470270 >> v2:172.21.15.99:3300/0 conn(0x7f277406bc10 0x7f277406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 --2- 172.21.15.99:0/4059470270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f277415d9a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 -- 172.21.15.99:0/4059470270 >> 172.21.15.99:0/4059470270 conn(0x7f27740d1900 msgr2=0x7f27740d1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 -- 172.21.15.99:0/4059470270 shutdown_connections 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.816+0000 7f27798c9640 1 -- 172.21.15.99:0/4059470270 wait complete. 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 Processor -- start 2024-09-10T11:40:15.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 -- start start 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f277415d5c0 0x7f2774152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f27741f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f277406e660 con 0x7f277406bc10 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2774160960 con 0x7f2774161160 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.817+0000 7f27798c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f277406e7e0 con 0x7f277415d5c0 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35684/0 (socket says 172.21.15.99:35684) 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 -- 172.21.15.99:0/2390630612 learned_addr learned my addr 172.21.15.99:0/2390630612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f27737fe640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f27741f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f276bfff640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f277415d5c0 0x7f2774152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 -- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f277415d5c0 msgr2=0x7f2774152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f277415d5c0 0x7f2774152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 -- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 msgr2=0x7f27741f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f27741f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 -- 172.21.15.99:0/2390630612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277415b500 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f27737fe640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f27741f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2772ffd640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f276400c500 tx=0x7f276400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764018070 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764007500 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277415b7e0 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764004a20 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.818+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277415bda0 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.819+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277406c040 con 0x7f277406bc10 2024-09-10T11:40:15.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.820+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f276401b070 con 0x7f277406bc10 2024-09-10T11:40:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.821+0000 7f2770ff9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 0x7f275406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.821+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f2764089d20 con 0x7f277406bc10 2024-09-10T11:40:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.821+0000 7f276bfff640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 0x7f275406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:15.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.821+0000 7f276bfff640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 0x7f275406e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f275c034000 tx=0x7f275c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:15.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.825+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2764058410 con 0x7f277406bc10 2024-09-10T11:40:15.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.998+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f277415b2b0 con 0x7f275406be30 2024-09-10T11:40:15.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:15.998+0000 7f2770ff9640 1 -- 172.21.15.99:0/2390630612 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f277415b2b0 con 0x7f275406be30 2024-09-10T11:40:15.999 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T11:40:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 msgr2=0x7f275406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 0x7f275406e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f275c034000 tx=0x7f275c02e000 comp rx=0 tx=0).stop 2024-09-10T11:40:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 msgr2=0x7f2774152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f276400c500 tx=0x7f276400c900 comp rx=0 tx=0).stop 2024-09-10T11:40:16.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 shutdown_connections 2024-09-10T11:40:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f275406be30 0x7f275406e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2774161160 0x7f27741f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f277415d5c0 0x7f2774152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 --2- 172.21.15.99:0/2390630612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f277406bc10 0x7f2774152800 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 >> 172.21.15.99:0/2390630612 conn(0x7f27740d1900 msgr2=0x7f277415d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 shutdown_connections 2024-09-10T11:40:16.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.000+0000 7f27798c9640 1 -- 172.21.15.99:0/2390630612 wait complete. 2024-09-10T11:40:16.012 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T11:40:16.012 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:16.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.114+0000 7f0743e8d640 1 Processor -- start 2024-09-10T11:40:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.115+0000 7f0743e8d640 1 -- start start 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.115+0000 7f0743e8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.115+0000 7f0743e8d640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c0b2510 con 0x7f073c15cab0 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.115+0000 7f0743e8d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f073c0b27f0 con 0x7f073c0b6510 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.115+0000 7f0743e8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0b2ad0 con 0x7f073c0b1fa0 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f073c15cab0 0x7f073c0b5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35920/0 (socket says 172.21.15.99:35920) 2024-09-10T11:40:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 -- 172.21.15.99:0/2684650905 learned_addr learned my addr 172.21.15.99:0/2684650905 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741428640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538269023 0 0) 0x7f073c0b2510 con 0x7f073c15cab0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0724003520 con 0x7f073c15cab0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2707856805 0 0) 0x7f0724003520 con 0x7f073c15cab0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 msgr2=0x7f073c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 >> v1:172.21.15.196:6789/0 conn(0x7f073c0b6510 legacy=0x7f073c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c0b2db0 con 0x7f073c15cab0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f072c0019f0 con 0x7f073c15cab0 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 >> v1:172.21.15.99:6789/0 conn(0x7f073c15cab0 legacy=0x7f073c0b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f07240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0724003f20 0x7f0724004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724004d40 0x7f07240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0b2ad0 con 0x7f0724004d40 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f072400ba20 con 0x7f0724003f20 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f072400bc20 con 0x7f073c0b1fa0 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f07240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f074242a640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724004d40 0x7f07240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741428640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0724003f20 0x7f0724004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 -- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0724003f20 msgr2=0x7f0724004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0724003f20 0x7f0724004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 -- 172.21.15.99:0/2684650905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724004d40 msgr2=0x7f07240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.116+0000 7f0741c29640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724004d40 0x7f07240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f0741c29640 1 -- 172.21.15.99:0/2684650905 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0724003520 con 0x7f073c0b1fa0 2024-09-10T11:40:16.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f074242a640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0724004d40 0x7f07240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:16.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f0741c29640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f07240039e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f072c00a880 tx=0x7f0724005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b175e020a16e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c00a4a0 con 0x7f073c0b1fa0 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c02f070 con 0x7f073c0b1fa0 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.117+0000 7f0740c27640 1 -- 172.21.15.99:0/2684650905 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c02bc70 con 0x7f073c0b1fa0 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 -- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 msgr2=0x7f07240039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 --2- 172.21.15.99:0/2684650905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f07240039e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f072c00a880 tx=0x7f0724005830 comp rx=0 tx=0).stop 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f074242a640 1 -- 172.21.15.99:0/2684650905 reap_dead start 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 -- 172.21.15.99:0/2684650905 shutdown_connections 2024-09-10T11:40:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 -- 172.21.15.99:0/2684650905 >> 172.21.15.99:0/2684650905 conn(0x7f073c073c60 msgr2=0x7f073c09ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 -- 172.21.15.99:0/2684650905 shutdown_connections 2024-09-10T11:40:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.118+0000 7f0743e8d640 1 -- 172.21.15.99:0/2684650905 wait complete. 2024-09-10T11:40:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.119+0000 7f0743e8d640 1 Processor -- start 2024-09-10T11:40:16.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.119+0000 7f0743e8d640 1 -- start start 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 0x7f073c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f073c1542c0 0x7f073c0710a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0b6ae0 con 0x7f073c14b9c0 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f073c0b6960 con 0x7f073c1542c0 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0743e8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f073c0b6c60 con 0x7f073c0b1fa0 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 0x7f073c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 0x7f073c070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35704/0 (socket says 172.21.15.99:35704) 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 -- 172.21.15.99:0/1689847526 learned_addr learned my addr 172.21.15.99:0/1689847526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741c29640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f074242a640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f073c1542c0 0x7f073c0710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 -- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 msgr2=0x7f073c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f073c0b1fa0 0x7f073c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 -- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f073c1542c0 msgr2=0x7f073c0710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f073c1542c0 0x7f073c0710a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c154820 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.120+0000 7f0741428640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 0x7f073c070b60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0730011990 tx=0x7f0730011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073000fa10 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073000fbb0 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c154b00 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730014440 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c1550c0 con 0x7f073c14b9c0 2024-09-10T11:40:16.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.121+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f073c0b5be0 con 0x7f073c14b9c0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.122+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0730012040 con 0x7f073c14b9c0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.122+0000 7f073a7fc640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f072806be30 0x7f072806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f073008b080 con 0x7f073c14b9c0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f0741c29640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f072806be30 0x7f072806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f073a7fc640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7f0728071cf0 0x7f0728074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0730025ec0 con 0x7f0728071cf0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f0730092050 con 0x7f073c14b9c0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f074242a640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7f0728071cf0 0x7f0728074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f0741c29640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f072806be30 0x7f072806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f072c033000 tx=0x7f072c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.123+0000 7f074242a640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7f0728071cf0 0x7f0728074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.124+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== osd.3 v2:172.21.15.99:6816/1027928425 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f0730025ec0 con 0x7f0728071cf0 2024-09-10T11:40:16.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 --> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f073c15bda0 con 0x7f0728071cf0 2024-09-10T11:40:16.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f073a7fc640 1 -- 172.21.15.99:0/1689847526 <== osd.3 v2:172.21.15.99:6816/1027928425 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f073c15bda0 con 0x7f0728071cf0 2024-09-10T11:40:16.136 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:16.136 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:16.136 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:16.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7f0728071cf0 msgr2=0x7f0728074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6816/1027928425,v1:172.21.15.99:6818/1027928425] conn(0x7f0728071cf0 0x7f0728074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f072806be30 msgr2=0x7f072806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f072806be30 0x7f072806e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f072c033000 tx=0x7f072c009030 comp rx=0 tx=0).stop 2024-09-10T11:40:16.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.135+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 msgr2=0x7f073c070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f0743e8d640 1 --2- 172.21.15.99:0/1689847526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f073c14b9c0 0x7f073c070b60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0730011990 tx=0x7f0730011e50 comp rx=0 tx=0).stop 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f074242a640 1 -- 172.21.15.99:0/1689847526 reap_dead start 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 shutdown_connections 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 >> 172.21.15.99:0/1689847526 conn(0x7f073c073c60 msgr2=0x7f073c0b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 shutdown_connections 2024-09-10T11:40:16.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.136+0000 7f0743e8d640 1 -- 172.21.15.99:0/1689847526 wait complete. 2024-09-10T11:40:16.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T11:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 Processor -- start 2024-09-10T11:40:16.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 -- start start 2024-09-10T11:40:16.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdce00d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce015ff70 con 0x7fdce01626c0 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce00d1620 con 0x7fdce015eb80 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.246+0000 7fdce5f8f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdce00d30c0 con 0x7fdce00d5010 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdeffd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdce01626c0 0x7fdce00d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35930/0 (socket says 172.21.15.99:35930) 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdeffd640 1 -- 172.21.15.99:0/2981299343 learned_addr learned my addr 172.21.15.99:0/2981299343 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdffff640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdce00d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 696322095 0 0) 0x7fdce015ff70 con 0x7fdce01626c0 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcc8003520 con 0x7fdce01626c0 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 356369276 0 0) 0x7fdcc8003520 con 0x7fdce01626c0 2024-09-10T11:40:16.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 msgr2=0x7fdce00d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdce00d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 >> v1:172.21.15.196:6789/0 conn(0x7fdce015eb80 legacy=0x7fdce00d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce0162bc0 con 0x7fdce01626c0 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7fdcd4001670 con 0x7fdce01626c0 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 >> v1:172.21.15.99:6789/0 conn(0x7fdce01626c0 legacy=0x7fdce00d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdcc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc8004d40 0x7fdcc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdce00d30c0 con 0x7fdcc80039e0 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc800ba70 con 0x7fdcc8004d40 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc800bc70 con 0x7fdce00d5010 2024-09-10T11:40:16.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdeffd640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdcc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.247+0000 7fdcdffff640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc8004d40 0x7fdcc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 -- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 msgr2=0x7fdcc8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce00d5010 0x7fdcc8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 -- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc8004d40 msgr2=0x7fdcc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc8004d40 0x7fdcc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 -- 172.21.15.99:0/2981299343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc8003520 con 0x7fdcc80039e0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003dc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0001860 tx=0x7fdcc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a577bcfe9a5593fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0037070 con 0x7fdcc80039e0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd002ac30 con 0x7fdcc80039e0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.248+0000 7fdcde7fc640 1 -- 172.21.15.99:0/2981299343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0001430 con 0x7fdcc80039e0 2024-09-10T11:40:16.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2981299343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc80039e0 msgr2=0x7fdcc8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2981299343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcc80039e0 0x7fdcc8003dc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0001860 tx=0x7fdcc8005830 comp rx=0 tx=0).stop 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdcdffff640 1 -- 172.21.15.99:0/2981299343 reap_dead start 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2981299343 shutdown_connections 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2981299343 >> 172.21.15.99:0/2981299343 conn(0x7fdce008e680 msgr2=0x7fdce009dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2981299343 shutdown_connections 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.249+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2981299343 wait complete. 2024-09-10T11:40:16.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.250+0000 7fdce5f8f640 1 Processor -- start 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.250+0000 7fdce5f8f640 1 -- start start 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc80039e0 0x7fdce00ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce015ba90 0x7fdce015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0163460 con 0x7fdce00b4970 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdce0163180 con 0x7fdcc80039e0 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdce5f8f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fdce00d0770 con 0x7fdce015ba90 2024-09-10T11:40:16.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35726/0 (socket says 172.21.15.99:35726) 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 -- 172.21.15.99:0/2921737214 learned_addr learned my addr 172.21.15.99:0/2921737214 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc80039e0 0x7fdce00ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdffff640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce015ba90 0x7fdce015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 -- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce015ba90 msgr2=0x7fdce015bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce015ba90 0x7fdce015bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 -- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc80039e0 msgr2=0x7fdce00ba2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc80039e0 0x7fdce00ba2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.251+0000 7fdcdeffd640 1 -- 172.21.15.99:0/2921737214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce00cbe30 con 0x7fdce00b4970 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdcdeffd640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4004f50 tx=0x7fdcd40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400f8e0 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd4018070 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce00cc110 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400fce0 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.252+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce00cc6d0 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.253+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce00d0aa0 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.253+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcd401d460 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.253+0000 7fdcc7fff640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 0x7fdccc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.254+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7fdcd408a950 con 0x7fdce00b4970 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.254+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 0x7fdccc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.254+0000 7fdcdf7fe640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 0x7fdccc06e320 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0034000 tx=0x7fdcd0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.258+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdcd4059040 con 0x7fdce00b4970 2024-09-10T11:40:16.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.425+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fdce00b3980 con 0x7fdccc06be60 2024-09-10T11:40:16.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.425+0000 7fdcc7fff640 1 -- 172.21.15.99:0/2921737214 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdce00b3980 con 0x7fdccc06be60 2024-09-10T11:40:16.426 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T11:40:16.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 msgr2=0x7fdccc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 0x7fdccc06e320 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0034000 tx=0x7fdcd0009030 comp rx=0 tx=0).stop 2024-09-10T11:40:16.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 msgr2=0x7fdce00b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4004f50 tx=0x7fdcd40074f0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 shutdown_connections 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fdccc06be60 0x7fdccc06e320 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdce015ba90 0x7fdce015bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdce00b4970 0x7fdce00b6ea0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 --2- 172.21.15.99:0/2921737214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcc80039e0 0x7fdce00ba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 >> 172.21.15.99:0/2921737214 conn(0x7fdce008e680 msgr2=0x7fdce015ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 shutdown_connections 2024-09-10T11:40:16.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.427+0000 7fdce5f8f640 1 -- 172.21.15.99:0/2921737214 wait complete. 2024-09-10T11:40:16.439 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T11:40:16.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:16.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.540+0000 7f60cff4b640 1 Processor -- start 2024-09-10T11:40:16.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.541+0000 7f60cff4b640 1 -- start start 2024-09-10T11:40:16.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c80d1200 0x7f60c8073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 0x7f60c806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60c815c060 0x7f60c815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f60c806ef50 con 0x7f60c815c060 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f60c8152e60 con 0x7f60c806e5d0 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cff4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80d15e0 con 0x7f60c80d1200 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60c815c060 0x7f60c815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60c815c060 0x7f60c815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35734/0 (socket says 172.21.15.99:35734) 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 -- 172.21.15.99:0/2408422074 learned_addr learned my addr 172.21.15.99:0/2408422074 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cdce7640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c80d1200 0x7f60c8073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cd4e6640 1 --2- 172.21.15.99:0/2408422074 >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 0x7f60c806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 -- 172.21.15.99:0/2408422074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c80d1200 msgr2=0x7f60c8073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c80d1200 0x7f60c8073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 -- 172.21.15.99:0/2408422074 >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 msgr2=0x7f60c806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2408422074 >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 0x7f60c806ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60ce4e8640 1 -- 172.21.15.99:0/2408422074 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8072680 con 0x7f60c815c060 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.542+0000 7f60cd4e6640 1 --2- 172.21.15.99:0/2408422074 >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 0x7f60c806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c815c060 0x7f60c815c4a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f60c806f8e0 tx=0x7f60c002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5de2939c17a9cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60ccce5640 1 -- 172.21.15.99:0/2408422074 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c0036070 con 0x7f60c815c060 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60ccce5640 1 -- 172.21.15.99:0/2408422074 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c0031600 con 0x7f60c815c060 2024-09-10T11:40:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60ccce5640 1 -- 172.21.15.99:0/2408422074 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c00319d0 con 0x7f60c815c060 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 -- 172.21.15.99:0/2408422074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c815c060 msgr2=0x7f60c815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c815c060 0x7f60c815c4a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f60c806f8e0 tx=0x7f60c002a610 comp rx=0 tx=0).stop 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 -- 172.21.15.99:0/2408422074 shutdown_connections 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c815c060 0x7f60c815c4a0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2408422074 >> v2:172.21.15.196:3300/0 conn(0x7f60c806e5d0 0x7f60c806ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2408422074 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c80d1200 0x7f60c8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 -- 172.21.15.99:0/2408422074 >> 172.21.15.99:0/2408422074 conn(0x7f60c806bc10 msgr2=0x7f60c806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 -- 172.21.15.99:0/2408422074 shutdown_connections 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.543+0000 7f60cff4b640 1 -- 172.21.15.99:0/2408422074 wait complete. 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 Processor -- start 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 -- start start 2024-09-10T11:40:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c806e5d0 0x7f60c80aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 0x7f60c815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c815c060 0x7f60c815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8072f20 con 0x7f60c80d1200 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f60c8072c40 con 0x7f60c806e5d0 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cff4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f60c815cd90 con 0x7f60c815c060 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cd4e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 0x7f60c815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cd4e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 0x7f60c815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35736/0 (socket says 172.21.15.99:35736) 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cd4e6640 1 -- 172.21.15.99:0/2494425914 learned_addr learned my addr 172.21.15.99:0/2494425914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c815c060 0x7f60c815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.545+0000 7f60cdce7640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c806e5d0 0x7f60c80aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 -- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c815c060 msgr2=0x7f60c815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c815c060 0x7f60c815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 -- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c806e5d0 msgr2=0x7f60c80aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c806e5d0 0x7f60c80aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c8159790 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cd4e6640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 0x7f60c815e480 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c7f0 tx=0x7f60c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4015070 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c4007820 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c80ab210 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c4007c10 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.546+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c8159a30 con 0x7f60c80d1200 2024-09-10T11:40:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.547+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60c806ea10 con 0x7f60c80d1200 2024-09-10T11:40:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60c4018070 con 0x7f60c80d1200 2024-09-10T11:40:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60ac06be30 0x7f60ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f60c408ab80 con 0x7f60c80d1200 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60cdce7640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60ac06be30 0x7f60ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f60ac071cf0 0x7f60ac074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60c401de50 con 0x7f60ac071cf0 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f60c408f2a0 con 0x7f60c80d1200 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f60ac071cf0 0x7f60ac074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.548+0000 7f60cdce7640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60ac06be30 0x7f60ac06e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f60bc007920 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.549+0000 7f60ce4e8640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f60ac071cf0 0x7f60ac074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.550+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== osd.4 v2:172.21.15.196:6800/3885371275 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f60c401de50 con 0x7f60ac071cf0 2024-09-10T11:40:16.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.561+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 --> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f60c80b4ad0 con 0x7f60ac071cf0 2024-09-10T11:40:16.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.561+0000 7f60ba7fc640 1 -- 172.21.15.99:0/2494425914 <== osd.4 v2:172.21.15.196:6800/3885371275 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f60c80b4ad0 con 0x7f60ac071cf0 2024-09-10T11:40:16.562 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:16.562 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:16.562 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f60ac071cf0 msgr2=0x7f60ac074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.196:6800/3885371275,v1:172.21.15.196:6801/3885371275] conn(0x7f60ac071cf0 0x7f60ac074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60ac06be30 msgr2=0x7f60ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f60ac06be30 0x7f60ac06e2f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f60bc007920 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 msgr2=0x7f60c815e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 --2- 172.21.15.99:0/2494425914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60c80d1200 0x7f60c815e480 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f60c400c7f0 tx=0x7f60c400ccb0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60ce4e8640 1 -- 172.21.15.99:0/2494425914 reap_dead start 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 shutdown_connections 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 >> 172.21.15.99:0/2494425914 conn(0x7f60c806bc10 msgr2=0x7f60c815d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 shutdown_connections 2024-09-10T11:40:16.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.562+0000 7f60cff4b640 1 -- 172.21.15.99:0/2494425914 wait complete. 2024-09-10T11:40:16.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T11:40:16.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.672+0000 7f83a241e640 1 Processor -- start 2024-09-10T11:40:16.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.672+0000 7f83a241e640 1 -- start start 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 0x7f839c163620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f839c0b4930 0x7f839c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f839c0b5420 con 0x7f839c0b4930 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f839c06c530 con 0x7f839c06bc10 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a241e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0b5140 con 0x7f839c160030 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f839c0b4930 0x7f839c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f839c0b4930 0x7f839c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35738/0 (socket says 172.21.15.99:35738) 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 -- 172.21.15.99:0/1498434957 learned_addr learned my addr 172.21.15.99:0/1498434957 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f839bfff640 1 --2- 172.21.15.99:0/1498434957 >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 0x7f839c163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f839b7fe640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 -- 172.21.15.99:0/1498434957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 msgr2=0x7f839c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 -- 172.21.15.99:0/1498434957 >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 msgr2=0x7f839c163620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 --2- 172.21.15.99:0/1498434957 >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 0x7f839c163620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f83a09bb640 1 -- 172.21.15.99:0/1498434957 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c162b40 con 0x7f839c0b4930 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f839bfff640 1 --2- 172.21.15.99:0/1498434957 >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 0x7f839c163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.673+0000 7f839b7fe640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f83a09bb640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c06e5d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f839c073870 tx=0x7f839402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e00be260da1c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f839affd640 1 -- 172.21.15.99:0/1498434957 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394036070 con 0x7f839c0b4930 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f839affd640 1 -- 172.21.15.99:0/1498434957 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8394031600 con 0x7f839c0b4930 2024-09-10T11:40:16.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f839affd640 1 -- 172.21.15.99:0/1498434957 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83940319d0 con 0x7f839c0b4930 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f83a241e640 1 -- 172.21.15.99:0/1498434957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 msgr2=0x7f839c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.674+0000 7f83a241e640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c06e5d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f839c073870 tx=0x7f839402a610 comp rx=0 tx=0).stop 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 -- 172.21.15.99:0/1498434957 shutdown_connections 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c06e5d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 --2- 172.21.15.99:0/1498434957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 --2- 172.21.15.99:0/1498434957 >> v2:172.21.15.196:3300/0 conn(0x7f839c06bc10 0x7f839c163620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 -- 172.21.15.99:0/1498434957 >> 172.21.15.99:0/1498434957 conn(0x7f839c0b3630 msgr2=0x7f839c0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 -- 172.21.15.99:0/1498434957 shutdown_connections 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.675+0000 7f83a241e640 1 -- 172.21.15.99:0/1498434957 wait complete. 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.676+0000 7f83a241e640 1 Processor -- start 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.676+0000 7f83a241e640 1 -- start start 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06bc10 0x7f839c13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c13dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0b4d90 con 0x7f839c0b4930 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f839c163100 con 0x7f839c06bc10 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f839c13f8b0 con 0x7f839c160030 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35746/0 (socket says 172.21.15.99:35746) 2024-09-10T11:40:16.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 -- 172.21.15.99:0/2588726683 learned_addr learned my addr 172.21.15.99:0/2588726683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839bfff640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06bc10 0x7f839c13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a09bb640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c13dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 -- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 msgr2=0x7f839c13dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c13dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 -- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06bc10 msgr2=0x7f839c13f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06bc10 0x7f839c13f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 -- 172.21.15.99:0/2588726683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c13fb30 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f839b7fe640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7f0 tx=0x7f839000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390015070 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83900079d0 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c13fe10 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390007da0 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.677+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1426d0 con 0x7f839c0b4930 2024-09-10T11:40:16.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.678+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c0d1560 con 0x7f839c0b4930 2024-09-10T11:40:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.679+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8390018070 con 0x7f839c0b4930 2024-09-10T11:40:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.679+0000 7f8398ff9640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.679+0000 7f839bfff640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.679+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f839008a9c0 con 0x7f839c0b4930 2024-09-10T11:40:16.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.680+0000 7f839bfff640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 0x7f838806e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f838c00aac0 tx=0x7f838c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.683+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f839008ce20 con 0x7f839c0b4930 2024-09-10T11:40:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.849+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f839c142070 con 0x7f838806be30 2024-09-10T11:40:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.849+0000 7f8398ff9640 1 -- 172.21.15.99:0/2588726683 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f839c142070 con 0x7f838806be30 2024-09-10T11:40:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T11:40:16.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 msgr2=0x7f838806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 0x7f838806e2f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f838c00aac0 tx=0x7f838c008040 comp rx=0 tx=0).stop 2024-09-10T11:40:16.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 msgr2=0x7f839c1416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f839000c7f0 tx=0x7f839000cbf0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 shutdown_connections 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f838806be30 0x7f838806e2f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f839c160030 0x7f839c13dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f839c0b4930 0x7f839c1416b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 --2- 172.21.15.99:0/2588726683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f839c06bc10 0x7f839c13f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.851+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 >> 172.21.15.99:0/2588726683 conn(0x7f839c0b3630 msgr2=0x7f839c0b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.852+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 shutdown_connections 2024-09-10T11:40:16.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.852+0000 7f83a241e640 1 -- 172.21.15.99:0/2588726683 wait complete. 2024-09-10T11:40:16.863 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T11:40:16.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:16.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 Processor -- start 2024-09-10T11:40:16.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 -- start start 2024-09-10T11:40:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 0x7f38a80d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f38a80b59b0 0x7f38a80d5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38a80d2150 con 0x7f38a8162d10 2024-09-10T11:40:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80d0d30 con 0x7f38a80b59b0 2024-09-10T11:40:16.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38ad265640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f38a80b4d30 con 0x7f38a80d16f0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38a67fc640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f38a80b59b0 0x7f38a80d5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.965+0000 7f38a77fe640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f38a8162d10 0x7f38a8163140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35936/0 (socket says 172.21.15.99:35936) 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a77fe640 1 -- 172.21.15.99:0/1971758256 learned_addr learned my addr 172.21.15.99:0/1971758256 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a6ffd640 1 --2- 172.21.15.99:0/1971758256 >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 0x7f38a80d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a67fc640 1 -- 172.21.15.99:0/1971758256 >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 msgr2=0x7f38a80d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a67fc640 1 --2- 172.21.15.99:0/1971758256 >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 0x7f38a80d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a67fc640 1 -- 172.21.15.99:0/1971758256 >> v1:172.21.15.99:6789/0 conn(0x7f38a8162d10 legacy=0x7f38a8163140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a67fc640 1 -- 172.21.15.99:0/1971758256 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80b5780 con 0x7f38a80b59b0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a5ffb640 1 -- 172.21.15.99:0/1971758256 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559705541 0 0) 0x7f38a80d2150 con 0x7f38a8162d10 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a6ffd640 1 --2- 172.21.15.99:0/1971758256 >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 0x7f38a80d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.966+0000 7f38a67fc640 1 --2- 172.21.15.99:0/1971758256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80d5740 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f38a808e5e0 tx=0x7f389c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1387baa81b6076 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38a5ffb640 1 -- 172.21.15.99:0/1971758256 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c037070 con 0x7f38a80b59b0 2024-09-10T11:40:16.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38a5ffb640 1 -- 172.21.15.99:0/1971758256 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389c032440 con 0x7f38a80b59b0 2024-09-10T11:40:16.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38a5ffb640 1 -- 172.21.15.99:0/1971758256 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c02ae60 con 0x7f38a80b59b0 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 -- 172.21.15.99:0/1971758256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 msgr2=0x7f38a80d5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 --2- 172.21.15.99:0/1971758256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80d5740 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f38a808e5e0 tx=0x7f389c02a110 comp rx=0 tx=0).stop 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 -- 172.21.15.99:0/1971758256 shutdown_connections 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 --2- 172.21.15.99:0/1971758256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80d5740 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 --2- 172.21.15.99:0/1971758256 >> v2:172.21.15.196:3300/0 conn(0x7f38a80d16f0 0x7f38a80d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 -- 172.21.15.99:0/1971758256 >> 172.21.15.99:0/1971758256 conn(0x7f38a80d4c20 msgr2=0x7f38a80d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 -- 172.21.15.99:0/1971758256 shutdown_connections 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.967+0000 7f38ad265640 1 -- 172.21.15.99:0/1971758256 wait complete. 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 Processor -- start 2024-09-10T11:40:16.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 -- start start 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a80d16f0 0x7f38a80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a8162d10 0x7f38a80b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80b4f80 con 0x7f38a80b59b0 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f38a80b52f0 con 0x7f38a80d16f0 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38ad265640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f38a8163850 con 0x7f38a8162d10 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38a6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35756/0 (socket says 172.21.15.99:35756) 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.969+0000 7f38a6ffd640 1 -- 172.21.15.99:0/8457371 learned_addr learned my addr 172.21.15.99:0/8457371 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a77fe640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a8162d10 0x7f38a80b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a67fc640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a80d16f0 0x7f38a80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 -- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a8162d10 msgr2=0x7f38a80b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a8162d10 0x7f38a80b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 -- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a80d16f0 msgr2=0x7f38a80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a80d16f0 0x7f38a80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a80c8e40 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a67fc640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a80d16f0 0x7f38a80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38a6ffd640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80c90f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f389800cf90 tx=0x7f389800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801a070 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389800d840 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a80b89a0 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389800dc10 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.970+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a80c9a40 con 0x7f38a80b59b0 2024-09-10T11:40:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.971+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38a80d1ad0 con 0x7f38a80b59b0 2024-09-10T11:40:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f389801d070 con 0x7f38a80b59b0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f389406be30 0x7f389406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f38a67fc640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f389406be30 0x7f389406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f389808b2a0 con 0x7f38a80b59b0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7f3894071cf0 0x7f3894074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3898012e50 con 0x7f3894071cf0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f3898092050 con 0x7f38a80b59b0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.972+0000 7f38a67fc640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f389406be30 0x7f389406e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f389c034000 tx=0x7f389c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.973+0000 7f38a77fe640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7f3894071cf0 0x7f3894074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.973+0000 7f38a77fe640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7f3894071cf0 0x7f3894074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.974+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== osd.5 v2:172.21.15.196:6824/3797029994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f3898012e50 con 0x7f3894071cf0 2024-09-10T11:40:16.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.985+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 --> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38a80d5250 con 0x7f3894071cf0 2024-09-10T11:40:16.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.985+0000 7f388f7fe640 1 -- 172.21.15.99:0/8457371 <== osd.5 v2:172.21.15.196:6824/3797029994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f38a80d5250 con 0x7f3894071cf0 2024-09-10T11:40:16.986 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:16.986 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:16.986 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:16.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7f3894071cf0 msgr2=0x7f3894074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.196:6824/3797029994,v1:172.21.15.196:6825/3797029994] conn(0x7f3894071cf0 0x7f3894074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f389406be30 msgr2=0x7f389406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f389406be30 0x7f389406e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f389c034000 tx=0x7f389c02e000 comp rx=0 tx=0).stop 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 msgr2=0x7f38a80c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 --2- 172.21.15.99:0/8457371 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f38a80b59b0 0x7f38a80c90f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f389800cf90 tx=0x7f389800a7d0 comp rx=0 tx=0).stop 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38a77fe640 1 -- 172.21.15.99:0/8457371 reap_dead start 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 shutdown_connections 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 >> 172.21.15.99:0/8457371 conn(0x7f38a80d4c20 msgr2=0x7f38a815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 shutdown_connections 2024-09-10T11:40:16.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:16.986+0000 7f38ad265640 1 -- 172.21.15.99:0/8457371 wait complete. 2024-09-10T11:40:16.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T11:40:17.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 Processor -- start 2024-09-10T11:40:17.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 -- start start 2024-09-10T11:40:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 0x7f7bfc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 0x7f7bfc0d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bfc0d52f0 con 0x7f7bfc15eb80 2024-09-10T11:40:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f7bfc0d30c0 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.096+0000 7f7c028d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc15ff70 con 0x7f7bfc1626c0 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7c00e74640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 0x7f7bfc0d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfb7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 0x7f7bfc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfbfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7bfc15eb80 0x7f7bfc0d29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:56264/0 (socket says 172.21.15.99:56264) 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfbfff640 1 -- 172.21.15.99:0/4061484898 learned_addr learned my addr 172.21.15.99:0/4061484898 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfaffd640 1 -- 172.21.15.99:0/4061484898 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749815196 0 0) 0x7f7bec001110 con 0x7f7bfc15eb80 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfaffd640 1 -- 172.21.15.99:0/4061484898 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7be4003520 con 0x7f7bfc15eb80 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfb7fe640 1 -- 172.21.15.99:0/4061484898 >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 msgr2=0x7f7bfc0d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfb7fe640 1 --2- 172.21.15.99:0/4061484898 >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 0x7f7bfc0d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfb7fe640 1 -- 172.21.15.99:0/4061484898 >> v1:172.21.15.196:6789/0 conn(0x7f7bfc15eb80 legacy=0x7f7bfc0d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7bfb7fe640 1 -- 172.21.15.99:0/4061484898 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc0d1620 con 0x7f7bfc1626c0 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.097+0000 7f7c00e74640 1 --2- 172.21.15.99:0/4061484898 >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 0x7f7bfc0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:17.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.098+0000 7f7bfb7fe640 1 --2- 172.21.15.99:0/4061484898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 0x7f7bfc0d4900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00097a0 tx=0x7f7bf002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64668419fb3d2cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.098+0000 7f7bfaffd640 1 -- 172.21.15.99:0/4061484898 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0037070 con 0x7f7bfc1626c0 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.098+0000 7f7bfaffd640 1 -- 172.21.15.99:0/4061484898 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf002ae50 con 0x7f7bfc1626c0 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.098+0000 7f7bfaffd640 1 -- 172.21.15.99:0/4061484898 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0032640 con 0x7f7bfc1626c0 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 -- 172.21.15.99:0/4061484898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 msgr2=0x7f7bfc0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 --2- 172.21.15.99:0/4061484898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 0x7f7bfc0d4900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00097a0 tx=0x7f7bf002a100 comp rx=0 tx=0).stop 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 -- 172.21.15.99:0/4061484898 shutdown_connections 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 --2- 172.21.15.99:0/4061484898 >> v2:172.21.15.196:3300/0 conn(0x7f7bfc0d4e40 0x7f7bfc0d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 --2- 172.21.15.99:0/4061484898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc1626c0 0x7f7bfc0d4900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 -- 172.21.15.99:0/4061484898 >> 172.21.15.99:0/4061484898 conn(0x7f7bfc08e680 msgr2=0x7f7bfc09dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 -- 172.21.15.99:0/4061484898 shutdown_connections 2024-09-10T11:40:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.099+0000 7f7c028d7640 1 -- 172.21.15.99:0/4061484898 wait complete. 2024-09-10T11:40:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.100+0000 7f7c028d7640 1 Processor -- start 2024-09-10T11:40:17.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.100+0000 7f7c028d7640 1 -- start start 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc15eb80 0x7f7bfc0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc1626c0 0x7f7bfc1f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc163180 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc162ea0 con 0x7f7bfc1626c0 2024-09-10T11:40:17.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c028d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc163460 con 0x7f7bfc15eb80 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35762/0 (socket says 172.21.15.99:35762) 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 -- 172.21.15.99:0/1308830802 learned_addr learned my addr 172.21.15.99:0/1308830802 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfb7fe640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc15eb80 0x7f7bfc0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7c00e74640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc1626c0 0x7f7bfc1f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 -- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc15eb80 msgr2=0x7f7bfc0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc15eb80 0x7f7bfc0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 -- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc1626c0 msgr2=0x7f7bfc1f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc1626c0 0x7f7bfc1f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 -- 172.21.15.99:0/1308830802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc0b1ce0 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bfbfff640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c3a0 tx=0x7f7bec00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec017070 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.101+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec005ce0 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.102+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc0b1fc0 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.102+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec007310 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.102+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc0b2580 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.103+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc09e430 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.103+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7bec01a070 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.104+0000 7f7bf8ff9640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 0x7f7be806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.104+0000 7f7bfb7fe640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 0x7f7be806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.104+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f7bec089e70 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.104+0000 7f7bfb7fe640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 0x7f7be806e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0034000 tx=0x7f7bf002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.108+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7bfc0d52f0 con 0x7f7bfc0d4e40 2024-09-10T11:40:17.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.274+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7bfc0d4900 con 0x7f7be806be30 2024-09-10T11:40:17.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.274+0000 7f7bf8ff9640 1 -- 172.21.15.99:0/1308830802 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7bfc0d4900 con 0x7f7be806be30 2024-09-10T11:40:17.275 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T11:40:17.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 msgr2=0x7f7be806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 0x7f7be806e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0034000 tx=0x7f7bf002e000 comp rx=0 tx=0).stop 2024-09-10T11:40:17.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 msgr2=0x7f7bfc14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c3a0 tx=0x7f7bec00c7a0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 shutdown_connections 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f7be806be30 0x7f7be806e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7bfc1626c0 0x7f7bfc1f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7bfc15eb80 0x7f7bfc0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 --2- 172.21.15.99:0/1308830802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bfc0d4e40 0x7f7bfc14c110 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.276+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 >> 172.21.15.99:0/1308830802 conn(0x7f7bfc08e680 msgr2=0x7f7bfc15dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.277+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 shutdown_connections 2024-09-10T11:40:17.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.277+0000 7f7c028d7640 1 -- 172.21.15.99:0/1308830802 wait complete. 2024-09-10T11:40:17.288 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T11:40:17.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.389+0000 7fd89085e640 1 Processor -- start 2024-09-10T11:40:17.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.389+0000 7fd89085e640 1 -- start start 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.390+0000 7fd89085e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd88c0d3470 0x7fd88c0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.390+0000 7fd89085e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 0x7fd88c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.390+0000 7fd89085e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd88c0d2c50 con 0x7fd88c0c7de0 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.390+0000 7fd89085e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fd88c0d2a00 con 0x7fd88c0d3470 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.390+0000 7fd89085e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd88c0b62d0 con 0x7fd88c0d57b0 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd889d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 0x7fd88c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88ad9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fd88c0c7de0 0x7fd88c0d1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.99:56280/0 (socket says 172.21.15.99:56280) 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88ad9d640 1 -- 172.21.15.99:0/2250518483 learned_addr learned my addr 172.21.15.99:0/2250518483 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88a59c640 1 --2- 172.21.15.99:0/2250518483 >> v2:172.21.15.196:3300/0 conn(0x7fd88c0d3470 0x7fd88c0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88959a640 1 -- 172.21.15.99:0/2250518483 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 211400900 0 0) 0x7fd88c0d2c50 con 0x7fd88c0c7de0 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88959a640 1 -- 172.21.15.99:0/2250518483 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd878003520 con 0x7fd88c0c7de0 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88a59c640 1 -- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 msgr2=0x7fd88c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88a59c640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 0x7fd88c0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88a59c640 1 -- 172.21.15.99:0/2250518483 >> v1:172.21.15.196:6789/0 conn(0x7fd88c0c7de0 legacy=0x7fd88c0d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd88a59c640 1 -- 172.21.15.99:0/2250518483 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c0b5f60 con 0x7fd88c0d3470 2024-09-10T11:40:17.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.391+0000 7fd889d9b640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 0x7fd88c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:40:17.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.392+0000 7fd88a59c640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b7a70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009d90 tx=0x7fd87c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=603f17041b06232 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.392+0000 7fd88959a640 1 -- 172.21.15.99:0/2250518483 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007740 con 0x7fd88c0d3470 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.392+0000 7fd88959a640 1 -- 172.21.15.99:0/2250518483 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87c0078e0 con 0x7fd88c0d3470 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.392+0000 7fd88959a640 1 -- 172.21.15.99:0/2250518483 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c007cb0 con 0x7fd88c0d3470 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 -- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 msgr2=0x7fd88c0b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b7a70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009d90 tx=0x7fd87c02a660 comp rx=0 tx=0).stop 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 -- 172.21.15.99:0/2250518483 shutdown_connections 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0d57b0 0x7fd88c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 --2- 172.21.15.99:0/2250518483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b7a70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 -- 172.21.15.99:0/2250518483 >> 172.21.15.99:0/2250518483 conn(0x7fd88c0d4c20 msgr2=0x7fd88c0d5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 -- 172.21.15.99:0/2250518483 shutdown_connections 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.393+0000 7fd89085e640 1 -- 172.21.15.99:0/2250518483 wait complete. 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.394+0000 7fd89085e640 1 Processor -- start 2024-09-10T11:40:17.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.394+0000 7fd89085e640 1 -- start start 2024-09-10T11:40:17.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0c7de0 0x7fd88c1eb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 0x7fd88c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c0c78b0 con 0x7fd88c0d57b0 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd88c0b81c0 con 0x7fd88c0d3470 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd89085e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fd88c0c7a30 con 0x7fd88c0c7de0 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 0x7fd88c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 0x7fd88c0b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35768/0 (socket says 172.21.15.99:35768) 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 -- 172.21.15.99:0/4122009994 learned_addr learned my addr 172.21.15.99:0/4122009994 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88a59c640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0c7de0 0x7fd88c1eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd889d9b640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 -- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0c7de0 msgr2=0x7fd88c1eb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd88c0c7de0 0x7fd88c1eb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 -- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 msgr2=0x7fd88c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd88c0d3470 0x7fd88c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88c0b29f0 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.395+0000 7fd88ad9d640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 0x7fd88c0b31b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c390 tx=0x7fd88400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884017070 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd884005ce0 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007310 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88c0b44f0 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88c1eba70 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.396+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd88c0d3850 con 0x7fd88c0d57b0 2024-09-10T11:40:17.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.397+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd88401a070 con 0x7fd88c0d57b0 2024-09-10T11:40:17.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd882ffd640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd86c06be30 0x7fd86c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7fd884089f80 con 0x7fd88c0d57b0 2024-09-10T11:40:17.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd88a59c640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd86c06be30 0x7fd86c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd882ffd640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7fd86c071cf0 0x7fd86c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd884014ec0 con 0x7fd86c071cf0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7fd88408c620 con 0x7fd88c0d57b0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd88a59c640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd86c06be30 0x7fd86c06e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd87c034000 tx=0x7fd87c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd889d9b640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7fd86c071cf0 0x7fd86c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.398+0000 7fd889d9b640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7fd86c071cf0 0x7fd86c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.399+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== osd.6 v2:172.21.15.196:6806/4190591731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fd884014ec0 con 0x7fd86c071cf0 2024-09-10T11:40:17.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.410+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 --> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd88c0b7d70 con 0x7fd86c071cf0 2024-09-10T11:40:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd882ffd640 1 -- 172.21.15.99:0/4122009994 <== osd.6 v2:172.21.15.196:6806/4190591731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7fd88c0b7d70 con 0x7fd86c071cf0 2024-09-10T11:40:17.412 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:17.412 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:17.412 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7fd86c071cf0 msgr2=0x7fd86c074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.196:6806/4190591731,v1:172.21.15.196:6808/4190591731] conn(0x7fd86c071cf0 0x7fd86c074110 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd86c06be30 msgr2=0x7fd86c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fd86c06be30 0x7fd86c06e2f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd87c034000 tx=0x7fd87c02f000 comp rx=0 tx=0).stop 2024-09-10T11:40:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 msgr2=0x7fd88c0b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 --2- 172.21.15.99:0/4122009994 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd88c0d57b0 0x7fd88c0b31b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c390 tx=0x7fd88400c850 comp rx=0 tx=0).stop 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd88ad9d640 1 -- 172.21.15.99:0/4122009994 reap_dead start 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 shutdown_connections 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 >> 172.21.15.99:0/4122009994 conn(0x7fd88c0d4c20 msgr2=0x7fd88c0b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 shutdown_connections 2024-09-10T11:40:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.411+0000 7fd89085e640 1 -- 172.21.15.99:0/4122009994 wait complete. 2024-09-10T11:40:17.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T11:40:17.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.521+0000 7f814058d640 1 Processor -- start 2024-09-10T11:40:17.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.521+0000 7f814058d640 1 -- start start 2024-09-10T11:40:17.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f814058d640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81380d1200 0x7f8138073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f814058d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 0x7f813806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f814058d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8138152e60 con 0x7f813806e5d0 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f814058d640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f813806ef80 con 0x7f81380d1200 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f814058d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f81380d15e0 con 0x7f813815c770 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81380d1200 0x7f8138073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f81380d1200 0x7f8138073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35784/0 (socket says 172.21.15.99:35784) 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 -- 172.21.15.99:0/677751921 learned_addr learned my addr 172.21.15.99:0/677751921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813eb2a640 1 --2- 172.21.15.99:0/677751921 >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 0x7f813806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 -- 172.21.15.99:0/677751921 >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 msgr2=0x7f813806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 --2- 172.21.15.99:0/677751921 >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 0x7f813806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 -- 172.21.15.99:0/677751921 >> v1:172.21.15.196:6789/0 conn(0x7f813806e5d0 legacy=0x7f813815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813e329640 1 -- 172.21.15.99:0/677751921 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8138072680 con 0x7f81380d1200 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.522+0000 7f813eb2a640 1 --2- 172.21.15.99:0/677751921 >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 0x7f813806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.523+0000 7f813e329640 1 --2- 172.21.15.99:0/677751921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81380d1200 0x7f8138073440 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f81380720c0 tx=0x7f813402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad439323a9d671ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.523+0000 7f813d327640 1 -- 172.21.15.99:0/677751921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134037070 con 0x7f81380d1200 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.523+0000 7f813d327640 1 -- 172.21.15.99:0/677751921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8134032440 con 0x7f81380d1200 2024-09-10T11:40:17.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.523+0000 7f813d327640 1 -- 172.21.15.99:0/677751921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134032810 con 0x7f81380d1200 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 -- 172.21.15.99:0/677751921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81380d1200 msgr2=0x7f8138073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 --2- 172.21.15.99:0/677751921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81380d1200 0x7f8138073440 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f81380720c0 tx=0x7f813402a1a0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 -- 172.21.15.99:0/677751921 shutdown_connections 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 --2- 172.21.15.99:0/677751921 >> v2:172.21.15.196:3300/0 conn(0x7f813815c770 0x7f813806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 --2- 172.21.15.99:0/677751921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81380d1200 0x7f8138073440 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 -- 172.21.15.99:0/677751921 >> 172.21.15.99:0/677751921 conn(0x7f813806bc10 msgr2=0x7f813806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 -- 172.21.15.99:0/677751921 shutdown_connections 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.524+0000 7f814058d640 1 -- 172.21.15.99:0/677751921 wait complete. 2024-09-10T11:40:17.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.525+0000 7f814058d640 1 Processor -- start 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.525+0000 7f814058d640 1 -- start start 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813806e5d0 0x7f813814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81380d1200 0x7f813814d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8138072f20 con 0x7f813815c770 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8138072c40 con 0x7f813806e5d0 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f813815cd90 con 0x7f81380d1200 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35800/0 (socket says 172.21.15.99:35800) 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 -- 172.21.15.99:0/743610141 learned_addr learned my addr 172.21.15.99:0/743610141 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813e329640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813806e5d0 0x7f813814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813db28640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81380d1200 0x7f813814d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 -- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81380d1200 msgr2=0x7f813814d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81380d1200 0x7f813814d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 -- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813806e5d0 msgr2=0x7f813814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813806e5d0 0x7f813814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 -- 172.21.15.99:0/743610141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81381f2ad0 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f813eb2a640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f813000cef0 tx=0x7f813000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813001a070 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813000d860 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81381f2d90 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813000dc50 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.526+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81381f3350 con 0x7f813815c770 2024-09-10T11:40:17.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.527+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8138073440 con 0x7f813815c770 2024-09-10T11:40:17.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.528+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f813001d070 con 0x7f813815c770 2024-09-10T11:40:17.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.528+0000 7f812effd640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 0x7f811c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.529+0000 7f813e329640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 0x7f811c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.529+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f813008b390 con 0x7f813815c770 2024-09-10T11:40:17.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.529+0000 7f813e329640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 0x7f811c06e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8134034000 tx=0x7f813402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.533+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8130059ab0 con 0x7f813815c770 2024-09-10T11:40:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.696+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f8138152860 con 0x7f811c06be30 2024-09-10T11:40:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.696+0000 7f812effd640 1 -- 172.21.15.99:0/743610141 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8138152860 con 0x7f811c06be30 2024-09-10T11:40:17.697 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T11:40:17.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 msgr2=0x7f811c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 0x7f811c06e2f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8134034000 tx=0x7f813402e000 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 msgr2=0x7f8138155240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f813000cef0 tx=0x7f813000a710 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 shutdown_connections 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f811c06be30 0x7f811c06e2f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f813815c770 0x7f8138155240 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81380d1200 0x7f813814d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 --2- 172.21.15.99:0/743610141 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813806e5d0 0x7f813814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.698+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 >> 172.21.15.99:0/743610141 conn(0x7f813806bc10 msgr2=0x7f813815f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.699+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 shutdown_connections 2024-09-10T11:40:17.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.699+0000 7f814058d640 1 -- 172.21.15.99:0/743610141 wait complete. 2024-09-10T11:40:17.710 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T11:40:17.710 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T11:40:17.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.811+0000 7f595cb9f640 1 Processor -- start 2024-09-10T11:40:17.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.811+0000 7f595cb9f640 1 -- start start 2024-09-10T11:40:17.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f595cb9f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f595815b830 0x7f59580d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f595cb9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f595806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f595cb9f640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595815bc10 con 0x7f59580d0640 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f595cb9f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f59580d0ab0 con 0x7f595815b830 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f595cb9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f595806e420 con 0x7f59580b5e10 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f594effd640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f59580d0640 0x7f59580b5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35946/0 (socket says 172.21.15.99:35946) 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.812+0000 7f594effd640 1 -- 172.21.15.99:0/2556847759 learned_addr learned my addr 172.21.15.99:0/2556847759 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5957fff640 1 --2- 172.21.15.99:0/2556847759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f595806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f59577fe640 1 --2- 172.21.15.99:0/2556847759 >> v2:172.21.15.196:3300/0 conn(0x7f595815b830 0x7f59580d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5956ffd640 1 -- 172.21.15.99:0/2556847759 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2918409807 0 0) 0x7f595815bc10 con 0x7f59580d0640 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5956ffd640 1 -- 172.21.15.99:0/2556847759 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5944003520 con 0x7f59580d0640 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5957fff640 1 -- 172.21.15.99:0/2556847759 >> v2:172.21.15.196:3300/0 conn(0x7f595815b830 msgr2=0x7f59580d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5957fff640 1 --2- 172.21.15.99:0/2556847759 >> v2:172.21.15.196:3300/0 conn(0x7f595815b830 0x7f59580d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5957fff640 1 -- 172.21.15.99:0/2556847759 >> v1:172.21.15.99:6789/0 conn(0x7f59580d0640 legacy=0x7f59580b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.813+0000 7f5957fff640 1 -- 172.21.15.99:0/2556847759 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580b6430 con 0x7f59580b5e10 2024-09-10T11:40:17.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f5957fff640 1 --2- 172.21.15.99:0/2556847759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f595806eee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5950009d90 tx=0x7f595002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4dc42559da047a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f5956ffd640 1 -- 172.21.15.99:0/2556847759 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950007740 con 0x7f59580b5e10 2024-09-10T11:40:17.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f5956ffd640 1 -- 172.21.15.99:0/2556847759 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59500078e0 con 0x7f59580b5e10 2024-09-10T11:40:17.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f5956ffd640 1 -- 172.21.15.99:0/2556847759 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950007c90 con 0x7f59580b5e10 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 -- 172.21.15.99:0/2556847759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 msgr2=0x7f595806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 --2- 172.21.15.99:0/2556847759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f595806eee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5950009d90 tx=0x7f595002a660 comp rx=0 tx=0).stop 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 -- 172.21.15.99:0/2556847759 shutdown_connections 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 --2- 172.21.15.99:0/2556847759 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f595806eee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 --2- 172.21.15.99:0/2556847759 >> v2:172.21.15.196:3300/0 conn(0x7f595815b830 0x7f59580d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.814+0000 7f595cb9f640 1 -- 172.21.15.99:0/2556847759 >> 172.21.15.99:0/2556847759 conn(0x7f595802cd20 msgr2=0x7f59580b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.815+0000 7f595cb9f640 1 -- 172.21.15.99:0/2556847759 shutdown_connections 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.815+0000 7f595cb9f640 1 -- 172.21.15.99:0/2556847759 wait complete. 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.816+0000 7f595cb9f640 1 Processor -- start 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.816+0000 7f595cb9f640 1 -- start start 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f5958148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59580d0640 0x7f5958152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 0x7f595806bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f595806e9c0 con 0x7f595815b830 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f595806e6e0 con 0x7f59580d0640 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f595806f420 con 0x7f59580b5e10 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 0x7f595806bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 0x7f595806bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35804/0 (socket says 172.21.15.99:35804) 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 -- 172.21.15.99:0/3169745362 learned_addr learned my addr 172.21.15.99:0/3169745362 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f59577fe640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f5958148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f594effd640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59580d0640 0x7f5958152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 -- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 msgr2=0x7f5958148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59580b5e10 0x7f5958148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 -- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59580d0640 msgr2=0x7f5958152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59580d0640 0x7f5958152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59581f5910 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f594effd640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59580d0640 0x7f5958152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5957fff640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 0x7f595806bc10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f595002ac50 tx=0x7f5950004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950040070 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5950031c60 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59581f5bf0 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950009450 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.817+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59581f61b0 con 0x7f595815b830 2024-09-10T11:40:17.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.818+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f595806eee0 con 0x7f595815b830 2024-09-10T11:40:17.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5950039040 con 0x7f595815b830 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f593806be30 0x7f593806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f59577fe640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f593806be30 0x7f593806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f59500abe90 con 0x7f595815b830 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f5938071cf0 0x7f5938074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f595002ee50 con 0x7f5938071cf0 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.819+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=189) ==== 24+0+0 (secure 0 0 0) 0x7f59500ae590 con 0x7f595815b830 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.820+0000 7f59577fe640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f593806be30 0x7f593806e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5948009aa0 tx=0x7f5948009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.820+0000 7f594effd640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f5938071cf0 0x7f5938074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.820+0000 7f594effd640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f5938071cf0 0x7f5938074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.821+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== osd.7 v2:172.21.15.196:6816/106140719 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f595002ee50 con 0x7f5938071cf0 2024-09-10T11:40:17.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.832+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 --> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f595815c930 con 0x7f5938071cf0 2024-09-10T11:40:17.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f5954ff9640 1 -- 172.21.15.99:0/3169745362 <== osd.7 v2:172.21.15.196:6816/106140719 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f595815c930 con 0x7f5938071cf0 2024-09-10T11:40:17.834 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-10T11:40:17.834 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-10T11:40:17.834 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f5938071cf0 msgr2=0x7f5938074110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.196:6816/106140719,v1:172.21.15.196:6817/106140719] conn(0x7f5938071cf0 0x7f5938074110 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f593806be30 msgr2=0x7f593806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f593806be30 0x7f593806e2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5948009aa0 tx=0x7f5948009190 comp rx=0 tx=0).stop 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 msgr2=0x7f595806bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 --2- 172.21.15.99:0/3169745362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f595815b830 0x7f595806bc10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f595002ac50 tx=0x7f5950004a40 comp rx=0 tx=0).stop 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f5957fff640 1 -- 172.21.15.99:0/3169745362 reap_dead start 2024-09-10T11:40:17.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 shutdown_connections 2024-09-10T11:40:17.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 >> 172.21.15.99:0/3169745362 conn(0x7f595802cd20 msgr2=0x7f595815cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 shutdown_connections 2024-09-10T11:40:17.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.833+0000 7f595cb9f640 1 -- 172.21.15.99:0/3169745362 wait complete. 2024-09-10T11:40:17.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T11:40:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.944+0000 7fa661ae4640 1 Processor -- start 2024-09-10T11:40:17.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.944+0000 7fa661ae4640 1 -- start start 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa661ae4640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa65c15aef0 0x7fa65c0b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa661ae4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa65c15b2d0 0x7fa65c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa661ae4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa65c06ee10 con 0x7fa65c0cd290 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa661ae4640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7fa65c0b0290 con 0x7fa65c15aef0 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa661ae4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7fa65c06e570 con 0x7fa65c15b2d0 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa65b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa65c15aef0 0x7fa65c0b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa65b7fe640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa65c15aef0 0x7fa65c0b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35812/0 (socket says 172.21.15.99:35812) 2024-09-10T11:40:17.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.945+0000 7fa65b7fe640 1 -- 172.21.15.99:0/1486931397 learned_addr learned my addr 172.21.15.99:0/1486931397 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65b7fe640 1 -- 172.21.15.99:0/1486931397 >> v2:172.21.15.196:3300/0 conn(0x7fa65c15b2d0 msgr2=0x7fa65c0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65b7fe640 1 --2- 172.21.15.99:0/1486931397 >> v2:172.21.15.196:3300/0 conn(0x7fa65c15b2d0 0x7fa65c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65b7fe640 1 -- 172.21.15.99:0/1486931397 >> v1:172.21.15.196:6789/0 conn(0x7fa65c0cd290 legacy=0x7fa65c0714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65b7fe640 1 -- 172.21.15.99:0/1486931397 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c06e850 con 0x7fa65c15aef0 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65b7fe640 1 --2- 172.21.15.99:0/1486931397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c0b4510 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009870 tx=0x7fa64c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7bd0f1cbb7f587 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65a7fc640 1 -- 172.21.15.99:0/1486931397 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c037070 con 0x7fa65c15aef0 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65a7fc640 1 -- 172.21.15.99:0/1486931397 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c032440 con 0x7fa65c15aef0 2024-09-10T11:40:17.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.946+0000 7fa65a7fc640 1 -- 172.21.15.99:0/1486931397 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c032810 con 0x7fa65c15aef0 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 -- 172.21.15.99:0/1486931397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 msgr2=0x7fa65c0b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 --2- 172.21.15.99:0/1486931397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c0b4510 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009870 tx=0x7fa64c02a270 comp rx=0 tx=0).stop 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 -- 172.21.15.99:0/1486931397 shutdown_connections 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 --2- 172.21.15.99:0/1486931397 >> v2:172.21.15.196:3300/0 conn(0x7fa65c15b2d0 0x7fa65c0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 --2- 172.21.15.99:0/1486931397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c0b4510 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 -- 172.21.15.99:0/1486931397 >> 172.21.15.99:0/1486931397 conn(0x7fa65c06bc10 msgr2=0x7fa65c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:17.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 -- 172.21.15.99:0/1486931397 shutdown_connections 2024-09-10T11:40:17.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.947+0000 7fa661ae4640 1 -- 172.21.15.99:0/1486931397 wait complete. 2024-09-10T11:40:17.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.948+0000 7fa661ae4640 1 Processor -- start 2024-09-10T11:40:17.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.948+0000 7fa661ae4640 1 -- start start 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c0cd290 0x7fa65c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa65c15b2d0 0x7fa65c1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c071d30 con 0x7fa65c15aef0 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c071bb0 con 0x7fa65c0cd290 2024-09-10T11:40:17.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa661ae4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7fa65c071eb0 con 0x7fa65c15b2d0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35826/0 (socket says 172.21.15.99:35826) 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 -- 172.21.15.99:0/537966310 learned_addr learned my addr 172.21.15.99:0/537966310 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65b7fe640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c0cd290 0x7fa65c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 -- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa65c15b2d0 msgr2=0x7fa65c1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa65c15b2d0 0x7fa65c1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.949+0000 7fa65affd640 1 -- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c0cd290 msgr2=0x7fa65c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa65affd640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c0cd290 0x7fa65c14e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa65affd640 1 -- 172.21.15.99:0/537966310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c0aa490 con 0x7fa65c15aef0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa65affd640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b4a0 tx=0x7fa65000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650014070 con 0x7fa65c15aef0 2024-09-10T11:40:17.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa650010610 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c0aa730 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650010a00 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.950+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c14bb60 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.951+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65c0d07b0 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.951+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa650017070 con 0x7fa65c15aef0 2024-09-10T11:40:17.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.952+0000 7fa643fff640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.952+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7fa65008a540 con 0x7fa65c15aef0 2024-09-10T11:40:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.952+0000 7fa65b7fe640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:17.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.952+0000 7fa65b7fe640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 0x7fa64806e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa64c034000 tx=0x7fa64c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:17.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:17.956+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa65c0d07b0 con 0x7fa65c15aef0 2024-09-10T11:40:18.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.123+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa65c0aff10 con 0x7fa64806be30 2024-09-10T11:40:18.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.123+0000 7fa643fff640 1 -- 172.21.15.99:0/537966310 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa65c0aff10 con 0x7fa64806be30 2024-09-10T11:40:18.125 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 msgr2=0x7fa64806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 0x7fa64806e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa64c034000 tx=0x7fa64c02e000 comp rx=0 tx=0).stop 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 msgr2=0x7fa65c15ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b4a0 tx=0x7fa65000b960 comp rx=0 tx=0).stop 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 shutdown_connections 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7fa64806be30 0x7fa64806e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa65c15b2d0 0x7fa65c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa65c15aef0 0x7fa65c15ddd0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 --2- 172.21.15.99:0/537966310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa65c0cd290 0x7fa65c14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.125+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 >> 172.21.15.99:0/537966310 conn(0x7fa65c06bc10 msgr2=0x7fa65c158980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.126+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 shutdown_connections 2024-09-10T11:40:18.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.126+0000 7fa661ae4640 1 -- 172.21.15.99:0/537966310 wait complete. 2024-09-10T11:40:18.137 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T11:40:18.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.233+0000 7f1faef11640 1 Processor -- start 2024-09-10T11:40:18.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.233+0000 7f1faef11640 1 -- start start 2024-09-10T11:40:18.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1faef11640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1fa80d1760 0x7f1fa80cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1faef11640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fa80d0130 0x7f1fa80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1faef11640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa80ce430 con 0x7f1fa8160030 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1faef11640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa80ce710 con 0x7f1fa80d0130 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1faef11640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa80d1240 con 0x7f1fa80d1760 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1fad4ae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fa80d0130 0x7f1fa80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1fad4ae640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1fa80d0130 0x7f1fa80cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35840/0 (socket says 172.21.15.99:35840) 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.234+0000 7f1fad4ae640 1 -- 172.21.15.99:0/4233419483 learned_addr learned my addr 172.21.15.99:0/4233419483 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1f9ffff640 1 --2- 172.21.15.99:0/4233419483 >> v2:172.21.15.196:3300/0 conn(0x7f1fa80d1760 0x7f1fa80cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1fad4ae640 1 -- 172.21.15.99:0/4233419483 >> v2:172.21.15.196:3300/0 conn(0x7f1fa80d1760 msgr2=0x7f1fa80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1fad4ae640 1 --2- 172.21.15.99:0/4233419483 >> v2:172.21.15.196:3300/0 conn(0x7f1fa80d1760 0x7f1fa80cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1fad4ae640 1 -- 172.21.15.99:0/4233419483 >> v1:172.21.15.196:6789/0 conn(0x7f1fa8160030 legacy=0x7f1fa80b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1fad4ae640 1 -- 172.21.15.99:0/4233419483 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa80ce030 con 0x7f1fa80d0130 2024-09-10T11:40:18.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1fad4ae640 1 --2- 172.21.15.99:0/4233419483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cdaf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80cee70 tx=0x7f1fa002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63018ab8e87030a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.235+0000 7f1f9f7fe640 1 -- 172.21.15.99:0/4233419483 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0036070 con 0x7f1fa80d0130 2024-09-10T11:40:18.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1f9f7fe640 1 -- 172.21.15.99:0/4233419483 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa0031600 con 0x7f1fa80d0130 2024-09-10T11:40:18.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1f9f7fe640 1 -- 172.21.15.99:0/4233419483 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00319d0 con 0x7f1fa80d0130 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 -- 172.21.15.99:0/4233419483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d0130 msgr2=0x7f1fa80cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 --2- 172.21.15.99:0/4233419483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cdaf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80cee70 tx=0x7f1fa002a730 comp rx=0 tx=0).stop 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 -- 172.21.15.99:0/4233419483 shutdown_connections 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 --2- 172.21.15.99:0/4233419483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cdaf0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 --2- 172.21.15.99:0/4233419483 >> v2:172.21.15.196:3300/0 conn(0x7f1fa80d1760 0x7f1fa80cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 -- 172.21.15.99:0/4233419483 >> 172.21.15.99:0/4233419483 conn(0x7f1fa80b4650 msgr2=0x7f1fa80b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 -- 172.21.15.99:0/4233419483 shutdown_connections 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.236+0000 7f1faef11640 1 -- 172.21.15.99:0/4233419483 wait complete. 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.237+0000 7f1faef11640 1 Processor -- start 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.237+0000 7f1faef11640 1 -- start start 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 0x7f1fa80c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa81606a0 con 0x7f1fa80d1760 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa80b5950 con 0x7f1fa80d0130 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faef11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa81f8740 con 0x7f1fa8160030 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1f9ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1f9ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35844/0 (socket says 172.21.15.99:35844) 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1f9ffff640 1 -- 172.21.15.99:0/2463150923 learned_addr learned my addr 172.21.15.99:0/2463150923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:18.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1faccad640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1fad4ae640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 0x7f1fa80c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.238+0000 7f1f9ffff640 1 -- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 msgr2=0x7f1fa80c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9ffff640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 0x7f1fa80c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9ffff640 1 -- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fa80d0130 msgr2=0x7f1fa80cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9ffff640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9ffff640 1 -- 172.21.15.99:0/2463150923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa81f8a20 con 0x7f1fa80d1760 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1fad4ae640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 0x7f1fa80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9ffff640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d830 tx=0x7f1f9800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800b6c0 con 0x7f1fa80d1760 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98019070 con 0x7f1fa80d1760 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa81f8d00 con 0x7f1fa80d1760 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800ba80 con 0x7f1fa80d1760 2024-09-10T11:40:18.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.239+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa81f4c70 con 0x7f1fa80d1760 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.240+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa80b38f0 con 0x7f1fa80d1760 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.240+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f9801e460 con 0x7f1fa80d1760 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.241+0000 7f1f9d7fa640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.241+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f1f9808b0e0 con 0x7f1fa80d1760 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.241+0000 7f1faccad640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.241+0000 7f1faccad640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40016e0 tx=0x7f1fa4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.245+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f98059e20 con 0x7f1fa80d1760 2024-09-10T11:40:18.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.406+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 --> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fa806db80 con 0x7f1f9406be30 2024-09-10T11:40:18.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.407+0000 7f1f9d7fa640 1 -- 172.21.15.99:0/2463150923 <== mgr.4110 v2:172.21.15.99:6832/2898440632 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12592 (secure 0 0 0) 0x7f1fa806db80 con 0x7f1f9406be30 2024-09-10T11:40:18.407 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:18.408 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 msgr2=0x7f1f9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 0x7f1f9406e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1fa40016e0 tx=0x7f1fa4009040 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 msgr2=0x7f1fa80c69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800d830 tx=0x7f1f9800dcf0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 shutdown_connections 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f1f9406be30 0x7f1f9406e2f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1fa8160030 0x7f1fa80c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1fa80d1760 0x7f1fa80c69f0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 --2- 172.21.15.99:0/2463150923 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1fa80d0130 0x7f1fa80cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 >> 172.21.15.99:0/2463150923 conn(0x7f1fa80b4650 msgr2=0x7f1fa80b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 shutdown_connections 2024-09-10T11:40:18.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.409+0000 7f1faef11640 1 -- 172.21.15.99:0/2463150923 wait complete. 2024-09-10T11:40:18.420 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-10T11:40:17.038267+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9458,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553041,"statfs":{"total":1600000000,"available":1590314336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485521,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-131,"num_write_kb":-85083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001224"},"pg_stats":[{"pgid":"1.0","version":"189'119","reported_seq":520,"reported_epoch":189,"state":"active+clean","last_fresh":"2024-09-10T11:40:16.286032+0000","last_change":"2024-09-10T11:40:16.286054+0000","last_active":"2024-09-10T11:40:16.286032+0000","last_peered":"2024-09-10T11:40:16.286032+0000","last_clean":"2024-09-10T11:40:16.286032+0000","last_became_active":"2024-09-10T11:30:18.077050+0000","last_became_peered":"2024-09-10T11:30:18.077050+0000","last_unstale":"2024-09-10T11:40:16.286032+0000","last_undegraded":"2024-09-10T11:40:16.286032+0000","last_fullsized":"2024-09-10T11:40:16.286032+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"189'119","last_scrub_stamp":"2024-09-10T11:40:16.285994+0000","last_deep_scrub":"189'119","last_deep_scrub_stamp":"2024-09-10T11:40:16.285994+0000","last_clean_scrub_stamp":"2024-09-10T11:40:16.285994+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:56:56.220390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1614,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193697,"statfs":{"total":200000000,"available":198346516,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1038,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194274,"statfs":{"total":200000000,"available":198936884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-09-10T11:40:18.424 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T11:40:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.521+0000 7f5e8bf78640 1 Processor -- start 2024-09-10T11:40:18.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.521+0000 7f5e8bf78640 1 -- start start 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e8bf78640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e8415eb80 0x7f5e840d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e8bf78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e840d5010 0x7f5e840d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e8bf78640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8415ff70 con 0x7f5e841626c0 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e8bf78640 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 -- 0x7f5e840d1620 con 0x7f5e8415eb80 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e8bf78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e84162bc0 con 0x7f5e840d5010 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e89d14640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e8415eb80 0x7f5e840d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e89d14640 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5e8415eb80 0x7f5e840d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35860/0 (socket says 172.21.15.99:35860) 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e89513640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5e841626c0 0x7f5e840d4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35950/0 (socket says 172.21.15.99:35950) 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.522+0000 7f5e89d14640 1 -- 172.21.15.99:0/987878386 learned_addr learned my addr 172.21.15.99:0/987878386 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e89d14640 1 -- 172.21.15.99:0/987878386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e840d5010 msgr2=0x7f5e840d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:18.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e89d14640 1 --2- 172.21.15.99:0/987878386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e840d5010 0x7f5e840d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e89d14640 1 -- 172.21.15.99:0/987878386 >> v1:172.21.15.99:6789/0 conn(0x7f5e841626c0 legacy=0x7f5e840d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e88d12640 1 -- 172.21.15.99:0/987878386 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2197905320 0 0) 0x7f5e8415ff70 con 0x7f5e841626c0 2024-09-10T11:40:18.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e89d14640 1 -- 172.21.15.99:0/987878386 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e84162ea0 con 0x7f5e8415eb80 2024-09-10T11:40:18.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.523+0000 7f5e89d14640 1 --2- 172.21.15.99:0/987878386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e8415eb80 0x7f5e840d29b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098a0 tx=0x7f5e8002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440b16c3d7dbeb67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e88d12640 1 -- 172.21.15.99:0/987878386 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80037070 con 0x7f5e8415eb80 2024-09-10T11:40:18.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e88d12640 1 -- 172.21.15.99:0/987878386 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80032440 con 0x7f5e8415eb80 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e88d12640 1 -- 172.21.15.99:0/987878386 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80032810 con 0x7f5e8415eb80 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 -- 172.21.15.99:0/987878386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e8415eb80 msgr2=0x7f5e840d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/987878386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e8415eb80 0x7f5e840d29b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5e800098a0 tx=0x7f5e8002a200 comp rx=0 tx=0).stop 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 -- 172.21.15.99:0/987878386 shutdown_connections 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/987878386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e840d5010 0x7f5e840d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/987878386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e8415eb80 0x7f5e840d29b0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 -- 172.21.15.99:0/987878386 >> 172.21.15.99:0/987878386 conn(0x7f5e8408e680 msgr2=0x7f5e8409dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 -- 172.21.15.99:0/987878386 shutdown_connections 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.524+0000 7f5e8bf78640 1 -- 172.21.15.99:0/987878386 wait complete. 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.525+0000 7f5e8bf78640 1 Processor -- start 2024-09-10T11:40:18.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.525+0000 7f5e8bf78640 1 -- start start 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 0x7f5e841f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 0x7f5e840b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840d2ef0 con 0x7f5e841626c0 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84163460 con 0x7f5e840d5010 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8bf78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840d3070 con 0x7f5e8415eb80 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8a515640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8a515640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35874/0 (socket says 172.21.15.99:35874) 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.526+0000 7f5e8a515640 1 -- 172.21.15.99:0/3981009809 learned_addr learned my addr 172.21.15.99:0/3981009809 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 -- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 msgr2=0x7f5e840b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e89513640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 0x7f5e840b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e89d14640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 0x7f5e841f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 0x7f5e840b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 -- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 msgr2=0x7f5e841f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 0x7f5e841f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 -- 172.21.15.99:0/3981009809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e841f9340 con 0x7f5e841626c0 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e89d14640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 0x7f5e841f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e89513640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 0x7f5e840b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8a515640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b9e0 tx=0x7f5e7c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c019070 con 0x7f5e841626c0 2024-09-10T11:40:18.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c00cb20 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e840b3fe0 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c014440 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.527+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e840b2ce0 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.528+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e840d29b0 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.528+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e7c01c070 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.529+0000 7f5e7a7fc640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 0x7f5e6806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.529+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f5e7c08a830 con 0x7f5e841626c0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.529+0000 7f5e89d14640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 0x7f5e6806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.529+0000 7f5e89d14640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 0x7f5e6806e320 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5e80034000 tx=0x7f5e8002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.533+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e7c0594e0 con 0x7f5e841626c0 2024-09-10T11:40:18.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.698+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f5e840b5a60 con 0x7f5e841626c0 2024-09-10T11:40:18.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.701+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7f5e7c058de0 con 0x7f5e841626c0 2024-09-10T11:40:18.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.701+0000 7f5e7a7fc640 1 -- 172.21.15.99:0/3981009809 <== mon.0 v2:172.21.15.99:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5e7c05d190 con 0x7f5e841626c0 2024-09-10T11:40:18.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.703+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 msgr2=0x7f5e6806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.703+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 0x7f5e6806e320 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f5e80034000 tx=0x7f5e8002e000 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 msgr2=0x7f5e8415bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00b9e0 tx=0x7f5e7c00bea0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 shutdown_connections 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f5e6806be60 0x7f5e6806e320 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e841626c0 0x7f5e8415bc40 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5e8415eb80 0x7f5e840b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 --2- 172.21.15.99:0/3981009809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e840d5010 0x7f5e841f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 >> 172.21.15.99:0/3981009809 conn(0x7f5e8408e680 msgr2=0x7f5e840d1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 shutdown_connections 2024-09-10T11:40:18.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.704+0000 7f5e8bf78640 1 -- 172.21.15.99:0/3981009809 wait complete. 2024-09-10T11:40:18.715 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T11:40:18.715 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T11:40:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.811+0000 7f4c3b0ed640 1 Processor -- start 2024-09-10T11:40:18.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.812+0000 7f4c3b0ed640 1 -- start start 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.812+0000 7f4c3b0ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c34073440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.812+0000 7f4c3b0ed640 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c34152e60 con 0x7f4c3406e5d0 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.812+0000 7f4c3b0ed640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c340d15e0 con 0x7f4c3415c770 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.812+0000 7f4c3b0ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c34072680 con 0x7f4c340d1200 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c33fff640 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4c3406e5d0 0x7f4c3415c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35964/0 (socket says 172.21.15.99:35964) 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c33fff640 1 -- 172.21.15.99:0/2850890433 learned_addr learned my addr 172.21.15.99:0/2850890433 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c34073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44144114 0 0) 0x7f4c34152e60 con 0x7f4c3406e5d0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c24003520 con 0x7f4c3406e5d0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2792012308 0 0) 0x7f4c24003520 con 0x7f4c3406e5d0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 msgr2=0x7f4c34073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c34073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 >> v1:172.21.15.196:6789/0 conn(0x7f4c3415c770 legacy=0x7f4c34159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c34072960 con 0x7f4c3406e5d0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3737057499 0 0) 0x7f4c20001670 con 0x7f4c3406e5d0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 >> v1:172.21.15.99:6789/0 conn(0x7f4c3406e5d0 legacy=0x7f4c3415c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c24004d40 0x7f4c240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34072680 con 0x7f4c240039e0 2024-09-10T11:40:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2400baf0 con 0x7f4c24004d40 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2400bcf0 con 0x7f4c340d1200 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c33fff640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c3968a640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c24004d40 0x7f4c240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 -- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 msgr2=0x7f4c24004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c340d1200 0x7f4c24004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 -- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c24004d40 msgr2=0x7f4c240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c24004d40 0x7f4c240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c38e89640 1 -- 172.21.15.99:0/2850890433 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24003520 con 0x7f4c240039e0 2024-09-10T11:40:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.813+0000 7f4c3968a640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c24004d40 0x7f4c240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.814+0000 7f4c38e89640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009ac0 tx=0x7f4c24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea59d955b83355bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.814+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28036070 con 0x7f4c240039e0 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.814+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4c28031400 con 0x7f4c240039e0 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.814+0000 7f4c337fe640 1 -- 172.21.15.99:0/2850890433 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28009690 con 0x7f4c240039e0 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/2850890433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240039e0 msgr2=0x7f4c24003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/2850890433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c240039e0 0x7f4c24003e00 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009ac0 tx=0x7f4c24005830 comp rx=0 tx=0).stop 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3968a640 1 -- 172.21.15.99:0/2850890433 reap_dead start 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/2850890433 shutdown_connections 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/2850890433 >> 172.21.15.99:0/2850890433 conn(0x7f4c3406bc10 msgr2=0x7f4c3406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/2850890433 shutdown_connections 2024-09-10T11:40:18.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.815+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/2850890433 wait complete. 2024-09-10T11:40:18.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.816+0000 7f4c3b0ed640 1 Processor -- start 2024-09-10T11:40:18.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.816+0000 7f4c3b0ed640 1 -- start start 2024-09-10T11:40:18.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c240039e0 0x7f4c34145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c34143a60 0x7f4c34147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3415cd90 con 0x7f4c341558f0 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c34072f20 con 0x7f4c240039e0 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3b0ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 -- 0x7f4c34073440 con 0x7f4c34143a60 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35896/0 (socket says 172.21.15.99:35896) 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c38e89640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c240039e0 0x7f4c34145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 -- 172.21.15.99:0/1365951760 learned_addr learned my addr 172.21.15.99:0/1365951760 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c33fff640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c34143a60 0x7f4c34147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 -- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c34143a60 msgr2=0x7f4c34147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c34143a60 0x7f4c34147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 -- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c240039e0 msgr2=0x7f4c34145bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c240039e0 0x7f4c34145bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:18.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.817+0000 7f4c3968a640 1 -- 172.21.15.99:0/1365951760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c34146e10 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c3968a640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4c34071bb0 tx=0x7f4c2c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c003b80 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4c2c003d40 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c34146290 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c006b50 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.818+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c34146850 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.819+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3406ea10 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.819+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c2c01a020 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.820+0000 7f4c317fa640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 0x7f4c1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.820+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 4943+0+0 (secure 0 0 0) 0x7f4c2c08a0d0 con 0x7f4c341558f0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.820+0000 7f4c38e89640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 0x7f4c1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:18.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.820+0000 7f4c38e89640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 0x7f4c1806e320 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4c28033000 tx=0x7f4c28008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:18.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:18.824+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c340d15e0 con 0x7f4c341558f0 2024-09-10T11:40:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.025+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c340761b0 con 0x7f4c341558f0 2024-09-10T11:40:19.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.025+0000 7f4c317fa640 1 -- 172.21.15.99:0/1365951760 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3914 (secure 0 0 0) 0x7f4c2c058d80 con 0x7f4c341558f0 2024-09-10T11:40:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-10T11:40:19.026 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":76,"btime":"2024-09-10T11:40:08:634049+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7198,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/3082360172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3082360172},{"type":"v1","addr":"172.21.15.196:6834","nonce":3082360172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7201,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1178393018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":1178393018},{"type":"v1","addr":"172.21.15.196:6835","nonce":1178393018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/3980848265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":3980848265},{"type":"v1","addr":"172.21.15.99:6836","nonce":3980848265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/231359051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":231359051},{"type":"v1","addr":"172.21.15.99:6837","nonce":231359051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-09-10T11:40:19.027 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 76 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 msgr2=0x7f4c1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 0x7f4c1806e320 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4c28033000 tx=0x7f4c28008030 comp rx=0 tx=0).stop 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 msgr2=0x7f4c34155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f4c34071bb0 tx=0x7f4c2c00ead0 comp rx=0 tx=0).stop 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 shutdown_connections 2024-09-10T11:40:19.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:6832/2898440632,v1:172.21.15.99:6833/2898440632] conn(0x7f4c1806be60 0x7f4c1806e320 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c341558f0 0x7f4c34155d70 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c34143a60 0x7f4c34147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 --2- 172.21.15.99:0/1365951760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c240039e0 0x7f4c34145bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 >> 172.21.15.99:0/1365951760 conn(0x7f4c3406bc10 msgr2=0x7f4c340ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 shutdown_connections 2024-09-10T11:40:19.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-10T11:40:19.027+0000 7f4c3b0ed640 1 -- 172.21.15.99:0/1365951760 wait complete. 2024-09-10T11:40:19.039 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T11:40:19.039 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T11:40:19.040 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T11:40:19.040 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.048 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T11:40:19.048 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T11:40:19.048 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.097 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T11:40:19.098 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T11:40:19.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.111 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T11:40:19.111 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T11:40:19.111 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.164 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T11:40:19.164 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T11:40:19.164 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T11:40:19.164 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.182 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T11:40:19.182 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T11:40:19.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.239 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T11:40:19.240 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T11:40:19.240 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.297 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T11:40:19.297 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T11:40:19.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.355 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T11:40:19.355 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T11:40:19.355 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.375 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T11:40:19.375 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T11:40:19.375 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.436 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T11:40:19.436 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T11:40:19.436 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.500 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T11:40:19.500 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T11:40:19.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.560 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T11:40:19.560 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T11:40:19.560 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T11:40:19.560 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.586 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T11:40:19.586 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T11:40:19.586 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.627 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T11:40:19.627 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T11:40:19.628 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T11:40:19.628 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.639 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T11:40:19.639 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T11:40:19.639 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.679 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T11:40:19.680 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T11:40:19.680 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T11:40:19.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-10T11:40:19.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-10T11:40:19.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-10T11:40:19.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-10T11:40:19.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-10T11:40:19.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-10T11:40:19.732 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-10T11:40:19.732 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T11:40:19.732 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T11:40:19.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-09-10T11:40:19.773 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:19.774 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T11:40:20.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:20.126 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T11:40:20.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:20.211 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T11:40:20.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:20.271 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T11:40:20.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:20.319 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T11:40:20.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:20.517 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T11:40:20.586 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:20.586 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T11:40:20.644 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:20.644 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T11:40:20.705 INFO:tasks.ceph:Archiving mon data... 2024-09-10T11:40:20.706 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/data/mon.a.tgz 2024-09-10T11:40:20.707 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-10T11:40:20.724 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.a7zvPekDzs 2024-09-10T11:40:20.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.a7zvPekDzs 2024-09-10T11:40:20.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.a7zvPekDzs 2024-09-10T11:40:20.961 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.a7zvPekDzs is 397KB 2024-09-10T11:40:21.056 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.a7zvPekDzs 2024-09-10T11:40:21.071 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/data/mon.b.tgz 2024-09-10T11:40:21.072 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-10T11:40:21.089 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.ZEi1ngXb1X 2024-09-10T11:40:21.090 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ZEi1ngXb1X 2024-09-10T11:40:21.243 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.ZEi1ngXb1X 2024-09-10T11:40:21.327 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.ZEi1ngXb1X is 398KB 2024-09-10T11:40:21.422 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.ZEi1ngXb1X 2024-09-10T11:40:21.437 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/data/mon.c.tgz 2024-09-10T11:40:21.437 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-10T11:40:21.491 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.9tPgrvrEzQ 2024-09-10T11:40:21.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.9tPgrvrEzQ 2024-09-10T11:40:21.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.9tPgrvrEzQ 2024-09-10T11:40:21.710 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.9tPgrvrEzQ is 398KB 2024-09-10T11:40:21.806 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.9tPgrvrEzQ 2024-09-10T11:40:21.822 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T11:40:21.822 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T11:40:21.824 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T11:40:21.906 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T11:40:21.943 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T11:40:21.943 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T11:40:21.945 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi099/crash 2024-09-10T11:40:21.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T11:40:21.972 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi196/crash 2024-09-10T11:40:21.973 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T11:40:22.007 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T11:40:22.007 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:22.017 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:22.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:22.125 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T11:40:22.150 INFO:tasks.ceph:Compressing logs... 2024-09-10T11:40:22.150 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T11:40:22.168 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T11:40:22.278 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32119.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40256.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.32119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31828.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.40256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38886.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32119.log.gz 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.31828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33004.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31828.log.gz 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.38886.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr: 90.8%/var/log/ceph/ceph-client.1.33004.log: -- replaced with /var/log/ceph/ceph-client.1.40256.log.gz 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34814.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39591.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.34814.log: 87.3% -- replaced with /var/log/ceph/ceph-client.1.33004.log.gz 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T11:40:22.279 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.39591.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose 90.2% -- replaced with /var/log/ceph/ceph-client.1.34814.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: -- /var/log/ceph/ceph-client.admin.31948.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.1.38886.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38177.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.31948.log: 92.9% -- replaced with /var/log/ceph/ceph-client.1.39591.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32033.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31948.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.38177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31908.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.32033.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: 87.8%/var/log/ceph/ceph-client.admin.31908.log: -- replaced with /var/log/ceph/ceph-client.admin.32033.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31868.log 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31908.log.gz 2024-09-10T11:40:22.280 INFO:teuthology.orchestra.run.smithi196.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.1.38177.log.gz 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32079.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.31868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31988.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31868.log.gz 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.32079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33460.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.32079.log.gz 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.31988.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31988.log.gz 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.3.33460.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr: 94.2% -- replaced with /var/log/ceph/ceph.audit.log.gz/var/log/ceph/ceph-osd.6.log: 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:gzip 87.1% -- replaced with /var/log/ceph/ceph-client.3.33460.log.gz 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36823.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36123.log 2024-09-10T11:40:22.281 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.36823.log: 90.4% -- replaced with /var/log/ceph/ceph-client.1.36823.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.36123.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.1.36123.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.37534.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mgr.x.log: 94.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34176.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.37534.log: 88.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr: -- replaced with /var/log/ceph/ceph-client.1.37534.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.34176.log: 88.0% -- replaced with /var/log/ceph/ceph-client.1.34176.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35462.log 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T11:40:22.282 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.35462.log: 93.0% -- replaced with /var/log/ceph/ceph-client.1.35462.log.gz 2024-09-10T11:40:22.398 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T11:40:22.398 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32441.log 2024-09-10T11:40:22.398 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32799.log 2024-09-10T11:40:22.398 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.32441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34072.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32799.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32441.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51896.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34072.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.32799.log.gz --verbose 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr: -- /var/log/ceph/ceph-client.admin.39670.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51896.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34072.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36019.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51896.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57550.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39670.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56730.log 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36019.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57550.log.gz 2024-09-10T11:40:22.399 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59701.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59701.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.56730.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59701.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36486.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65823.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53810.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59577.log 2024-09-10T11:40:22.400 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65823.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55454.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59577.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59577.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42616.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55454.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64556.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45246.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52874.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41057.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52874.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.64556.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46638.log 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41057.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.52874.log.gz 2024-09-10T11:40:22.401 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40456.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41057.log.gz 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55370.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46638.log.gz/var/log/ceph/ceph-client.admin.40456.log: 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57283.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55370.log: gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40456.log.gz -- 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.39713.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55370.log.gz 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61161.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39713.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57283.log.gz 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60138.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61161.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.57634.log 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61161.log.gz 2024-09-10T11:40:22.402 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41950.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57634.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60138.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41950.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57634.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62333.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41950.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64504.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62333.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53641.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37692.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62333.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64504.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54336.log 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53641.log.gz 2024-09-10T11:40:22.403 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49072.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54336.log: gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37692.log.gz /var/log/ceph/ceph-client.admin.65192.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54336.log.gz 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54171.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40122.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54171.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49072.log.gz 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65192.log.gz 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56993.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54171.log.gz 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60012.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56993.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40122.log.gz 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-09-10T11:40:22.404 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56993.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60012.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54091.log 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33396.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60012.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33736.log 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33400.log 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33736.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54091.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33393.log 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33736.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48338.log 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33400.log.gz 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45964.log/var/log/ceph/ceph-client.admin.33393.log: 2024-09-10T11:40:22.405 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33393.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45964.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48338.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37553.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45964.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44039.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37553.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47513.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39442.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44039.log: /var/log/ceph/ceph-client.admin.47513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44039.log.gz 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2024-09-10T11:40:22.406 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47513.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39442.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45045.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54294.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39265.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40202.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31833.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45045.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40202.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39265.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56193.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.31833.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40202.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57462.log 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31833.log.gz 2024-09-10T11:40:22.407 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56193.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42802.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57462.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61861.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58678.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61861.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61861.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58678.log.gz 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-09-10T11:40:22.408 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48548.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39484.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50890.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39484.log: gzip 88.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz -- 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.64931.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50890.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.39484.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50890.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44967.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48634.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.64931.log.gz -5 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62682.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44967.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50512.log 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44967.log.gz 2024-09-10T11:40:22.409 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62682.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62682.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64973.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50512.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64973.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65531.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58285.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32221.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65531.log: /var/log/ceph/ceph-client.admin.58285.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65531.log.gz 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45884.log 2024-09-10T11:40:22.410 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58285.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32221.log: /var/log/ceph/ceph-client.admin.45884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48592.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32221.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46746.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45884.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33733.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47933.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48592.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33733.log: /var/log/ceph/ceph-client.admin.47933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38704.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33733.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34071.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47933.log.gz 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 2024-09-10T11:40:22.411 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54845.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38704.log: /var/log/ceph/ceph-client.admin.56268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65363.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38704.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62079.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54551.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54845.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65363.log: /var/log/ceph/ceph-client.admin.62079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65363.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44207.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1%/var/log/ceph/ceph-client.admin.54551.log: -- replaced with /var/log/ceph/ceph-client.admin.62079.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42700.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54551.log.gz 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36689.log 2024-09-10T11:40:22.412 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56646.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53455.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44207.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36689.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34269.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53455.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.43871.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56646.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53455.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40283.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34269.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40721.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43871.log.gz 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50806.log 2024-09-10T11:40:22.413 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40283.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65278.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51980.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40721.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50806.log: /var/log/ceph/ceph-client.admin.65278.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50806.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45559.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65278.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59192.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51980.log: /var/log/ceph/ceph-client.admin.45559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33135.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51980.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59192.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45559.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54425.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59192.log.gz 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58600.log 2024-09-10T11:40:22.414 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54425.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33135.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44315.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54425.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58600.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47723.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44315.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34665.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37597.log: /var/log/ceph/ceph-client.admin.47723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56898.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34665.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47723.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34665.log.gz 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61548.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61591.log 2024-09-10T11:40:22.415 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56898.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36081.log: /var/log/ceph/ceph-client.admin.61548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60390.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61548.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61591.log: /var/log/ceph/ceph-client.admin.60390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60650.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61591.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60390.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59276.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.60650.log: /var/log/ceph/ceph-client.admin.43741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35741.log 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-09-10T11:40:22.416 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51938.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59276.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35741.log: /var/log/ceph/ceph-client.admin.51938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61404.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35741.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42742.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51938.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46428.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56436.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55783.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46428.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61404.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46428.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65447.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56436.log.gz 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33399.log 2024-09-10T11:40:22.417 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55783.log: gzip/var/log/ceph/ceph-client.admin.65447.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65741.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55783.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35699.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65447.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33399.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47807.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65741.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40325.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35699.log: /var/log/ceph/ceph-client.admin.47807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50311.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35699.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40325.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47807.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62766.log 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50311.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40325.log.gz 2024-09-10T11:40:22.418 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33714.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50311.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44774.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62766.log: /var/log/ceph/ceph-client.admin.33714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42415.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62766.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33714.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52022.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42415.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44774.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49555.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42415.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42205.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49995.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52022.log: /var/log/ceph/ceph-client.admin.42205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58389.log 2024-09-10T11:40:22.419 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52022.log.gz -- replaced with /var/log/ceph/ceph-client.admin.42205.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62724.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34395.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49995.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58389.log: /var/log/ceph/ceph-client.admin.62724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65015.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58389.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62724.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48928.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34395.log: /var/log/ceph/ceph-client.admin.65015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34395.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48928.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65015.log.gz 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37043.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54593.log 2024-09-10T11:40:22.420 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39755.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37043.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52790.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37043.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52172.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54593.log: /var/log/ceph/ceph-client.admin.52790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59831.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54593.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52790.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65573.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5/var/log/ceph/ceph-client.admin.59831.log: 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52172.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.45517.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65573.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59831.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61633.log 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65573.log.gz 2024-09-10T11:40:22.421 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58921.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45517.log: gzip/var/log/ceph/ceph-client.admin.61633.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45517.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58921.log: gzip -5 88.8% --verbose -- /var/log/ceph/ceph-client.admin.63883.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61633.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58921.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60180.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50121.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63883.log: /var/log/ceph/ceph-client.admin.60180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62245.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63883.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50121.log: gzip 88.1% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60180.log.gz 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.54635.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32757.log 2024-09-10T11:40:22.422 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62245.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50121.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54635.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62245.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45432.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54635.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32757.log: /var/log/ceph/ceph-client.admin.45432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55666.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32757.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45432.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56310.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55666.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55666.log.gz 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63683.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55582.log 2024-09-10T11:40:22.423 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56310.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38981.log: /var/log/ceph/ceph-client.admin.63683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33397.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63683.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46088.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55582.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33397.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52432.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46512.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46088.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.52432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55741.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33737.log 2024-09-10T11:40:22.424 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51111.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9%/var/log/ceph/ceph-client.admin.33737.log: -- replaced with /var/log/ceph/ceph-client.admin.55741.log.gz 91.2% 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53581.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33737.log.gz 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51111.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51111.log.gz 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40637.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.52432.log.gz 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53581.log.gz 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54252.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52832.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40637.log.gz 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47765.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61278.log 2024-09-10T11:40:22.425 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54252.log: /var/log/ceph/ceph-client.admin.47765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64649.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61278.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47765.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52970.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38600.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61278.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2%/var/log/ceph/ceph-client.admin.52970.log: -- replaced with /var/log/ceph/ceph-client.admin.64649.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52970.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64889.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38600.log: /var/log/ceph/ceph-client.admin.46554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40549.log 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38600.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-09-10T11:40:22.426 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63821.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64889.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40549.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65236.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36312.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32399.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63821.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65236.log: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz -5 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54761.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65236.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63925.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36312.log: /var/log/ceph/ceph-client.admin.54761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49597.log 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36312.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54761.log.gz 2024-09-10T11:40:22.427 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41832.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63925.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49597.log: /var/log/ceph/ceph.audit.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49597.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36941.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41832.log: /var/log/ceph/ceph-client.admin.36941.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41832.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34068.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36941.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40679.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34068.log.gz 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44165.log 2024-09-10T11:40:22.428 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40679.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.42079.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65146.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44165.log: gzip 88.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.43997.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: 89.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44165.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43997.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53329.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65146.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43997.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58327.log 2024-09-10T11:40:22.429 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34069.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53329.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58327.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.49398.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58327.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48718.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63291.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.48718.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32715.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63291.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57718.log 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63291.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32715.log: gzip 93.6% -- replaced with /var/log/ceph/ceph-client.0.60650.log.gz 2024-09-10T11:40:22.430 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43829.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32715.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59787.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57718.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53936.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43829.log: /var/log/ceph/ceph-client.admin.59787.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43829.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55859.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59787.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38032.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53936.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48101.log 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55859.log: /var/log/ceph/ceph-client.admin.38032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63134.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55859.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48101.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38032.log.gz 2024-09-10T11:40:22.431 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45372.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48101.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35935.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.63134.log: /var/log/ceph/ceph-client.admin.45372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58963.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45372.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61488.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.58963.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35935.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60807.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61488.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58963.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61488.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-client.admin.55624.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61083.log 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60807.log.gz 2024-09-10T11:40:22.432 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51191.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55624.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33219.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9%/var/log/ceph/ceph-client.admin.51191.log: -- replaced with /var/log/ceph/ceph-client.admin.61083.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51191.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61236.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33219.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.36354.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61236.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62808.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61236.log.gz 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45685.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50596.log 2024-09-10T11:40:22.433 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62808.log: -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64462.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59234.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50596.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.62808.log.gz/var/log/ceph/ceph-client.admin.64462.log: 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35577.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50596.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64462.log.gz 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47681.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59234.log.gz 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.2.35577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40027.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.47681.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50233.log 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47681.log.gz 2024-09-10T11:40:22.434 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.2.35577.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35201.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40027.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65781.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40591.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65781.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50233.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37001.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65781.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35977.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40591.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60324.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35977.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37001.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56394.log 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35977.log.gz 2024-09-10T11:40:22.435 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60324.log: /var/log/ceph/ceph-client.admin.56394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59150.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56394.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60324.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48676.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59150.log: /var/log/ceph/ceph-client.admin.48676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43159.log 87.7% 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59150.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2%/var/log/ceph/ceph-client.admin.50974.log: -- replaced with /var/log/ceph/ceph-client.admin.48676.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43159.log.gz 2024-09-10T11:40:22.436 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.37641.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51291.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64294.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/ceph-client.admin.64294.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50428.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34073.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64294.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51686.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50428.log.gz 2024-09-10T11:40:22.437 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47849.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51686.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr: -- /var/log/ceph/ceph-client.admin.43699.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46130.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48017.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5/var/log/ceph/ceph-client.admin.43699.log: --verbose -- /var/log/ceph/ceph-client.admin.36176.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48017.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56772.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48017.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42887.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36176.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56772.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56772.log.gz 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr: /var/log/ceph/ceph-mds.a.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40412.log 2024-09-10T11:40:22.438 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mds.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42887.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51644.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33732.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40412.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51644.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50848.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33732.log: /var/log/ceph/ceph-client.admin.42490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33389.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50848.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33732.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39610.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50848.log.gz 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35851.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33389.log: /var/log/ceph/ceph-client.admin.39610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-09-10T11:40:22.439 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35851.log: 87.5% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33389.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39610.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35851.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35082.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54803.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48059.log: /var/log/ceph/ceph-client.0.35082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37734.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54803.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47006.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54803.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.0.35082.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32883.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.47006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45643.log 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37734.log.gz 2024-09-10T11:40:22.440 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59743.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32883.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56115.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5%/var/log/ceph/ceph-client.admin.59743.log: -- replaced with /var/log/ceph/ceph-client.admin.45643.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59743.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47555.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47163.log: /var/log/ceph/ceph-client.admin.56115.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56115.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63333.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50638.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47555.log: /var/log/ceph/ceph-client.admin.63333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47555.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63333.log.gz 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43913.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-09-10T11:40:22.441 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50638.log: /var/log/ceph/ceph-client.admin.43913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50638.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43913.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51812.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.35893.log: /var/log/ceph/ceph-client.admin.53894.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59062.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50470.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59062.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51812.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59062.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36773.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61903.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50470.log: /var/log/ceph/ceph-client.admin.36773.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50470.log.gz 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45162.log 2024-09-10T11:40:22.442 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36773.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32925.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61903.log: /var/log/ceph/ceph-client.admin.45162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61903.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45162.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30031.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32925.log: /var/log/ceph/ceph.tmp-client.admin.30031.log: 0.0% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32925.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph.tmp-client.admin.30031.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50764.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39187.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43544.log: /var/log/ceph/ceph-client.admin.50764.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51016.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50764.log.gz 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64607.log 2024-09-10T11:40:22.443 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51016.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39187.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51016.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57327.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43416.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64607.log: /var/log/ceph/ceph-client.admin.57327.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64607.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57327.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43416.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51514.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39839.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64420.log 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51514.log.gz 2024-09-10T11:40:22.444 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64733.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64420.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33735.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59402.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5%/var/log/ceph/ceph-client.admin.33735.log: -- replaced with /var/log/ceph/ceph-client.admin.64733.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49725.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7%/var/log/ceph/ceph-client.admin.59402.log: -- replaced with /var/log/ceph/ceph-client.admin.33735.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62514.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59402.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36234.log 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49725.log.gz 2024-09-10T11:40:22.445 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62514.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53978.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.36234.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36234.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49843.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53978.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53978.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39881.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49843.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60849.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39881.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39881.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60849.log: 89.5% 95.0% -- replaced with /var/log/ceph/ceph-client.0.63134.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60849.log.gz 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-10T11:40:22.446 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53254.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40367.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33401.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53254.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33401.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40367.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33401.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59108.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54719.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50722.log: /var/log/ceph/ceph-client.admin.59108.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59108.log.gz 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41624.log 2024-09-10T11:40:22.447 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54719.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54719.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57760.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41624.log: /var/log/ceph/ceph-client.admin.57760.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41624.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63524.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57760.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37822.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56856.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63524.log: /var/log/ceph/ceph-client.admin.34563.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63524.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59005.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56856.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56856.log.gz 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57676.log 2024-09-10T11:40:22.448 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59005.log: /var/log/ceph/ceph-client.admin.37205.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59005.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46218.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57676.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57676.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52064.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55213.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46218.log: /var/log/ceph/ceph-client.admin.52064.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46218.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52064.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46046.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.55213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46046.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46046.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39965.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46680.log 2024-09-10T11:40:22.449 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33734.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56604.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46680.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46680.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38074.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56604.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56604.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38074.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38074.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57073.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41015.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48461.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57073.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44081.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48461.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48461.log.gz 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33933.log 2024-09-10T11:40:22.450 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44081.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44081.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44123.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33933.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33933.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44123.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44123.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56352.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55919.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56352.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56352.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38443.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57420.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55919.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.38443.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.38443.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.55213.log.gz 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2024-09-10T11:40:22.451 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57420.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57420.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47975.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50932.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42532.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61446.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49012.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61446.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61446.log.gz 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64177.log 2024-09-10T11:40:22.452 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46470.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61675.log 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64177.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64177.log.gz 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50386.log 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61675.log.gz 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56562.log 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50386.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50386.log.gz 2024-09-10T11:40:22.464 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36815.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56562.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56562.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34070.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36815.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36815.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34067.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34070.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34070.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53684.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34067.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34067.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33177.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53684.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53684.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32571.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33177.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33177.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32571.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32571.log.gz 2024-09-10T11:40:22.465 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50680.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36647.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50680.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50680.log.gz 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63779.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37427.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63779.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63779.log.gz 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37906.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37427.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37427.log.gz 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37906.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37906.log.gz 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62289.log 2024-09-10T11:40:22.466 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51598.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gz 2024-09-10T11:40:22.467 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-09-10T11:40:22.467 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62289.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62289.log.gz 2024-09-10T11:40:22.481 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56688.log 2024-09-10T11:40:22.482 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40507.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-09-10T11:40:22.495 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-09-10T11:40:22.496 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56688.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56688.log.gz 2024-09-10T11:40:22.506 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64252.log 2024-09-10T11:40:22.507 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37776.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 2024-09-10T11:40:22.514 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T11:40:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-09-10T11:40:22.517 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64252.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64252.log.gz 2024-09-10T11:40:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63566.log 2024-09-10T11:40:22.535 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34353.log 2024-09-10T11:40:22.535 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63566.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63566.log.gz 2024-09-10T11:40:22.537 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-09-10T11:40:22.543 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2024-09-10T11:40:22.549 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34353.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T11:40:22.549 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34353.log.gz 2024-09-10T11:40:22.552 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-09-10T11:40:22.552 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51556.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2024-09-10T11:40:22.552 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T11:40:22.552 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51333.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-09-10T11:40:22.552 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60911.log 2024-09-10T11:40:22.554 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph.log: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T11:40:22.556 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36857.log 2024-09-10T11:40:22.556 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60911.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60911.log.gz 2024-09-10T11:40:22.556 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65321.log 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36857.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36857.log.gz 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65321.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65321.log.gz 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41099.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-10T11:40:22.566 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37298.log 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53413.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37298.log.gz 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57868.log 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42574.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-09-10T11:40:22.567 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51375.log 2024-09-10T11:40:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57868.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57868.log.gz 2024-09-10T11:40:22.573 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53176.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51375.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51375.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58837.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53176.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53176.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58837.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58837.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32527.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48970.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32527.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32527.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51770.log 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65058.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-09-10T11:40:22.574 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59318.log 2024-09-10T11:40:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51770.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51770.log.gz 2024-09-10T11:40:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-09-10T11:40:22.575 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59318.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59318.log.gz 2024-09-10T11:40:22.582 INFO:teuthology.orchestra.run.smithi196.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60264.log 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33725.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60264.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60264.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49138.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61320.log 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41165.log: 89.5% 95.4% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.0.49398.log.gz 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57508.log 2024-09-10T11:40:22.583 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32841.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61320.log.gz 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61717.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32841.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57508.log.gz 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32841.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43013.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41425.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43013.log: /var/log/ceph/ceph-client.admin.61717.log: 88.5% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43013.log.gz 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61717.log.gz 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33051.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.41425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44575.log 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33051.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33051.log.gz 2024-09-10T11:40:22.584 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64691.log 2024-09-10T11:40:22.597 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.44575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57155.log 2024-09-10T11:40:22.598 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64691.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64691.log.gz 2024-09-10T11:40:22.606 INFO:teuthology.orchestra.run.smithi196.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T11:40:22.609 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57378.log 2024-09-10T11:40:22.609 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57155.log.gz 2024-09-10T11:40:22.621 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54953.log 2024-09-10T11:40:22.622 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57378.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57378.log.gz 2024-09-10T11:40:22.628 INFO:teuthology.orchestra.run.smithi099.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T11:40:22.630 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39568.log 2024-09-10T11:40:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54953.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54953.log.gz 2024-09-10T11:40:22.631 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-10T11:40:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39568.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39568.log.gz 2024-09-10T11:40:22.633 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40763.log 2024-09-10T11:40:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39797.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-10T11:40:22.634 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61759.log 2024-09-10T11:40:22.635 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40763.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40763.log.gz 2024-09-10T11:40:22.635 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60054.log 2024-09-10T11:40:22.636 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61759.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61759.log.gz 2024-09-10T11:40:22.636 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50037.log 2024-09-10T11:40:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60054.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60054.log.gz 2024-09-10T11:40:22.638 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-09-10T11:40:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50037.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50037.log.gz 2024-09-10T11:40:22.639 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45120.log 2024-09-10T11:40:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46386.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-09-10T11:40:22.640 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39923.log 2024-09-10T11:40:22.641 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45120.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45120.log.gz 2024-09-10T11:40:22.642 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42971.log 2024-09-10T11:40:22.642 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39923.log.gz 2024-09-10T11:40:22.643 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-09-10T11:40:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42971.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-09-10T11:40:22.644 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51463.log 2024-09-10T11:40:22.646 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48886.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-09-10T11:40:22.646 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44249.log 2024-09-10T11:40:22.647 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51463.log.gz 2024-09-10T11:40:22.647 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38642.log 2024-09-10T11:40:22.649 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.44249.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44249.log.gz 2024-09-10T11:40:22.649 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32613.log 2024-09-10T11:40:22.650 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38642.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38642.log.gz 2024-09-10T11:40:22.650 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-09-10T11:40:22.651 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32613.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32613.log.gz 2024-09-10T11:40:22.651 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61999.log 2024-09-10T11:40:22.653 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42163.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-09-10T11:40:22.653 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52631.log 2024-09-10T11:40:22.654 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61999.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61999.log.gz 2024-09-10T11:40:22.654 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34521.log 2024-09-10T11:40:22.655 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52631.log.gz 2024-09-10T11:40:22.655 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55412.log 2024-09-10T11:40:22.657 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34521.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34521.log.gz 2024-09-10T11:40:22.657 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31570.log 2024-09-10T11:40:22.658 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.55412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55412.log.gz 2024-09-10T11:40:22.658 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36563.log 2024-09-10T11:40:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.31570.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31570.log.gz 2024-09-10T11:40:22.659 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62874.log 2024-09-10T11:40:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36563.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36563.log.gz 2024-09-10T11:40:22.661 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-09-10T11:40:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62874.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62874.log.gz 2024-09-10T11:40:22.662 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-09-10T11:40:22.663 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56520.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz 2024-09-10T11:40:22.663 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63395.log 2024-09-10T11:40:22.665 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53539.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-09-10T11:40:22.665 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59360.log 2024-09-10T11:40:22.666 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63395.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63395.log.gz 2024-09-10T11:40:22.666 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-09-10T11:40:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59360.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59360.log.gz 2024-09-10T11:40:22.667 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43588.log 2024-09-10T11:40:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62640.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-09-10T11:40:22.669 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57239.log 2024-09-10T11:40:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43588.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43588.log.gz 2024-09-10T11:40:22.670 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62598.log 2024-09-10T11:40:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57239.log.gz 2024-09-10T11:40:22.671 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-09-10T11:40:22.672 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62598.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62598.log.gz 2024-09-10T11:40:22.673 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65489.log 2024-09-10T11:40:22.674 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43458.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-09-10T11:40:22.674 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-09-10T11:40:22.675 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65489.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65489.log.gz 2024-09-10T11:40:22.675 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-09-10T11:40:22.676 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37248.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-09-10T11:40:22.677 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45789.log 2024-09-10T11:40:22.678 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45330.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-09-10T11:40:22.678 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34437.log 2024-09-10T11:40:22.679 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45789.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45789.log.gz 2024-09-10T11:40:22.679 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33093.log 2024-09-10T11:40:22.680 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34437.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34437.log.gz 2024-09-10T11:40:22.680 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39340.log 2024-09-10T11:40:22.681 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33093.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33093.log.gz 2024-09-10T11:40:22.682 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34479.log 2024-09-10T11:40:22.683 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39340.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39340.log.gz 2024-09-10T11:40:22.683 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43787.log 2024-09-10T11:40:22.684 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34479.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34479.log.gz 2024-09-10T11:40:22.685 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59966.log 2024-09-10T11:40:22.686 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43787.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43787.log.gz 2024-09-10T11:40:22.686 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41582.log 2024-09-10T11:40:22.687 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59966.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59966.log.gz 2024-09-10T11:40:22.687 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59497.log 2024-09-10T11:40:22.688 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.41582.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41582.log.gz 2024-09-10T11:40:22.688 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2024-09-10T11:40:22.689 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59497.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59497.log.gz 2024-09-10T11:40:22.690 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61362.log 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45475.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.47006.log.gz 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47360.log 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64099.log 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47360.log: /var/log/ceph/ceph-client.admin.61362.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61362.log.gz 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47360.log.gz 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62472.log 2024-09-10T11:40:22.693 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62384.log 2024-09-10T11:40:22.694 INFO:teuthology.orchestra.run.smithi099.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64099.log.gz 2024-09-10T11:40:22.694 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65657.log 2024-09-10T11:40:22.708 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62472.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62161.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65657.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62384.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57802.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65657.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50554.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59882.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62161.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57802.log: /var/log/ceph/ceph-client.admin.50554.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50554.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57802.log.gz 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61801.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63641.log 2024-09-10T11:40:22.709 INFO:teuthology.orchestra.run.smithi099.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59882.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.61801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62426.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 89.4%/var/log/ceph/ceph-client.admin.63641.log: -- replaced with /var/log/ceph/ceph-client.admin.61801.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64336.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63641.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46344.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64336.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62426.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64336.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37143.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46344.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37990.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37143.log.gz 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56814.log 2024-09-10T11:40:22.710 INFO:teuthology.orchestra.run.smithi099.stderr: 88.9%/var/log/ceph/ceph-client.admin.37990.log: -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39382.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37990.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56814.log: /var/log/ceph/ceph-client.admin.39382.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56814.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39382.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46174.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57197.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58128.log 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57197.log.gz 2024-09-10T11:40:22.711 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45288.log 2024-09-10T11:40:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.0.58128.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-09-10T11:40:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65615.log 2024-09-10T11:40:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45288.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45288.log.gz 2024-09-10T11:40:22.712 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32967.log 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65615.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65615.log.gz 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54677.log 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32967.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32967.log.gz 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43097.log 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54677.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54677.log.gz 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33398.log 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43097.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43097.log.gz 2024-09-10T11:40:22.714 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47639.log 2024-09-10T11:40:22.715 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33398.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33398.log.gz 2024-09-10T11:40:22.715 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59924.log 2024-09-10T11:40:22.715 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47639.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47639.log.gz 2024-09-10T11:40:22.716 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60222.log 2024-09-10T11:40:22.716 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.59924.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59924.log.gz 2024-09-10T11:40:22.716 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53497.log 2024-09-10T11:40:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60222.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60222.log.gz 2024-09-10T11:40:22.717 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42929.log 2024-09-10T11:40:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53497.log.gz 2024-09-10T11:40:22.718 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-09-10T11:40:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.42929.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42929.log.gz 2024-09-10T11:40:22.719 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46260.log 2024-09-10T11:40:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36731.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-09-10T11:40:22.720 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38801.log 2024-09-10T11:40:22.721 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46260.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46260.log.gz 2024-09-10T11:40:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56478.log 2024-09-10T11:40:22.722 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38801.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38801.log.gz 2024-09-10T11:40:22.723 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49767.log 2024-09-10T11:40:22.723 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56478.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56478.log.gz 2024-09-10T11:40:22.724 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54381.log 2024-09-10T11:40:22.724 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49767.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49767.log.gz 2024-09-10T11:40:22.725 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32483.log 2024-09-10T11:40:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54381.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54381.log.gz 2024-09-10T11:40:22.726 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53726.log 2024-09-10T11:40:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32483.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32483.log.gz 2024-09-10T11:40:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60096.log 2024-09-10T11:40:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53726.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53726.log.gz 2024-09-10T11:40:22.728 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53371.log 2024-09-10T11:40:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.60096.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60096.log.gz 2024-09-10T11:40:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62203.log 2024-09-10T11:40:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53371.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53371.log.gz 2024-09-10T11:40:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58753.log 2024-09-10T11:40:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62203.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62203.log.gz 2024-09-10T11:40:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54467.log 2024-09-10T11:40:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58753.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58753.log.gz 2024-09-10T11:40:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64378.log 2024-09-10T11:40:22.733 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.54467.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54467.log.gz 2024-09-10T11:40:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37948.log 2024-09-10T11:40:22.734 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.64378.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64378.log.gz 2024-09-10T11:40:22.735 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65699.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37948.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.58128.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37948.log.gz 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53852.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34311.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65699.log.gz 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53852.log.gz 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.34311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62556.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48802.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34311.log.gz 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47891.log 2024-09-10T11:40:22.743 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.62556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65405.log 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62556.log.gz 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59659.log 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.65405.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47891.log.gz 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65405.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47205.log: /var/log/ceph/ceph-client.admin.59659.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59659.log.gz 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52589.log 2024-09-10T11:40:22.744 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40889.log 2024-09-10T11:40:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52589.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52589.log.gz 2024-09-10T11:40:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63967.log 2024-09-10T11:40:22.745 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40889.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40889.log.gz 2024-09-10T11:40:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-09-10T11:40:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.63967.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63967.log.gz 2024-09-10T11:40:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36428.log 2024-09-10T11:40:22.746 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-09-10T11:40:22.747 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32673.log 2024-09-10T11:40:22.747 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36428.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36428.log.gz 2024-09-10T11:40:22.748 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40847.log 2024-09-10T11:40:22.748 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32673.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32673.log.gz 2024-09-10T11:40:22.749 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T11:40:22.749 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40847.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40847.log.gz 2024-09-10T11:40:22.750 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-09-10T11:40:22.759 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-10T11:40:22.760 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45601.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-09-10T11:40:22.773 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50079.log 2024-09-10T11:40:22.774 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36605.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-10T11:40:22.774 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49885.log 2024-09-10T11:40:22.775 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.50079.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50079.log.gz 2024-09-10T11:40:22.785 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53768.log 2024-09-10T11:40:22.786 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.49885.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49885.log.gz 2024-09-10T11:40:22.791 INFO:teuthology.orchestra.run.smithi099.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T11:40:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39526.log 2024-09-10T11:40:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.53768.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53768.log.gz 2024-09-10T11:40:22.792 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58795.log 2024-09-10T11:40:22.793 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.39526.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39526.log.gz 2024-09-10T11:40:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-09-10T11:40:22.794 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58795.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58795.log.gz 2024-09-10T11:40:22.795 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58879.log 2024-09-10T11:40:22.795 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.46302.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-09-10T11:40:22.796 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48258.log 2024-09-10T11:40:22.796 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.58879.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58879.log.gz 2024-09-10T11:40:22.797 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52693.log 2024-09-10T11:40:22.797 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.48258.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48258.log.gz 2024-09-10T11:40:22.798 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45204.log 2024-09-10T11:40:22.799 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.52693.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52693.log.gz 2024-09-10T11:40:22.799 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33009.log 2024-09-10T11:40:22.800 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45204.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45204.log.gz 2024-09-10T11:40:22.800 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38885.log 2024-09-10T11:40:22.800 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.33009.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33009.log.gz 2024-09-10T11:40:22.801 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40805.log 2024-09-10T11:40:22.802 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.38885.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38885.log.gz 2024-09-10T11:40:22.802 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56003.log 2024-09-10T11:40:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.40805.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40805.log.gz 2024-09-10T11:40:22.803 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-09-10T11:40:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.56003.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56003.log.gz 2024-09-10T11:40:22.804 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-10T11:40:22.805 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37864.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-09-10T11:40:22.805 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45727.log 2024-09-10T11:40:22.806 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.43955.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-10T11:40:22.806 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36899.log 2024-09-10T11:40:22.807 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.45727.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45727.log.gz 2024-09-10T11:40:22.807 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57592.log 2024-09-10T11:40:22.809 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.36899.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36899.log.gz 2024-09-10T11:40:22.809 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47597.log 2024-09-10T11:40:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.57592.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57592.log.gz 2024-09-10T11:40:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32317.log 2024-09-10T11:40:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.47597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47597.log.gz 2024-09-10T11:40:22.811 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2024-09-10T11:40:22.811 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32317.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32317.log.gz 2024-09-10T11:40:22.812 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32080.log 2024-09-10T11:40:22.812 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.51728.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2024-09-10T11:40:22.813 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-09-10T11:40:22.813 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.32080.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32080.log.gz 2024-09-10T11:40:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/ceph-client.admin.37385.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-09-10T11:40:22.870 INFO:teuthology.orchestra.run.smithi099.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.44575.log.gz 2024-09-10T11:40:22.896 INFO:teuthology.orchestra.run.smithi099.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.41425.log.gz 2024-09-10T11:40:22.945 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T11:40:22.947 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T11:40:23.027 INFO:teuthology.orchestra.run.smithi196.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T11:40:23.100 INFO:teuthology.orchestra.run.smithi196.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T11:40:23.102 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-10T11:40:23.102 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m0.892s 2024-09-10T11:40:23.102 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m4.035s 2024-09-10T11:40:23.102 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.177s 2024-09-10T11:40:23.108 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T11:40:23.152 INFO:teuthology.orchestra.run.smithi099.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T11:40:23.167 INFO:teuthology.orchestra.run.smithi099.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T11:40:23.223 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T11:40:23.428 INFO:teuthology.orchestra.run.smithi099.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T11:40:23.430 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-10T11:40:23.430 INFO:teuthology.orchestra.run.smithi099.stderr:real 0m1.249s 2024-09-10T11:40:23.430 INFO:teuthology.orchestra.run.smithi099.stderr:user 0m6.290s 2024-09-10T11:40:23.430 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m0.568s 2024-09-10T11:40:23.430 INFO:tasks.ceph:Archiving logs... 2024-09-10T11:40:23.431 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi099/log 2024-09-10T11:40:23.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:40:24.628 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi196/log 2024-09-10T11:40:24.629 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T11:40:24.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-10T11:40:24.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-10T11:40:24.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-10T11:40:24.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-10T11:40:24.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-10T11:40:24.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-10T11:40:24.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-10T11:40:25.208 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T11:40:25.217 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T11:40:25.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:40:25.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T11:40:25.314 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T11:40:25.323 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T11:40:25.323 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:40:25.326 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:40:25.341 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:40:25.343 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -191us[ -181us] +/- 66ms 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 7 -122us[ -112us] +/- 77ms 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +323us[ +333us] +/- 63ms 2024-09-10T11:40:25.346 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 -190us[ -159us] +/- 66ms 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -122us[ -122us] +/- 77ms 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +337us[ +337us] +/- 63ms 2024-09-10T11:40:25.348 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:40:25.348 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T11:40:25.358 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T11:40:25.358 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T11:40:25.382 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:40:25.497 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T11:40:25.570 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:40:25.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:25.600 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:40:25.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:25.629 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T11:40:25.639 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T11:40:25.665 INFO:teuthology.task.internal:Duration was 948.903390 seconds 2024-09-10T11:40:25.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T11:40:25.699 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T11:40:25.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:40:25.701 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T11:40:25.738 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:40:25.741 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:40:26.055 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T11:40:26.055 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-10T11:40:26.055 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:40:26.116 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-10T11:40:26.116 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T11:40:26.142 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T11:40:26.142 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:40:26.159 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T11:40:26.209 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T11:40:26.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:40:26.226 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T11:40:26.361 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T11:40:26.371 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T11:40:26.371 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:40:26.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T11:40:26.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T11:40:26.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:40:26.443 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:40:26.463 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-10T11:40:26.483 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-10T11:40:26.497 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:40:26.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:26.529 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T11:40:26.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:40:26.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T11:40:26.561 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T11:40:26.562 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi099 2024-09-10T11:40:26.562 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:40:26.606 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899086/remote/smithi196 2024-09-10T11:40:26.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T11:40:26.641 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T11:40:26.641 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:40:26.644 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T11:40:26.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T11:40:26.702 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T11:40:26.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T11:40:26.725 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T11:40:26.725 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:40:26.727 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T11:40:26.740 INFO:teuthology.orchestra.run.smithi099.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:40 /home/ubuntu/cephtest 2024-09-10T11:40:26.746 INFO:teuthology.orchestra.run.smithi196.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 11:40 /home/ubuntu/cephtest 2024-09-10T11:40:26.748 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T11:40:26.758 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T11:40:26.828 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/mds-full} duration: 948.9033899307251 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T11:40:26.828 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T11:40:26.921 INFO:teuthology.run:pass